25 Oct
2021
25 Oct
'21
3:21 p.m.
A couple of issues with mirroring as R14.0.11 approaches.
(1) Over the last two days a substantial backlog has developed.
Only one mirror was up to date last night with the others
between 6K and 45K files behind.
(2) Over the last four days we're starting to see files that the
secondaries fail to take from the primary. They appear to
have pathnames similar to those shown below. The backlog
of these is now approaching 600. I'm wondering if the
master server rsync filter should block these from coming to
the primary mirror.
--Mike
deb/trinity-sb/dists/bionic/main-r14/binary-amd64/Packages.diff/.~tmp~/2021-10-08-0325.48+2021-10-10-1724.30.gz
deb/trinity-sb/dists/bionic/main-r14/binary-amd64/Packages.diff/.~tmp~/2021-10-08-0019.34+2021-10-10-1724.30.gz
deb/trinity-sb/dists/bionic/main-r14/binary-amd64/Packages.diff/.~tmp~/2021-10-08-0017.21+2021-10-10-1724.30.gz
25 Oct
25 Oct
4:55 p.m.
Hi Mike,
thank you for your excellent monitoring of mirrors!
On Monday 25 of October 2021 17:21:57 Mike Bird wrote:
A couple of issues with mirroring as R14.0.11
approaches.
(1) Over the last two days a substantial backlog has developed.
Only one mirror was up to date last night with the others
between 6K and 45K files behind.
Yes, in connection with the build of the final packages for R14.0.11 in
repository occurs large updates. Dependencies, core modules, libraries and
KOffice were built. During this day, I deliberately preserved a quiet
state that mirrors get time to complete synchronization. Building
applications and meta-packages will follow.
(2) Over the last four days we're starting to see
files that the
secondaries fail to take from the primary. They appear to
have pathnames similar to those shown below. The backlog
of these is now approaching 600. I'm wondering if the
master server rsync filter should block these from coming to
the primary mirror.
As the updates are now in progress more often, there may occur that the
synchronization of the primary mirror is started when the built packages
are being published into the repository. During the publication of each
package, apt metadata files are continuously updated, including files in
the "Packages.diff" directory that is used for faster downloading changes
in repository.
During publishing packages, I try to follow the dates of start and end of
synchronization to primary mirror to minimize such concurrents. However,
sometimes such a situation can occur.
Helped today's quiet state as I hoped?
--Mike
deb/trinity-sb/dists/bionic/main-r14/binary-amd64/Packages.diff/.~tmp~/2
021-10-08-0325.48+2021-10-10-1724.30.gz
deb/trinity-sb/dists/bionic/main-r14/binary-amd64/Packages.diff/.~tmp~/2
021-10-08-0019.34+2021-10-10-1724.30.gz
deb/trinity-sb/dists/bionic/main-r14/binary-amd64/Packages.diff/.~tmp~/2
021-10-08-0017.21+2021-10-10-1724.30.gz
____________________________________________________
Cheers
--
Slávek
5:42 p.m.
On Mon October 25 2021 09:55:16 Slávek Banko wrote:
During publishing packages, I try to follow the dates
of start and end of
synchronization to primary mirror to minimize such concurrents. However,
sometimes such a situation can occur.
I think it would be worth filtering .~tmp~ at your end.
Helped today's quiet state as I hoped?
In the last 10 hours kuiper has only gone from 44395 files backlog
to 44354 files backlog. Looks like a problem on his end. I'll contact
him if he doesn't fix it in the next few days.
The others are missing the 546 .~tmp~ files, except for copernicus which
is missing the .~tmp~ files plus 9 more.
All in all I'd say we're as up to date as we can be.
--Mike
6:21 p.m.
On Monday 25 of October 2021 19:42:15 Mike Bird wrote:
Great, soon I will start publishing applications packages.
On Mon October 25 2021 09:55:16 Slávek Banko wrote:
This is strange because ".~tmp~" is listed in the patters to be excluded. I
tried a while ago to manually create a folder of such a name and the rsync
call statement did not show it.
During publishing packages, I try to follow the
dates of start and end
of synchronization to primary mirror to minimize such concurrents.
However, sometimes such a situation can occur.
I think it would be worth filtering .~tmp~ at your end.
Helped
today's quiet state as I hoped?
In the last 10 hours kuiper has only gone from 44395 files backlog
to 44354 files backlog. Looks like a problem on his end. I'll contact
him if he doesn't fix it in the next few days.
The others are missing the 546 .~tmp~ files, except for copernicus which
is missing the .~tmp~ files plus 9 more.
All in all I'd say we're as up to date as we can be.
--Mike
____________________________________________________
Thank you!
--
Slávek
6:53 p.m.
On Mon October 25 2021 11:21:35 Slávek Banko wrote:
This is strange because ".~tmp~" is listed
in the patters to be excluded. I
tried a while ago to manually create a folder of such a name and the rsync
call statement did not show it.
Looks like we picked them up over the last few days. They seem
to be filtered now but a rsync bug is preventing them from being
deleted.
We use --delay-updates but that seems to be blocking their deletion.
I'll do a one-time pass without --delay-updates to get rid of them.
--Mike
915
days inactive
915
days old
5 comments
2 participants
participants (2)
-
Mike Bird -
Slávek Banko