21 Jul
2018
21 Jul
'18
11:55 p.m.
On Sunday 22 of July 2018 00:12:21 Mike Bird wrote:
Yes, releasing the version will always cause a load of bandwidth to the
primary server for several days. However, thanks to the cache on the
redirector, this load can be lower and the packages are accessible to
users earlier.
Note: If I remember correctly, the RPM packages cache on the redirector
have not yet been set at time of release R14.0.4. Therefore, when we
release R14.0.5, I expect the additional benefit of this cache.
Hi Slavek,
Thank you for the comprehensive response. A few points come to mind.
(1) TDE mirrors are not synced "daily".
Due to master server bandwidth limitations, primary sync runs can
take anywhere from a minute to a week.
The primary mirror normally syncs from Tim's master server
continuously subject to a bandwidth limit selected by Tim and
with a short delay between the completion of one sync run
and the start of the next. During major release rollouts
syncing is changed e.g. by reducing delay between sync runs
and by deferring dbg/debuginfo/dev/devel/tar/src files when more
urgent files need to be mirrored.
Secondary mirrors sync from the primary on their own schedules.
Currently some secondaries sync three times per day and some four.
Interesting. I supposed that the synchronization is initialized once a
day, and then it only depends on whether it has finished or that it will
be delayed when previous synchronization is still running.
On Sat July 21 2018 10:20:33 Slávek Banko wrote:
Do not worry, this caching is not counterproductive, just the opposite.
The synchronization of packages into the redirector cache is done before
the packages are published to the official repository == before the
synchronization to the primary mirror starts. This means that
synchronization into redirector cache uses the time when the bandwidth on
the primary server is not used. This is the main trick of this caching -
that's why this caching is very beneficial.
Separate substantial benefit is caching of RPM packages. RPM packages are
being prepared by François on their own repository. From this repository,
the packages are first synchronized into the primary server => load of
bandwidth on the incoming. Then are synchronized to the primary mirror =>
load of bandwidth on the outgoing. This both represents a long time. On
the redirector, packages are cached directly from the François repository
== this does not cause any bandwidth load on the primary server, and the
packages may be available before they are downloaded to the primary
server.
Therefore, I would like to say that caching on the redirector is very
beneficial both for saving bandwidth to the primary server and also for
faster availability of packages to users. Both sides wins :)
On Sunday 24 of June 2018 20:59:46 Mike Bird
wrote:
(2) This seems counter-productive. The reason for implementing a
distinct primary mirror was so that each file would only be
transferred once from Tim's master server due to bandwidth
limitations.
(6) To avoid overloading build farm bandwidth, it
is best not to
announce releases until they are fully mirrored. If users start
pulling unmirrored files they are served from the build farm,
overloading its bandwidth and slowing mirroring, so things get very
slow for everyone.
This is no longer necessary. As I mentioned above, the redirector has
a cache that is filled with new packages before the official release.
Therefore, at the time of release, if packages are not yet available
on official mirrors, they may be served directly from this cache.
This makes the packages available to users quickly and without
overloading the line to the primary server. Already
R14.0.4 has successfully used this cache at the time of
release.
(3) There were severe slowdowns for several days during R14.0.4 rollout
which appeared to be due to master server bandwidth limitations.
--Mike
Cheers
--
Slávek