30 Jul
2018
30 Jul
'18
2:18 a.m.
On Sunday 29 of July 2018 12:15:07 Mike Bird wrote:
On Sun July 29 2018 02:09:22 Slávek Banko wrote:
Hi Mike,
you mention "Indeed Trinity successfully used a single repo incorporating
trinity-nightly-builds for many years." But here is one basic mistake.
The repository trinity-nightly-builds is one of the separate repositories
on QuickBuild. At the time of publishing the new release, the contents of
this repository is on QuickBuild copied into the official repository. And
this is the moment when synchronization of complete content to the mirror
begins. There is no "It reduces the number of downloads and upgrades
needed for new minor releases and generally provides a smoother and more
efficient transition from testing to release." It just does not work in
that way.
To make the situation even more complicated, you know that I also have the
repositories for preliminary stable packages that are on QuickBuild.
However, these repositories is not normally used by any user just for
reasons that were with the 'axis' repository.
These repositories on QuickBuild I mainly use to prepare packages that
will then be published to the official repositories as a new release.
Into these repositories, I'm uploading identical source packages as I
uploaded into Preliminary Stable Builds repository. But on QuickBuild
packages must be built again because into QuickBuild is not possible to
upload binary packages, only source packages.
It is just these repositories that contain the final packages for new TDE
release before they are published into the official repository. And just
these packages are downloaded into the cache on the redirector already
during the build == while the bandwidth to the primary server is not
loaded! And that is the main trick why this cache is so important when
releasing a new version. While to the primary mirror synchronization
starts at the moment of publishing packages, these packages are already
available on the redirector's cache before that time. Simple and very
effective.
Since this synchronization of packages from the primary server to the
redirector cache is for deb packages, the advantate is to use apt-mirror
for this purpose. Apt-mirror works in a way that repositories remain
consistent during the update. This is why I use apt-mirror for DEB
packages, while for rpm packages I use rsync.
Here is a small example of the trick. The current stable version is
R14.0.4. Therefore, listing the tdelibs folder from apt repository
displays packages for R14.0.4:
http://mirror.ppa.trinitydesktop.org/trinity/trinity-r14.0.0/debian/pool/ma…
For example DSC file for Debian 8 (Jessie) is as follows:
http://mirror.ppa.trinitydesktop.org/trinity/trinity-r14.0.0/debian/pool/ma…
Since the builds of R14.0.5 final packages are currently underway and
these are continuously synchronized to the cache on the redirector, it is
now already possible to get files for the future R14.0.5 version. For
example DSC file for Debian 8 (Jessie) is as follows:
http://mirror.ppa.trinitydesktop.org/trinity/trinity-r14.0.0/debian/pool/ma…
Once the packages will be published to the official repository, the folder
listing starts showing new files, the meta-information repositories will
refer to new files, and these files will be ready in the cache on the
redirector. Simply because they are there in advance.
Is it now clear what I mean? Is it now clear why the cache on the
redirector is very useful? And it is clear that keeping Preliminary
Stable Builds independent of QuickBuild is no duplication because the
same duplication is also going on at QuickBuild?
Cheers
--
Slávek
As I mentioned above, all official repositories
(signed by the
official Trinity key) are managed on QuickBuild. As I explained in
previous emails, Preliminary Stable Builds are created independently
of QuickBuild - independent builders (used pbuilder on my builders),
independent repository maintenance (used reprepro on my server).
That's why PSB repositories simply can not be signed with an official
Trinity key that is integrated with QuickBuild. I could create some
general GPG key for signing repository instead of my personal, but it
would still be a key other than the official Trinity key. So it seems
like a futile change.
Regarding the propose to rename from 'stable' to 'testing' builds,
this change is not possible. Existing Preliminary Stable Builds are
built on a 'stable' branch (now r14.0.x) == this is preliminary
packages for the next maintenance relase - therefore 'stable' in the
repository name. Additionally, the second repository named
Preliminary Testing Builds has already been prepared. This second
repository is built on a 'master' branch == this is preliminary
packages for the next major release, which rightfully deserves the
naming of 'testing'. The official announcement of this 'testing'
repository can be expected soon.
Hi Slávek,
I am not yet understanding why we have so much duplication. Why is
there not a "testing" release in the main repository instead of a
having a main repo and a DEB PSB repo and a RPM PSB repo in different
locations and using different tooling? (And soon in addition a new DEB
testing repo?)
I understand that PSB updates every two hours whereas the primary
mirror currently checks for updates three hours after the completion of
the previous run, but I could change the three hour delay to a ten
minute delay without adversely impacting Tim's bandwidth. What is the
average volume (GB) of DEB PSB updates published per day?
The standard single Debian repository structure is well understood by
all concerned. It reduces the number of downloads and upgrades needed
for new minor releases and generally provides a smoother and more
efficient transition from testing to release. Indeed Trinity
successfully used a single repo incorporating trinity-nightly-builds
for many years.
--Mike