16 Jun
2023
16 Jun
'23
3:51 a.m.
On Thu June 15 2023 19:25:53 Michele Calgaro wrote:
if I remember correctly, we are still providing
R14.0.13 on the mirror to
ease user migration, am I wrong? So R14.0.13 and R14.1.0 would take up lot
of space.
If that is the case, would make more sense to drop off R14.0.13 from the
mirror and keep trinity-testing? What do you think?
If my one-liner to analyze web logs on the primary mirror while filtering
out the redirector's status checks is accurate than I think you're right
Michele. Here's the top 20 by second-level directory, counting accesses
not bytes, over the last 15 days or so.
145327 /deb/trinity-sb
129511 /deb/trinity-testing
48093 /deb/trinity-r14.0.x
25878 /deb/trinity-r14.1.x
22105 /rpm/osstw
21067 /ulab/ubuntu
17410 /deb/trinity-builddeps-r14.0.x
11493 /rpm/f38
10715 /trinity-r14.0.0/debian
10461 /rpm/el6
10172 /rpm/el5
9957 /trinity-r14.0.0/ubuntu
9336 /rpm/el8
9008 /rpm/el7
8930 /rpm/pclinuxos
7487 /rpm/mga8
7233 /rpm/opensuse15.5
6659 /trinity/rpm
6291 /rpm/mga9
5885 /rpm/el9
Unless someone comes up with a better idea in the next 24 hours I'll
mask r14.0.x and builddeps on the mirror (on the assumption that most
of that traffic will soon move from 14.0.x to 14.1.x) and restore
trinity-testing.
FWIW here are the directory sizes:
2.0G ./releases/R14.0.8
2.4G ./releases/R14.0.0
1.9G ./releases/R14.0.11
1.9G ./releases/R14.0.7
2.3G ./releases/R14.0.5
2.3G ./releases/R14.0.4
1.9G ./releases/R14.1.0
2.0G ./releases/R14.0.9
1.9G ./releases/R14.0.6
5.3G ./releases/R14.0.1
2.4G ./releases/R14.0.3
1.9G ./releases/R14.0.12
2.0G ./releases/R14.0.10
1.9G ./releases/R14.0.13
2.4G ./releases/R14.0.2
34G ./releases
650M ./archlinux/x86_64
299M ./archlinux/aarch64
296M ./archlinux/armv7h
432M ./archlinux/powerpc64le
1.7G ./archlinux
78G ./deb/trinity-r14.1.x
52G ./deb/trinity-sb
115G ./deb/trinity-r14.0.x
1.9G ./deb/trinity-builddeps-r14.0.x
246G ./deb
1.3G ./ulab/ubuntu
1.3G ./ulab
3.6G ./rpm/osstw
2.4G ./rpm/el7
2.2G ./rpm/opensuse15.5
2.5G ./rpm/el9
3.5G ./rpm/mga8
2.5G ./rpm/el8
2.5G ./rpm/f37
3.3G ./rpm/el6
4.2G ./rpm/el5
1.7G ./rpm/pclinuxos
2.5G ./rpm/f38
2.2G ./rpm/opensuse15.4
2.5G ./rpm/mga9
2.5G ./rpm/f36
38G ./rpm
321G .
--Mike
19 Jun
19 Jun
11:40 p.m.
New subject: Mirror Growth
On Friday 16 of June 2023 05:51:45 Mike Bird via tde-devels wrote:
On Thu June 15 2023 19:25:53 Michele Calgaro wrote:
Hi Mike,
thank you for your supervision and maintenance of the mirror. During that
time there were several things:
Before the release of Bookworm, I noticed that a staging repository was
accidentally published on the mirror, where the final packages for
Bookworm and Raspbian Bookworm were being prepared. This caused these
packages to be there twice for a while. In any case, publishing the final
packages for Bookworm and Raspbian Bookworm means that a few more GiB have
been consumed there.
During the preparation of the special update of TQt3 and tdelibs for
R14.1.0, there was another temporary increase in the occupied space.
However, it has already been cleaned up again.
We dropped Stretch, Xenial and Impish in PTB and Impish in PSB. This led to
some reduction in the space occupied. However, there will be separate
packages for Trixie and Mantic in a while, so it will take up some space
again.
You may recall that when there was a plan to create a new repository for
R14.1.x, I mentioned that it would later represent an increase in volume
of about 80 GiB. In short, it could be said that increasing the size of
the mirror is exactly as expected.
At the time R14.1.0 was being prepared, there was a very unusual situation
where the volume was reduced because the PTB and PSB became identical. As
well as the subsequently released R14.1.0 in the new repository. But at
the moment of release, new packages for PSB and new packages for PTB
started to appear there. That's why these repositories started to take on
their usual size again. Even at this point, there are still some packages
identical to R14.1.x, PSB and PTB, so you can still expect to consume more
space there. So it's not an incorrect or unexpected state.
To reduce the volume on the mirror, we must therefore consider excluding
some older data from the mirror. Originally, I assumed that the R14.0.x
repository would be removed from the mirror after the release of R14.1.1.
If it seems appropriate, we can exclude it earlier. What is your opinion?
Probably no one will mind if we exclude the source tarballs for the
R14.0.x series. This will help free up some additional space. For
cdimmages, I can set excludes on the primary archive.
It will certainly be useful to organize the exclusion of the old data so
that the PTB repository can be back on the mirrors.
Cheers
--
Slávek
if I remember correctly, we are still providing
R14.0.13 on the mirror
to ease user migration, am I wrong? So R14.0.13 and R14.1.0 would take
up lot of space.
If that is the case, would make more sense to drop off R14.0.13 from
the mirror and keep trinity-testing? What do you think?
If my one-liner to analyze web logs on the primary mirror while
filtering out the redirector's status checks is accurate than I think
you're right Michele. Here's the top 20 by second-level directory,
counting accesses not bytes, over the last 15 days or so.
145327 /deb/trinity-sb
129511 /deb/trinity-testing
48093 /deb/trinity-r14.0.x
25878 /deb/trinity-r14.1.x
22105 /rpm/osstw
21067 /ulab/ubuntu
17410 /deb/trinity-builddeps-r14.0.x
11493 /rpm/f38
10715 /trinity-r14.0.0/debian
10461 /rpm/el6
10172 /rpm/el5
9957 /trinity-r14.0.0/ubuntu
9336 /rpm/el8
9008 /rpm/el7
8930 /rpm/pclinuxos
7487 /rpm/mga8
7233 /rpm/opensuse15.5
6659 /trinity/rpm
6291 /rpm/mga9
5885 /rpm/el9
Unless someone comes up with a better idea in the next 24 hours I'll
mask r14.0.x and builddeps on the mirror (on the assumption that most
of that traffic will soon move from 14.0.x to 14.1.x) and restore
trinity-testing.
FWIW here are the directory sizes:
2.0G ./releases/R14.0.8
2.4G ./releases/R14.0.0
1.9G ./releases/R14.0.11
1.9G ./releases/R14.0.7
2.3G ./releases/R14.0.5
2.3G ./releases/R14.0.4
1.9G ./releases/R14.1.0
2.0G ./releases/R14.0.9
1.9G ./releases/R14.0.6
5.3G ./releases/R14.0.1
2.4G ./releases/R14.0.3
1.9G ./releases/R14.0.12
2.0G ./releases/R14.0.10
1.9G ./releases/R14.0.13
2.4G ./releases/R14.0.2
34G ./releases
650M ./archlinux/x86_64
299M ./archlinux/aarch64
296M ./archlinux/armv7h
432M ./archlinux/powerpc64le
1.7G ./archlinux
78G ./deb/trinity-r14.1.x
52G ./deb/trinity-sb
115G ./deb/trinity-r14.0.x
1.9G ./deb/trinity-builddeps-r14.0.x
246G ./deb
1.3G ./ulab/ubuntu
1.3G ./ulab
3.6G ./rpm/osstw
2.4G ./rpm/el7
2.2G ./rpm/opensuse15.5
2.5G ./rpm/el9
3.5G ./rpm/mga8
2.5G ./rpm/el8
2.5G ./rpm/f37
3.3G ./rpm/el6
4.2G ./rpm/el5
1.7G ./rpm/pclinuxos
2.5G ./rpm/f38
2.2G ./rpm/opensuse15.4
2.5G ./rpm/mga9
2.5G ./rpm/f36
38G ./rpm
321G .
--Mike
____________________________________________________
20 Jun
20 Jun
12:22 a.m.
New subject: Mirror Growth
On Mon June 19 2023 16:40:06 Slávek Banko via tde-devels wrote:
To reduce the volume on the mirror, we must therefore
consider excluding
some older data from the mirror. Originally, I assumed that the R14.0.x
repository would be removed from the mirror after the release of R14.1.1.
If it seems appropriate, we can exclude it earlier. What is your opinion?
Probably no one will mind if we exclude the source tarballs for the
R14.0.x series. This will help free up some additional space. For
cdimmages, I can set excludes on the primary archive.
It will certainly be useful to organize the exclusion of the old data so
that the PTB repository can be back on the mirrors.
I'm in an odd state where I prepay for the mirror VPS a year
at a time and there are still several more months left but the
provider is discontinuing this particular service so I will
have to switch to another service by November.
For now the mirrors have PSB and PTB and R14.1.x but R14.0.x
has been removed.
The source tarballs for R14.1.x amount to only 2GB of the 72GB
of R14.1.x. I doubt that excluding sources from R14.0.x would
reduce the 115GB of R14.0.x enough for it to fit.
Meantime I'm keeping my eyes open for good deals on large VPSs.
They don't come up often. This will determine whether I can
resume mirroring R14.0.x.
--Mike
21 Jun
21 Jun
12:38 a.m.
New subject: Mirror Growth
On Tuesday 20 of June 2023 02:22:06 Mike Bird via tde-devels wrote:
On Mon June 19 2023 16:40:06 Slávek Banko via
tde-devels wrote:
Oh, I see, no need to think about it, it's already been excluded...
I'll add an exclude on the primary archive, so that it matches :)
To reduce the volume on the mirror, we must
therefore consider
excluding some older data from the mirror. Originally, I assumed that
the R14.0.x repository would be removed from the mirror after the
release of R14.1.1. If it seems appropriate, we can exclude it
earlier. What is your opinion? Probably no one will mind if we exclude
the source tarballs for the R14.0.x series. This will help free up
some additional space. For cdimmages, I can set excludes on the
primary archive.
It will certainly be useful to organize the exclusion of the old data
so that the PTB repository can be back on the mirrors.
I'm in an odd state where I prepay for the mirror VPS a year
at a time and there are still several more months left but the
provider is discontinuing this particular service so I will
have to switch to another service by November.
For now the mirrors have PSB and PTB and R14.1.x but R14.0.x
has been removed.
The source tarballs for R14.1.x amount to only 2GB of
the 72GB
of R14.1.x. I doubt that excluding sources from R14.0.x would
reduce the 115GB of R14.0.x enough for it to fit.
I meant the source tarballs located in the /releases/ folder. There it
represents a larger volume than just 2 GiB.
Meantime I'm keeping my eyes open for good deals
on large VPSs.
They don't come up often. This will determine whether I can
resume mirroring R14.0.x.
The intention here was to exclude R14.0.x from mirroring. It just happened
sooner than I expected. But it does not matter. It is useful to keep the
mirror at some reasonably acceptable size. I hope that you will soon
manage to find an alternative for your VPS. Thank you for your
contribution to the project!
--Mike
____________________________________________________
Cheers
--
Slávek
3:11 a.m.
New subject: Mirror Growth
On Tue June 20 2023 17:38:52 Slávek Banko via tde-devels wrote:
The intention here was to exclude R14.0.x from
mirroring. It just happened
sooner than I expected. But it does not matter. It is useful to keep the
mirror at some reasonably acceptable size. I hope that you will soon
manage to find an alternative for your VPS. Thank you for your
contribution to the project!
I have a larger replacement mirror now but it will take a few days to
configure and sync it and replace the old mirror.
--Mike
24 Jun
24 Jun
9:44 a.m.
New subject: Mirror Growth
The exclude-to-mirror setting is failing to mirror a important thing: trinity-keyring.deb
is not mirrored.
Only three items inside "deb" are now mirrored:
trinity-r14.1.x
trinity-sb
trinity-testing
See:
http://tde-mirror.yosemite.net/trinity/deb/
http://kuiper.mirrorservice.org/sites/trinitydesktop.org/trinity/deb/
http://copernicus.mirrorservice.org/sites/trinitydesktop.org/trinity/deb/
http://ftp.fau.de/trinity/deb/
https://ftp.fau.de/trinity/deb/
http://tde-mirror-01.inet-design.com/trinity/deb/
I note a exception: http://mirror.nasutek.com/trinity/deb/ does mirror
trinity-keyring.deb
Please do "deb/trinity-keyring.deb" mirrorable. Thank you.
En miércoles, 21 de junio de 2023, 02:40:56 CEST, Slávek Banko via tde-devels
<devels(a)trinitydesktop.org> escribió:
On Tuesday 20 of June 2023 02:22:06 Mike Bird via tde-devels wrote:
On Mon June 19 2023 16:40:06 Slávek Banko via
tde-devels wrote:
Oh, I see, no need to think about it, it's already been excluded...
I'll add an exclude on the primary archive, so that it matches :)
To reduce the volume on the mirror, we must
therefore consider
excluding some older data from the mirror. Originally, I assumed that
the R14.0.x repository would be removed from the mirror after the
release of R14.1.1. If it seems appropriate, we can exclude it
earlier. What is your opinion? Probably no one will mind if we exclude
the source tarballs for the R14.0.x series. This will help free up
some additional space. For cdimmages, I can set excludes on the
primary archive.
It will certainly be useful to organize the exclusion of the old data
so that the PTB repository can be back on the mirrors.
I'm in an odd state where I prepay for the mirror VPS a year
at a time and there are still several more months left but the
provider is discontinuing this particular service so I will
have to switch to another service by November.
For now the mirrors have PSB and PTB and R14.1.x but R14.0.x
has been removed.
The source tarballs for R14.1.x amount to only 2GB of
the 72GB
of R14.1.x. I doubt that excluding sources from R14.0.x would
reduce the 115GB of R14.0.x enough for it to fit.
I meant the source tarballs located in the /releases/ folder. There it
represents a larger volume than just 2 GiB.
Meantime I'm keeping my eyes open for good deals
on large VPSs.
They don't come up often. This will determine whether I can
resume mirroring R14.0.x.
The intention here was to exclude R14.0.x from mirroring. It just happened
sooner than I expected. But it does not matter. It is useful to keep the
mirror at some reasonably acceptable size. I hope that you will soon
manage to find an alternative for your VPS. Thank you for your
contribution to the project!
--Mike
____________________________________________________
Cheers
--
Slávek
____________________________________________________
tde-devels mailing list -- devels(a)trinitydesktop.org
To unsubscribe send an email to devels-leave(a)trinitydesktop.org
Web mail archive available at
https://mail.trinitydesktop.org/mailman3/hyperkitty/list/devels@trinitydesk…
10:10 a.m.
New subject: Mirror Growth
On Sat June 24 2023 02:44:52 Alexis PM via tde-devels wrote:
The exclude-to-mirror setting is failing to mirror a
important thing:
trinity-keyring.deb is not mirrored. Only three items inside "deb" are now
mirrored:
trinity-r14.1.x
trinity-sb
trinity-testing
See:
http://tde-mirror.yosemite.net/trinity/deb/
http://kuiper.mirrorservice.org/sites/trinitydesktop.org/trinity/deb/
http://copernicus.mirrorservice.org/sites/trinitydesktop.org/trinity/deb/
http://ftp.fau.de/trinity/deb/
https://ftp.fau.de/trinity/deb/
http://tde-mirror-01.inet-design.com/trinity/deb/
I note a exception: http://mirror.nasutek.com/trinity/deb/ does mirror
trinity-keyring.deb Please do "deb/trinity-keyring.deb" mirrorable. Thank
you.
Hi Alexis,
Thank you for bringing this to our attention.
It turns out that trinity-keyring.deb is actually mirrored but it's just
a symlink into the r14.0.x repository. As r14.0.x is not currently mirrored
the symlink is dead and not shown.
I've just finished changing the primary mirror to a larger VPS. After a
day or two to check it is running smoothly I will resume mirroring r14.0.x.
Meanwhile Slávek may want to consider changing the master copy of the link to
point into the current r14.1.x repository.
FWIW the "official" path to TDE is http://mirror.ppa.trinitydesktop.org/ which
is a redirector which knows which mirrors currently have which files. It can
always find the keyring for you.
--Mike
25 Jun
25 Jun
8:10 a.m.
New subject: Mirror Growth
On Saturday 24 of June 2023 12:10:31 Mike Bird via tde-devels wrote:
On Sat June 24 2023 02:44:52 Alexis PM via tde-devels
wrote:
Hi,
yes, Mike accurately identified the problem. I did not realize that symlink
referred to R14.0.x repository, and that after exclude R14.0.x will be
invalid on mirrors. I changed symlink to refer to R14.1.x repository.
Cheers
--
Slávek
The exclude-to-mirror setting is failing to
mirror a important thing:
trinity-keyring.deb is not mirrored. Only three items inside "deb" are
now mirrored:
trinity-r14.1.x
trinity-sb
trinity-testing
See:
http://tde-mirror.yosemite.net/trinity/deb/
http://kuiper.mirrorservice.org/sites/trinitydesktop.org/trinity/deb/
http://copernicus.mirrorservice.org/sites/trinitydesktop.org/trinity/d
eb/ http://ftp.fau.de/trinity/deb/
https://ftp.fau.de/trinity/deb/
http://tde-mirror-01.inet-design.com/trinity/deb/
I note a exception: http://mirror.nasutek.com/trinity/deb/ does mirror
trinity-keyring.deb Please do "deb/trinity-keyring.deb" mirrorable.
Thank you.
Hi Alexis,
Thank you for bringing this to our attention.
It turns out that trinity-keyring.deb is actually mirrored but it's just
a symlink into the r14.0.x repository. As r14.0.x is not currently
mirrored the symlink is dead and not shown.
I've just finished changing the primary mirror to a larger VPS. After a
day or two to check it is running smoothly I will resume mirroring
r14.0.x.
Meanwhile Slávek may want to consider changing the master copy of the
link to point into the current r14.1.x repository.
FWIW the "official" path to TDE is http://mirror.ppa.trinitydesktop.org/
which is a redirector which knows which mirrors currently have which
files. It can always find the keyring for you.
--Mike
____________________________________________________ 
11:47 a.m.
New subject: Mirror Growth
yes, Mike accurately identified the problem. I did not
realize that symlink
referred to R14.0.x repository, and that after exclude R14.0.x will be
invalid on mirrors. I changed symlink to refer to R14.1.x repository.
would be even better if the keyring package was not bound to a specific branch but just
general, IMO.
Cheers
Michele
546
days inactive
555
days old
8 comments
4 participants
participants (4)
-
Alexis PM -
Michele Calgaro -
Mike Bird -
Slávek Banko