Hi all,
the tenth service on the migration list is the primary archive.
-- Migration status --
The change of the primary archive was made before the release of R14.0.7 (in October 2019). This removed the dependency of the primary archive on QuickBuild, which made it possible to make a change in the archive layout and cleanup. The reprepro tool is now used to manage repositories of deb packages, which allows us to publish new releases very smoothly. Thanks to the use of reprepro, Contents files for apt-file are also available.
Because the primary archive is too large to be placed on a tde-box or tde-vps, it is located on my home server. In order to reduce the size for the mirror system, packages of older versions are excluded.
I would like to thank those who provides mirrors for our repositories, and especially Mike Bird for his efforts and attention in managing the primary mirror and overseeing the status of other mirrors.
-- What needs to be done --
We may consider placing the primary archive in a hosting center - for example, by purchasing larger disks for tde-box or by increasing the space provided for tde-vps. However, it will probably be a good idea to consider such changes only after we are able to manage the project funds and accept donations.
-- More ideas and suggestions?
Do you have any other ideas and suggestions?
Cheers
On 2020/12/20 04:00 AM, Slávek Banko via tde-users wrote:
We may consider placing the primary archive in a hosting center - for example, by purchasing larger disks for tde-box or by increasing the space provided for tde-vps. However, it will probably be a good idea to consider such changes only after we are able to manage the project funds and accept donations.
This is defintely something we will need to do in future, to move away from possible one-person-dependant services. I agree with you we need to figure out how to handle donations and project funds, then we can relook at this.
Cheers Michele
On Sunday 20 December 2020 12:02:26 am Michele Calgaro via tde-users wrote:
On 2020/12/20 04:00 AM, Slávek Banko via tde-users wrote:
We may consider placing the primary archive in a hosting center - for example, by purchasing larger disks for tde-box or by increasing the space provided for tde-vps. However, it will probably be a good idea to consider such changes only after we are able to manage the project funds and accept donations.
This is defintely something we will need to do in future, to move away from possible one-person-dependant services. I agree with you we need to figure out how to handle donations and project funds, then we can relook at this.
In that vein (away from possible one-person-dependant services):
- How big is the archive? (<250GB? ~4TB?) - How simple is it to back it up? (a few rsync’s? tarball?)
I have no issues storing a copy, or two, of the archive on one of my backup servers (which are in hosting centers).
Best, Michael
* If it’s in the 10-15GB range I can dump multiple copies on my DNS servers as well.
Dne Sunday 20 of December 2020 18:12:25 Michael via tde-users napsal(a):
On Sunday 20 December 2020 12:02:26 am Michele Calgaro via tde-users
wrote:
On 2020/12/20 04:00 AM, Slávek Banko via tde-users wrote:
We may consider placing the primary archive in a hosting center - for example, by purchasing larger disks for tde-box or by increasing the space provided for tde-vps. However, it will probably be a good idea to consider such changes only after we are able to manage the project funds and accept donations.
This is defintely something we will need to do in future, to move away from possible one-person-dependant services. I agree with you we need to figure out how to handle donations and project funds, then we can relook at this.
In that vein (away from possible one-person-dependant services):
- How big is the archive? (<250GB? ~4TB?)
- How simple is it to back it up? (a few rsync’s? tarball?)
I have no issues storing a copy, or two, of the archive on one of my backup servers (which are in hosting centers).
I'm glad you're interested in the details.
Current size and occupied space of the partition: /dev/mapper/lvm1-tde_data 400G 336G 65G 84% /srv/tde
A subset of this volume is synchronized to the mirror system. First to the primary mirror, from which the other mirrors are then synchronized. This synchronization usually takes place several times a day, with pauses of 3 hours between the individual synchronizations.
The size always decreases significantly at the moment of release, when the contents of the official repository and PSB are identical. The size of the repository is now constantly growing again as the packages in PSB are built for R14.0.10~pre. Such an increase can be almost 100 GiB. At the same time, there is a gradual increase in volume when adding support for new distributions.
During the outage, which I announced in early December, I replaced my home server's hardware with slightly newer components, including the purchase of new, larger disks. So now I have a sufficient reserve of disk capacity again.
The specified partition is on two disks (RAID1). In addition to these disks, I make a regular daily local backup on an external usb disk. The backup disk requires significantly more capacity, as the daily differences can be large - from hundreds of MiBs to tens of GiBs. Next to this, I'm making a copy of the archive on my next machine in a remote location (two disks in RAID1). Copying to this remote location is not regular yet.
Best, Michael
- If it’s in the 10-15GB range I can dump multiple copies on my DNS
servers as well. ____________________________________________________
Cheers
On Sunday 20 December 2020 01:34:29 pm Slávek Banko via tde-users wrote:
Dne Sunday 20 of December 2020 18:12:25 Michael via tde-users napsal(a):
On Sunday 20 December 2020 12:02:26 am Michele Calgaro via tde-users
wrote:
On 2020/12/20 04:00 AM, Slávek Banko via tde-users wrote:
We may consider placing the primary archive in a hosting center - for example, by purchasing larger disks for tde-box or by increasing the space provided for tde-vps. However, it will probably be a good idea to consider such changes only after we are able to manage the project funds and accept donations.
This is defintely something we will need to do in future, to move away from possible one-person-dependant services. I agree with you we need to figure out how to handle donations and project funds, then we can relook at this.
In that vein (away from possible one-person-dependant services):
- How big is the archive? (<250GB? ~4TB?)
- How simple is it to back it up? (a few rsync’s? tarball?)
I have no issues storing a copy, or two, of the archive on one of my backup servers (which are in hosting centers).
I'm glad you're interested in the details.
Current size and occupied space of the partition: /dev/mapper/lvm1-tde_data 400G 336G 65G 84% /srv/tde
A subset of this volume is synchronized to the mirror system. First to the primary mirror, from which the other mirrors are then synchronized. This synchronization usually takes place several times a day, with pauses of 3 hours between the individual synchronizations.
The size always decreases significantly at the moment of release, when the contents of the official repository and PSB are identical. The size of the repository is now constantly growing again as the packages in PSB are built for R14.0.10~pre. Such an increase can be almost 100 GiB. At the same time, there is a gradual increase in volume when adding support for new distributions.
During the outage, which I announced in early December, I replaced my home server's hardware with slightly newer components, including the purchase of new, larger disks. So now I have a sufficient reserve of disk capacity again.
The specified partition is on two disks (RAID1). In addition to these disks, I make a regular daily local backup on an external usb disk. The backup disk requires significantly more capacity, as the daily differences can be large - from hundreds of MiBs to tens of GiBs. Next to this, I'm making a copy of the archive on my next machine in a remote location (two disks in RAID1). Copying to this remote location is not regular yet.
Hi Slávek,
Lol, interesting, but TMI ;)
- What’s the max size you’ve ever needed? - How many unique backups do you want? * - Rsync fine? Push? Pull?(what are the paths and creds?) ** - cron every 4 hours okay?
It’ll take half an hour to setup, and I guess we can stop spamming the rest of the list, so just send to me personally for the rest...
Best, Michael michael@
* I’ll do one here at home off of one of my servers as well. ** I prefer pull, but just from a security issue of not giving people access to my servers. PS: I’ll send you a copy of my final script so you can automate the remote location you were talking about too.
On Sunday 20 of December 2020 21:28:11 Michael via tde-users wrote:
On Sunday 20 December 2020 01:34:29 pm Slávek Banko via tde-users wrote:
Dne Sunday 20 of December 2020 18:12:25 Michael via tde-users
napsal(a):
On Sunday 20 December 2020 12:02:26 am Michele Calgaro via tde-users
wrote:
On 2020/12/20 04:00 AM, Slávek Banko via tde-users wrote:
We may consider placing the primary archive in a hosting center
- for example, by purchasing larger disks for tde-box or by
increasing the space provided for tde-vps. However, it will probably be a good idea to consider such changes only after we are able to manage the project funds and accept donations.
This is defintely something we will need to do in future, to move away from possible one-person-dependant services. I agree with you we need to figure out how to handle donations and project funds, then we can relook at this.
In that vein (away from possible one-person-dependant services):
- How big is the archive? (<250GB? ~4TB?)
- How simple is it to back it up? (a few rsync’s? tarball?)
I have no issues storing a copy, or two, of the archive on one of my backup servers (which are in hosting centers).
I'm glad you're interested in the details.
Current size and occupied space of the partition: /dev/mapper/lvm1-tde_data 400G 336G 65G 84% /srv/tde
A subset of this volume is synchronized to the mirror system. First to the primary mirror, from which the other mirrors are then synchronized. This synchronization usually takes place several times a day, with pauses of 3 hours between the individual synchronizations.
The size always decreases significantly at the moment of release, when the contents of the official repository and PSB are identical. The size of the repository is now constantly growing again as the packages in PSB are built for R14.0.10~pre. Such an increase can be almost 100 GiB. At the same time, there is a gradual increase in volume when adding support for new distributions.
During the outage, which I announced in early December, I replaced my home server's hardware with slightly newer components, including the purchase of new, larger disks. So now I have a sufficient reserve of disk capacity again.
The specified partition is on two disks (RAID1). In addition to these disks, I make a regular daily local backup on an external usb disk. The backup disk requires significantly more capacity, as the daily differences can be large - from hundreds of MiBs to tens of GiBs. Next to this, I'm making a copy of the archive on my next machine in a remote location (two disks in RAID1). Copying to this remote location is not regular yet.
Hi Slávek,
Lol, interesting, but TMI ;)
- What’s the max size you’ve ever needed?
The current partition size is to be understood as "beginning size". Each newly added distribution adds 1.5 to 12 GiB (depending on the number of architectures). Once we release R14.1.x, it will mean new repositories, an estimated new 80 GiB. So here it is necessary to take into account that the required space will continue to increase.
- How many unique backups do you want? *
I use dirvish for backup. It uses the principle of hardlinks so that each backup serves as an independent full backup, although it is performed as an incremental one. Due to this independence, backups can have different expiration times. A normal daily backup has an expiration date of 15 days, weekly backups have an expiration date of 3 months, and the first weekly backup of the month has an expiration date of one year.
- Rsync fine? Push? Pull?(what are the paths and creds?) **
- cron every 4 hours okay?
An rsync server is provided for synchronization to the primary mirror. So the primary mirror does the pull. As I mentioned, there are some parts excluded from syncing to mirrors.
If we want to solve the backup of the entire primary archive and at the same time keep the exclusion so that the mirror system does not grow beyond the accepted limit and contains currently useful data, it is possible to set the second path for rsync.
It’ll take half an hour to setup, and I guess we can stop spamming the rest of the list, so just send to me personally for the rest...
It will certainly be good for Mike, as the administrator of the primary mirror, to take part in the discussion here as well.
Best, Michael michael@
- I’ll do one here at home off of one of my servers as well.
** I prefer pull, but just from a security issue of not giving people access to my servers. PS: I’ll send you a copy of my final script so you can automate the remote location you were talking about too. ____________________________________________________
Cheers
On Sun December 20 2020 11:34:29 Slávek Banko via tde-users wrote:
Current size and occupied space of the partition: /dev/mapper/lvm1-tde_data 400G 336G 65G 84% /srv/tde
The primary mirror is currently: /dev/mapper/DISK-TDE 298G 250G 49G 84% /ARCHIVE/TDE
Since the upgrade a year ago we can handle 400G no problem so the easiest solution might be just to mirror everything. I'm not sure how much the secondaries can take but it's not like it's a massive change.
I wonder if Michael mb_trinity_desktop@inet-design.com would be interested in becoming a mirror, which would use less primary archive bandwidth than pulling backups directly.
--Mike
On Sunday 20 of December 2020 22:01:33 Mike Bird via tde-users wrote:
On Sun December 20 2020 11:34:29 Slávek Banko via tde-users wrote:
Current size and occupied space of the partition: /dev/mapper/lvm1-tde_data 400G 336G 65G 84% /srv/tde
The primary mirror is currently: /dev/mapper/DISK-TDE 298G 250G 49G 84% /ARCHIVE/TDE
Since the upgrade a year ago we can handle 400G no problem so the easiest solution might be just to mirror everything. I'm not sure how much the secondaries can take but it's not like it's a massive change.
As I mentioned in the other mail in this thread - here it can be expected that the size will continue to increase. So it is possible that 400 GiB will not be enough again next year. Therefore, it seems like a good idea that only a subset is published to the mirror system.
I wonder if Michael mb_trinity_desktop@inet-design.com would be interested in becoming a mirror, which would use less primary archive bandwidth than pulling backups directly.
Yes, this would definitely be good. If Michael has enough free capacity, he could join as another official mirror.
--Mike ____________________________________________________
Cheers
On Sunday 20 December 2020 08:53:16 pm Slávek Banko via tde-users wrote:
I wonder if Michael mb_trinity_desktop@inet-design.com would be interested in becoming a mirror, which would use less primary archive bandwidth than pulling backups directly.
Yes, this would definitely be good. If Michael has enough free capacity, he could join as another official mirror.
K, cap for next 12-18 months is 600GB? Doable.
Taking this to private mail, no need to keep spaming the users list ;)
Best, Michael
On Sun December 20 2020 19:18:03 Michael via tde-users wrote:
K, cap for next 12-18 months is 600GB? Doable.
Taking this to private mail, no need to keep spaming the users list ;)
Keeping this on list for the archive for the benefit of future historians.
The primary mirror currently has 400GB assigned to TDE but it's an 800GB VM with Linux and LVM and Apache and not much else so if we can keep TDE to 600GB or so there shouldn't be a problem.
If Slávek keeps a backup of his rsync filters we could maybe use them if any secondary mirrors complain about the new size.
---Mike
On Monday 21 of December 2020 05:24:51 Mike Bird via tde-users wrote:
On Sun December 20 2020 19:18:03 Michael via tde-users wrote:
K, cap for next 12-18 months is 600GB? Doable.
Taking this to private mail, no need to keep spaming the users list ;)
Keeping this on list for the archive for the benefit of future historians.
The primary mirror currently has 400GB assigned to TDE but it's an 800GB VM with Linux and LVM and Apache and not much else so if we can keep TDE to 600GB or so there shouldn't be a problem.
If Slávek keeps a backup of his rsync filters we could maybe use them if any secondary mirrors complain about the new size.
---Mike ____________________________________________________
Ok, that sounds good, so I will gradually enable the directories that are now excluded. We will see how the traffic will increase as soon as the PTB repository is available on the mirror system.
Cheers
On Sunday 20 December 2020 10:24:51 pm Mike Bird via tde-users wrote:
On Sun December 20 2020 19:18:03 Michael via tde-users wrote:
K, cap for next 12-18 months is 600GB? Doable.
Taking this to private mail, no need to keep spaming the users list ;)
Keeping this on list for the archive for the benefit of future historians.
Okay, makes sense.
The primary mirror currently has 400GB assigned to TDE but it's an 800GB VM with Linux and LVM and Apache and not much else so if we can keep TDE to 600GB or so there shouldn't be a problem.
If Slávek keeps a backup of his rsync filters we could maybe use them if any secondary mirrors complain about the new size.
Ignore the NVMe host from earlier, they’re nice people, but they don't customize their plans.
If this meets the needs for a mirror:
http://turnkeyinternet.net/linux-cpanel-web-hosting/
C10 cPanel Hosting $2.49per month (it’s $90 for 3 years.)
I’ll go talk with them and see if we fit their ‘Unlimited’ using these specs.
Max HD: 800GB Max Data: 3TB/month Pipe Needs: 50Mbps *
* They say Gbps, but does a mirror needs that? If not, let them cap data rates for the storage size?
As we’ve had a lot of different numbers thrown around, Mike/Slávek please verify the above.
Ref: 50 Mbit/s = ~16 TB per month
# # #
Alternatively:
In March we’re switching to T-Mobile 5G internet at our house and doing away with cable TV entirely. To support that, I’m going to build a ~30TB file server for the house.
* 1U Xeon server ($150) w/ 30TB of drives ($160), plus other junk (~$100).
The file server certainly won’t be ‘fast,’ but the up/down bandwidth is 100+ Mbps.
I will have to look into T-Mobile’s TOS, but with the plethora of Plex servers out their I don’t really see any issues.
The other thing would be the use of a dynamic DNS, but would the mirror software in use by TDE care or have issues with that?
All for now, best, Michael
On Mon December 21 2020 09:40:43 Michael via tde-users wrote:
Ignore the NVMe host from earlier, they’re nice people, but they don't customize their plans.
If this meets the needs for a mirror:
http://turnkeyinternet.net/linux-cpanel-web-hosting/
C10 cPanel Hosting $2.49per month (it’s $90 for 3 years.)
I’ll go talk with them and see if we fit their ‘Unlimited’ using these specs.
Generally "web hosting" plans only allow you to run pre-approved software.
There's a lot listed so I may have missed it but I didn't notice ssh server, rsync server, or bash scripting.
Max HD: 800GB Max Data: 3TB/month Pipe Needs: 50Mbps *
- They say Gbps, but does a mirror needs that? If not, let them cap data
rates for the storage size?
As we’ve had a lot of different numbers thrown around, Mike/Slávek please verify the above.
Ref: 50 Mbit/s = ~16 TB per month
# # #
Alternatively:
In March we’re switching to T-Mobile 5G internet at our house and doing away with cable TV entirely. To support that, I’m going to build a ~30TB file server for the house.
- 1U Xeon server ($150) w/ 30TB of drives ($160), plus other junk (~$100).
The file server certainly won’t be ‘fast,’ but the up/down bandwidth is 100+ Mbps.
I will have to look into T-Mobile’s TOS, but with the plethora of Plex servers out their I don’t really see any issues.
The other thing would be the use of a dynamic DNS, but would the mirror software in use by TDE care or have issues with that?
Dynamic DNS is not good for servers as it leads to a trade off between excessive DNS traffic and temporary failures.
We're doing OK for mirrors right now so you might want to focus on your new file server and later if you feel like creating a mirror you could do so and once you have it working reliably contact Slávek to add you to the list. FWIW I customize our installs to always pull from our own mirror so this way I test it and get a heads-up if it's misbehaving.
I would suggest that if you get into mirroring TDE you join the TDE devel mailing list so we can discuss mirroring there. Best,
--Mike
I would suggest that if you get into mirroring TDE you join the TDE devel mailing list so we can discuss mirroring there. Best,
--Mike
Thanks Mike and Michael for the excellent contribution and discussion. It will be good to have a full mirror (and maybe more than one) of the TDE archive on Slavek machine.
Nevertheless (and perhaps my fault for not explaining it well), the point of "moving away from a single person structure" is that we need the archive (and all TDE tools) on public machines accessible from more than one person. Slavek archive/computer is only accessible by him, so it is still a week point, although having a full mirror will improve the current situation. Longer term we need to move all remaining services to public machines, definitely something to discuss again after sorting out the project license and donation status.
It would also be good to have at least 2 people who knows what are the required tasks for maintenance of the various services. I do get involved in some of them, but mostly it is Slavek doing the maintenance of the machines, while I focus more on the development side. It would be good if more people steps up and help out Slavek's on the maintenance burden :-)
Cheers Michele
P.S: perhaps we should discuss on the devels ML rather than users?? so adding devs ML here too
On Tuesday 22 December 2020 09:02:31 pm Michele Calgaro via tde-users wrote:
Thanks Mike and Michael for the excellent contribution and discussion. It will be good to have a full mirror (and maybe more than one) of the TDE archive on Slavek machine.
Hello All TDE Developers,
This started out as a discussion about a TDE archive, then it morphed into a discussion about a TDE mirror. I believe I know what the difference is, and I’m willing to do both, but possibly one of the TDE Dev’s should give us users an ‘official’ definition of each? Just so everyone knows what is what.
Archive Definition:
Mirror Definition:
Why? A full up VPS, or webhosting service, is less than $100/year[1], which I, and I’m sure several others on this user’s list, have no issue either outright ‘donating’, or adding funds for, to TDE[2]. It also completely solves:
the point of "moving away from a single person structure" is that we need the archive (and all TDE tools) on public machines accessible from more than one person.
But at this point I am completely and utterly frustrated [3], as I have asked repeatedly what the requirements for each are and have not been given a concrete answer. If we (the users) don’t know what the requirements are, we can’t help you (the dev’s).
Just to put everyone on the same page and not having anyone guessing, a ‘my best guess at Archive needs’ for an example (see prior post for my guess at Mirror):
Requirements for Archive:
Time frame good for: 24 Months HD: 2TB (max) Data: 3TB / month (max) Pipe: 15 Mbps (min) CPU: 1 (min) Software: SSH, rsync (plus standard Linux stack, cron, bash, tar, etc.)
Number of unique Backups w/ time frames: ==Raw data== 1 raw rsync, synced once per day (2 AM) ==Tarballs== Daily: 1 for ea of last 4 days (2 AM) Weekly: 1 for ea of last 3 weeks (2 AM Sunday) Monthly: 1 for ea of last 3 months (2 AM, 1st of Month) Yearly: 1 for ea of last 2 years (2 AM, Jan 1)
(don’t like tarballs? Fill in the Software requirement above with whatever floats your boat...)
# # #
Now, PLEASE!, fill in the blanks... (Do add a criteria if I’ve missed it)
Requirements for Archive:
Time frame good for: HD: Data: Pipe: CPU: Software: Number of unique Backups w/ time frames:
Requirements for Mirror:
Time frame good for: HD: Data: Pipe: CPU: Software:
# # #
Okay, that’s it from me. If I can get something concrete to work from, I can help you, otherwise, not so much.
May everyone have a Happy & Merry holiday season!
Best Regards All,
Michael
[1] Yes you have to know how to dig these up, as you won’t find them ‘retail.’ [2] I could give a rats ass about a non-profit donation receipt. [3] This discussion time alone has ‘cost’ me 5-10x in time of just straight up buying and donating a 3 year VPS to the ‘cause.’
PS: My source for bulk ~$15 / 3TB SAS2 drives had a warehouse fire, so that alternative is off the table atm. I really didn’t want a 1U airplane in an upstairs closet anyway (sour grapes)...
Hi Michael,
As discussed we're now mirroring the whole thing so there's no separate "backup" option any more. A key factor in mirroring is to minimize traffic to and from the primary server.
On Wed December 23 2020 08:35:05 Michael via tde-users wrote:
Requirements for Mirror: Time frame good for: HD: Data: Pipe: CPU: Software:
Slávek posted the current size:
Current size and occupied space of the partition: /dev/mapper/lvm1-tde_data 400G 336G 65G 84% /srv/tde
- and there was some discussion of trying to stay under 600GB over the next few years.
I posted the bandwidth info:
Primary mirror has 4TB/month bandwidth and has used 876GB about two thirds of the way thru this month. So about 1.3TB in a quiet month for the primary mirror. Secondary mirrors will probably see a third of that. There's more when a release is finalizing - maybe double or triple. So a 2TB/month cap should be OK for a secondary mirror and you might even get by with 1TB/month.
All other decisions are up to the mirror operator. I did post some details of the script I use. Other mirrors use different scripts. I use Debian Buster and Apache2 currently but that may change and other mirrors may use entirely different software. The CPU and RAM will of course depend upon the bandwidth which we have discussed and the software which you choose but in reality it would probably run fine on a twenty year old Pentium with a gig of RAM.
Please join the TDE devel list if you wish to discuss mirroring further as that is a more appropriate forum.
Thanks,
--Mike
On Sunday 20 December 2020 03:01:33 pm Mike Bird via tde-users wrote:
On Sun December 20 2020 11:34:29 Slávek Banko via tde-users wrote:
Current size and occupied space of the partition: /dev/mapper/lvm1-tde_data 400G 336G 65G 84% /srv/tde
The primary mirror is currently: /dev/mapper/DISK-TDE 298G 250G 49G 84% /ARCHIVE/TDE
Since the upgrade a year ago we can handle 400G no problem so the easiest solution might be just to mirror everything. I'm not sure how much the secondaries can take but it's not like it's a massive change.
I wonder if Michael would be interested in becoming a mirror, which would use less primary archive bandwidth than pulling backups directly.
Hi Mike,
I asked Tim about becoming a mirror years ago, never got a response...
So, yes there is interest, but first I'd need to know what's the maximum monthly bandwidth I'd expect to see?
All of my pipes are things like 10 TB / month on a 1Gbps port, so if it's under 1TB / month easy peasy.
My least used server has 275GB of never used, free space, so if at all possible I’d like to stay within that limit. If not, I’ll look at the loads on my other ones and pick one of those (most have TB drives).
Best, Michael
On Sun December 20 2020 18:55:46 Michael via tde-users wrote:
So, yes there is interest, but first I'd need to know what's the maximum monthly bandwidth I'd expect to see?
Primary mirror has 4TB/month bandwidth and has used 876GB about two thirds of the way thru this month. So about 1.3TB in a quiet month for the primary mirror. Secondary mirrors will probably see a third of that. There's more when a release is finalizing - maybe double or triple. So a 2TB/month cap should be OK for a secondary mirror and you might even get by with 1TB/month.
CAVEAT: I'm wondering if the proposed broadening of mirror scope to full backup will expose more volatile files which would increase bandwidth.
--Mike