Tim,
The manhunt had begun -- What had happened to Tim? TDE GIT -> dead, mailing list -> dead, Tim -> ? We were beginning to worry... What's up?
We need a contingency plan. Snail mail ;)
Calvin On May 20, 2012 1:24 AM, "David C. Rankin" drankinatty@suddenlinkmail.com wrote:
Tim,
The manhunt had begun -- What had happened to Tim? TDE GIT -> dead, mailing list -> dead, Tim -> ? We were beginning to worry... What's up?
-- David C. Rankin, J.D.,P.E.
To unsubscribe, e-mail: trinity-devel-unsubscribe@lists.pearsoncomputing.net For additional commands, e-mail: trinity-devel-help@lists.pearsoncomputing.net Read list messages on the web archive: http://trinity-devel.pearsoncomputing.net/ Please remember not to top-post: http://trinity.pearsoncomputing.net/mailing_lists/#top-posting
Tim,
The manhunt had begun -- What had happened to Tim? TDE GIT -> dead, mailing list -> dead, Tim -> ? We were beginning to worry... What's up?
-- David C. Rankin, J.D.,P.E.
Nothing serious on this end fortunately. :-)
The ISP that services the TDE server cluster had a localized equipment failure, but instead of trying to fix the outage they decided to blame the customer (me). This prolonged the outage significantly, even though in the end it did turn out to be upstream ISP equipment failure.
I do apologize for the outage; as you are probably aware the TDE services can only utilize a single carrier due to the long-term contractual expense of bringing out a redundant service provider to the server location.
Tim
On 05/20/2012 01:05 AM, Timothy Pearson wrote:
Nothing serious on this end fortunately. :-)
The ISP that services the TDE server cluster had a localized equipment failure, but instead of trying to fix the outage they decided to blame the customer (me). This prolonged the outage significantly, even though in the end it did turn out to be upstream ISP equipment failure.
I do apologize for the outage; as you are probably aware the TDE services can only utilize a single carrier due to the long-term contractual expense of bringing out a redundant service provider to the server location.
Tim
Have we approached any of the high-bandwidth Universities or OpenSource outfits to see if they would be willing to provide mirror services? I see so many run-of-the-mill projects that have mirror services all over the world. It sure seems like a favorable project like TDE would be a prime candidate to be accepted and helped by some of these institutions.
For my learning, if somebody can mirror, would all that would need to be done is to have some cron job push TDE to the mirror to keep it updated, or is there more to it than that? I see a mirror as just (storage + band width). For TDE, that's about 8G in storage. Is that about it?
On 05/20/2012 01:05 AM, Timothy Pearson wrote:
Nothing serious on this end fortunately. :-)
The ISP that services the TDE server cluster had a localized equipment failure, but instead of trying to fix the outage they decided to blame the customer (me). This prolonged the outage significantly, even though in the end it did turn out to be upstream ISP equipment failure.
I do apologize for the outage; as you are probably aware the TDE services can only utilize a single carrier due to the long-term contractual expense of bringing out a redundant service provider to the server location.
Tim
Have we approached any of the high-bandwidth Universities or OpenSource outfits to see if they would be willing to provide mirror services? I see so many run-of-the-mill projects that have mirror services all over the world. It sure seems like a favorable project like TDE would be a prime candidate to be accepted and helped by some of these institutions.
<snip>
We do mirror most of our static binary content on a worldwide mirror system. However, most of the TDE services are dynamic (GIT, Wiki, mailing lists, searchable documentation, IRC, bugtracker, patch lists, etc.) and therefore cannot (easily/safely) be mirrored.
Good idea though!
Tim
--- On Sun, 5/20/12, Timothy Pearson kb9vqf@pearsoncomputing.net wrote:
From: Timothy Pearson kb9vqf@pearsoncomputing.net Subject: Re: [trinity-devel] Que Pasa Bro -- What be the Story Tim? To: trinity-devel@lists.pearsoncomputing.net Date: Sunday, May 20, 2012, 1:31 PM
On 05/20/2012 01:05 AM, Timothy
Pearson wrote:
Nothing serious on this end fortunately. :-)
The ISP that services the TDE server cluster had a
localized equipment
failure, but instead of trying to fix the outage
they decided to blame
the customer (me). This prolonged the outage
significantly, even though in
the end it did turn out to be upstream ISP
equipment failure.
I do apologize for the outage; as you are probably
aware the TDE
services can only utilize a single carrier due to the
long-term contractual
expense of bringing out a redundant service provider to the
server location.
Tim
Have we approached any of the high-bandwidth
Universities or OpenSource
outfits to see if they would be willing to provide mirror
services? I see so many
run-of-the-mill projects that have mirror services all
over the world. It
sure seems like a favorable project like TDE would be a
prime candidate to be
accepted and helped by some of these institutions.
<snip>
We do mirror most of our static binary content on a worldwide mirror system. However, most of the TDE services are dynamic (GIT, Wiki, mailing lists, searchable documentation, IRC, bugtracker, patch lists, etc.) and therefore cannot (easily/safely) be mirrored.
Good idea though!
I'm not concerned about the type of outage we experienced. Those things happen.
If the recent outage was a hosting plan squabble and you still had normal customer ISP access, then a personal email to somebody would have helped to pass the word as much as possible. If you lost personal services too, then perhaps a trip to the library or local coffee shop would have been in order to send an email or leave a message in IRC.
The recent outage leaves me concerned about what we can do should something serious happen long-term. Nasty weather and natural disasters could disrupt an ISP connection for days, weeks, or months. Serious illness could keep you out of touch too.
We need a basic contingency plan. Something so the project stays alive and the remaining participants are not left hanging.
When the outage occurred I had the latest (known) GIT sources. Worst case then the project could have move toward some kind of restoration with that, minus a few patches. Yet when I think about the bug tracker, etherpad, wiki, and mail list archives disappearing then I do not feel so comfortable.
Granted, part of what we do is dynamic, but not so much that a basic contingency plan can't restore.
Currently we have two domains: www.pearsoncomputing.net and www.trinitydesktop.org. If both are housed from the same location/ISP, then we become dependent upon those people not to get a bug up their backside. Perhaps we can move one domain elsewhere. Basically, diversify --- don't put all our eggs in one basket.
We do not need to mirror everything in the traditional sense. Some work-arounds would save the project.
For example, some of us can rsync our local GIT tree at least daily or more often. We don't need to "mirror" those sources if several of us do that. GIT is a distributed system so all we need are people keeping synced.
Losing the bug tracker would be a huge challenge. Is there a way some of us can rsync or spider the bug tracker to our personal systems? Likewise for the wiki, etherpad, mail list archives. Such efforts would be the equivalent of a backup system.
None of this need cost additional funds. We just need a way for several project members to rsync/spider files. Should disaster strike we then have a way to get us back on our feet.
I don't want to make mountains out of molehills. I'm just asking that as a group we consider basic stewardship. :-)
Darrell
The bug tracker can be exported and imported as XML files!
I'm a firm believer in a VPS for trinity in the long run. Most offer 99.9% guarantee uptime and are cheap enough that for probably around hundred to two hundred a year we could have a stable web server. This would hold bugzilla, website whatever else. Also the connections are business class so no more slow website under heavy load.
I know funding would be the major blocker would in that idea.
Calvin On May 20, 2012 8:23 PM, "Darrell Anderson" humanreadable@yahoo.com wrote:
--- On Sun, 5/20/12, Timothy Pearson kb9vqf@pearsoncomputing.net wrote:
From: Timothy Pearson kb9vqf@pearsoncomputing.net Subject: Re: [trinity-devel] Que Pasa Bro -- What be the Story Tim? To: trinity-devel@lists.pearsoncomputing.net Date: Sunday, May 20, 2012, 1:31 PM
On 05/20/2012 01:05 AM, Timothy
Pearson wrote:
Nothing serious on this end fortunately. :-)
The ISP that services the TDE server cluster had a
localized equipment
failure, but instead of trying to fix the outage
they decided to blame
the customer (me). This prolonged the outage
significantly, even though in
the end it did turn out to be upstream ISP
equipment failure.
I do apologize for the outage; as you are probably
aware the TDE
services can only utilize a single carrier due to the
long-term contractual
expense of bringing out a redundant service provider to the
server location.
Tim
Have we approached any of the high-bandwidth
Universities or OpenSource
outfits to see if they would be willing to provide mirror
services? I see so many
run-of-the-mill projects that have mirror services all
over the world. It
sure seems like a favorable project like TDE would be a
prime candidate to be
accepted and helped by some of these institutions.
<snip>
We do mirror most of our static binary content on a worldwide mirror system. However, most of the TDE services are dynamic (GIT, Wiki, mailing lists, searchable documentation, IRC, bugtracker,
patch
lists, etc.) and therefore cannot (easily/safely) be mirrored.
Good idea though!
I'm not concerned about the type of outage we experienced. Those things happen.
If the recent outage was a hosting plan squabble and you still had normal customer ISP access, then a personal email to somebody would have helped to pass the word as much as possible. If you lost personal services too, then perhaps a trip to the library or local coffee shop would have been in order to send an email or leave a message in IRC.
The recent outage leaves me concerned about what we can do should something serious happen long-term. Nasty weather and natural disasters could disrupt an ISP connection for days, weeks, or months. Serious illness could keep you out of touch too.
We need a basic contingency plan. Something so the project stays alive and the remaining participants are not left hanging.
When the outage occurred I had the latest (known) GIT sources. Worst case then the project could have move toward some kind of restoration with that, minus a few patches. Yet when I think about the bug tracker, etherpad, wiki, and mail list archives disappearing then I do not feel so comfortable.
Granted, part of what we do is dynamic, but not so much that a basic contingency plan can't restore.
Currently we have two domains: www.pearsoncomputing.net and www.trinitydesktop.org. If both are housed from the same location/ISP, then we become dependent upon those people not to get a bug up their backside. Perhaps we can move one domain elsewhere. Basically, diversify --- don't put all our eggs in one basket.
We do not need to mirror everything in the traditional sense. Some work-arounds would save the project.
For example, some of us can rsync our local GIT tree at least daily or more often. We don't need to "mirror" those sources if several of us do that. GIT is a distributed system so all we need are people keeping synced.
Losing the bug tracker would be a huge challenge. Is there a way some of us can rsync or spider the bug tracker to our personal systems? Likewise for the wiki, etherpad, mail list archives. Such efforts would be the equivalent of a backup system.
None of this need cost additional funds. We just need a way for several project members to rsync/spider files. Should disaster strike we then have a way to get us back on our feet.
I don't want to make mountains out of molehills. I'm just asking that as a group we consider basic stewardship. :-)
Darrell
To unsubscribe, e-mail: trinity-devel-unsubscribe@lists.pearsoncomputing.net For additional commands, e-mail: trinity-devel-help@lists.pearsoncomputing.net Read list messages on the web archive: http://trinity-devel.pearsoncomputing.net/ Please remember not to top-post: http://trinity.pearsoncomputing.net/mailing_lists/#top-posting
The bug tracker can be exported and imported as XML files!
I'm a firm believer in a VPS for trinity in the long run. Most offer 99.9% guarantee uptime and are cheap enough that for probably around hundred to two hundred a year we could have a stable web server. This would hold bugzilla, website whatever else. Also the connections are business class so no more slow website under heavy load.
I know funding would be the major blocker would in that idea.
Calvin
"Business class" does not mean "fast". I have "business class" to the TDE servers already.
The cost will be much, much higher BTW, as you would also need to factor in a very high bandwidth connection from the servers that remain here (build farm, etc.) to the VPS. This could easily run upwards of $1000/month.
Tim
On 21/05/12 02:42, Timothy Pearson wrote:
The bug tracker can be exported and imported as XML files!
I'm a firm believer in a VPS for trinity in the long run. Most offer 99.9% guarantee uptime and are cheap enough that for probably around hundred to two hundred a year we could have a stable web server. This would hold bugzilla, website whatever else. Also the connections are business class so no more slow website under heavy load.
I know funding would be the major blocker would in that idea.
Calvin
"Business class" does not mean "fast". I have "business class" to the TDE servers already.
The cost will be much, much higher BTW, as you would also need to factor in a very high bandwidth connection from the servers that remain here (build farm, etc.) to the VPS. This could easily run upwards of $1000/month.
How much bandwidth and storage are needed by Trinity, or its various sections?
On 21/05/12 02:42, Timothy Pearson wrote:
The bug tracker can be exported and imported as XML files!
I'm a firm believer in a VPS for trinity in the long run. Most offer 99.9% guarantee uptime and are cheap enough that for probably around hundred to two hundred a year we could have a stable web server. This would hold bugzilla, website whatever else. Also the connections are business class so no more slow website under heavy load.
I know funding would be the major blocker would in that idea.
Calvin
"Business class" does not mean "fast". I have "business class" to the TDE servers already.
The cost will be much, much higher BTW, as you would also need to factor in a very high bandwidth connection from the servers that remain here (build farm, etc.) to the VPS. This could easily run upwards of $1000/month.
How much bandwidth and storage are needed by Trinity, or its various sections?
Storage runs into the Terabyte range. Upload bandwidth runs at a *minimum* 300GB/month and download is several times that.
Needless to say, not cheap. ;-)
Tim
On 05/21/2012 04:01 PM, Timothy Pearson wrote:
Storage runs into the Terabyte range. Upload bandwidth runs at a *minimum* 300GB/month and download is several times that.
Needless to say, not cheap. ;-)
Tim
That's what sparked the thought of approaching some of the Univ. with strong comp. sci. programs to see if any were interested in (1) using the project as part of some undergraduate/graduate level desktop design curriculum and (2) providing a mirror for the code. I see the benefit as many-fold, (1) TDE benefits from the brainpower of the students and faculty involved, (2) young minds, fresh ideas, (3) exposure of many, many more young folks to TDE/expansion of the user base, and (4) the tangential storage and bandwidth contribution if they pick up the project.
Best of all -- the cost...
Needless to say, not cheap. ;-)
That's what sparked the thought of approaching some of the Univ. with strong comp. sci. programs to see if any were interested in (1) using the project as part of some undergraduate/graduate level desktop design curriculum and (2) providing a mirror for the code. I see the benefit as many-fold, (1) TDE benefits from the brainpower of the students and faculty involved, (2) young minds, fresh ideas, (3) exposure of many, many more young folks to TDE/expansion of the user base, and (4) the tangential storage and bandwidth contribution if they pick up the project.
Best of all -- the cost...
Consider contacting the people at the Oregon State University Open Source Lab. If we offer to meet costs, even partial, I suspect they will be willing to host our services. They might still be willing even if we don't offer simply because of the nature of our project. But as we now do pay our own expenses, more or less, I believe we should offer to cover costs as much as we can pay.
OSUOSL is popular and dependable.
I'm not affiliated with them in any way. OSUOSL is one of the primary mirrors for Slackware. Hence my familiarity.
Darrell
On 05/22/2012 11:32 AM, Darrell Anderson wrote:
Consider contacting the people at the Oregon State University Open Source Lab. If we offer to meet costs, even partial, I suspect they will be willing to host our services. They might still be willing even if we don't offer simply because of the nature of our project. But as we now do pay our own expenses, more or less, I believe we should offer to cover costs as much as we can pay.
OSUOSL is popular and dependable.
I'm not affiliated with them in any way. OSUOSL is one of the primary mirrors for Slackware. Hence my familiarity.
Darrell
All right,
I should have a little time next week to put out a few feelers to the univ. folks. I'll report back with the efforts.
-
Andrew Young -
Calvin Morrison -
Darrell Anderson -
David C. Rankin -
Timothy Pearson