Re: [trinity-devel] UK Mirror Service mirrors

Hi Mike, The two servers sync independently. Whilst this is not an ideal situation, and can lead to them having slightly out of step content, it's the simplest approach and requires the least development and maintenance work. In theory they should sync within an hour of each other, so the likely reason for them to have different files for an extended period is that one of them is failing to sync. Obviously this is something that's quite easy to investigate at the time, but impossible to work out later. So if you see this please let us know at the time so we can see why it's happening. The load balancer doesn't take much action regarding this. It doesn't know which mirrors are up to date or not, just whether they're functioning. However, it does have "stickiness" enabled, which means a client from the same IP should repeatedly hit the same backend unless there's a failure. So they should at least get consistent content. My main gripe is about publishing the details of the backend systems directly. This leads to people using them not via the load balancer and then complaining when one is down. In your case you have a smart redirector in front, so presumably that'd deal with that problem without inconveniencing the users. I don't have any problem with you doing that but I'd be happier if the backend URLs weren't so obviously advertised. The period of downtime in early June was because we had a complete data centre outage for the first time in over 10 years. The UPS systems were being upgraded and a failure occurred which powered everything off - the systems supposed to save us were the ones that failed us :-( The Mirror Service sadly isn't a priority system for the University, compared to core teaching and research systems, so we had to wait until full power was restored before we could turn it back on. That issue also killed copernicus, or more specifically its RAID controller. I have now replaced that and spent the past couple of weeks rebuilding it and resyncing the data. This has proved quite time consuming but it is starting to come back online this weekend. Tim. On Sat, Jul 18, 2020 at 11:35:18AM -0700, Mike Bird wrote: > Hi Tim, > > Thank you for mirroring TDE! > > I don't maintain that web page so this is not a direct response > to your request but rather a request for further information. > > FYI https://www.trinitydesktop.org/mirrorstatus.php is not the > normal route whereby users retrieve files. Users work instead > via a smart redirector http://mirror.ppa.trinitydesktop.org/ > > The status page is more of a debugging tool. > > We monitor mirrors to determine when they have pulled updates. > There have been several extended periods when copernicus and > kuiper have held different files. We use this information in > the smart redirector to only send user traffic to mirrors which > are known to have the file the user needs. > > I was wondering to what extent the UK Mirror Service load balancer > addresses this as the last time I saw copernicus rsync from us > was June 19th? And earlier in June there was a period of a few days > when neither kuiper nor copernicus were updating? > > Thanks again! > > --Mike