[Notice] Outage

[EricF]

Just wanted to apologise for the outage that occurred from about 0530 BST (0430 UTC) this morning (so that would have been about midnight in NewYork, and ten PM in Los Angeles, midday in Australia I guess). Looks like we lost network connectivity for about 20 minutes from one datacentre on both links to the server in question. We're not sure yet what caused the outage, but it does appear to have been upstream of us at this point.

 

[EricF]

I don't have 100% of the details yet, but it appears the datacentre we're currently hosting TrekBBS out of lost incoming mains power, and switched over smoothly to the generators. However due to a battery fault failed to switch back to the restored mains power and due to the battery issue two of the UPS banks depleted more rapidly than anticipated whilst the centre staff were debugging the issue. So kit on those two banks ultimately lost power. We didn't loose power ourselves, but the upstream switches and routers were on the feeds that did loose power. Replacement batteries have been delivered today and they are planning to restore to mains power at midnight (BST) tonight once those batteries are charged.

Fingers crossed!