Infrastructure, resources & services status updates

Maintenance works & outages progress details

2017-12-12

Valid from: 2017-12-09 (approx.)
Expected end time: 2017-12-16 (approx)
Actual end time: TBD

The prolonged outage resulted in our internal services flooding our edge servers, most notably the e-mail services.
No-delay service was restored on 11 Dec. However, e-mails submitted betwen 8 and 11 Dec are still being assessed & re-scheduled for delivery.
At the current pace, they should be examined, re-scheduled and delivered within approx 24 hours.


2017-12-10

Valid from: 2017-10-10 (approx.)
Expected end time: 2017-12-16 (approx)
Actual end time: TBD

As a follow-up to the previous mntn notices, all systems are go. However, see above.

 


2017-10-30

Valid from: 2017-10-30 00:00 CET
Expected end time: 2017-11-05 00:00 CET
Actual end time: RESOLVED

As a follow-up to maintenance announcements from 25.10, 24.10 and 12.10, we are glad to announce to have had managed to set up two parallel VM pools, either of those either using FC, SAS, NFS, iSCSI, or a mix of the above.
PS. DR is about 90% complete.

 


2017-10-25

Valid from: 2017-10-25 15:00 CEST
Expected end time: 2017-10-29 18:00 CEST
Actual end time: 2017-10-28 03:00 CET

As a follow-up to maintenance announcements from 24.10 and 12.10, we have begun to migrate salvaged resources from cold storage to a newly provisioned VM pool.

 


2017-10-24

Valid from: 2017-10-24 15:00 CEST
Expected end time: 2017-10-24 18:00 CEST
Actual end time: 2017-10-24 18:45 CEST

SAS array controller maintenance is scheduled between 3:00 PM and 6:00 PM. Intermittent 10-20 minute outages of some services might occur.

UPDATE: 2017-10-24 23:30 CEST
One of the SAS controllers is unoperational, most of data is served from cold storage backup array. Performance of much of web resources will be degraded until further notice.
We are performing upgrades & re-provisioning of the FC SAN infrastructure (metadata & topology recovery).

 


2017-10-12

Valid from: 2017-10-12 13:00 CEST
Expected end time: 2017-10-11 15:00 CEST
Actual end time: RESOLVED

Due to yesterday's outage, intermittent outages during electric repairs may occur.


2017-10-11

Valid from: 2017-10-11 06:00 CEST
Expected end time: 2017-10-11 17:00 CEST
Actual end time: 2017-10-11 21:00 CEST

Due to electrical circuit breaker that tripped we lost redundancy on our FC SAN and most front-end services were experiencing difficulties.
We were able to resolve the problem mostly.

UPDATE: 2017-10-11 17:00 CEST
Intermittent outages during electric repairs may occur tomorrow. A small handful of services remains unaccessible - on a storage array that needs a reliable power supply to perform data recovery.


2017-09-29

Valid from: 2017-09-29 00:00 CEST
Expected end time: 2017-10-15 20:00 CEST
Actual end time: RESOLVED

Due to maintenance works on several of our storrage arrays, some services may be temporarily unavailable.

UPDATE: 2017-10-12 10:00 CEST
Yesterday's outage had us postpone the operation.

UPDATE: 2017-10-05 18:00 CEST
Replacement parts arrived late today. Therefore the operation time will be prolonged. At present no services are directly affected.