Friday 30th December 2011, 10:40 to 11:30 GMT (50 minutes)
Reason for Downtime:
A large log file relating to the mail queue caused a partition to become full within seconds, this caused the primary database server to respond slowly. This was detected at 10:30 GMT before the system was affected but the steps taken to resolve the issue caused the database server to go down. To prevent confusion the myRepono website and system were shutdown while the log file was removed and related issues could be assessed and resolved.
Downtime Affects:
Some user backups were cancelled and will be automatically restarted within 1 hour of service being resumed. Any restoration or combined backup download processes which were in progress were cancelled and will not be automatically restarted.
Future Prevention:
The issue was detected in advance of it's effects but attempts to resolve the issue caused additional problems. The mail queue partition size and monitoring frequency will be increased, and a dedicated mail server will be introduced to prevent such issues from re-occurring.
We sincerely apologise for any inconvenience caused by this downtime and will endeavour to eliminate the chances of such issues re-occurring in the future!