At 13:00 UTC 26 July 2012 we’ll be taking the server offline for some scheduled maintenance. We don’t anticipate the downtime to be more than 1 hour. We’ll update the post as the maintenance progresses.
2012-07-26 14:05 UTC: Web119 is back up and functioning properly.
Web238 is currently down for emergency maintenance. We’ll continue to update this post with additional information as it is available.
2012-07-22 20:40:34 UTC: We are running fsck as it went read only to prevent any data corruption.
2012-07-22 22:19 UTC: The file system was unable to be recovered after numerous file system corruptions that were unable to be re-paired. We’re now work working to recover the data from the /home directory and any other data that is not corrupt.
2012-07-22 23:47 UTC: We are still working to recover all available data from the corrupted filesystem. We’ve been able to recover almost all of the data stored in the /home directory.
2012-07-22 00:45 UTC: The data recovery has finished. We’re now re-installing the operating system.
2012-07-22 01:45 UTC: The operating system has been re-installed and we are now installing our platform and tools.
2012-07-22 02:46 UTC: We’re now syncing the saved data back to the machine.
2012-07-22 04:19 UTC: The sync of user data back to the machine has completed. We are now working on restoring the MySQL and PostgreSQL databases.
2012-07-22 05:27 UTC: We are still restoring the MySQL and PostgreSQL databases.
2012-07-22 05:55 UTC: The databases have been restored and the server is now back online and working normally. We’ll continue to monitor the server to verify that there are no further problems.