On each of these days, the maintenance will take place between 7:30 am and 9:30 am EDT, and will require a network switch reboot that will result in a loss of network connectivity for up to 30 minutes (but probably less). We’ll update this post as the maintenance progresses.
Web238 is currently down for emergency maintenance. We’ll continue to update this post with additional information as it is available.
2012-07-22 20:40:34 UTC: We are running fsck as it went read only to prevent any data corruption.
2012-07-22 22:19 UTC: The file system was unable to be recovered after numerous file system corruptions that were unable to be re-paired. We’re now work working to recover the data from the /home directory and any other data that is not corrupt.
2012-07-22 23:47 UTC: We are still working to recover all available data from the corrupted filesystem. We’ve been able to recover almost all of the data stored in the /home directory.
2012-07-22 00:45 UTC: The data recovery has finished. We’re now re-installing the operating system.
2012-07-22 01:45 UTC: The operating system has been re-installed and we are now installing our platform and tools.
2012-07-22 02:46 UTC: We’re now syncing the saved data back to the machine.
2012-07-22 04:19 UTC: The sync of user data back to the machine has completed. We are now working on restoring the MySQL and PostgreSQL databases.
2012-07-22 05:27 UTC: We are still restoring the MySQL and PostgreSQL databases.
2012-07-22 05:55 UTC: The databases have been restored and the server is now back online and working normally. We’ll continue to monitor the server to verify that there are no further problems.