WebFaction: network maintenance affecting several EU servers on July 1, 2015

Posted in Scheduled downtime by

Our upstream provider will be performing network maintenance on July 1, 2015 between 22:00 and 03:00 UTC the following morning. The following servers will be affected:

Dweb186, Web209, Web210, Web211, Web217, Web218, Web223, Web224, Web225, Web303, Web304, Web305, Web306, Web314, Web316, Web317, Web327, Web331, Web332, Web333, Web334, Web357, Web358, Web359, Web364, Web370, Web371, Web381, Web383, Web385, Web392, Web394, Web395, Web398, Web403, Web412, Web416, Web420, Web421, Web427, Web431, Web433, Web435, Web437, Web441, Web447, Web448, Web451, Web456, Web457, Web459, Web461, Web464, Web466, Web478, Web480, Web483, Web488

Connectivity to the above servers will be interrupted for upwards of 20 minutes during the maintenance window.

We apologize in advance for the inconvenience. Should you have any questions about this maintenance, feel free to open a support ticket or email support@webfaction.com and our support team will be happy to help.

-
-

[Fixed] Web225 down

Posted in Downtime by

Web225 stopped responding several minutes ago and has not come back online following several attempted reboots. We’re working to restore service at this time and will update this post when we have more information.

2012-12-22 02:33 UTC: The server is still offline and not responding.

2012-12-22 03:51 UTC: We’ve booted the server into a rescue environment and we’re checking the state of the RAID, disks, and filesystems.

2012-12-22 05:01 UTC: We’re currently checking the filesystem for errors by running the fsck command.

2012-12-22 07:20 UTC: The file system checks have finished and the file systems are now in a clean state. We’re now working to get the RAID controller card replaced.

2012-12-22 10:14 UTC: At this time the server will not boot because of a bad hard drive. We’re replacing the hard drive that is causing the boot errors and we will continue working to bring the server back online as soon as possible.

2012-12-22 11:38 UTC: The hard drive has been replaced and server is now booting. However, the boot process is stopped at a kernel panic. We are working to resolve the cause of the panic now.

2012-12-22 12:39 UTC: The server is now booting correctly and the kernel panics have been resolved. We’re now working to restore full access to the machine and verify its functionality.

2012-12-22 14:17 UTC: In verifying the functionality of the server we’ve found that there was widespread corruption in both the PostgreSQL and MySQL databases. We’ve reinstalled and reinitialized both database servers and we’re now working to restore the databases from backups.

2012-12-22 15:20 UTC: We are still restoring the databases from backups.

2012-12-22 17:11 UTC: The MySQL databases have been mostly restored and we’re beginning to work on the PostgreSQL databases.

2012-12-22 19:11 UTC: The MySQL and PostgreSQL databases have been restored and the server is back online

2012-12-23 10:22 UTC: The server’s disks have gone read-only again and we are working on the server now.

2012-12-23 12:23 UTC: The server is back as of now but in case the problem returns we have started a copy full copy of the data to a spare server.

-
-

[Fixed] Web225 down

Posted in Downtime by

Web225 stopped responding to requests a few minutes ago. We’re looking into the problem and should have service restored soon.

2012-04-10 19:01 UTC: The server is currently performing a forced filesystem check following a reboot.

2012-04-10 19:29 UTC: Web225 is back online and responding to requests.

-
-