One of the disks in Web183’s RAID array has failed, which has lead to decreased I/O performance. Our datacenter is in the process of replacing the failed disk and rebuilding the RAID. We will update this post with more information as it becomes available.
2015-11-28 12:15 UTC: The bad disk has been swapped, and the RAID array is rebuilding.
2015-11-28 17:40 UTC: RAID array is still rebuilding.
2015-11-29 10:50 UTC: RAID rebuild is 27% complete.
2015-11-29 17:20 UTC: RAID rebuild is 36% complete.
2015-11-29 22:10 UTC: RAID rebuild is 41% complete.
2015-11-30 08:10 UTC: RAID rebuild is 53% complete.
2015-11-30 17:52 UTC: The machine has stopped responding, we are looking into the issue.
2015-12-01 00:18 UTC: Apologies for the lack of updates. Web183 has been back online, but under high load. We’re still looking into the cause.
2015-12-01 00:45 UTC: It appears that Web183 may have another failing disk. We’re planning a complete migration to new hardware.
2015-12-01 15:03 UTC: We have provisioned a new machine and are in the process syncing the customer data.
2015-12-01 18:31 UTC: The initial sync is approximately 25% complete, and the buildout of the new server is almost finished.
2015-12-01 23:19 UTC: The initial sync is approximately 50% complete.
2015-12-01 23:29 UTC: The migration to the new hardware will take place on Wednesday, December 2 2015, between 3pm and 6pm UTC, after the initial sync is complete. As part of this migration the server’s main IP will change from 18.104.22.168 to 22.214.171.124. If you’re using WebFaction’s DNS servers for all your domains then your DNS A records will be updated automatically. If you’re using non-WebFaction DNS servers for some of your domains then you will have to update the A records for those domains yourself.
2015-12-02 15:00 UTC: The migration is starting now.
2015-12-02 18:24 UTC: The web183 migration is over.
2015-12-02 19:53 UTC: If you experience any issues with your sites open a ticket and we’ll look into it asap. We have setup a proxy on the old server to redirect all HTTP and HTTPS traffic to the new server. The proxy will stay for 5 days