Web17 stopped responding several minutes ago and is currently being rebooter. We should have service restored soon.
2009-08-27 11:12 – Web17 is back online. The problem appears to have been extremely high load due to a massive number of incoming FTP connections.
At 6pm GMT today we will take down Web61 to check the filesystem with fsck. We will update this post with our progress.
Update 1.10pm CDT time: fsck has now started
Update 3pm CDT time: the fsck run is now over and the server is back online
The filesystem on Web61 went into a read-only state at around 19:30 CDT. We rebooted the machine at that time and it has not yet come back online.
We’re working with the data center team to resolve this issue and hope to have service restored soon. We’ll update this post when we have more info.
2009-08-15 20:46 CDT – web61 is back online, but the filesystem problem remains. We are still investigating the problem.
2009-08-16 04:26 CDT – web61 is now back online and working. We will schedule a fsck run for later in the day.
The filesystem on Web8 went into a read-only state several minutes ago. We rebooted the server, and it is currently running a filesystem check. We hope to have service restored soon.
We are currently experiencing an issue with our “mail2script” feature (see
http://blog.webfaction.com/mail2script) which is causing it to miss a small percentage of emails.
We will update this post as soon as we have a resolution.
On Sunday August 31 between 11:00 AM and 3:00 PM CDT we will be rebooting all web servers to deploy a kernel update. We do not expect any server to see more than 10 minutes of down time.
web41.webfaction.com is currently down. We are looking into the problem and hope to have service restored soon.
Update [011:41 AM CST]: Upon rebooting the server we were faced with a kernel panic. We are still working on restoring service.
Update [02:35 PM CST]: We are still working on restoring service.
Update [04:40 PM CST]: We are still working on restoring service. If you would like you can submit a support ticket and we will give you access to another server. If we are not able to restore service soon we will setup another server to take its place.
Update [08:35 PM CST]: Service has been restored. The problem was file system configuration related and has been fixed.
At the moment Web 92 is inaccessible. We are looking into it.
Update [01:50 PM CST]: The problem was a spike in load. Everything is under control now.
At the moment Web 4 is inaccessible. We are looking into it.
Update [08:25 PM CST]: The problem was a spike in load. Everything is under control now.
Dweb 13, Dweb 30, Dweb 33, Web 8, and Web 16 are currently inaccessible.
Update [05:05 PM CST]: The problem has been tracked down to The Planet’s Dallas 5 data center.
Update [06:15 PM CST]: The problem has been fixed.