On Sunday at…
…4 PM GMT we are going to replace Web 1’s /dev/sdc with a 300 GB HDD. Expected down time is an hour.
…5 PM GMT we are going to reboot Dweb 30, Dweb 57, Dweb 58, Dweb 59, Mail 2, and Mail 3 to install a kernel update. Expected down time is 15 minutes.
Update [...]
A problem with one of our DNS servers briefly interrupted access to most of our servers. The issue has been resolved and all servers are now accessible.
Network problems in the data center are currently causing degraded performance and intermittent outages on multiple servers. We’re looking into the problem and hope to have it resolved soon. We will update this post as the situation develops.
Update (7.13pm CST): Servers currently affected are: Web32, Web12, Dweb21, Dweb28, Web14, Dweb52, Web24, Web54, Dweb19, Web13, Mail4.
Update [...]
We will be investigating the cause of visual alarms on mail5.webfaction.com and web11.webfaction.com at 30 Dec 2009 0400 GMT-6.
Depending on the severity we might have to take the server down and if so it will be down between a few minutes and a few hours. We will update this ticket as soon as we have [...]
See http://statusblog.webfaction.com/2009/12/14/kernel-upgrade/.
On Sunday December 20 at 11 AM CST we will begin rebooting the following servers to install a kernel upgrade:
Dweb19
Dweb21
Dweb23
Dweb26-dweb30
Dweb32
Dweb33
Dweb39
Dweb42-dweb47
Dweb49-dweb54
Mail1
Mail5-Mail8
Mailbox1-Mailbox4
Mamba
panel.webfaction.com
Taipan
Viper
Web21-Web34
Web36-Web43
Web46-Web48
Web50
Web52
Web54
Web56-Web66
Web68
Web69
Web71-Web83
Web85-Web89
Web91-Web100
Web102-Web108
Web110
We expect to be done by 7 PM CST and we do not expect any server to be down for more than 10 minutes.
Update [06:35 PM CST]: Done
We are looking into the issue and will update this post when we get more info.
Update 1: We rebooted the server and it is currently running fsck.
Update 2: fsck is currently on a second pass
[09:10 PM CST] Update: The HD appears to be failing so we are going to attempt to back up what we [...]
