Web 35 Down (fixed)

Posted in Downtime by

Web 35’s root partition has gone read only. We are looking
into it now.

[06:16 PM CST] Update: Web 35 is up and running again. There do not appear to be any software problems so we are running a diagnostic test on the server’s hardware.

[08:58 PM CST] Update: We are currently running fsck on Web 35.

[010:48 PM CST] Update: The fsck is complete and the server is back online.

-
-

New Webmail interface available

Posted in Service change by

We’ve setup a new Webmail interface (AtMail) at https://atmail.webfaction.com. Feel free to try it out and to send us your feedback at support@webfaction.com.

The plan is to make it the default Webmail interface in the near future.

-
-

Network issue in our datacenters

Posted in Downtime by

Several of the datacenters hosting our servers are having network issues and a large chunk of our servers is currently unreachable.

Update: the network issues are now fixed and all servers are back online.

Update 2009-05-13 10:38 CDT: the datacenters just experienced another brief network outage, but all servers are back online at this time.

-
-

Read-only filesystem on Web71 (fixed)

Posted in Downtime by

The filesystem on Web71 went into a read-only state a few minutes ago. We are currently rebooting the server and will investigate the problem further when it is back online.

Update 2009-05-07 08:43 CDT – Web71 is back online.

-
-

Problem on Web54 (fixed)

Posted in Downtime by

We’re currently investigating a problem that is affecting sites hosted on Web54. We’ll update this entry when we have more information.

Update 2009-05-07 11:33 CDT – Web54 is back online. The problem was high load due to excessive CPU utilization by our backup script. The problem with the script has been resolved.

-
-

Services down on several servers (fixed)

Posted in Downtime by

We are currently investigating an issue following an upgrade to CentOS-5.3. Services are down on several servers and we are currently working on restoring them. The servers are:

  • web23
  • web25
  • web27
  • web31
  • web32
  • web33
  • web34
  • web35
  • web36
  • web38
  • web39
  • web43
  • web46
  • web49
  • web52
  • web53
  • web55
  • web57
  • web58
  • web68
  • web75
  • dweb40

Update 2009-05-06 11:10 CDT: We are in the process of reinstalling some RPMs on the servers in order to bring them back online.

Update 2009-05-06 11.50 CDT: Two servers are back online (web36 and web52). We are continuing to work on the other servers.

Update 2009-05-06 12:39 CDT: Web32, Web35 and Web39 are back online. We are continuing to work on the other servers.

Update 2009-05-06 12:55 CDT: Web52 is down again.

Update 2009-05-06 13:23 CDT: Web25, Web27 and Web58 are back online. We are continuing to work on the other servers.

Update 2009-05-06 13:28 CDT: Web55 and Web68 are back online. We are continuing to work on the other servers.

Update 2009-05-06 14:06 CDT: Web38 and Web57 are back online. We are continuing to work on the other servers.

Update 2009-05-06 14:12 CDT: Web52 is back online. We are continuing to work on the other servers.

Update 2009-05-06 16:11 CDT: Web31, Web34 and Web46 are back online. We are continuing to work on the other servers.

Update 2009-05-06 16:48 CDT: Dweb40 and Web43 are back online. We are continuing to work on the other servers.

Update 2009-05-06 16:55 CDT: Web23 and Web53 are back online. We are continuing to work on the other servers.

Update 2009-05-06 17:04 CDT: Web33 and Web75 are back online. We are continuing to work on Web49 (the last server!).

Update 2009-05-06 17:26 CDT: Web49 is back online – that was the last server affected by the problem, so we’re calling this one fixed. Sorry for the trouble, folks!

-
-