[Fixed] Web48 Connectivity Issues

Posted in Downtime by

We are currently experiencing connectivity issues with the server. We are investigating the issue and will update this post when we have more information.

2013-05-31 1345 UTC Service to Web48 has been restored.  We are continuing to investigate the root cause for the outage.

2013-05-31 1400 UTC The server is operating normally after a preemptive hardware check passed.

-
-

[Fixed]DDoS on Web371

Posted in Downtime by

Web371 server is experiencing a massive DDoS attack which our DDoS protection system is having trouble mitigating.

We’re doing everything we can to restore service at this time, and will update this post as progress is made.

Update 10:08 am: Our DDoS protection is now doing a much better job at mitigating the attack. All sites are now up but performance is slightly degraded.

Update 1pm: The attack has stopped and the server is back to normal

-
-

[Fixed] Intermittent SMTP connectivity

Posted in Problems by

Connectivity to smtp.webfaction.com is currently intermittent for some customers due to DDOS mitigation services that were activated to protect the server from an inbound attack.

We’ll update this post when the attack has subsided and when normal connectivity has been restored.

2013-05-28 22:43 UTC: The attack has subsided and normal connectivity has been restored.

-
-

[Fixed] Web371 down

Posted in Downtime by

Web371 became unresponsive about 30 minutes ago. We’ve attempted to reboot it, but the machine has not come back online yet.

We’re working on restoring service and will update this post when we have more information.

2013-05-28 18:06 UTC: Web371 is back online.

-
-

[Done] Postgres upgrade on Web239-Web249

Posted in Downtime by

We are going to upgrade Postgresql 8.3 / PostGIS 1.3.6 to Postgresql 8.4 / PostGIS 1.5 on the following servers in the

upgrade interval mentioned alongside.

Web239    May 29, 2am to 4am UTC

Web240    May 29, 2am to 4am UTC

Web241    May 29, 2am to 4am UTC

Web242    May 29, 3am to 5am UTC

Web243    May 29, 3am to 5am UTC

Web244    May 29, 3am to 5am UTC

Web245    May 29, 3am to 5am UTC

Web246    May 29, 4am to 6am UTC

Web247    May 29, 4am to 6am UTC

Web248    May 29, 4am to 6am UTC

Web249    May 29, 4am to 6am UTC

Postgresql will be unavailable for the duration of the upgrade.

We will update this post as the upgrades progress.

[May 29 06:08] The upgrade is done on all the servers except Web244 where we are having some problems, so we have decided to delay it for a later date.

-
-

[Done] Filesystem Upgrade on Web24

Posted in Scheduled downtime by

Web24’s filesystem will be upgraded from ext3 to ext4 on Tuesday 28th May 2013

between 5am to 9 am UTC.

We will be updating this post as the upgrade progresses.

[May 28 06:48] The upgrade is done and the server is back and OK now.

-
-

[Done]Emergency maintenance on Web344

Posted in Downtime by

Web44 became unresponsive due to high load. We’ve taken the server down to add some load-limiting software.

Update 12:32 UTC: The server is now back to normal

-
-

[Done] IP address change for mx9.webfaction.com on May 27th

Posted in General by

On Monday May 27th, 2013 the IP address of mx9.webfaction.com (one of our inbound mail servers) will change from 67.228.154.2 to 108.168.242.152. The old IP will still be working for several days after that while the IP change propagates everywhere.

-
-

[Done]Scheduled maintenance on Web223, May 24th, 2013.

Posted in Scheduled downtime by

Web223 will be taken down for a RAID controller swap Friday, May 24th, 2013 between 07:00UTC and 10:00UTC. We will update this post as maintenance progresses.

2013-05-24 08:20 UTC RAID controller and cables have been replaced, the server is back to operational status.

-
-

[Done]Emergency maintenance on Web223, May 22nd, 2013.

Posted in Downtime by

Web223 filesystem went read-only. We’re taking the server down and run a fsck. We will update this post as maintenance progresses.

2013-05-22 07:46 UTC the server is back at operational status

-
-