[Fixed] Web408 and Web416 offline

Posted in Downtime by

Web408 and Web416 are currently offline. We’re working to restore service and will update this post when we have more information.

2014-06-25 23:26 UTC: Web408 was offline due to a kernel panic. It’s been rebooted and is now back in service.

2014-06-25 23:57 UTC: Web416 is still offline. We’re rebooting the machine for hardware diagnostics.

2014-06-25 23:33 UTC: Web416 is back online after a filesystem check, but mysql is still offline. We are working to get mysql back online.

2014-06-26 00:57 UTC: Web408 is offline again. We’re looking into it.

2014-06-26 02:22 UTC: Web408 is back online.  We used some defensive networking to mitigate a DDOS attack.

2014-06-26 05:56 UTC: Web416 MySQL is back online.

-
-

[Fixed] Issue with MySQL on Web377

Posted in Downtime by

The MySQL service on web377 is currently down. We’re working to restore service and will update this post when we have more information.

2014-06-25 15:38 UTC:  All databases have been restored and MySQL is back online. If you have any problems, please open a support ticket.

-
-

[Done] Network maintenance on July 2, 2014

Posted in Scheduled downtime by

On July 2 2014, our upstream provider will be performing network maintenance that will briefly interrupt network connectivity on the following servers:

  • Web247
  • Web345
  • Web347

The maintenance is scheduled for a 1-hour window, beginning at 7:00 am EDT.

We’ll update this post as the maintenance progresses.

2014-07-02 11:52 UTC The maintenance is over and the servers have been restored to service.

-
-

[Done] Network maintenance July 8-10 2014

Posted in Scheduled downtime by

On July 8th through July 10th, our upstream provider will be performing network maintenance that will briefly interrupt network connectivity for several servers:

  • July 8:  Dweb120, Dweb121
  • July 9: Dweb127
  • July 10: Web232, Web233, Web234, Web236, Web238, Web239, Web240, Web241, Web310, Web311

On each of these days, the maintenance will take place between 7:30 am and 9:30 am EDT, and will require a network switch reboot that will result in a loss of network connectivity for up to 30 minutes (but probably less). We’ll update this post as the maintenance progresses.

 

2014-07-10 This maintenance was completed.

-
-

[Fixed] DDoS attack on web27

Posted in Downtime by

The server is currently under a heavy DDoS attack. Mitigation has been enabled to help with the issue. We will continue to monitor and will update when we have more information.

2014-06-22 06:06 UTC:  The server is back online and functioning normally.

2014-06-22 11:43 UTC:  The server is experiencing another attack.

2014-06-22 16:20 UTC:  The server is back online and functioning normally.

-
-

[Cancelled] Migration of Web452 on June 22, 2014, IP address change.

Posted in Downtime by

Due to some stability issues on Web452 we are going to migrate it to new hardware on June 22 between 03:00AM and 06:00AM UTC.

The server’s main IP will change from 75.126.173.142 to 50.97.173.23.

If you’re using WebFaction’s DNS servers for all your domains then you don’t need to do anything.

If you’re using non-WebFaction DNS servers for some of your domains then you will have to update their A records yourself. We will leave a proxy running on the old server for 5 days to redirect all HTTP and HTTPS traffic to the new server.

We’ll update this post as the work progresses.

2014-06-18 17:56 UTC: We have cancelled the migration.

-
-

[Done] Emergency fsck on Web452

Posted in Downtime by

Web452 is being taken down for an emergency file system check.

[Jun 18, 02:41 UTC] The file system check is complete and the server is back online.

-
-

[Done]Migration of Web37 to faster hardware on June 19th, 2014, IP address change.

Posted in Scheduled downtime by

Due to performance issues on Web37 we are going to migrate it to faster hardware on June 19th between 07:00AM and 09:00AM UTC.

The server’s main IP will change from 199.58.84.80 to 50.97.173.24.

If you’re using WebFaction’s DNS servers for all your domains then you don’t need to do anything.

If you’re using non-WebFaction DNS servers for some of your domains then you will have to update their A records yourself. We will leave a proxy running on the old server for 5 days to redirect all HTTP and HTTPS traffic to the new server.

We’ll update this post as the work progresses.

2014-06-19 07:16 UTC The migration is starting now.

2014-06-19 12:18 UTC The migration is now over. If you experience any issues with your sites open a ticket and we’ll look into it asap. We have setup a proxy on the old server to redirect all HTTP and HTTPS traffic to the new server. The proxy will stay for 5 days.

-
-

[Fixed] Web347, Web247, Web345 offline

Posted in Downtime by

Web347. Web247, Web345 are currently offline. We’re working to restore service at this time and will update this post as soon as we have more information.

2014-06-14 06:18 UTC: Services have been restored, we are looking as to how  this happened. Apologies for any inconveniences.

-
-

[Fixed] Issue with MySQL on Web309

Posted in Downtime by

The MySQL service on web309 is currently down. We’re working to restore service and will update this post when we have more information.

2014-06-13 14:35 UTC: We discovered an issue at filesystem level of database and we are dumping all data from it.

2014-06-13 14:41 UTC: We finished dumping the data and we are reloading data into a new mysql instance.

2014-06-13 14:58 UTC: All databases have been restored and MySQL is back online. If you have any problems, please open a support ticket.

2014-06-13 21:36 UTC: The MySQL service on web309 is currently down again. We’re working to restore service and will update this post when we have more information.

2014-06-14 01:30 UTC: All databases have been restored and MySQL is back online. If you have any problems, please open a support ticket.

-
-