[Fixed] Mail5 down

Posted in Downtime by

Our mail5 server is currently offline due to filesystem errors. We’re working to back up the data before we attempt any filesystem repairs. We’ll update this post as soon as we have more information.

2014-02-26 02:42 UTC: We’re currently checking mail5 RAID status and provisioning a new RAID card. We’ll update this post as soon as we have more information.

2014-02-26 04:04 UTC: We replaced the RAID card on mail5, at this moment all services on mail5 are functional.

-
-

[Done] Web155 down due to filesystem errors

Posted in Downtime by

Web155 is currently down due to filesystem errors. We’re working to correct them and will update this post when we have more information.

Update 18:56 UTC: The filesystem appears to be partially corrupted. We have started an emergency migration of the server to new hardware. We will update this post as the migration progresses.

Update: 21:26 UTC: All MySQL databases and home directories starting with “a” have been migrated. The MySQL server and web servers have been started on the new server and the first websites are back online. We are working on migrating the PostgreSQL data and the rest of the home directories.

Note that the server’s main IP will change from 108.59.9.147 to 75.126.149.4

If you’re using WebFaction’s DNS servers for all your domains then you don’t need to do anything.

If you’re using non-WebFaction DNS servers for some of your domains then you will have to update their A records yourself. We will leave a proxy running on the old server for 10 days to redirect all HTTP and HTTPS traffic to the new server

Update 23:16 UTC: All PostgreSQL databases and home directories up to the letter “c” have been migrated. We are working on the rest of the home directories.

Update 07:21 UTC: Home directories up to the letter “i” have been migrated. We are working on the rest of the home directories.

Update 09:52 UTC: Home directories up to the letter “p” have been migrated. We are working on the rest of the home directories.

Update 14:30 UTC: The migration is now over. If you notice any problems with your account open a ticket and we’ll look into it asap.

-
-

[Done] Migration of Web324 to faster hardware on February 27th, IP address change

Posted in Downtime by

Due to performance issues on Web324 we are going to migrate it to faster hardware on February 27th between 10am and 12pm UTC.

The server’s main IP will change from 199.58.84.68 to 75.126.149.5

If you’re using WebFaction’s DNS servers for all your domains then you don’t need to do anything.

If you’re using non-WebFaction DNS servers for some of your domains then you will have to update their A records yourself. We will leave a proxy running on the old server for 5 days to redirect all HTTP and HTTPS traffic to the new server.

We’ll update this post as the work progresses.

Update Jan 27th, 10am UTC: The migration is starting now

Update 12.15pm UTC: The migration is now over. If you experience any issues with your sites open a ticket and we’ll look into it asap. We have setup a proxy on the old server to redirect all HTTP and HTTPS traffic to the new server. The proxy will stay for 5 days.

-
-

[Done] Migration of Web314 to faster hardware on February 27th, IP address change

Posted in Downtime by

Due to performance issues on Web314 we are going to migrate it to faster hardware on February 27th between 10am and 12pm UTC.

The server’s main IP will change from 85.17.23.70 to 37.58.75.250.

If you’re using WebFaction’s DNS servers for all your domains then you don’t need to do anything.

If you’re using non-WebFaction DNS servers for some of your domains then you will have to update their A records yourself. We will leave a proxy running on the old server for 5 days to redirect all HTTP and HTTPS traffic to the new server.

We’ll update this post as the work progresses.

Update 2014-02-27 12:06 UTC: The migration is starting now

Update 2014-02-27 14:47 UTC: The migration is taking a bit longer than expected.

Update 2014-02-27 19:15 UTC: The server has been migrated to the new machine, but there was a problem with the old server and we were unable to recover the last 24 hours of changes. Also the MySQL data on the new server is incomplete, we are in the process of restoring from backup.

Update 2014-02-27 19:58 UTC: We have setup a proxy on the old server to redirect all HTTP and HTTPS traffic to the new server. The proxy will stay for 5 days.

Update 2014-02-27 20:20 UTC: PostgreSQL is also missing and we are working on restoring.

Update 2014-02-27 21:58 UTC: We have finished restoring Postgres and MySQL databases from backup.

Update 2014-02-27 22:01 UTC: The migration is now over. If you experience any issues with your sites open a ticket and we’ll look into it asap.

-
-

[Fixed] MySQL down on Web74

Posted in Downtime by

We are experiencing problems with MySQL on web74 at the moment, so MySQL is not accessible.  We are working urgently to bring it back online.

2014-02-20 14:01 UTC: We discovered mysql structure is inconsistent on the server, we are doing data dump and restore

2014-02-20 14:42 UTC: Mysql on web174 is up and running at the moment

-
-

[Fixed] Web231 connectivity issues

Posted in Problems by

We are currently experiencing connectivity issues with web231.

We are investigating and will restore service as quickly as possible.

2014-02-17 22:22 UTC There is a hardware issue on the server and we are going to migrate the data to new hardware. As part of the migration the server’s main IP will change from 108.59.2.82 to 184.172.15.231.  If you’re using WebFaction’s DNS servers for all your domains then you don’t need to do anything. If you’re using non-WebFaction DNS servers for some of your domains then you will have to update their A records yourself. We will leave a proxy running on the old server for 10 days to redirect all HTTP and HTTPS traffic to the new server.

2014-02-18 02:00 UTC: All MySQL and PostgreSQL data has been migrated to the new server and the database servers have been started. All home directories up to the letter ‘c’ have been migrated and their sites are back up. We are working on the rest of the home directories.

2014-02-18 08:42 UTC: All home directories up to the letter ‘j’ have been migrated.

2014-02-18 12:53 UTC: All home directories up to the letter ‘m’ have been migrated.

2014-02-18 14:37 UTC: All home directories up to the letter ‘r’ have been migrated.

2014-02-18 18:00 UTC: All data has now been migrated and everything should be back to normal. If you notice any problems open a ticket and we’ll look into it asap.

-
-

[Done] Migration of Web200 to faster hardware on February 18th, IP address change

Posted in Service change by

Due to performance issues on Web200 we are going to migrate it to faster hardware on February 18th between 10am and 12pm UTC.

The server’s main IP will change from 108.59.11.77 to 184.172.15.230.

If you’re using WebFaction’s DNS servers for all your domains then you don’t need to do anything.

If you’re using non-WebFaction DNS servers for some of your domains then you will have to update their A records yourself. We will leave a proxy running on the old server for 5 days to redirect all HTTP and HTTPS traffic to the new server.

We’ll update this post as the work progresses.

2014-02-18 11:00 UTC The migration is starting now.

2014-02-18 16:32 UTC While migrating the latest data the filesystem on the old hardware crashed, some sites are currently down, we’re working to restore them asap.

2014-02-18 20:00 UTC: The migration is now over. If you experience any issues with your sites open a ticket and we’ll look into it asap. We have setup a proxy on the old server to redirect all HTTP and HTTPS traffic to the new server. The proxy will stay for 5 days.

-
-

[Cancelled] Scheduled maintenance on webmail.webfaction.com, February 17th, 2014.

Posted in Downtime by

webmail.webfaction.com will be taken down for upgrading, Monday February 17th, 2014 between 09:00 UTC and 10:00 UTC. We will update this post as maintenance will progress.

2014-02-17 10:35 UTC This maintenance has been cancelled.

-
-

[Done] Emergency maintenance on Web200

Posted in Downtime by

Web200 is currently offline. We’re investigating the issue and working to restore service at this time and will update this post as soon as we have more information.

2014-02-13 02:39 UTC:  Our data center is putting the server into a rescue environment so we can repair the filesystem.

2014-02-13 04:35 UTC: The filesystem check is running.

2014-02-13 05:25 UTC: The server is back online.

-
-

[Done] Scheduled maintenance on Web217, February 13th, 2014

Posted in Downtime by

Web217 will be taken down for a firmware upgrade Thursday, February 13th 2014 between 15:00 UTC and 16:00 UTC. We will update this post as maintenance progresses.

-
-