[Done] Emergency maintenance on Web246

Posted in Downtime by

Web246 is currently offline. We’re working to restore service at this time and will update this post as soon as we have more information.

Update: The server is showing signs of hardware issues. At this point we’re going to migrate the data to new (and faster) hardware. As part of the migration process the main IP address will change from 199.115.114.40 to 108.168.205.72. We will update this blog post with the migration progress.

Update 4am UTC: The data on the old hardware appears to be corrupted. At this point we are restoring all data from backup on the new hardware and we are trying to recover the data on the old hardware (both in parallel).

Update 10am UTC: We are still working on recovering the data and putting it on the new hardware.

Update 6pm UTC: At this point all MySql and PostgreSQL data has been migrated to the new hardware. Some home directories have been migrated and some sites are back online. We are working on the rest of the home directories.

Update 9pm UTC: We are continuing to recover the home directories from backup and bring more sites online.

-
-

[Done] Emergency migration of Web336 to faster hardware

Posted in Service change by

Due to performance issues on Web336 (and suspected hardware issues) we are starting an emergency migration to faster hardware.

The server’s main IP will change from 108.59.2.70 to 75.126.149.10.

If you’re using WebFaction’s DNS servers for all your domains then you don’t need to do anything.

If you’re using non-WebFaction DNS servers for some of your domains then you will have to update their A records yourself. We will leave a proxy running on the old server for 5 days to redirect all HTTP and HTTPS traffic to the new server.

Update 1pm UTC: The migration is now over. We have setup a proxy on the old server to redirect all HTTP and HTTPS traffic to the new server. The proxy will stay for 5 days. Note that during the migration MySql refused to start on either hardware and we had to restore all MySql data from backup.

-
-

[Done]Scheduled maintenance on Web119, 30th December 2013.

Posted in Scheduled downtime by

Web119 will be taken down for a RAID firmware upgrade Monday 30th December 2013 at 13:30 UTC. We will update this post as maintenance will progress.

2013-12-30 13:52 UTC The RAID firmware has been updated. The server is back to operational status.

-
-

[Fixed] Web432 down

Posted in Downtime by

Web432 is currently down due to what appears to be a hardware issue. We’re working on restoring service asap.

Update 2.45pm UTC: We are unable to get the current hardware to boot so we are going to migrate the server to new hardware. The IP address will change from 75.126.113.172 to 75.126.149.12

Update: 5.11pm UTC: We have managed to fix the current server and all services are back to normal. The migration to new hardware is cancelled and the server keeps its existing IP (75.126.113.172)

-
-

[Done]Scheduled maintenance on Web343, December 19th, 2013.

Posted in Scheduled downtime by

Web343 will be taken down for a RAID firmware upgrade Thursday December 19th, 2013 between 13:00 UTC and 15:00 UTC. We will update this post as maintenance will progress.

2013-12-19 13:28 UTC The RAID firmware upgrade completed. The server is back to operational status.

-
-

[Fixed] Network switch failure affecting several dedicated servers

Posted in Downtime by

The following dedicated servers are currently inaccessible due to a failed network switch:

  • Dweb140
  • Dweb141
  • Dweb142
  • Dweb152
  • Dweb153
  • Dweb158
  • Dweb160
  • Dweb161
  • Dweb162

The data center is working to replace the switch at this time. We will update this post when we have more information.

2013-12-17 02:06 UTC: The network hardware is repaired and the machines are back online.

-
-

[Done] Migration of Web337 to faster hardware on December 23th, IP address change

Posted in Service change by

Due to performance and hardware issues on Web337 we are going to migrate it to faster hardware on December 23th between 10am and 12pm UTC.

The server’s main IP will change from 108.59.2.71 to 108.168.213.88

If you’re using WebFaction’s DNS servers for all your domains then you don’t need to do anything.

If you’re using non-WebFaction DNS servers for some of your domains then you will have to update their A records yourself. We will leave a proxy running on the old server for 5 days to redirect all HTTP and HTTPS traffic to the new server.

We’ll update this post as the work progresses.

Update Dec 23rd, 10am UTC: The migration is starting now.

Update Dec 23rd, 11.20am UTD: The migration is now over. If you experience any issues with your sites open a ticket and we’ll look into it asap. We have setup a proxy on the old server to redirect all HTTP and HTTPS traffic to the new server. The proxy will stay for 5 days.

-
-

[Fixed] Emergency maintenance on Web337

Posted in Downtime by

Web337 is currently offline. We’re working to restore service at this time and will update this post as soon as we have more information.

Update 21:11 UTC: We ran fsck on the server. The server is now back online but the mysql and postgresql data appears to be corrupted. We are currently restoring that data from our backups.

Update 23:29 UTC: We have now restored all mysql and postgresql databases from backup. If you notice any problem with your data open a ticket and we’ll look into it asap

-
-

[Fixed] MySQL problems on Web211

Posted in Downtime by

We are experiencing problems on MySQL on web211 at the moment.  Some users may encounter problems updating or deleting data or databases.  We are working on fixing this right now.

2013-12-14 07:11 UTC: We are importing all of the current MySQL data into a new MySQL instance which should resolve the issues.  The re-import is about halfway complete at this stage.

2013-12-14 19:00 UTC: All MySQL data has been restored.

-
-

[Done] Migration of Web245 to faster hardware on December 20th, IP address change

Posted in Service change by

Due to performance issues on Web245 we are going to migrate it to faster hardware on December 20th between 10am and 12pm UTC.

The server’s main IP will change from 199.115.114.39 to 75.126.173.137

If you’re using WebFaction’s DNS servers for all your domains then you don’t need to do anything.

If you’re using non-WebFaction DNS servers for some of your domains then you will have to update their A records yourself. We will leave a proxy running on the old server for 5 days to redirect all HTTP and HTTPS traffic to the new server.

We’ll update this post as the work progresses.

Update Dec 20th, 10am UTC: The migration is starting now.

Update Dec 20th, 11.36am UTC: The migration is now over. If you experience any issues with your sites open a ticket and we’ll look into it asap. We have setup a proxy on the old server to redirect all HTTP and HTTPS traffic to the new server. The proxy will stay for 5 days.

-
-