[Resolved] DDoS attack against Web331

Posted in General by

Web331 is under a heavy DDoS attack at the moment, which is affecting its network connectivity.

Cisco Guard has been enabled to help reduce the impact. We are continuing to monitor.

 

-
-

[Resolved] Web456 connectivity issues

Posted in Problems by

We are currently experiencing connectivity issues with the server. The issue is being investigated. We will update this post when we have more information.

2014-05-26 9:20 UTC: We detected a DDoS attack against Web456. The server has been placed under our mitigation system which has improved the situation, but packet loss and other network problems are still to be expected. We will keep monitoring the server.

2014-05-27 08:16 UTC: The DDoS attack is still going on.

2014-05-28 18:50 UTC: The DDoS attack has subsided and we haven’t seen significant packet loss to or from the server. DDOS protection has been removed for several hours now and the server is operating properly. We will update this post if the issue resurfaces.

-
-

[Done] Scheduled maintenance on web385 on Sunday, May 25th at 11am UTC

Posted in Downtime by

On Sunday, May 25th at 10am UTC, web385 will be brought down for a firmware upgrade on the hard drives. Expected downtime is less than 1 hour.

We’ll update this post as the maintenance progresses.

10:15 UTC The maintenance has begun.
10:50 UTC The maintenance is complete and server is back online.

-
-

[Done] Scheduled maintenance on webmail.webfaction.com, May 26th, 2014.

Posted in Scheduled downtime by

webmail.webfaction.com will be taken down for upgrading, Monday May 26th 2014, between 07:00 UTC and 09:00 UTC. The maintenance should be less than 2 hours. We will update this post as maintenance will progress.

2014-05-26 08:32 UTC webmail.webfaction.com is back to operational status.

-
-

[Done] Migration of Web210 to faster hardware on May 26th, IP address change

Posted in Service change by

Due to performance issues on Web210 we are going to migrate it to faster hardware on May 26th between 11am and 4pm UTC.

The server’s main IP will change from 94.75.245.143 to 81.95.157.166

If you’re using WebFaction’s DNS servers for all your domains then you don’t need to do anything.

If you’re using non-WebFaction DNS servers for some of your domains then you will have to update their A records yourself. We will leave a proxy running on the old server for 5 days to redirect all HTTP and HTTPS traffic to the new server.

We’ll update this post as the work progresses.

Update 2014-05-26 11:30 UTC: The migration is starting now

Update 2014-05-26 16:00 UTC: The migration is taking a bit longer than planned and we are still syncing the data to the new machine.

Update 2014-05-26 17:45 UTC: As there is still quite a bit of data to migrate we have started re-enabling the services on the new machine and we have set up a proxy on the old machine to forward all HTTP and HTTPS requests to the new machine. So far all MySQL and PostgreSQL databases have been migrated and all home directories up to the letter “q” have been migrated and their sites have been re-enabled. We’re still working on migrating the rest of the home directories.

Update 2014-05-26 21:30 UTC: Home directories up to the letter “s” have been migrated and their websites have been re-enabled. We’re still working on migrating the rest of the home directories.

Update 2014-05-26 23:30 UTC: The migration is now over. If you experience any issues with your sites open a ticket and we’ll look into it asap. We have setup a proxy on the old server to redirect all HTTP and HTTPS traffic to the new server. The proxy will stay for 5 days.

-
-

[Update] Issue with MySQL on Web210

Posted in Downtime by

The MySQL service on web210 is currently intermittent. We’re working to restore service and will update this post when we have more information.

2014-05-22 19:46 UTC: Server is currently online and all services are running.

2014-05-23 20:19 UTC: Due to the continuing issues, we’ve scheduled a migration: http://statusblog.webfaction.com/2014/05/22/migration-of-web210-to-faster-hardware-on-may-26th-ip-address-change/

-
-

[Fixed] Web416 Inaccessible

Posted in Downtime by

Currently Web416 is down and websites are not working. We are investigating the issue and working to bring all services back online ASAP.

2014-05-19 18:42 UTC: We’re checking server connectivity

2014-05-19 18:55 UTC: Server is currently online and all services are running

-
-

[Fixed] Issue with MySQL on Web420

Posted in Downtime by

The MySQL service on web420 is currently down. We’re working to restore service and will update this post when we have more information.

2014-05-19 16:10 UTC: We have restored MySQL from backups and working on the MySql user table.

2014-05-19 20:30 UTC: All databases have been restored and MySQL is back online. If you have any problems, please open a support ticket.

-
-

[Fixed] Network connectivity issues in Amsterdam

Posted in Downtime by

There are network issues in our European datacenter, and we are working with our datacenter engineers to resolve the problem.

The problems are caused by power outage in one of the server rooms. We have people on-site working on a solution.

2014-05-19 14:41 UTC: power has been restored to the server room and servers are starting to come back online.

2014-05-19 15:02 UTC: Most of the servers are back online and we are scrutinizing them.

2014-05-19 18:10 UTC : all servers affected by the power outage are back online, with two exceptions:

2014-05-19 20:25 UTC: all servers are back online

We will update this post as more information becomes available.

-
-

[Fixed] Web433 having sporadic connectivity issues

Posted in Problems by

Web433 is having sporadic connectivity issues which can cause sites to be down, and difficulty with SSH/FTP connections. We are working to restore normal service and will update this post when more information is available.

2014-05-18 20:15 UTC: We detected a DDoS attack on server, it is under our mitigation system and the it is fully accessible at this moment. We will keep monitoring the server.

2014-05-19 12:38 UTC: The server has been stable for about the last 8.5 hours. We will continue to monitor.

 

-
-