[Done] MySQL on Web95 down

Posted in Problems by

The MySQL service on Web95 is currently down. We’re looking into the problem and will update this post as soon as we have more information.

2015-02-13 00:09 UTC: We’re currently working to dump and restore the MySQL data.

2015-02-13 01:51 UTC: The filesystem on Web95 has failed, so we’re preparing a new server so that we can perform an emergency migration of Web95 to more-reliable hardware.

2015-02-13 04:57 UTC: We have setup the new machine and have starting syncing the data. As part of the migration process the main IP address will change from 108.59.9.132 to 50.97.95.211. We will update this blog post with the migration progress.

2015-02-13 07:02 UTC: Home directories up to the letter “a” have been copied to the new server. We’re continuing to copy the rest of the home directories.

2015-02-13 09:07 UTC: Home directories up to the letter “c” have been copied to the new server. We’re continuing to copy the rest of the home directories.

2015-02-13 10:56 UTC: Home directories up to the letter “d” have been copied to the new server. We’re continuing to copy the rest of the home directories.

2015-02-13 15:37 UTC: The mysql data was corrupt on the old machine was not recoverable, we restored all databases from backup.

2015-02-13 17:03 UTC: Home directories up to the letter “f”  have been copied to the new server. We’re continuing to copy the rest of the home directories.

2015-02-13 17:50 UTC: Home directories up to the letter “t” – “z”  have been copied to the new server. We’re continuing to copy the rest of the home directories.

2015-02-13 19:16 UTC: Home directories up to the letter “i” and “r” -“z”  have been copied to the new server. We’re continuing to copy the rest of the home directories.

2015-02-13 21:06 UTC: Home directories up to the letter “l” and “p” -“z”  have been copied to the new server. We’re continuing to copy the rest of the home directories.

2015-02-13 23:09 UTC: We are down to syncing the final 2 home directories.

2015-02-14 07:11 UTC: The migration is now over. If you experience any issues with your sites open a ticket and we’ll look into it asap. We have setup a proxy on the old server to redirect all HTTP and HTTPS traffic to the new server. The proxy will stay for 5 days.

-
-

[Fixed] Web403 emergency MySQL maintenance

Posted in Downtime by

We are performing an urgent maintenance on web403’s MySQL. It will appear offline until the maintenance is over.

We will update this post when there is progress on the subject.

The MySQL user table had crashed and had to be repaired. The operation was successful and now the MySQL service is functioning properly.

-
-

[Fixed] Web349 MySQL urgent maintenance

Posted in Downtime by

We detected a problem with one of MySQL’s tables on web349 and we shut down the database service in an emergency to carry out repair procedures.

We will update this post when there is progress on the issue.

2014-03-08 11:29 UTC: The problem MySQL table was repaired and the server is back online.

-
-

[Done] MySQL down on Web235, January 16th, 2014.

Posted in Downtime by

We are experiencing problems with MySQL on web235 at the moment, so MySQL is not accessible.  We are working urgently to bring it back online.

2014-01-16 13:21 UTC The problems have been fixed and server is back at operational status.

-
-

[Fixed] MySQL problems on Web211

Posted in Downtime by

We are experiencing problems on MySQL on web211 at the moment.  Some users may encounter problems updating or deleting data or databases.  We are working on fixing this right now.

2013-12-14 07:11 UTC: We are importing all of the current MySQL data into a new MySQL instance which should resolve the issues.  The re-import is about halfway complete at this stage.

2013-12-14 19:00 UTC: All MySQL data has been restored.

-
-

[Fixed] MySQL down on Web336

Posted in Downtime by

We are experiencing problems with MySQL on web336 at the moment, so MySQL is not accessible at the moment.  We are working urgently to bring it back online.

2013-12-07 04:10 UTC: We found some file system errors which may be related to the root issue, so we are now in the process of taking an immediate backup of MySQL as a precaution.

2013-12-07 05:36 UTC: Data dump is still in progress.

2013-12-07 08:00 UTC: We’ve managed to fix the problems and MySQL and the server are back online and fully operational.

-
-

[Fixed] MySQL/Filesystem problems on Web312

Posted in Downtime by

The MySQL service on Web312 is currently down. We’re working to restore service and will update this post as soon as we have more information.

2013-11-21 20:50 UTC: Several corrupt databases are interfering with the restoration of the MySQL service. We’re working to isolate those databases to restore the service.

2013-11-21 22:15 UTC: We’re checking the Web312 filesystem at this moment because its filesystem became corrupt when restoring databases.

2013-11-21 23:12 UTC: The filesystem was too damaged to be recovered. We’re preparing a new server to restore backups.

013-11-22 00:50 UTC: We are still setting up the new server. We will update this post as soon as we have more information.

2013-11-22 02:11 UTC: The new server is up and running and data is being restored.

2013-11-22 05:07 UTC: PostgreSQL and MySQL Databases were restored. Home data is still being restored.

2013-11-22 13:53 UTC: Most home directories have been restored. We’re working on restoring the remaining data.

2013-11-23 09:30 UTC: 95% of home directories have now been restored. If your home directory is still missing data please open a support ticket.

2013-11-23 11:30 UTC: All home directories have now been restored. If you’re still experiencing issues please contact support.

-
-

[Fixed] MySQL problems on Web319

Posted in Problems by

The MySQL service on Web319 is currently down. We’re working to restore service and will update this post as soon as we have more information.

Update: The data was corrupted beyond repair so we restored all databases from backup. If you notice any problem with your database open a support ticket and we’ll look into it asap

-
-

[Fixed] Web335 MySQL problems

Posted in Downtime by

We are currently experiencing problems with MySQL on web335, so users may be unable to connect.  We are working on bringing it back ASAP.

2013-09-16 02:07 UTC: MySQL is now back up and running.

2013-09-16 02:43 UTC: Problems have re-occurred.  We are going to upgrade some system components and restart the server shortly.

2013-09-16 03:39 UTC: The problems have been sorted out and Web335 is now stable again.

-
-

[FIXED] MySQL Down on Web327

Posted in Downtime by

The MySQL service is currently down on Web327. We’re working to restore service and will update this post when we have more information

2013-02-23 1923 UTC: Users may see intermittent issues with the Webserver in general as we need to stop web services to bring MySQL back online.

2013-02-23 2308 UTC: Web327 is back on-line and MySQL service has been restored. The machine is now online and functioning normally

-
-