[fixed] web340 inaccessible

Posted in Downtime by

We are investigating the issue and will update the post as soon as we have further details.

2017-01-23 23:46 UTC: We’re running a filesystem check on Web340 at this time.

2017-01-24 00:40 UTC: The disk on Web340 is completely unrecoverable. We’re going to use the most recent backup to restore the machine to new hardware. We’ll email affected customers with the new server details.

2017-01-24 02:18 UTC: We’re transferring data from the backup server to the new hardware at this time.

2017-01-24 02:28 UTC: The new server details were just emailed to all affected customers.

2017-01-24 02:45 UTC: About 30% of the affected customers have been restored so far.

2017-01-24 05:37 UTC: About 62% of the affected customers have been restored so far.

2017-01-24 09:20 UTC: Most sites should now be back online. Large files are still being restored

2017-01-24 12:00 UTC: Large files are still being restored.

2017-01-24 19:06 UTC: Large files are still being restored.

2017-01-24 23:15 UTC: Some very large files for a very small number of accounts are still being restored.

2017-01-25 01:58 UTC: All accounts on Web340 have been restored to the new server. If you’re having any problems with your site on the new hardware, then please open a support ticket and our team will assist you.

-
-

[Resolved] Web467 is inaccessible

Posted in Downtime by

2016-12-30 13:33 UTC: The server is currently not responding. We are investigating the issue and will update this post as soon as we have further details.

2016-12-30 13:41 UTC: We were able to reboot the server to resolve the issue. The server is back online.

-
-

[Fixed] Web339 inaccessible

Posted in Downtime by

The server is currently not accessible. We are investigating the issue and will update as soon as we have more information.

2016-10-28 14:01 UTC: The server is back online again. One of the disks was replaced.

2016-10-31 17:51 UTC: The server has been taken down for emergency maintenance.

2016-10-31 22:48 UTC: We’re preparing to reload the OS on Web339 and will restore customer data from a backup made earlier today.

2016-11-01 02:41 UTC: The server has been reloaded and is in the process of being setup.

2016-11-01 06:23 UTC: The server has been setup and databases restored.

2016-11-01 13:46 UTC: User accounts have been restored up to letter “P”.

2016-11-01 20:35 UTC: All user accounts have been restored. If you have any issues, please open a support ticket.

-
-

[Fixed] web244 inaccessible

Posted in Downtime by

The server is currently running a filesystem check as there were errors found.

We will update the post as soon as it has completed.

2016-10-18 17:28 UTC: The filesystem check is 67% complete.

2016-10-18 18:02 UTC: The filesystem check is complete and the server is back online. Our apologies for the trouble.

-
-

[Fixed] Web416 inaccessible

Posted in Uncategorized by

We are currently investigating the issue. As soon as we have any updates, we will post here.

2016-09-09 21:43 UTC: There was a networking issue that caused the server to be unavailable. This has been resolved and the server is back online.

-
-

Web162 inaccessible

Posted in Uncategorized by

The server is currently not responding. We are investigating the issue and will update as soon as we have further details.

2016-09-07 12:51 UTC: There was a temporary network issue that caused this. The server is back online.

2016-09-10 18:45 UTC: The server is currently not responding. We are investigating the issue and will update as soon as we have further details.

2016-09-10 19:15 The server is back online. We are monitoring.

-
-

[fixed] Web319 connectivity issues

Posted in Downtime by

We are currently investigating the issue. Further details will be posted as soon as we can.

2016-07-21 20:43 UTC: The issue has been narrowed to the primary IP of the server. We are still investigating the cause.

2016-07-21 22:49 UTC: Connectivity has been restored and the server is back online.

-
-

[Fixed] Web162 inaccessible

Posted in Downtime by

We are currently investigating the issue and will post an update as soon as we have further details.

2016-07-06 18:08 UTC: Web162 is back online.

-
-

[Fixed] Web519 inaccessible

Posted in Downtime by

There is an issue with the network card in the server.

We are investigating the issue with our upstream provider.

2016-06-21 22:52 UTC: We have identified the problem and are working with the datacenter to resolve the issue.

2016-06-21 23:38 UTC: The network issue has been resolve and the server is back online.

 

-
-

[Fixed] Web75 inaccessible

Posted in Downtime by

The server is currently not responding. We are looking into the issue and will post any updates as soon as possible.

2016-06-13 21:50 UTC: We are still working with our upstream provider to get the server back online.

2016-06-14 00:40 UTC: We have identified the RAID issue and are rebuilding in rescue mode to prevent any data loss.

2016-06-14 03:29 UTC: One of the failed drives in the RAID array has finished rebuilding, we are replacing the other and letting it rebuild before bringing back online.

2016-06-14 9:30 UTC: Some additional issues with the RAID setup on the server were discovered after our previous update. We have already resolved them and now we are waiting for the second drive in the RAID array to finish rebuilding. The process is currently at 20%.

2016-06-14 9:50 UTC: The process of rebuilding the second drive is currently at 40%.

2016-06-14 10:20 UTC: The process of rebuilding the second drive is currently at 60%.

2016-06-14 10:45 UTC: The process of rebuilding the second drive is currently at 76%.

2016-06-14 11:05 UTC: The process of rebuilding the second drive is currently at 88%.

2016-06-14 13:22 UTC: The servers is back online.

2016-06-14 19:20 UTC: The server went read-only, we are working on a solution.

2016-06-14 22:27 UTC: We are still working on a solution to prevent data loss.

2016-06-15 04:21 UTC: The RAID was beyond the state of recovery, we are replacing the server and will restore from backup.

2016-06-15 06:50 UTC: The server is setup and online. We have restored all databases and home directories starting with the letters ‘a’ through ‘d’.

2016-06-15 08:25 UTC: All home directories up to ‘f’ have been restored.

2016-06-15 09:25 UTC: All home directories up to ‘g’ have been restored.

2016-06-15 10:25 UTC: All home directories up to ‘i’ have been restored.

2016-06-15 14:46 UTC: All home directories up to ‘l’ have been restored.

2016-06-15 17:18 UTC: All home directories up to ‘p’ have been restored.

2016-06-15 20:17 UTC: All home directories up to ‘s’ have been restored.

2016-06-15 22:21 UTC: All accounts have been restored from backup. If you experience any issues with your sites, open a ticket and we’ll look into it ASAP.

-
-