[Done]Emergency maintenance on Web226.

Posted in Downtime by

Currently Web226 is offline due to a read only file system. We have taken the server offline to perform a full file system check. We will update this post as we have more information.

2013-08-30 23:04 UTC: The file system check is still in progress at 45% finished on the first pass.

2013-08-30 23:46 UTC: The file system check has finished and the server is now back online. We are continuing to monitor the server closely to verify that the file system is now stable.

-
-

[Done]Emergency maintenance on Web223 Agust 30th, 2013.

Posted in Downtime by

Web223 filesystem went R/O. We’re currently taking the server down for running a fsck. We’ll update this status blog post as maintenance progresses.

2013-08-30 06:42 UTC Fsck completed and the server is back at operational status.

-
-

[Done] Chassis swap on web364 on Monday, August 26th

Posted in Downtime by

web364 will be taken down on Monday, August 26th for a routine chassis swap between 6am-11am UTC. Expected downtime is less 30 minutes.

2013-08-26 08:20 UTC Maintenance is complete and the server is back online.

-
-

[Fixed] Web364 down

Posted in Downtime by

Web364 has suddenly gone down and we are currently working to bring it back online ASAP.

2013-08-25 16:53 UTC: Web364 is back and operating normally.

-
-

[Done] Emergency maintenance on Web226

Posted in Downtime by

Filesystem on Web226 had gone read-only so we have booted it into rescue now to run a full filesystem check.

We will update this post as the check progresses.

[Aug 25 10:14] The check is done and the server is back and OK now.

-
-

[Fixed] Intermittent network routing issues affecting several servers

Posted in Downtime by

Several of our servers are currently experiencing intermittent network routing issues. Affected servers may include:
Web34, Web42, Web48, Web83, Web91, Web95, Web99, Web102, Web105, Web143, Web148, Web151, Web155, Web162, Web182, Web243

We’re working to resolve this issue and will update this post when we have more information.

2013-08-24 17:45 UTC: The problem has been identified as a rack switch that failed and has gone offline.

2013-08-24 18:05 UTC: The problem was an issue with an upstream network carrier and has been resolved.

-
-

[Fixed] Control panel and API offline

Posted in Downtime by

Our control panel and API server is currently offline. We’re working to restore service and will update this post as soon as we have more information.

2013-08-22 21:35 UTC – The control panel and API are back online.

-
-

[Done]Web313 down

Posted in Downtime by

Web313 has gone down and we are working urgently to bring it back online ASAP.

09:34:17 Aug 15 2013 UTC: Web313 is back and operating normally.

-
-

[Done]Emergency maintenance on Web329.

Posted in Downtime by

The file system on Web329 has gone read only. We’ve taken the machine offline and we are starting a file system check. We’ll update this post as we have more information.

2013-08-11 00:26 UTC: The file system check has finished and the server is now back online and operational.

-
-

[Done] Emergency maintenance on Web247

Posted in Downtime by

Web247 filesystem is showing inconsistencies. We’re taking the server down to run a fsck. We’ll update this post as the maintenance progresses.

2013-08-10 11:59 UTC: The fsck first pass is complete, second pass underway.
2013-08-10 12:34 UTC: Second pass at 60%.
2013-08-10 13:48 UTC: Second pass complete, scan 1C is underway.
2013-08-10 14:46 UTC: Fsck completed and the server is back at operational status.

-
-