[Fixed] Mailbox17 inaccessible

Posted in Uncategorized by

The server is currently not accessible. We are looking into the issue and will resolve as quickly as possible.

2018-06-30 22:34 UTC: Mailbox17 is back online.

-
-

[Fixed] Router issues in Singapore

Posted in Uncategorized by

Mon Jun 18 00:02:08 UTC 2018: Some servers in our Singapore datacenters are not accessible. We’re working with them to resolve the situation as soon as possible. Please stand by.

Mon Jun 18 00:27:18 UTC 2018: There is an issue with a router that serves  some servers in Singapore:

Web321, Web322, Web323, Web339, Web360, Web375, Web379, Web409, Web424, Web429, Web442, Web463, Web474, Web486, Web505, Web508, Web517

Our techs are looking at resolving this as soon as possible. Please stand by for updates.

Mon Jun 18 04:17:10 UTC 2018: Some servers are starting to become accessible. We’re still investigating the problem and will make another post as soon as we are sure services have been restored. We thank you all for your patience.

Mon Jun 18 04:58:07 UTC 2018: The issue has been mitigated. Services should be restored. We are still in talks with the datacenter in regards to the cause and ways to prevent future occurrences. You may need to manually start any apps that are not currently running. If you still encounter issues please feel free to open a ticket for us to investigate.

Mon Jun 18 21:12:50 UTC 2018: This issue has been marked as resolved.

-
-

[Fixed] Mailbox26 inaccessible

Posted in Uncategorized by

Mailbox26 is currently not accessible. We are looking into the issue and will resolve as quickly as possible.

2018-05-28 03:20 UTC: Mailbox26  is back online and all services are operational.

-
-

[Fixed] Mailbox17 inaccessible

Posted in Uncategorized by

The server is currently not accessible. We are looking into the issue and will resolve as quickly as possible.

2018-05-22 17:08 UTC: We have identified the problem and are working as fast as possible to bring the machine back online.

2018-05-22 18:36 UTC: We have done a chassis swap and importing the data.

2018-05-22 19:56 UTC: The server is back online.

-
-

[Done] Scheduled maintenance on our ticketing system on Sunday, April the 22nd

Posted in Uncategorized by

On Sunday, 22nd of April, between 11:00 and 12:00 UTC, we will be migrating our ticketing system to new hardware.

During this time our ticketing system will be unavailable. You will not be able to raise new tickets, nor reply or review old ones.

We expect total downtime to be less than 30 minutes.

2018-04-22 12:10 UTC: We are starting the maintenance.

2018-04-22 12:42 UTC: The maintenance has been successfully completed.

-
-

[Done] Emergency Europe network maintenance on March 27-28, 2018

Posted in Uncategorized by

On March 27-28, 2018 between 22:30 UTC and 1:00 UTC we will be performing
emergency network maintenance on our Europe based servers.

Connectivity to the servers will be interrupted for up to 30 minutes during the
maintenance window.

2018-03-28 03:12 UTC: The maintenance has been successfully completed.

-
-

Unexpected downtime for Web463.

Posted in Uncategorized by

We had a brief period of downtime while we restarted the hardware. Web463 should be back online.

-
-

[Done] Emergency maintenance on Web572 on Sunday, March 4, 2018

Posted in Uncategorized by

Web572 will be taken down to replace a failing memory module Sunday, March 4, 2018 between 00:00 and 01:00. The expected downtime should be 15 minutes or less.

2018-03-04 00:58 UTC: The maintenance is complete and the server is back online.

-
-

[Fixed] Control panel, API, and website down

Posted in Uncategorized by

Our control panel server is currently not responding. This also affects the control panel API and www.webfaction.com. We’re working to restore service and will update this post when more info is available.

2018-01-06 07:04 UTC: The Control panel, API, and website are back online.

-
-

CPU Vulnerabilities: Meltdown & Spectre

Posted in Uncategorized by

Researchers disclosed a security vulnerability affecting side-channel analysis of speculative execution on modern computer processors (CVE-2017-5715, CVE-2017-5753, and CVE-2017-5754). This issue affects home computers, phones, tablets and servers. As such, nearly everyone who uses such a device is affected. Our team is working on mitigating this, but we expect to have to reboot every machine shortly in order to protect your data.

We are looking to reboot every machine in our fleet at non-business hours over the next few days, we will soon contact you to let you know of the emergency maintenance window we have allocated for your machine.

We will keep you updated on this serious issue and give you as much notice for any emergency reboots as we possibly can.

References:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5715
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5753
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5754

We will keep this thread updated and apologize in advance for the inconvenience. Should you have any questions about this maintenance, feel free to open a support ticket and our team will be happy to help.

2018-01-05 17:28 UTC: All Singapore shared servers have been updated and rebooted.

2018-01-05 22:44 UTC: All EU shared and dedicated servers have been updated and rebooted.

2018-01-06 13:56 UTC: All USA shared and dedicated servers have been updated and rebooted.

-
-