[Fixed] Delays in processing payments

Posted in Problems by

We are currently experiencing a temporary issue with our payment pre-processor.

Due to an issue on their side we are not receiving payment notifications in a timely manner, thus any payments, while received, are not instantly recorded to our control panel’s Payment Transaction History page.

This issue started today at 05:16 UTC and after communicating with them they said that it should be fixed by 14:30 UTC.

When they do resolve this issue we’ll go through all the payments made today, double check the transactions and add them to our panel.

If you did make a payment during that time but it is missing, feel free to raise a ticket and we’ll look into this.

2018-02-19 15:30 UTC: Our payment pre-processor have gotten back to us, letting us know that everything is fixed on their side and we are seeing payments going through successfully in the last hour. We are not double checking all transactions that happened during the window and adding any missing one to our panel.

2018-02-19 17:00 UTC: All transactions should be visible in our control panel now.

-
-

[Fixed] Web530 inaccessible

Posted in Downtime by

Web530 is currently not accessible. We are working to restore services as soon as possible.

2018-02-07 16:51 UTC: We are still working with our data center to bring the machine back online as soon as possible.

2018-02-07 20:04 UTC: The servers is back online.

2018-02-08 07:39 UTC: The server is experiencing network issues again and is inaccessible. We’re looking into this issue and will update this post as soon as we have more info.

2018-02-08 10:12 UTC: The server is up and running and looks stable. We’ll be monitoring it for a while before marking this issue as resolved. Please let us know if you notice any further issues from your end.

2018-02-08 10:49 UTC: Some customers have reported issues accessing the server via IPv4. We are looking into this. Customers using IPv6 should be able to access the server.

2018-02-08 14:30 UTC: After monitoring the server for some time, all services look stable so we’ll mark this issue as fixed.

-
-

[fixed] Web571 inaccessible

Posted in Downtime by

Web571 is currently not accessible. We are looking into the issue and will resolve as quickly as possible.

2018-01-07 08:45 UTC: The issue has been resolved with a reboot and the server should be responding to requests.

-
-

[Fixed] Web623 inaccessible

Posted in Downtime by

Web623 didn’t came back after our recent reboot to patch the ‘Meltdown’ and ‘Spectre’ security vulnerabilites.

We are working with our datacenter technicians to resolve this issue and get the server up and running again as quickly as possible.

2018-01-06 07:30 UTC: The server is back online.

-
-

[Fixed] Control panel, API, and website down

Posted in Uncategorized by

Our control panel server is currently not responding. This also affects the control panel API and www.webfaction.com. We’re working to restore service and will update this post when more info is available.

2018-01-06 07:04 UTC: The Control panel, API, and website are back online.

-
-

CPU Vulnerabilities: Meltdown & Spectre

Posted in Uncategorized by

Researchers disclosed a security vulnerability affecting side-channel analysis of speculative execution on modern computer processors (CVE-2017-5715, CVE-2017-5753, and CVE-2017-5754). This issue affects home computers, phones, tablets and servers. As such, nearly everyone who uses such a device is affected. Our team is working on mitigating this, but we expect to have to reboot every machine shortly in order to protect your data.

We are looking to reboot every machine in our fleet at non-business hours over the next few days, we will soon contact you to let you know of the emergency maintenance window we have allocated for your machine.

We will keep you updated on this serious issue and give you as much notice for any emergency reboots as we possibly can.

References:
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5715
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5753
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5754

We will keep this thread updated and apologize in advance for the inconvenience. Should you have any questions about this maintenance, feel free to open a support ticket and our team will be happy to help.

2018-01-05 17:28 UTC: All Singapore shared servers have been updated and rebooted.

2018-01-05 22:44 UTC: All EU shared and dedicated servers have been updated and rebooted.

2018-01-06 13:56 UTC: All USA shared and dedicated servers have been updated and rebooted.

-
-

[fixed] web544 inaccessible

Posted in Downtime by

The server is currently not accessible. We are looking into the issue and will resolve as quickly as possible.

2017-12-27 13:36 UTC: The issue has been resolved with a reboot and the server is back online.

-
-

[fixed] Server connectivity issues

Posted in Problems by

2017-12-18 10:24 UTC: We’re currently addressing connection issues affecting customers on some of our servers. Please watch this area for updates and progress on the incident. We appreciate your patience.

2017-12-18 13:09 UTC: The problem was a bug in our web application firewall setup. It’s been corrected and connectivity has been restored as of about 30 minutes ago.

-
-

[Fixed] Partial access to mail service disruption

Posted in Problems by

2017-11-22 10:58 UTC An issue with one of our mail servers is causing some of our customres to access their mailboxes. We’re working to restore service and will update this post when more information is available.

2017-11-22 13:02 UTC Access to the faulty mail server is restored. A network issue between some nodes of our mailing infrastructure was causing login attempts to timeout. Please raise a ticket if you are still seeing an issue.

-
-

[fixed] Delayed payment transactions

Posted in Problems by

2017-11-10 19:12 UTC: For the past few hours, we have been experiencing delays between the time a customer makes a payment and the time that the payment is posted to the customer’s account. The delay is occurring with our billing processor. If you make a payment and do not see it posted to your account immediately, please do not make an additional payment. The original payment should be posted within a few hours. We’ll update this post when the issue is resolved. Please contact WebFaction support if you have any questions or concerns.

2017-11-10 21:19 UTC: The issue has been resolved.

-
-