[Fixed] Several cloud servers offline

Posted in Downtime by

2017-10-06 17:13 UTC: Several of our cloud servers are currently offline due to an unknown issue. We’re investigating at this time and will update this post when more information is available. The affected servers are:

  • wf-173-255-197-72
  • wf-173-255-203-84
  • wf-198-58-100-37
  • wf-198-58-120-80
  • wf-198-58-98-119
  • wf-207-38-86-187
  • wf-207-38-86-192
  • wf-207-38-92-115
  • wf-207-38-92-14
  • wf-207-38-92-176
  • wf-207-38-92-182
  • wf-207-38-92-183
  • wf-207-38-92-184
  • wf-207-38-92-215
  • wf-207-38-92-244
  • wf-207-38-92-55
  • wf-207-38-92-93
  • wf-207-38-92-94
  • wf-207-38-93-134
  • wf-207-38-93-174
  • wf-207-38-93-187
  • wf-207-38-93-252
  • wf-207-38-93-253
  • wf-207-38-93-254
  • wf-207-38-94-111
  • wf-207-38-94-206
  • wf-207-38-94-230
  • wf-207-38-94-235
  • wf-207-38-94-251
  • wf-23-239-25-54
  • wf-45-33-113-161
  • wf-45-33-14-240
  • wf-45-33-18-166
  • wf-45-33-24-216
  • wf-45-33-7-73
  • wf-45-56-122-41
  • wf-45-56-123-50
  • wf-45-56-77-37
  • wf-45-79-0-162
  • wf-45-79-1-12
  • wf-45-79-10-158
  • wf-45-79-4-136
  • wf-45-79-7-111
  • wf-66-228-50-12
  • wf-72-14-182-215
  • wf-96-126-117-174

2017-10-06 17:32 UTC: Several cloud servers are back online after a reboot of the host hardware. The servers down at this time are:

  • wf-173-255-197-72
  • wf-173-255-203-84
  • wf-198-58-100-37
  • wf-198-58-120-80
  • wf-198-58-98-119
  • wf-23-239-25-54
  • wf-45-33-113-161
  • wf-45-33-14-240
  • wf-45-33-18-166
  • wf-45-33-24-216
  • wf-45-33-7-73
  • wf-45-56-122-41
  • wf-45-56-123-50
  • wf-45-56-77-37
  • wf-45-79-0-162
  • wf-45-79-1-12
  • wf-45-79-10-158
  • wf-45-79-4-136
  • wf-45-79-7-111
  • wf-66-228-50-12
  • wf-72-14-182-215
  • wf-96-126-117-174

2017-10-06 18:32 UTC: All of the affected servers are back online.

2017-10-11 12:59 UTC: We are taking the machine down for emergency maintenance to do a chassis swap.

2017-10-11 15:48 UTC: The server is back online.

-
-

[Done] Scheduled maintenance on nameservers Monday 9th, October

Posted in Maintenance by

On Monday, 9th October at 9am UTC we will be migrating our nameservers to new hardware.

If you are using the IP address of the servers please be aware that the IP address will change from:

ns1: 108.168.213.81
ns2: 178.79.142.142
ns3: 191.101.157.188
ns4: 173.230.141.144

To:

ns1: 185.20.51.42
ns2: 103.44.220.74
ns3: 62.138.130.11
ns4: 148.72.160.4

If you are using the domain name (e.g. ns1.webfaction.com) then you do not need to do anything.

The old nameservers will be kept online till October 30, after which they will be decommissioned and no longer available.

No service interruption is expected. We will update this post as maintenance progresses.

2017-10-09 09:00 UTC: Maintenance has begun
2017-10-09 09:10 UTC: Maintenance is complete

-
-

[fixed] web607 inaccessible

Posted in Downtime by

The server is currently not responding. We are investigating the issue and will post updates as soon as we have further details.

2017-09-07 18:35 UTC: There is an issue with the RAID controller. We are working to have this resolved as quickly as possible.

2017-09-07 19:15 UTC: The server is currently back online. However, we are still investigating the issue further.

2017-09-07 20:57 UTC: There have been no further issues since the server reboot.

-
-

[Done] Control Panel and API maintenance Monday, September 4th 2017

Posted in Uncategorized by

On Monday, 4 September 2017  between 11:00 and 12:00 UTC, we will perform maintenance on our Control Panel.

During this time, the Control Panel and API will be temporarily unavailable. We apologize in advance for the inconvenience. Should you have any questions about this maintenance, feel free to open a support ticket and our support team will be happy to help.

2017-09-04 12:01 UTC: We are starting the maintenance.

2017-09-04 13:49 UTC: The maintenance is now over. Thank you for your patience, and if there are any issues please feel free to contact support by opening a ticket.

-
-

[Fixed] Web573 inaccessible

Posted in Uncategorized by

Web573 did not come back after a quick reboot. We are looking into this and will resolve as quickly as possible.

2017-08-24 10:48 UTC: The server is up and running and all services look stable. Thank you for your patience, and if there are any issues please feel free to contact support by opening a ticket.

-
-

[Done] Emergency maintenance on Web564

Posted in Uncategorized by

We are taking down Web564 to replace a failed memory module. Downtime should be 15 minutes or less.

2017-08-20 16:13 UTC: Shutting down the machine.

2017-08-20 16:25 UTC: The memory has been replaced and server is back online.

-
-

[fixed] web518 inaccessible

Posted in Uncategorized by

Web518 did not come back after a quick reboot. We are looking into this and will resolve as quickly as possible.

2017-08-20 06:02 UTC: All services should be back to normal. Thank you for your patience, and if there are any issues please feel free to contact support by opening a ticket.

 

-
-

[fixed] Web512 and Web534 inaccessible

Posted in Uncategorized by

The servers are currently not responding. We are looking into the issue and will provide an update as soon as possible.

2017-08-16 03:26 UTC: Access to Web512 has been restored. We’re still monitoring.

2017-08-16 03:28 UTC: Access to Web534 has been restored. Please let us know if issues persist by opening a support ticket.

-
-

[fixed] web527 inaccessible

Posted in Downtime by

Web527 did not come back after a reboot. We are looking into this and will resolve as quickly as possible.

2017-08-15 13:48 UTC: The server is back online. The issue was with the installed kernel. It was changed and the server rebooted.

-
-

[fixed] IMAP/POP authentication issues

Posted in Problems by

2017-08-12 04:27 UTC: Users with email mailboxes on Mailbox8 and Mailbox9 servers may encounter intermittent authentication failures. Please stand by for further updates.

2017-08-16 22:54 UTC: We’ve determined that this issue is related to an ongoing migration of mailboxes to new hardware. We’re continuing to troubleshoot.

2017-08-28 22:55 UTC: A permission problem related to the ongoing mailbox migrations is currently causing IMAP/POP login failure. We’re fixing the permissions at this time and will update this post when complete.

2017-08-28 23:42 UTC: The permission problem has been resolved, and IMAP/POP authentication should be working normally at this time. We’ll continue to monitor.

2017-08-30 21:53 UTC: We’re still seeing intermittent IMAP/POP authentication failures related to the ongoing mailbox migrations. This also affects webmail logins. Any failures should be temporary – if they don’t seem resolved within a few minutes, then please contact WebFaction support for further assistance.

2017-09-07 18:45 UTC: We’ve updated our migration process to minimize its impact on the IMAP/POP service, and are no longer seeing intermittent IMAP/POP authentication issues.

-
-