[fixed] Web576 offline

Posted in Downtime by

2017-06-20 01:41 UTC: Web576 is currently offline due to a DDoS attack. We’re working to restore service and will update this post when more information is available.

2017-06-20 02:09 UTC: We’ve mitigated the attack upstream. The server appears to be stable, but the mitigation may affect UDP traffic. Please contact WebFaction support if you’re not able to access the server. We’ll continue to monitor.

2017-06-20 14:33 UTC: At the present time, Web576 is showing very high packet loss. We’re investigating at this time.

2017-06-20 17:48 UTC: Web576 is stable at this time. We’ll continue to monitor.

2017-06-21 14:37 UTC:  The attack has resumed, and we’re working to mitigate it.

2017-06-21 18:21 UTC: We’ve possibly identified the intended target of the attack and have isolated them from other customers. The server is stable at this time and we’ll continue to monitor.

2017-06-22 18:23 UTC: The server has been stable for the past 24 hours. We’ll continue to monitor.

2017-06-25 15:23 UTC: The server has been stable for over 72 hours, and there has been no further sign of any attack.

-
-

[Done] Scheduled maintenance on Sunday, June 25, 2017

Posted in Uncategorized by

We will be performing power maintenance on the following servers between 16:00 UTC June 25, 2017 and 04:00 UTC June 26, 2017:

wf-207-38-86-39
wf-207-38-86-40
wf-207-38-86-43
wf-207-38-86-47
wf-207-38-86-48
wf-207-38-86-49
wf-207-38-86-73
wf-207-38-86-75
wf-207-38-86-76
wf-207-38-86-78
wf-207-38-86-79
wf-207-38-86-85
wf-207-38-86-88
wf-207-38-86-90
wf-207-38-86-92
wf-207-38-86-93
wf-207-38-86-94
wf-207-38-86-98
wf-207-38-86-102
wf-207-38-86-103
wf-207-38-86-104
wf-207-38-86-105
wf-207-38-86-106
wf-207-38-86-107
wf-207-38-92-110
wf-207-38-86-178
wf-207-38-86-180
wf-207-38-86-191
wf-207-38-86-193
wf-207-38-86-194
wf-207-38-86-195
wf-207-38-86-196
wf-207-38-86-197
wf-207-38-86-198
wf-207-38-86-199
wf-207-38-93-249
wf-207-38-86-77
wf-207-38-86-135
wf-207-38-86-138
wf-207-38-86-140
wf-207-38-86-145
wf-207-38-86-147
wf-207-38-86-149
wf-207-38-86-150
wf-207-38-86-159
wf-207-38-86-160
wf-207-38-86-164
wf-207-38-86-165
wf-207-38-86-166
wf-207-38-86-167
wf-207-38-86-168
wf-207-38-86-170
wf-207-38-86-171
wf-207-38-86-175
web590
web591
web592
web593
web594
web595
web596
web597
web598
web600
web601
web602
web603

Connectivity to the servers will be interrupted for up to one hour during the maintenance window. We apologize in advance for the inconvenience. Should you have any questions about this maintenance, feel free to open a support ticket and our support team will be happy to help.

 

2017-06-25 22:26 UTC: The maintenance has begun.

2017-06-25 23:48 UTC: The maintenance had to be postponed.

-
-

[Rescheduled] Scheduled maintenance on Sunday, June 18, 2017

Posted in Downtime by

We will be performing a power maintenance on the following servers at 16:00 UTC June 18, 2017 between and 4:00 UTC June 19, 2017:

wf-207-38-86-39
wf-207-38-86-40
wf-207-38-86-43
wf-207-38-86-47
wf-207-38-86-48
wf-207-38-86-49
wf-207-38-86-73
wf-207-38-86-75
wf-207-38-86-76
wf-207-38-86-78
wf-207-38-86-79
wf-207-38-86-85
wf-207-38-86-88
wf-207-38-86-90
wf-207-38-86-92
wf-207-38-86-93
wf-207-38-86-94
wf-207-38-86-98
wf-207-38-86-102
wf-207-38-86-103
wf-207-38-86-104
wf-207-38-86-105
wf-207-38-86-106
wf-207-38-86-107
wf-207-38-92-110
wf-207-38-86-178
wf-207-38-86-180
wf-207-38-86-191
wf-207-38-86-193
wf-207-38-86-194
wf-207-38-86-195
wf-207-38-86-196
wf-207-38-86-197
wf-207-38-86-198
wf-207-38-86-199
wf-207-38-93-249
wf-207-38-86-77
wf-207-38-86-135
wf-207-38-86-138
wf-207-38-86-140
wf-207-38-86-145
wf-207-38-86-147
wf-207-38-86-149
wf-207-38-86-150
wf-207-38-86-159
wf-207-38-86-160
wf-207-38-86-164
wf-207-38-86-165
wf-207-38-86-166
wf-207-38-86-167
wf-207-38-86-168
wf-207-38-86-170
wf-207-38-86-171
wf-207-38-86-175
web590
web591
web592
web593
web594
web595
web596
web597
web598
web600
web601
web602
web603

Connectivity to the servers will be interrupted for up to one hour during the maintenance window. We apologize in advance for the inconvenience. Should you have any questions about this maintenance, feel free to open a support ticket and our support team will be happy to help.

2017-06-19 16:20 UTC:  The maintenance had to be rescheduled and we’ll post the new schedule when it is ready.

2017-06-19 22:47 UTC: Rescheduled to Sunday, June 25, 2017

-
-

[Monitoring] Control panel, API, and website down

Posted in Downtime by

2017-06-05 19:36 UTC: our control panel server is currently not responding. This also affects the control panel API and www.webfaction.com. We’re working to restore service and will update this post when more info is available.

2017-06-05 19:47 UTC: the panel, API, and www.webfaction.com are back online. We’re monitoring for further issues.

-
-

[Done] Scheduled maintenance on mailing list servers Saturday 10th June

Posted in Uncategorized by

Between 07:00 UTC and 11:00 UTC we will be performing maintenance on our mailing list servers. During this time list archives and administration will be unavailable and any incoming mail will be delayed until after the maintenance is complete.

07:00 UTC: The maintenance has begun

08:20 UTC: Maintenance has been completed. Mailing lists are back online

-
-

[fixed] MySQL down on Web231

Posted in Problems by

2017-05-26 00:59 UTC: MySQL on Web231 is currently down. We’re working to restore service and will update this post when more info is available.

2017-05-26 01:13 UTC: We’re dumping and reloading all MySQL databases on Web231. The reload is halfway complete.

2017-05-26 01:25 UTC: All MySQL databases on Web231 have been restored and the MySQL service is now operational.

-
-

[Fixed] Multiple US servers offline

Posted in Problems by

2017-05-17 22:01 UTC: Several US shared and cloud servers in our St. Louis data center are currently offline We’ll update this post as soon as more info is available.

2017-05-17 22:07 UTC: The following servers are affected:

  • Shared: web537, web538, web539, web540, web541, web542, web543, web544, web545, web546, web547, web548, web549, web550, web551, web552, web553, web554, web555, web556, web557, web559, web560, web563, web590, web591, web592, web593, web594, web595, web596, web597, web598
  • Cloud: wf-207-38-86-39, wf-207-38-86-40, wf-207-38-86-43, wf-207-38-86-47, wf-207-38-86-48, wf-207-38-86-49, wf-207-38-86-73, wf-207-38-86-75, wf-207-38-86-76, wf-207-38-86-78, wf-207-38-86-79, wf-207-38-86-85, wf-207-38-86-88, wf-207-38-86-90, wf-207-38-86-92, wf-207-38-86-93, wf-207-38-86-94, wf-207-38-86-98, wf-207-38-86-102, wf-207-38-86-103, wf-207-38-86-104, wf-207-38-86-105, wf-207-38-86-106, wf-207-38-86-107, wf-207-38-92-110, wf-207-38-86-110, wf-207-38-86-111, wf-207-38-86-112, wf-207-38-86-114, wf-207-38-86-115, wf-207-38-86-116, wf-207-38-86-117, wf-207-38-86-118, wf-207-38-86-120, wf-207-38-86-121, wf-207-38-86-122, wf-207-38-86-123, wf-207-38-86-124, wf-207-38-86-125, wf-207-38-86-126, wf-207-38-86-127, wf-207-38-86-128, wf-207-38-86-129, wf-207-38-86-130, wf-207-38-86-131, wf-207-38-86-132, wf-207-38-86-134, wf-207-38-86-137, wf-207-38-86-139, wf-207-38-86-77, wf-207-38-86-135, wf-207-38-86-138, wf-207-38-86-140, wf-207-38-86-145, wf-207-38-86-147, wf-207-38-86-149, wf-207-38-86-150, wf-207-38-86-159, wf-207-38-86-160, wf-207-38-86-164, wf-207-38-86-165, wf-207-38-86-166, wf-207-38-86-167, wf-207-38-86-168, wf-207-38-86-170, wf-207-38-86-171, wf-207-38-86-175, wf-207-38-86-178, wf-207-38-86-180, wf-207-38-86-191, wf-207-38-86-193, wf-207-38-86-194, wf-207-38-86-195, wf-207-38-86-196, wf-207-38-86-197, wf-207-38-86-198, wf-207-38-86-199, wf-207-38-92-219, wf-207-38-92-220, wf-207-38-92-221, wf-207-38-92-222, wf-207-38-92-223, wf-207-38-92-224

2017-05-17 23:09 UTC: We’re still working with the data center to identify and resolve this issue. We’ll update this post as soon as more information is available.

2017-05-17 23:20 UTC: web537, web538, web539, web540, web541, web542, web543, web544, web545, web546, web547, web548, web549, web550, web551, web552, web553, web554, web555, web556, web557, web559, web560, web563, web590, web591, web592, web593, web594, web595, web596, web597, web598 are all back online.

2017-05-17 23:40 UTC: All US servers are back online at this time. We will continue to monitor and update this post if there are further developments.

-
-

Network maintenance on May 30th – minimal downtime expected

Posted in Maintenance by

Our upstream provider will be performing emergency maintenance one of their routers on May 30th starting at 08:00 UTC.

A three hour maintenance window has been scheduled. The total expected downtime should not exceed three minutes.

The affected servers are:

Mailbox12, Mailbox13, Mailbox14, Mailbox15, Mailbox16

 

-
-

[fixed] web513 inaccessible

Posted in Downtime by

The server is currently not accessible. We are looking into this and will update as soon as we have further details.

2017-05-15 14:59 UTC: We are still investigating the issue.

2017-05-15 17:20 UTC: Our datacenter technicians are working directly to power on the server.

2017-05-15 20:17 UTC: The server is accessible again.

-
-

Network maintenance on May 22, 24, and 25 – minimal downtime expected

Posted in Maintenance by

Our upstream provider will be performing emergency maintenance on three separate routers on May 22, 24, and 25.

A three hour maintenance window has been scheduled for each router. The total expected downtime within each window should not exceed three minutes.

The maintenance windows and affected servers are:

  • 22-May-2017 22:00 UTC through 23-May-2017 01:00 UTC: dweb186, web209, web210, web211, web217, web218, web223, web224, web225, web303, web304, web305, web306, web314, web316, web317, web327, web331, web332, web333, web334, web357, web358, web359, web364, web383, web392, web395, web416, web421, web435, web448, web457, web459, web464, web480, web483, web488
  • 24-May-2017 05:00 UTC through 08:00 UTC: ns1, web22, web23, web39, web40, web69, web70, web72, web74, web75, web119, web122, web183, web200, web213, web228, web230, web231, web233, web234, web235, web242, web248, web249, web338, web391, web397, web413, web453, web484, web487
  • 25-May-2017 05:00 UTC through 08:00 UTC: mail, mx7, mx8, web31, web42, web48, web155, web174, web178, web186, web226, web237, web241, web351, web354, web355, web356, web363, web365, web366, web367, web372, web378, web382, web387, web408, web450, web492, webmail
-
-