[Fixed] Webmail issues

Posted in Problems by

There is currently an issue with our webmail interfaces at https://webmail.webfaction.com and https://squirrelmail.webfaction.com/ which is resulting in sporadic 500 Internal Server Errors for some email users.

We are currently looking into the problem and hope to have normal service restored soon.

If you need to access your mailbox, please consider https://docs.webfaction.com/user-guide/email.html#configuring-an-email-client until this issue is resolved.

We will update this entry as soon as we have more information.

Update 2018-12-12 14:48UTC: We applied a patch to our webmail instance a few days ago and haven’t seen an issue since, so we are going to go ahead and mark this as fixed. Feel free to raise a ticket if you are still experiencing issues.

-
-

Changes in our payment system

Posted in Service change by

We recently made some changes to the way customers should make their payments or add their credit cards for monthly recurring charges. The major customer facing change was that Paypal’s UI would be used instead of our control panel.

That decision was made to further secure our systems, where we wouldn’t capture customer’s data at all in our platform, not even temporary, before transmitting them to Paypal’s API to be processed.

Unfortunately, this caused some issues with payments, for example customers were being asked to create a Paypal account, even though one shouldn’t be needed, or being charged extra fees from Paypal’s side.

Those were issues we haven’t seen during our testing period or for the last few years that Paypal has been our default gateway. Taking them into account, we’ve reverted our changes and you should be able to make payments or add your payment sources using our control panel directly again.

We will re-deploy the changes as soon as further testing is done, with an ETA of mid-January.

We will update this post as soon as this is done.

-
-

[Done] Emergency maintenance on our ticketing system

Posted in Downtime by

Our ticketing system is down for some emergency maintenance.

During this time our ticketing system will be unavailable. You will not be able to raise new tickets, nor reply or review old ones.

We expect total downtime to be about an hour.

2018-11-30 18:40 UTC: The maintenance is complete and the ticket system is back online.

-
-

[Fixed] Intermittent mail issues

Posted in Problems by

We are seeing issues with a few customers being unable to access their mailboxes, with their attempts timing out as the authentication process takes a long time.

The issues are sporadic and only affect a small number of users are affected, but we are looking into this as our top priority.

We’ll update this status post as soon as we have more feedback.

UPDATE: The underlying issues have been fixed.

-
-

[Fixed] Payment system trouble

Posted in Problems by

2018-07-18 18:15 UTC: At this time, we’re unable to process manual payments and cannot set up new payment sources for automatic monthly billing. We’re working to resolve the issue and will update this post when more info is available.

2018-07-18 19:15 UTC: We’ve track this issue down to our payment pre-processor. We are working with them to resolve this.

2018-07-19 15:56 UTC: The issues with our payment pre-processor have been resolved.

-
-

Deprecating TLS 1.0 and TLS_RSA_WITH_3DES_EDE_CBC_SHA ciphers

Posted in Service change by

In order to meet the latest PCI Data Security Standards (PCI DSS) we will be dropping support for TLS 1.0 and the TLS_RSA_WITH_3DES_EDE_CBC_SHA ciphers on all servers in our infrastructure.

The deadline set from the PCI Security Standards Council is the 30th of June, 2018. You can read more about it here.

After this update, customers using legacy browsers and user agents will be unable to access any SSL secured website hosted in our platform.

These user agents include but not limited to:

  1. any browser running on Windows XP
  2. any browser running on Android 4.4 or earlier
  3. Internet Explorer 10 and below
  4. Firefox 23 and below
  5. Chrome 21 and below
  6. any clients using Java 1.7 or earlier
  7. curl and wget command line clients using OpenSSL versions priors to v1.0.1

If your browser and operating system is up to date, then you should probably be OK.

If you are unsure whether this change will affect you or not you can visit this test page.

If you have any questions regarding this security change, please raise a ticket and our support team will gladly answer your questions.

-
-

[Fixed] MySQL service down on Web536

Posted in Downtime by

2018-06-01 14:35 UTC: The shared MySQL database service on Web536 is currently down. We’re working to restore service at this time. We’ll update this post when more information is available.

2018-06-01 17:59 UTC: All databases have been restored.

-
-

Regarding GLIBC_PRIVATE errors in your python application

Posted in Problems by

A recent system update to the core glibc package seems to have caused a small number of python apps hosted on our servers to stop working, showing the following in their error logs:

ImportError: /lib64/libresolv.so.2: symbol __h_errno, version
GLIBC_PRIVATE not defined in file libc.so.6 with link time reference

The Apache stack needs to fully stop for this to be fixed. You can do that by starting an ssh session, traversing to the directory hosting your app, and using the `stop` and `start` scripts:

cd ~/webapps/your_app
./apache2/bin/stop
./apache2/bin/start

If you are having any trouble or need further assistance with this, please raise aticket and our support team will gladly help you with.

-
-

[Done] SOCKS proxy for outgoing connections disabled

Posted in Service change by

Starting 30th of March 2018, all outgoing SOCKS proxies from servers will be disabled.

You will still be able to connect to the server your account is on via SSH and create a tunnel to our MySQL and PostgreSQL databases or your long running application like the Apache or Nginx instance your Django or Rails application runs.

The most important reason for this is that an SSH tunnel or SOCKS connection can be used to mask malicious traffic and attacks to other servers, within or outside of the WebFaction network, which could lead to the server being blacklisted or even having its IP address null rooted by the datacenter, causing a disruption to all the clients hosted there.

The majority of our users won’t be affected by the new policy but if you have any questions or concerns regarding this change, please open a support ticket via our control panel to let us know.

2018-03-30 10:02 UTC: This has been deployed everywhere.

-
-

[Fixed] Several US servers inaccessible

Posted in Downtime by

We’ve just got an alert about the following US based servers being inaccessible.

We’re investigating at this time and will update this post when more information is available.

The affected shared servers are: web500, web501, web518, web521, web524, web590, web591, web592, web594, web595, web596, web597, web598, web599, web600, web601, web602, web603, web608, web609, web610, web611, web612, web613, web614, web615, web616, web617, web618, web619, web620, web621, web622, web623, web624, web625, web626, web627, web628, web629, web630, web631, web632, web634.

The affected cloud instances are: wf-148-72-160-10, wf-148-72-160-103, wf-148-72-160-104, wf-148-72-160-105, wf-148-72-160-106, wf-148-72-160-107, wf-148-72-160-108, wf-148-72-160-109, wf-148-72-160-110, wf-148-72-160-111, wf-148-72-160-114, wf-148-72-160-115, wf-148-72-160-117, wf-148-72-160-118, wf-148-72-160-119, wf-148-72-160-12, wf-148-72-160-120, wf-148-72-160-13, wf-148-72-160-14, wf-148-72-160-16, wf-148-72-160-17, wf-148-72-160-5, wf-148-72-160-6, wf-148-72-160-7, wf-148-72-160-8, wf-148-72-160-9, wf-207-38-86-102, wf-207-38-86-125, wf-207-38-86-135, wf-207-38-86-138, wf-207-38-86-140, wf-207-38-86-145, wf-207-38-86-147, wf-207-38-86-149, wf-207-38-86-150, wf-207-38-86-159, wf-207-38-86-160, wf-207-38-86-164, wf-207-38-86-165, wf-207-38-86-166, wf-207-38-86-167, wf-207-38-86-168, wf-207-38-86-170, wf-207-38-86-171, wf-207-38-86-175, wf-207-38-86-187, wf-207-38-86-192, wf-207-38-86-194, wf-207-38-86-67, wf-207-38-86-68, wf-207-38-86-77, wf-207-38-92-106, wf-207-38-92-115, wf-207-38-92-122, wf-207-38-92-123, wf-207-38-92-124, wf-207-38-92-125, wf-207-38-92-135, wf-207-38-92-14, wf-207-38-92-144, wf-207-38-92-149, wf-207-38-92-176, wf-207-38-92-18, wf-207-38-92-182, wf-207-38-92-183, wf-207-38-92-184, wf-207-38-92-215, wf-207-38-92-244, wf-207-38-92-5, wf-207-38-92-55, wf-207-38-92-6, wf-207-38-92-61, wf-207-38-92-7, wf-207-38-92-93, wf-207-38-92-94, wf-207-38-92-95, wf-207-38-93-134, wf-207-38-93-174, wf-207-38-93-187, wf-207-38-93-216, wf-207-38-93-252, wf-207-38-93-253, wf-207-38-93-254, wf-207-38-93-35, wf-207-38-93-36, wf-207-38-93-93, wf-207-38-94-108, wf-207-38-94-111, wf-207-38-94-140, wf-207-38-94-159, wf-207-38-94-181, wf-207-38-94-206, wf-207-38-94-21, wf-207-38-94-22, wf-207-38-94-224, wf-207-38-94-230, wf-207-38-94-235, wf-207-38-94-241, wf-207-38-94-251, wf-207-38-94-69, wf-207-38-94-70, wf-207-38-94-78, wf-207-38-94-91

2018-03-03 16:19 UTC We’ve identified the cause as a power outage. Servers are beginning to come back online now. We’ll update this post if anything changes.

2018-03-03 17:14 UTC The power outage was caused by a cooling failure. This is being worked on now. We’ll update this post when more information becomes available.

2018-03-03 18:50 UTC The issue is resolved and all servers have been restarted. We’ll continue to monitor. If you see any issues, please open a support ticket.

-
-