[Done] Upgrading psycopg2

Posted in Scheduled downtime by

We are going to be upgrading the server-wide install of psycopg2, to begin on Monday, July 09, 2018 around 03:00 UTC. This will be a minor version upgrade that may affect running Python applications which interface with PostgreSQL databases, most notably Django, and issues are usually resolved by restarting the application. If you need any assistance, simply open a support ticket and we will be happy to help.

If you are running any Python+PostgreSQL applications, please be aware of this upgrade and inspect the behavior of your app following the roll-out. You can check the version of psycopg2 by printing psycopg2.__version__ from within a python environment.

Note that the latest version of psycopg2 is not compatible with python2 below 2.6, nor with python3 below 3.2. We will not be upgrading psycopg2 for legacy versions in order to avoid breaking operation of applications that still use them.

-
-

GDPR Updates

Posted in General by

The General Data Protection Regulation is taking effect soon, and we’ll be deploying related updates to various pieces of our infrastructure, including the Control Panel and Community sites, over the next 24 hours. We expect downtime of affected services to last less than an hour, and we’ll mark this post complete as soon as all changes have been fully deployed.

-
-

[Fixed] Widespread network issues

Posted in Downtime by

We are currently seeing widespread connectivity issues across our hosting platform, and this is affecting both webserver-to-webserver communication as well as external website accessibility for hosted sites. We’re working on restoring normal network functionality as quickly as possible.

Update Thu Apr 19 07:20 UTC 2018: At this point we believe that we’ve fixed all network issues and that full service has been restored. If you’re still seeing network problems at this time, please let our Support team know.

Update Thu Apr 19 10:11 UTC 2018: We haven’t received any more reports of issues and everything looks good, so we’re marking this Fixed.

-
-

[Done] Emergency maintenance on Mailbox10, April 16, 2017

Posted in Downtime by

Mailbox10 will be taken down for an HDD swap and possibly RAID controller replacement,
Sunday April 16th, 2017 between 1:00 UTC and 5:00 UTC.

Sun Apr 16 01:15:32 UTC 2017: The server has been taken offline for maintenance

Sun Apr 16 02:15:32 UTC 2017: The failing drive has been replaced and the RAID is rebuilding (currently 3%).

Sun Apr 16 02:45:34 UTC 2017: The rebuild has failed, which indicates a faulty RAID controller, which is now being replaced.

Sun Apr 16 03:29:52 UTC 2017: The RAID controller has been replaced; now doing build/verify and rebuild.

Sun Apr 16 06:22:50 UTC 2017: The RAID controller initial build/verify is done and new HDD is rebuilding (currently 15%).

Sun Apr 16 07:00:51 UTC 2017: HDD is rebuilding (currently 35%).

Sun Apr 16 08:22:08 UTC 2017: HDD is rebuilding (currently 80%).

Sun Apr 16 08:52:54 UTC 2017: The rebuild has completed and the server is back online.

-
-

[Completed] Emergency maintenance on web474, April 02, 2017

Posted in Downtime by

Web474 will be taken down for hard disk replacement on Sunday April 2nd, 2017 between 04:00 UTC and 08:00 UTC.

We will update this post as maintenance progresses.

Sun Apr  2 04:18:02 UTC 2017: The server has been taken offline for maintenance.

Sun Apr  2 05:39:44 UTC 2017: The first of two hard disks has been replaced and the RAID is rebuilding (25% currently)

Sun Apr  2 07:34:41 UTC 2017: The RAID has finished rebuilding after the first disk replacement, and the second disk is now being replaced.

Sun Apr  2 07:45:17 UTC 2017: The server is back online. The second disk was replaced and the RAID is rebuilding, which will decrease performance until it finishes.

Sun Apr  2 12:14:17 UTC 2017: All drives in the RAID have finished rebuilding. Maintenance is now complete.

-
-

[Fixed] Web366 unreachable

Posted in Uncategorized by

web366.webfaction.com is unreachable by the external network, and initial evidence suggests possible router failure. Our system administrators are looking into it.

[2016-05-23] The router issue has been addressed and the server is back online.

-
-

[Fixed] Web409 read-only filesystem

Posted in Downtime by

Web409 is down with a read-only filesystem. We are currently looking into the issue and will resolve it as quickly as possible.

[Sun Oct  4 02:00:34 UTC 2015] The machine has been taken offline for a filesystem check

[Sun Oct  4 02:45:42 UTC 2015] The machine is back online

-
-

[Fixed]DDoS on Web371

Posted in Downtime by

Web371 server is experiencing a massive DDoS attack which our DDoS protection system is having trouble mitigating.

We’re doing everything we can to restore service at this time, and will update this post as progress is made.

Update 10:08 am: Our DDoS protection is now doing a much better job at mitigating the attack. All sites are now up but performance is slightly degraded.

Update 1pm: The attack has stopped and the server is back to normal

-
-

[Done]Emergency maintenance on web338

Posted in Downtime by

Web338 has shown poor performance and stopped responding, so we have taken the server offline for a filesystem check and hardware inspection. We will update this post as progress completes.

2013-04-28 05:09 UTC: The fsck has finished and the server is back online, but there are still some issues which need to be addressed, so the server will be rebooted again once more soon.

2013-04-28 05:44 UTC: We are now rebooting the machine after some changes to bring it fully back online.

2013-04-28 06:49 UTC: The server is now online and functioning normally.

-
-