Due to recently-announced exceptionally serious vulnerabilities in all versions of PHP, I have decided that my only safe option is to shut down all Servology web hosting which makes use of PHP for the time being.
I have not taken this decision lightly, and I do not know how soon I will be able to turn PHP-based web hosting back on again.
I will update this page as and when I have more information.
Please accept my apologies for this outage. All indications at the present are that this is one of the most serious security vulnerabilities in history and could result in the complete take-over of any vulnerable server. I would prefer to be safe than sorry.
For more information see the Wikipedia page on PHP2Shell and references linked from that Wikipedia page.
Please note: At the moment, almost all Servology web hosting is offline as I have not been able to identify quickly enough exactly which sites use PHP and which do not, and to separate the configurations for vulnerable PHP-based sites from unaffected sites. Further updates as information is released.
[15:20] I am reading up on the mechanisms, impact and possible mitigation strategies for the vulnerabilities.
[16:15] It looks like the sury PHP repository may have fixed versions of older PHP versions, so it may be possible to patch a number of systems despite them being older than supported, and bring sites back online.
[18:00] The proxy frontend has been mostly unaffected, so hosted redirects for several customers have not had an outage. The same server hosts a few sites which are now patched and back online. This status site has also been patched and is now back online. There are several more web hosts to update and/or mitigate, some of which may not be possible to fix. In case they cannot be fixed it may be necessary to build a new web host and migrate sites across, which may take some considerable time.
[23:50] Thank you all for your patience today with the near-total outage of Servology web hosting.
As you will undoubtedly have seen, there has been a disclosure, with less than 24 hours’ warning, of a vulnerability in the PHP web programming language, which allows an attacker to run code of their choosing on any server running an unpatched version of PHP and in combination with other exploits potentially take full control of the server.
https://www.bbc.co.uk/news/articles/cmly0lg3w5x2o
https://en.wikipedia.org/wiki/PHP2Shell
There are multiple methods of mitigating the problems, but they are both quite complex to implement and incomplete, and attackers will likely work around them to break into servers anyway. Other than isolating servers from the internet, the only complete protection is to update the PHP interpreter to a patched version.
The problem here is that most of Servology’s web hosting is inherited from older businesses and I haven’t yet completed a build of a replacement web service to migrate all the many existing sites over to. I find myself in the situation that almost every site is on servers which are too old to accept the published PHP patches, and also too old or complex to be straightforwardly updated to software versions that can be patched.
There are two main potential ways forward:
1. Find a way to update the existing virtual servers hosting the web services to a sufficiently recent version that they are then patchable.
2. Build a new web service and migrate customer sites to it one at a time.
There is the option of using mitigations, either local configuration changes or external web firewall services or both, to attempt to mitigate the problem while working on option 1 or 2, but I feel this is likely to take as long as fixing the problem properly and be more risky (both because attackers might bypass the mitigations anyway, and because it would involve a lot of bulk reconfiguration which might also break things) and be considerably more work in total.
I have spent much of this afternoon examining possible ways to update the servers, as I have at various times in the last few years, but I almost immediately run into multiple problems, and I am wary of devoting more time to this as don’t even know whether there is a route through the obstacles which will actually achieve the goal of upgrading the servers enough to accept the patches. At the moment I am therefore minded to go with option 2 above.
I have previously spent time working on a new Servology web service which I believe is fairly close to being usable, so my plan for tomorrow is to resume work on that with the goal of getting it ready to accept sites hopefully by the end of the day tomorrow. It will probably be considerably less easy to work with and lack many of the features I had hoped for but my immediate goal will be just to make it functional and secure. Once it is working and I have validated it by migrating some test sites to it, I would then start migrating customer sites to the new platform and thus bringing them back online.
I am sorry that this will mean considerably more downtime for your web sites, but I feel this is the best way forward that I can manage. As one individual I unfortunately do not have the resources to get this done as quickly as I would like.
Finally, I would like to introduce you to my status web site, where I will endeavour to post timely updates for this and any future problems at Servology.
This site is hosted entirely separately from Servology with the aim that it should not be affected by outages on the Servology network and that I can still post information to it during such outages to keep you informed. The status site has already had the PHP updates applied to it. I will post further updates there.
Thank you for your understanding,