Web Services on the Palamedes server temporarily disabled

TrackBacks (0) Comments (3)

Notification Type

Emergency Maintenance

Date

December 0, 0000 12:00 AM

Service Affecting

Yes

Message

Our engineers here have had to bring down the webservice (IIS) on the shared windows server Palamedes (81.17.248.55).  This will cause your website to go down, however all email services will be unaffected.

The reason this service has been brought down is because it looks like many sites on the server were compromised last night and had their index pages defaced.  In order to make the server secure and see how this hacker(s) compromised so many sites on the one server we have had to disable the webservice temporarily.

We hope to have this service enabled again as soon as possible and in the meantime we apologise for any inconveniences this may cause.

Update: 12:15

Currently we're working to delete all the files that contain the string of text that was put in place. This will cause many sites to show blank pages, but it'll also re-enable many sites on this server.

The restore is going to take some time to run as we can't filer out the index files. We don't have an estimated time to fix for everyone, but many of your .net apps will be back very shortly.

Update: 21:00 Sat August 9th

After a full days investigation we've found the hole that allowed this attack. A high profile site had an upload feature which allowed malicious attackers to upload arbitrary code. This code was an asp.net "shell" which was a basic web page which allowed the attackers access via to customers folders. We're unsure yet if there's any real protection in shared windows hosting regarding an attack vector like this, it's unlikely without restricting .net apps and causing functionality issues.

The site in question has been shut down and the owner contacted. We've also crawled through millions of files on the server to find any/all traces of the offensive index.html files placed on customers domains. We've also found some other copies of the ASP.net shell that the attackers left incase we found their primary entry point.

There will be one further update to this blog post in the coming week with further analysis of this attack vector and our solution to preventing it from happening again.

Final Update: Friday August 15 09:30

During our investigations of this intrusion we've noticed a few security implications. We've now taken measures to ensure that the default applcation pools for .net 1.1 and 2.0 do not run as the service Network Service. They now run as their own unprivileged users. We've also ensured that customers in their own application pools now run as their own web user also.

We've taken other measures to mitigate other attacks via other coding systems such as perl and php also to further strengthen the shared hosting platform.

0 TrackBacks

Listed below are links to blogs that reference this entry: Web Services on the Palamedes server temporarily disabled.

TrackBack URL for this entry: http://www.blacknightstatus.com/cgi-bin/mt/mt-tb.cgi/44

3 Comments

When can we expect the servers to be back up?

Can we expect a report on how this happened and what steps Blacknight are taking to ensure this kind of interruption to service is prevented in the future?

Kind regards

Dara Robinson

Brendan Moran on August 8, 2008 11:41 AM

I was trying to show client updates on website but cannot which is a disaster for me.

I am at client and need to access files by ftp but cannot.

When will I be able to do?

Regards

Brendan Moran

@Dara only one web server was affected and our technical staff will update the blog post with information as soon as they have it.

@Brendan we are working as quickly as possible to restore from a clean backup

Leave a comment