Summary: Windows node pemvzwin02 is experiencing issues with internal virtuzzo API calls. This is causing a task backlog in our provisioning system. In order to diagnose the issue further we've been asked to reboot the node.

We're taking the opertunity to run windows updates while we do this, so at least some good will come of it.

The following list of VPSs are affected:

vps-238.cp.blacknight.com, vps-258.cp.blacknight.com, vps-262.cp.blacknight.com, vps-265.cp.blacknight.com, vps-271.blacknight.com, vps-282.cp.blacknight.com, vps-287.cp.blacknight.com, vps-288.cp.blacknight.com, vps-302.cp.blacknight.com, vps-317.cp.blacknight.com, vps-331.cp.blacknight.com, vps-337.cp.blacknight.com, vps-341.cp.blacknight.com, vps-342.cp.blacknight.com, vps-343.cp.blacknight.com, vps-344.cp.blacknight.com, vps-346.cp.blacknight.com, vps-347.blacknight.com, vps-353.cp.blacknight.com, vps-354.cp.blacknight.com, vps-355.cp.blacknight.com, vps-357.cp.blacknight.com, vps-361.blacknight.com, vps-362.cp.blacknight.com, vps-386.cp.blacknight.com, vps-387.cp.blacknight.com, vps-390.cp.blacknight.com, vps-407.cp.blacknight.com, vps-409.cp.blacknight.com, vps-410.cp.blacknight.com,
vps-584.cp.blacknight.com, vps-589.cp.blacknight.com, vps-612.cp.blacknight.com, vps-620.cp.blacknight.com, vps-626.cp.blacknight.com, vps-635.cp.blacknight.com, vps-642.cp.blacknight.com, vps-645.cp.blacknight.com, vps-664.cp.blacknight.com, vps-667.cp.blacknight.com, vps-672.cp.blacknight.com, vps-684.cp.blacknight.com, vps-689.cp.blacknight.com, vps-704.cp.blacknight.com, vps-709.cp.blacknight.com,
vps-710.cp.blacknight.com, vps-716.cp.blacknight.com, vps-718.cp.blacknight.com, test.zerotouch.ie

I'll update this ticket once it comes back up and all VPSs are back up. It'll probably take upto an hour, but it's completely necessary.

Update: 21:30

The node rebooted successfully and about 70% of the VPSs have restarted. The rest should be back online in the next 10 to 15 minutes or so.

We're continuing our investigation of the issues we've been encountering in our backend systems.

Tomorrow morning our software vendor will be patching our back end provisioning systems. This will affect multiple customer facing systems including cp.blacknight.com, access to plesk and virtuozzo power panel on Plesk loaded VPSs and potentially other systems. All sites, services will remain in place, just admin access to customers will not be available during the upgrade.

It should be starting around 4am and will last for upto 1 hour. I'll post an update tomorrow morning post upgrade with news of fixes to issues we've been having.

Update: 22:05 - 5th Jan 09

Due to issues beyond our control this has been re-scheduled until Thursday 8th at the same time.

There was an issue with cp.blacknight.com earlier this morning which was preventing users from logging in.

Once we became aware of the issue we took action to fix it and it is now working as normal.

Hosting services were not affected.

We are currently experiencing issues processing payments via credit card / Laser.

We will update you once this issue has been resolved.

In the interim we are more than happy to accept payment via Paypal, wire transfer etc.,

Apologies for the inconvenience

UPDATE: 2320 - this issue should now be resolved

We've added a couple of more tutorials to assist users of our shared Windows hosting plans.

The new tutorials cover using the File Manager on Windows as well as managing FTP.

They may be found in the tutorials section of our site here

If there are any topics that people would like to see us provide tutorials for please let us know

If you are running a Plesk based VPS DO NOT upgrade to Plesk 9.

There are issues with Plesk 9 on our system which may render your VPS unusable

So please do not install Plesk 9 until we give the "all clear"

The domain search function in our online store is currently taking a long time to provide results. We're aware of this issue and working to resolve it as soon as possible, we believe one of the registrars/registrys we use may currently be having some problems. Once we figure out which one we'll disable it and the store domain search function should return to normal.

Update: 12:22

The domain search function is now working correctly. We're still investigating the cause, but as of 12:22 it's back working. We suspect large logfiles (2GB file boundry) may have caused this, but we're awaiting clarification from Parallels.

We have been informed that Nominet, which is the registry operator for *.uk domains, will be conducting maintenance tomorrow morning from around 8am until 9am.

During this time period we will not be able to process any new co.uk domain registrations or updates.

Full details on their site

NB: This will not affect existing co.uk domains or any other services

UPDATE: 9.33 am
Nominet have been experiencing issues with this upgrade and now expect normal service to be restored around 10am

Summary: Mail on our new shared hosting platform is not working 100%, there was a change made by Parallels to our control panel systems this morning which should not have affected e-mail but it looks to have had.

We're currently working to resolve this issue and we'll post updates when we have an ETA.

Update: 09:55

This appears to be an LDAP problem, we're working to resolve it at the moment. Right now the service is running but keeps locking up. The knock on effect is that the mail servers can't auth pop3 or imap users currently.

Update: 10:35

We've been monitoring ldap now for some time and for the last 18 or so minutes we've been successfully authing users. We have the capability for LDAP replicas which we'll implement to prevent us having to rely on 1 LDAP server in the future.

Problem resolution:

The ldap indexes were corrupt, we stopped ldap, moved the indexes out, started ldap and the indexes have been rebuilt. POP, IMAP and SMTP are now functioning correctly. 

We are currently investigating some network issues that are having some impact on our network.

Sorry if this is vague, but I'm waiting on more concrete information from our technical staff

Just be aware that we are looking into it