When: Wednesday September 16th at 4am
What: Control panel and backend service upgrade to POA 2.9 HF02 which will prevent access to cp.blacknight.com for the duration of the upgrade.
Overview: We're currently running quite a new version of POA 2.9 and there are a number of bugs in this version. We'll be upgrading to POA HF02 which will fix a number of UI, provisioning and general system bugs that crept into our system since the last upgrade around 6 weeks ago. There's also a couple of new features introduced however these won't affect our existing customers for the time being.
Update: This update is a reminder that this maintenance window is going ahead tomorrow morning.
Update: 09:45
There are two issues since the upgrade this morning that are causing people issues.
Firstly webmail.$domain.tld isn't currently working. As a work around, you can login to https://webmail.blacknight.com and use the mail server "mail.blacknight.com" with your normal e-mail address and password.
Secondly all webspaces with php5 as apache mode seem to have somehow reverted to php4. We understand that this is a very serious issue and we've raised it with the highest possible priority with Parallels to get it fixed. The only short term work around is to change your webspace to use php5 cgi mode, but this won't suit everyone so it's not recomended unless your site is badly effected.
Update: 16:25 - all issues resolved and the upgrade ticket is now closed.
As of around midday the above issues were resolved. We were just waiting to see if any other issues cropped up before we gave the all clear on this upgrade.
Summary: March 3rd @ 4am to 8am we'll be upgrading our new shared hosting platform and some back end services.
About 75 bugs are going to be fixed tomorrow morning starting at 4am. The result of this upgrade will mean that cp.blacknight.com and some other services will be down during the upgrade.
The maintenance window from parallels is approx 4 hours, so it should be fully completed by 8am.
A list of affected services and some well known bugs are below:
1) Hosted exchange bug fixes
2) linux shared hosting bug fixes
3) windows shared hosting bug fixes
4) shared e-mail, there will be some upgrades to our qmail servers which will cause pop3/imap issues during the maintenance window.
5) Bug#264931 upgradeSubscription for container if capacity VPS Number per HW node is over - currently if the limit we've set per hw node is reached and a customer gets an upgrade, their VPS will be shutdown and migrated to another node. This has been resolved.
6) Bug#271059 Webmail login does not work for 'webmail.yourdomain'
Update: March 3rd @ 09:45
This updated was delayed for a few hours but is currently being performed. It should be completed by 10:15 at the latest.
Update 2: 1253 - All services should have returned to normal before 11am. However if you are experiencing issues that you think may be related please let us know.
A few people have told us that over the past while they've noticed the new store can be a bit slow at times. There are two parts to the current store, the part on blacknight.com that allows to you choose to build a package and the bit that handles domain registrations, account creation and the checkout process (the part where we take your money, which is very important).
Since the entire platform is virtualised we've got the ability to tweak access to the underlying hardware for each part of the platform.
Today we've done the following:
1) We've given store.blacknight.com more access to the Intel multi-core CPUs on the hardware nodes that it runs across.
2) We've given bm.blacknight.com access to all cores on the nodes that it runs across as the store makes xmlrpc calls to bm and bm processes around 20k tasks a day (domain lookups, billing customers, creating accounts, sending provisioning commands to the provisioning system, checking on transfers, registrations etc etc).
3) We've re-written both the vps and shared hosting shops so that they now cache the plans instead of hitting the XMLrpcd on each customer view. The shared hosting one is live and the VPS one will go live in the coming week.
The changes above should make your experience with us more pleasant and also cause less frustration. We can also re-look at the allocation of cores at a later date to better tweak both systems.
Today we're upgrading our billing system to version 4.3.4 of Parallels Business Automation. This will happen at 10:30am today Tuesday 7th of October.
Your billing panel, our store and other services will be intermittently affected during this software upgrade.
No critical customer services will be affected by this.
Sorry for the short notice on this, but it was planned for today however it contains some critical fixes for our software that need to be applied.
Summary: On Sunday morning next @ 02:00 am one of our Fibre providers (Auroa) are carrying out maintenance on the backup ring that they provide to us.This will be a total cut of the fibre while they replace several joins to make them more flexible. During this 8 hour window our traffic will be on our primary link which does not go near the affected physical area and the route is direct to DEG.
If we do experience an outage on our primary ring due to this window, we've a tertiary path that we may use and this should automatically failover.
Any questions please address them to our technical team https://support.blacknight.ie/
update: Sunday June 22 @ 10:10am
This upgrade was successful, Aurora report that circa 04:30 they completed the work on the fibre that our CWDM ring runs over. We got the all clear at 10am to confirm the fibre was back in the man hole and that the window was completed successfully with no snags.
On Thursday morning starting at 7am we'll be upgrading our CP system to PBA 4.3 from 4.2. This upgrade will take between 1 and 2 hours and for the duration of it the Customer Control Panel (https://cp.blacknight.com) will be unavailable.
You'll be able to access your VPS via SSH or Plesk/Vzpp by going to it's IP address and port 8443 or 4643 using https.
If you have any questions please let us know.
Hi there,
PEM(PSA as its now called) will be upgraded to version 2.7 tomorrow morning starting at 6am. PEM has some UI and Backend bugs that needed to be fixed that Parallels have included in the 2.7 release. As we're affected by 2 of these bugs we're having them update our install to 2.7 from 2.6.5.
The upgrade of our current system should take about 3 hours. Parallels development/engineering team will be doing this upgrade. There will be approx 3 minutes of downtime per Windows VPS node and upto 35 minutes of downtime to our CP.
We'll post tomorrow with an "all clear" once we've tested everything after the update.
As usual, if you notice any issues please contact us via e-mail (support at blacknight dot ie) or directly on our helpdesk.