Essential Maintenance PEMVZMPS20, PEMVZMPS23

Message

We have been alerted to a disk failure in both PEMVZMPS20 and PEMVZMPS23. Due to it's RAID configuration no data loss has occured of course.

To get the RAID back up and running to it's full potential we are going to replace the failed disks tonight.

When: 3rd of March 2010 at 21:00 (Expected downtime 15 minutes per node.)

Affected nodes are:

81.17.254.85    pemlinweb04.blacknight.com     
81.17.254.86    pemlinweb05.blacknight.com
81.17.254.45    mysql71.cp.blacknight.com

We will update this blog post once completed.


Update 21:32 - PEMVZMPS23 has been completed, both pemlinweb04/05 are back online. We are waiting on a disk check to finish on PEMVZMPS20, once complete the node will be brought online. ETA 15 minutes.


Update: 22:20 - both nodes are back up and the services are restored. As the raid arrays are rebuilding thing sill be a little sluggish until it's completed. This will take several hours.