Tuesday, 15 May 2012
Planned Maintenance and Network Upgrades
Details of maintenance
As part of CCS’s ongoing commitment to provide a quality service to our customers, we are implementing these upgrades to add capacity and resilience to our core network locations.
This will involve the removal of existing core routing hardware and its replacement with new hardware to provide a thirty fold increase in core routing capacity and a twenty fold increase in core switching capacity, adding additional diversity between the locations as a benefit.
On the night of the 1st June, we will be working in the location housing the largest percentage of our leased line customers, our SharedBand core servers, many of our Helm based web servers, and much of our Voice over IP Platform.
While each service will only be hard down for a short period – a few minutes per platform is anticipated – the sheer number of services makes for a lengthy downtime window during which one or more services will be unavailable at any one time.
Traffic re-routing is anticipated at the beginning of the maintenance as we re-route as many customer services as we possibly can to maintain connectivity while the core routers are swapped. Following this, core switching and customer routing will be re-connected, then VoIP platforms, leased lines, servers and firewalls.
We anticipate to be complete considerably earlier than 4am, however this is a very extensive piece of work, and it is quite possible we may require the full six hour window.
On the night of the 2nd June, we will be working in the location primarily housing routing equipment, DNS servers and the larger percentage of our external connectivity plus a smaller number of leased line customers.
At the start of the maintenance window we will be changing routing to force all traffic to another part of the network, which may result in a brief loss of connectivity. Following this, the core routers will be removed and replaced and customer services terminating at this site brought back into service.
We anticipate the works on the 2nd June to be completed considerably earlier with perhaps only a total window of disruption in the order of two hours. However we are once again advising a six hour maintenance window to cover the unexpected.
1st June: Customers with none resilient Leased Line, IP Transit Services, Voice over IP services, SharedBand users and Helm based web hosting WILL be affected by service outages on at least one occasion. ADSL users may notice brief interruptions in service too.
2nd June: Customers with resilient Leased Line services may notice a loss of resilience. IP Transit Services single homed on Telehouse East WILL notice a loss of service. ADSL users may notice brief interruptions in service.
Customers with fully resilient services, delivered from either London, Manchester or Leeds will notice only a loss of resilience.
Customers based at either of the Leeds Datacentres will not be affected, although a lowering of resilience will take place during the change and on the evening of the 2nd June placing the site “at risk”. We have more than adequate resilient capacity in both the Leeds sites to handle the traffic throughout without interruption.
To protect the network in advance of major events over the coming months and to increase capacity and resilience ongoing as we add further customers and services.
Rollback Plan and Precautions:
In case a problem is discovered during or immediately post implementation, we benefit from having all existing hardware in an un-touched state to return to service. Our NOC will monitor the process throughout and provide early advice of any apparent issues.
The new hardware is presently on a bench in complete testing in Leeds to burn in before being pressed into live service.
Once complete, or indeed aborted for any reason, customers will be advised via their registered email addresses.
Although this is planned maintenance, CCS will continue to monitor customer services throughout and will take every precaution to return services to an operational state in a speedy manner. Never the less, if you do feel there is an issue that is unattended to post the maintenance window, please do not hesitate to get in contact via the normal methods.