Engineers at our datacenter in Washington, DC are actively working on mitigating the issues customers are experiencing. At this time, the Denial of Service attack targeting customers on some of our servers has been mitigated; however, due to the high packet count along with the number of interfaces that reset during the event, the router is now displaying signs of instability resulting in further intermittent packetloss for come customers.
More information will be relayed to customers as it is obtained.
We do apologize for the issues and are actively working on a resolution for this event.
[UPDATE]
SoftLayer Engineers along with Cisco TAC have identified a software bug on FCR01.WDC01 which is causing forwarding issues for public connectivity. Engineers will be performing an EMERGENCY code change on Jun 2nd, 2011 at 10:00 PM EDT to resolve this issue. The expected downtime is 20 minutes with the maintenance window being scheduled for up to 4 hours.
Start Time: 10:00pm EDT (6/2/2011)
End Time: 02:00am EDT (6/3/2011)
Expected Duration: 20 minutes
Customer Impact:
During this maintenance, customers will notice a complete loss of connectivity to their servers on the frontend network (public network). Backend network (private network) connectivity will NOT be impacted during this maintenance. While the upgrade duration is scheduled for 4 hours, we only expect around 20 minutes of downtime as the code is changed. Again, this will NOT impact the backend network (private network) for customer servers.
[UPDATE]
The reload of FCR01.WDC01 has been completed. SoftLayer Engineers will continue to monitor the router to determine if this has resolved the issue. More information will follow as it is obtained.
[RESOLVED]
At this time all services are wroking normally. Softlayer engineers will continuew to monitor servers. If you have any connectivity issue, pelase be sure to submit a support ticket.
Thursday, June 2, 2011
