Network outage: core switch and UMTS

Network outage: core switch and UMTS

23-04-2015 01:47:00 - 23-04-2015 14:42:00
Status: opgelost

At 1.47h we observed an outage on one of the core switches in the BIT core network: its module automaically reloaded. As a result, a number of services, most notably VLANs between BIT-2 and locations in Amsterdam were disrupted for several minutes.

In addition, all network traffic passing through this switch was shortly impacted before this was rerouted via other paths.

At 1.49h the switch was operating normally again and all connections were restored.

We're currrently investigating the cause of this unexpected reload.

Update placed at 23-04-2015, 2:51h
There seems to be a problem with the interconncetion between BIT and Vodafone. As a result customers using the BIT APN do not have working internet connections. Although this interconnection is not connected to the reloaded module (or even at the same location), the time of the outages are identical. We suspect that a safeguard in the Vodafone network was triggered which disabled our interconnection. We have contacted Vodafone and opened a case and are currently awaiting their investigations.
Update placed at 23-04-2015, 13:32h
Unfortunately, Vodafone seems to have problems identifying this connection and hasn't been able to help us repair this interconnection yet. We're doing all we can to escalate this problem at Vodafone and get this connection working again.
Update placed at 23-04-2015, 14:55h
Since 14:42h the interconnect with Vodafone is restored.