Network incident

Network incident

25-02-2016 12:06:24 - 25-02-2016 18:00:00
Status: opgelost

There are clues that there might be disruption in our network. Engineers have this problem under investigation. Any news on this matter will be published on our website.


Update placed at 12:20 The problem seems to be caused by one of our core routers. Some connections work and some don't.

Update placed at 12:24

The cause is found and a patch is being made at this moment.

Update placed at 12:37

The patch does not seem to solve all problems yet, further investigation is on its way.

Update placed at 12:59

News on this incident will be published on www.bit.org because www.bit.nl is not available for everybody.

Update placed at 13:10

One of the core routers is disabled at the moment. This improves the situation, but makes the network less redundant.

Update placed at 13:15

The problem is isolated for now. Our network is completely available. For the moment any updates will be published on this site.

Update placed at 14:10

Customers with internet connections on access networks (NDIX, Eurofiber, etc.) stil experience problems.

Update placed at 14:30

Internet connections on fiber networks are available as should be.

Update placed at 14:40

The faulted router will be brought back online step by step and taken into production. After completion the network will be fully redundant again.

Update placed at 17:30

A human error made during configuration updates on one of our core routers caused a problem in routing to parts of the BIT network: the traffic got filtered instead of routed. This error was detected early and restored swiftly. As a result a new problem arose: the router's processor that takes care of changes in the routing tables got overloaded. This resulted in a situation where routing tables were not updated correctly anymore. Therefore the router mistakenly routed traffic to parts of the internet and the BIT network that could not be reached by that router. As a counter measure BIT engineers routed away all traffic from this router and afterwards brought traffic back to this router.

The incident has been resolved. If you have any questions regarding this incident, please contact the Customer Care department by phone on +31 (0)318 648 688 or by email on support@bit.nl.