This page compares the different LINX IX and explains some of the underlying technologies in the context of how they are used at LINX.
Platform
Juniper MX series with JunOS - A proven platform from Juniper which has been powering LON1 since 2011. It currently provides 1G - 400G ports. (1G - 100G only supported on NoVA currently).
Nokia SR-s series with SR-OS - The first Nokia SR7-s was deployed in 2022 in LON1 to provide 400G and 100G services.
Edgecore AS7712, AS5812 with OcNOS - The Edgecore routers are white box networking devices based on the Open Network Install Environment (ONIE) and are used in combination with IPI’s OcNOS at LINX. Ports from 1G - 100G are supported. The Edgecore AS7946 was deployed in 2023 to enable 400G capability on LON2.
Nokia 7220 IXR-D3 with SR LINUX - The first Nokia SR LINUX devices were deployed in 2023 in LINX Nairobi to provide 10G - 100G ports services.
Platform
Traditionally, L2 networks were based on Ethernet switching with spanning tree for loop and fault prevention. By today’s standards this is no longer adequate for large IX like LINX, and technologies have been developed to improve the failover times in events of link failures and to reduce BUM (broadcast, unknown unicast and multicast) traffic.
The following is a list of those technologies and how they are deployed at LINX.
BGP session culling (BCP214)
BGP session culling has been implemented at all major LINX peering LANs to reduce the impact of LINX maintenance work on member traffic.
ARP/ND flooding mitigation and filtering
ARP/ND Proxy - All EVPN-based LINX peering LANs implement proxy ARP/ND on the LINX edge routers:
All edge routers are either statically (OcNOS - configuration of MAC/IP mappings) or dynamically (JunOS - snooping ARP replies or gratuitous ARPs) configured with MAC/IP mappings of the local member ports.
The MAC/IP mappings are propagated to all other edge routers via BGP type 2 EVPN routes.
The edge routers snoop ARP requests/NS from the members. If the ARP request/NS is regarding a known MAC/IP mapping, it proxy replies to the member and does not flood the ARP request/NS throughout the network. In case the MAC/IP mapping is unknown, the ARP request/NS is flooded as usual.
Invalid ARP Ingress Filtering - In EVPN-based JunOS LINX peering LANs, proxy ARP MAC/IP mappings are learnt dynamically by snooping ARP replies or gratuitous ARPs. To avoid members accidentally “poisoning” the LINX ARP cache with invalid ARP replies or gratuitous ARPs, LINX filters all ARP packets where the ARP sender IP does not match the member’s IP.
ARP Sponge - After a LINX member has been decommissioned or its port went down temporarily, unknown unicast traffic or ARP requests directed to that member is being flooded throughout the LINX peering LAN. To mitigate this undesired flooding, in every peering LAN, there is one ARP sponge configured with unused IP addresses. It advertises its own MAC address together with the unused IP address and therefore prevents unknown unicast flooding and flooding of ARP requests.
Private VLANs and closed user groups (PVLAN/CUG)
Private VLANs or Closed User Groups are separate L2 domains in addition to the standard peering LAN at every IX.
Traffic in those L2 domains is not filtered and the “ARP/ND Flooding Mitigation and Filtering” as well as “BGP Session Culling (BCP214)” measures do not apply.
Network automation
LINX uses network automation to configure most of its IX network devices which enables faster and more consistent deployment of configuration changes. In modern networks, complicated configuration changes on multiple devices are frequently required, which caused LINX to implement network automation in 2017 for LON1 and LINX is now using it for all major IX.
LINX’s network automation is known internally as network configuration automation (NCA) and has been developed in-house. It is based on ansible and Python for the orchestration and Napalm and Netconf for the interaction with the network devices.
Features like self-service provisioning and BCP214 rely heavily on automation and would not have been possible without it.
Broadcast, Multicast, Unknown Unicast
To mitigate negative effects of flooded traffic on the peering LAN, BUM (Broadcast, Multicast, Unknown Unicast) traffic is either policed at the member port or per routing instance on edge switches.
It is possible to send low amounts of Multicast traffic via the peering LAN but in case higher amounts of Multicast traffic are going to be sent, a closed user group (CUG) is recommended.
