LINX Exchanges
SOFT Admin
SOFT Admin

LINX Interface on Cisco

Configuring your LINX interface on a Cisco device

The technical rules are detailed in the Technical Appendix 1 of the LINX Memorandum of Understanding (MoU).

This document is a guideline from LINX Engineering, to assist members in complying with these rules and offers other useful tips, but does not in itself form part of the MoU.

This document does not discuss BGP routing policy configuration.

With regards to BGP, LINX does not require MD5 passwords configuring on peerings to AS5459 (224/226 LAN collectors) and AS8714 (route-servers). If you prefer to have MD5 passwords configured however, we are happy to do this.

 

Port Speed and Duplex Settings

  • Your port should not use auto negotiation, it must be set to full duplex at 100Mb/sec, 1Gb or 10Gb as appropriate.

  • The only possible exception to this is gigabit over copper, which may need to have auto-negotiation enabled in order for the link to work. (Nailed config is not supported on some devices.)

  • Flow control should be switched off.


Other Interface settings to disable:

  • ICMP Redirects (no ip redirects)

  • PIM

ARP

  • Proxy ARP must be switched off. (no ip proxy-arp) a host - generally a router - replies to ARP requests for IP addresses other than itself. Very dangerous and you should disable it explicitly on your LINX interfaces. Particularly Cisco routers where, if on our /22 networks, you accidentally configured a /24 mask, the Cisco may then start to reply to ARP requests for hosts that are not in your interface's configured /24 "subnet"

  • On some devices, particularly cisco versions, Proxy ARP is enabled implicitly by default on the interface.

  • Generally, as the MAC addresses on the LINX LAN remain fairly static, set your ARP cache to a reasonably high timeout value. (But not too high as to cause problems with flooded traffic) When most ARP implementations see a different MAC address for an already cached IP address, it will delete the old one from its arp cache and use the new value instead.

  • Use something other than the default ARP age timers. As many devices have the same ARP timeout values, an outage or switch reboot can trigger waves of ARP requests and ARP broadcast storms, by causing these ARP timers to "synchronize". If routers are configured with slightly different values, this problem becomes reduced.

  • In an outage, you should shut down your BGP sessions instead of shutting down your interface, unless this is absolutely necessary. This will mean that your router is still responding to ARP requests and is not contributing to floods of ARP traffic from all your peers attempting to re-establish BGP sessions and hence sending broadcasted ARP requests.

  • If an IP address is not used for some time, LINX may "sponge" the address by binding the IP address to one of our interfaces. This means that ARP broadcast requests are replied to, and the overall broadcast traffic is reduced.

Other non-IP protocols

Includes the following:-

  • DECnet, IPX, Loopback, SNA, NETBIOS, Spanning Tree

  • Vendor proprietary discovery protocols (CDP, DTP/VTP, EDP, FDP, Dynamic VLAN discovery.)

  • Corrupted frames

Disable the offending protocol

Only a single MAC address per port is permitted. Many of these protocols (especially where a switch is involved) will cause multiple MAC addresses to be learned on the interface, and hence may also cause you to fall foul of this rule.

  • Disable the offending protocol.
    i.e. Configure 'no cdp enable' on the LINX interface. (Cisco)

  • DECNET LAT/MOP. Configure 'no mop enabled' on your LINX Interface. (Cisco) We have had reports that some versions of IOS send out these frames, but there is no command to turn it off. (i.e. DECNet was removed from the CLI as it is not supported in the version, but some of it still is in the code!)

  • BOOTP/DHCP and Broadcast DNS requests etc. (Check Cisco startup config for correct tftp location and DNS servers. Turn off unconfigured interfaces rather than using default DHCP.)

  • DTP/VTP. Possibly dangerous and, fiddly or impossible to switch off on some IOS versions. You may find some of the following documents useful:
    Cisco Catalyst 6500 Series: Configuring VTP
    Completely disabling VTP in CatOS 7.x
    Per-Port control of VTP is only possible in CatOS 8.1(1)
    Some general notes which identify some of the other protocols you might want to control and the best-practices

    Cisco IOS Switches 29xx and 35xx:

    vtp mode transparent

vlan
name LINX

interface
 switchport access vlan
 switchport mode access
 switchport nonegotiate
 no keepalive
 speed nonegotiate
 no cdp enable
 no udld enable

    Cisco Catalyst CatOS based switches:

    set vtp mode off
 set port name  LINX
 set cdp disable
 set udld disable
 set trunk  off dot1q
 set spantree bpdu-filter enable
 set vlan  name LINX
 set vlan

  • LOOP Loopback traffic - Keepalives. Legacy keepalive/AUI/heartbeat frames being sent. Used to determine if a transceiver is alive. Most modern interfaces spot carrier transitions without the need for these heartbeat frames. Configure 'no keepalive' on your LINX Interface. (Cisco)

  • Garbage? LINX will generally contact you if your router is emitting large amounts of corrupted frames. Since the MAC Addresses are corrupted, i.e. "0e:a4:04:b8:02:95 -> 95:a7:d7:e0:28:50" LINX will have to investigate these MAC addresses on the switches to determine where the corrupted frames are coming from.

  • Check for overloaded interfaces and duplex mismatches. (All LINX interfaces are no auto negotiation, full duplex.)

  • We have also seen corruped frames caused by software bugs and leaky VLANS, broken tunelling protocols, overloaded CPU/buffers, bugs in CEF and MAC Accounting etc, and particular revisions of line cards.

Non-unicast (Link Local) traffic

Any packets directed to a multicast or broadcast address, not permitted by the LINX MoU.
Includes:

  • OSPF, RIP, EIGRP, ISIS, IRDP etc.

  • DHCP, Broadcast DNS or NTP

  • PIM

  • ICMPv6 ND-RA

Troubleshooting non-unicast traffic:

  • Disable the IGP on your LINX interface (passive-interface)

  • Disable DHCP requests (unconfigured interfaces etc) : (no service dhcp or no ip bootp server)

  • Unconfigured Cisco DNS settings sends DNS requests to 255.255.255.255

  • Unconfigured Cisco TFTP tries to tftp to 255.255.255.255
    (set the ip tftp source-interface to be the correct interface or use 'no service config' globally)

  • IPv6 Router Advertisements (ipv6 nd ra suppress all)

Sample cisco router interface config:

interface GigabitEthernet0/0
 description LINX Interface
 ip address 195.66.224.n 255.255.254.0
 no ip redirects
 no ip proxy-arp
 no mop enable
 no cdp enable
 ipv6 nd ra suppress all

On switch interfaces, you may also need the "no keepalive" setting.

Monitoring

  • If you poll your router with SNMP, check which interface you are using. Ideally you should poll a loopback interfaceand not the LINX peering LAN address. If you shut down or move IP addresses, your SNMP packets may become visible to everybody on the peering LAN (including your SNMP community string!)

  • Please do not block ICMP requests from the peering LAN. Ping is used by LINX for monitoring purposes. The LINX monitoring system will ping your interface about every 2 minutes. If you are concerned about abuse of ICMP, then you should consider rate-limiting ICMP requests rather than blocking, so that LINX - and your peers - can use ping for troubleshooting purposes.

Other policies

Your attention is also drawn to sections 4 and 5 of the Technical Appendix of the MoU:

Routing

  1. All exchange of routes across the LINX network shall be via BGP4(+).

  2. AS numbers used in BGP4(+) sessions across the LINX network shall not be from ranges reserved for private use.

  3. LINX supports good engineering practice and LINX Members are encouraged to aggregate their routes in accordance with RFC2519 "A Framework for Inter-Domain Route Aggregation".

  4. IP address space assigned to LINX peering LAN shall not be advertised to other networks without explicit permission of LINX.

  5. All routes advertised across the LINX network shall point to the router advertising it UNLESS agreement has been made in advance in writing by LINX and the two members involved.

  6. All routes to be advertised in a peering session across LINX shall be registered in the RIPE or other public routing registry.

  7. Members may use more than one ASN for their LINX peering provided that each ASN presented shares the same NOC and peering contact details.

Forwarding

  1. Traffic shall only be forwarded to a LINX Member when permission has been given by the receiving Member either:

    1. by advertising a route across the LINX network

    2. or explicitly in writing

  2. Traffic shall not be routinely exchanged between two LINX ports owned by the same LINX Member.

If you have any questions or would like to discuss further, please contact [email protected]