eVPN Primer:

The first thing to note is that eVPN is not a new technology – it’s new to NSX-T, also eVPN generally runs in two different modes (L2 vs. L3) – more on that later – but the focus for NSX-T is the L3 mode, more specifically the use case of mapping a VRF through Multi Protocol BGP (MP-BGP), eVPN Address Family (AF) between an NSX-T Tier 0 (T0) router and a DataCenter Gateway (DCGW) implementation (Nexus 9000, Cisco ASR9k etc.).

One key thing to note is that the implementation of eVPN requires two new constructs for NSX-T

• Virtual eXtensible Local Area Networks (VXLAN) enablement
• Tier 0 Virtualised Routing & Forwarding Instances (VRFs)

Yes, you read that right, for the implementation of eVPN, NSX-T leverages VXLANs between the T0 and the DCGW, this is primarily due to the fact that the number of upstream DCGWs supporting GENEVE encapsulation for this is low (and may require a hardware refresh).

So, back to eVPN, MultiProtocol BGP (MP-eBGP) is implementated as a control plane protocol for VXLANS, this model introduces the capability of control-plane learning for hosts / networks connected beyond the Tunnel EndPoint (TEP) – this gives the ability for control / data plane separation (watch this space in the future) for L3 forwarding over a VXLAN Network.

In the case of the NSX-T implementation, because we are focusing on a L3 eVPN interconnect the routes we exchange are eVPN type 5 Prefixes, referencing back to the iETF bess draft we can see that this Type 5 IP Prefix looks as can be seen below:

With this structure, we have the ability to transport IP (IPv4 and IPv6) prefixes over a MP-BGP session, to do this we also leverage the EVPN address family (aka bgp l2vpn evpn AF in Cisco terminology).

To quote from the RFP the eVPN RT 5 decouples IP prefix advertisements from the MAC/IP route advertisements (RT2) in eVPN (RFC 7432)

We will get into more detail later in the post with examples of these prefixes (both IPv4 and IPv6) as well as decoding the BGP NRLI so that we can reference back to the draft RFC and see how this implementation actually works.

MP-EBGP (eVPN) Use Cases

A common telco use-case for eVPN involves transportation of Corporate APNs(C-APN) over a L3 Data Center fabric, in the Evolved Packet Core (EPC) use cases each Corporate Access Point Network (APN) has its own VRF that typically extends into the (now virtualized) packet-gateway node.
The challenge as shown below is how to maintain this routing isolation on top of a common / shared L3 fabric that does not implement MPLS, basically how to extend the VRF from the VM to the PE / DCGW… Sure you could provide a regular VLAN and then create an overlay on the L3 Fabric, but that’s not entirely a simple or scalable solution – the idea being to deploy the underlying fabric and (as much as possible) – leave it alone.

Hence MP-BGP eVPN and NSX-T VRF capability, although the former diagram doesn’t include the T0 / Edge Nodes in the diagram, this is a fundamental element of the NSX-T eVPN solution, basically the connectivity model looks similar to the model below: (Image to be added)…

Note: this diagram shows only a single NSX-T Edge node, however the solution fully works in active/active mode (you just need to repeat the configuration on multiple NSX-T Edge Nodes) and configure BGP Multipath on the VM, T0 and DCGW.

Lets break this down a little bit so that is clearer to understand, what we have is a Virtual Network Function (VNF – VM) that is running VRFs (in the example configs we will see later this is a Cisco CSR1Kv node), this runs vanilla eBGP between the VRF (or global construct) and the new T0-VRF construct on NSX-T – Note: this needs to be a .1q tagged interface on the VM, thus we need a Geneve segment with Guest VLAN tagging (GVT) enabled – the GVT is used to distinguish the VRF constructs on the VNF.

That takes care pretty much of the VNF side of things, but what about the T0 and this new VRF construct

Here we see the new configuration elements of the Tier0 Gateway – as you can see we have a VRF defined: VRFBlue, this is the new NSX-T T0 VRF construct that provides:

• Tenant Routing isolation
• Increased T0 Scale on a single edge node –

Prior to T0 VRFs we could only have a single T0 per edge node – this now increases the effective number of T0s to 100 through the use of VRFs).

In addition to creating these T0 VRFs, we also have some specific VRF settings that are required for configuration of the evPN service, here under the new section VRF settings we see some options:

Route-Distinguisher (RD): RD leveraged to create unique RD+IPvX prefixes in the routing tables
Route-Targets (RT): well known BGP EXT-Community to determine route import and export targets
eVPN Transit Vxlan Network Interface (VNI): this is the upstream VXLAN (between T0 and DCGW that will be used for transporting prefixes (and traffic) between the T0 and the DCGW — this is the VLXAN encapsulation addition to NSX-T that allows interop between the T0 and the DCGW for eVPN.

So, thus far we have seen the draft that is used for the eVPN implementation, a common use-case and a little snippet of detail around the T0 VRF construct, lets start a walkthrough of how to get this configured from scratch on NSX-T.

I will make the assumption here that you have NSX-T deployed, a host cluster prepared as well as having deployed an Edge Cluster with at least one edge-node and a regular T0 configured.
Note: All configuration is under the Policy UI (not the legacy Manager UI – in NSX- you won’t see the option for toggling the UI unless you have something configured that relies on it anyway)

Step 1: Create a VXLAN pool used for the upstream connectivity between the T0 and the DCGW:

Under Network settings goto VNI pool, here create a new VNI pool, give it a name and a start-end range, these are the VXLAN segments that will be used upstream towards the DCGW

Once this is created, go to the main T0 and select this pool under the eVPN settings, if you don’t do this you will have an issue creating your T0 VRF later, also enable BGP and allocate your BGP Local-AS , but do not add any neighbours (yet)…

Step 2: Create the Geneve GVT segment

Under segments, create a new segment in your Overlay domain, note here we have set the VLAN range (100-300), this is the VLANS that will be used as sub-interfaces towards the VNF (VM), remember all traffic from the T0 to the VM is sent as a VLAN inside the Geneve overlay – we need to make sure we set a valid range here.

Step 3: Create a new Tier 0 VRF construct

Under Tier-0 Gateways select Add Gateway and select VRF, this will deploy our first T0 VRF, you need to fill in the following details:

• Name – Give the VRF a name
• T0 Gateway – Connect this to your main (Parent) T0 Gateway
• VRF Settings: RD – Give this VRF a unique RD value
• eVPN Transit VNI – Select a VXLAN number you wish to use upstream for this VRF: Note this must exist within the range configured in step 1.
• Route Targets – Here you have a choice to used Auto RD or manually select route-targets

Personally, I prefer to set the route-targets manually that way I have a little more control and i can track what values are used between the T0 and the DCGW, add your route-targets here in the form of BGP ASN:xxx where xxx represents a number you wish to use to define how prefixes are imported / exported between peers.

Step 4: Create an interface between the T0 VRF and the VNF (VM)

Here we will build out interface between our T0 VRF and the VM, the following information needs to be configured:

• Name – Interface Name
• Type – This must be configured as a Service interface
• IP Address – this is the IPv4 and / or IPv6 address of the T0 Interface
• Connected to (Segment) – use the Segment we built in step 2 that has guest VLAN tags configured
• Edge Node – which edge node you want this interface to belong to (in my case EN1)
• Access VLAN ID – this is the Guest VLAN Tag (GVT) that is used between the T0 and the VM, this needs to be within the permitted range of VLANS allowed when the segment was created in step 2.

Note: for now i’m only configuring an interface between NSX-Edge Node 01 and the VM, if I wanted to have Edge Node redundancy I would create second service interface on the VRF with a different set of IP addresses AND a different Access VLAN ID and bind this to NSX-Edge Node 02 – this would mean from a single vNIC of the VM there would be two GVT’s one which would terminate on EN1 and the other on EN2.

At this point we can hop over to our VM and check connectivity towards this new T0 VRF interface, in the case of this lab, I’ll connect to my CSR-1Kv node:

CSR-1KV#show run int gig 2.197
Building configuration…
Current configuration : 121 bytes
!
interface GigabitEthernet2.197
 encapsulation dot1Q 197
 vrf forwarding blue
 ip address 10.1.1.1 255.255.255.252
end

Here we can see the CSR-1Kv is using gigabitethernet 2 (under vSphere this interface is connected to the Geneve segment we built in step 2), we have a matching encapsulation (VLAN 197).
Note: here i’ve configured VRFs on the CSR-1Kv that way I can create many VRF and test 1,2, 10 VRFs with a single VM, you don’t have to do this, but its recommended if you want to use a single VM and demonstrate multiple isolated routing tables end-to-end between VM and DCGW.

CSR-1KV#ping vrf blue 10.1.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/3 ms
CSR-1KV#ping vrf blue ipv6 aaaa:bbbb::2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to AAAA:BBBB::2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
CSR-1KV#

Success! , we have both IPv4 AND IPv6 reachability between our VM and the T0 VRF, lets add some BGP to this to make it interesting

Step 5 – Configuring BGP between VNF-VM and Tier-0 workload

Under the VRF Tier0 goto BGP and configure your BGP neighbours, be sure to set the matching Remote AS and so on.
Note: the BGP configuration of the T0-VRF (in terms of BGP-AS# etc.) is inherited from the Parent T0 – so for now at least make sure you have BGP enabled and a BGP-AS configured there (we will start to add the eVPN part of the BGP configuration to the parent T0 shortly).

Below is the configuration from the CSR-1Kv, remember we have the interface in a VRF here, so we need to configure BGP IPv4 and IPv6 inside the vRF, you need to enable the command vrf upgrade-cli on the CSR-1Kv to be able to implement this – i’ve also added the global VRF configuration as well as a loopback interface I want to advertise over BGP to the DCGW VRF constructs.

vrf definition blue
 rd 65100:1
!
address-family ipv4
 exit-address-family
!
address-family ipv6
 exit-address-family
!
interface Loopback6
 vrf forwarding blue
 ip address 11.11.11.11 255.255.255.255
 ipv6 address 6666:6666::1/128
end
!
router bgp 65100
bgp router-id 192.168.100.194
bgp log-neighbor-changes
!
address-family ipv4 vrf blue
 network 11.11.11.11 mask 255.255.255.255
 neighbor 10.1.1.2 remote-as 65000
 neighbor 10.1.1.2 update-source GigabitEthernet2.197
 neighbor 10.1.1.2 version 4
 neighbor 10.1.1.2 timers 5 15
 neighbor 10.1.1.2 activate
 neighbor 10.1.1.2 send-community both
 neighbor 10.1.1.2 soft-reconfiguration inbound
exit-address-family
!
address-family ipv6 vrf blue
 network 6666:6666::1/128
 neighbor AAAA:AAAA::1 remote-as 65000
 neighbor AAAA:AAAA::1 activate
 neighbor AAAA:AAAA::1 send-community both
 neighbor AAAA:BBBB::2 remote-as 65000
 neighbor AAAA:BBBB::2 update-source GigabitEthernet2.197
 neighbor AAAA:BBBB::2 timers 5 15
 neighbor AAAA:BBBB::2 activate
 neighbor AAAA:BBBB::2 send-community both
 neighbor AAAA:BBBB::2 soft-reconfiguration inbound
exit-address-family
!

Looking at some common BGP commands on the CSR-1Kv I can see that BGP v4 and v6 neighbours are up!
Note: here i’m receiving prefixes because my lab is already finished end-to-end at this point you shouldn’t be seeing any prefixes (although you could go to the T0 VRF and configure a loopback and advertise that).

CSR-1KV#show bgp vrf blue all sum
For address family: IPv4 Unicast


For address family: IPv6 Unicast


For address family: VPNv4 Unicast
BGP router identifier 192.168.100.194, local AS number 65100
BGP table version is 8, main routing table version 8
2 network entries using 512 bytes of memory
3 path entries using 384 bytes of memory
3/2 BGP path/bestpath attribute entries using 840 bytes of memory
1 BGP AS-PATH entries using 40 bytes of memory
2 BGP extended community entries using 80 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 1856 total bytes of memory
1 received paths for inbound soft reconfiguration
BGP activity 16/12 prefixes, 27/21 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.1.1.2        4        65000    1992    1948        8    0    0 02:45:48        1

For address family: VPNv6 Unicast
BGP router identifier 192.168.100.194, local AS number 65100
BGP table version is 5, main routing table version 5
2 network entries using 560 bytes of memory
3 path entries using 468 bytes of memory
3/2 BGP path/bestpath attribute entries using 840 bytes of memory
1 BGP AS-PATH entries using 40 bytes of memory
2 BGP extended community entries using 80 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 1988 total bytes of memory
1 received paths for inbound soft reconfiguration
BGP activity 16/12 prefixes, 27/21 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
AAAA:BBBB::2    4        65000    1992    1947        5    0    0 02:45:48        1

Step 6 – Build the T0 Uplink (towards the fabric)

Here we are using a VLAN uplink (in my case VL102) towards the fabric – in my cases the T0 Edge Node and Cisco ASR9000 are connected on the same segment – in most cases this will NOT be the case thus the use of eBGP MultiHop is possible for the VXLAN use case.
Remember this is just a VLAN for the uplink, the VXLAN is IP routed and NOT L2 switched.

Right now, this lab is set only to use a single ASR9000, so in this case i’ve bound this segment to NSX-Edge Node 01, if you wanted a redundant uplink you would build a second uplink to NSX-Edge Node 02 and obviously give that a different IP address.

Step 7 – Set the eVPN TEP address between (Parent) Tier-0 and the DCGW (Cisco XRv in this lab)

First lets go back to the parent T0 gateway (not the VRF), edit this and you can set the eVPN tunnel endpoint, in my case i am just using the same address as the interface, although you could use an altogether different address (more on this shortly).

Note: if you set the tunnel endpoint to a different IP address, then you need to make sure this is reachable from the DCGW, effectively converting the MP-eBGP session into a multi-hop session
(which is fully supported), in order to do this you will see a new option to advertise eVPN TEP address across the IPv4 portion of the MP-BGP session between the T0 and the DCGW, under route-redistribution (which is a little different in 3.0) you will see an option to advertise the TEP address (here as i’m using the interface address i dont need to do this).

Step 8 – Configuring MP-eBGP on the Tier 0

First lets take a look at regular BGP configuration, nothing special – we already configured the BGP AS, configure ECMP (if you want to have multipath), Inter-SR iBGP allows for an iBGP session to be established between the Edge Nodes for uplink resiliency (may be important if your VM is not dual-connected to both edge nodes).

Once this is created, go to the main T0 and select this pool under the eVPN settings, if you don’t do this you will have an issue creating your T0 VRF later, also enable BGP and allocate your BGP Local-AS , but do not add any neighbours (yet)…

What’s important is to configure the Route-Filter, click on this and now we can see a new L2VPN_EVPN address family that we need to enable to ensure that the eVPN address family is configured.

Voila! By this point if everything is good you should have an MP-eBGP session between your T0 and the ASR9k, wait whats that you say how do I configure the ASR9K, oh yeah…

Step 9 – Configure your ASR9000 (or Xrv Node)

So lets take a little look at the ASR9K configuration and go through this is some detail so we can see what each section is doing, first we need to build our VRFs on the ASR9k, I’ve been very creative and used the same VRF name here throughout (keep it simple), note here we need the stitching keyword to redistribute prefixes between the eVPN and the IP BGP (VRF).

vrf blue
 address-family ipv4 unicast
  import route-target
   65000:120 stitching
  !
  export route-target
   65000:120 stitching
  !
 !
 address-family ipv6 unicast
  import route-target
   65000:120
   65000:120 stitching
  !
  export route-target
   65000:120
   65000:120 stitching
  !
 !
!

For good measure i aso added a loopback (to test if this prefix makes it all the way through to the VM we configured earlier)

interface Loopback66
 vrf blue
 ipv4 address 65.65.65.1 255.255.255.0
 ipv6 address 7777:7777::1/64
!

The interface towards the T0 in my lab is Gig 0/0/0/0 with a simple configuration:

interface GigabitEthernet0/0/0/0
 description VM NetworkAdapter4
 mtu 9000
 ipv4 address 192.168.102.243 255.255.255.0
!

Next i need some new constructs on the ASR-9k a BVI and and NVE interface, the NVE is the Network Virtualisation interface, we also need to configure the L2VPN section.

interface BVI100
 host-routing
 ipv4 address 22.222.22.1 255.255.255.0     <-- Can use any IP here.
!
interface nve1
 member vni 120020
  vrf blue
  host-reachability protocol bgp
 !
 overlay-encapsulation vxlan
 source-interface Loopback0
!
l2vpn
 bridge group blue
  bridge-domain blue
   routed interface BVI100
   !
   member vni 120020
   !
  !
 !

What we are doing here on the interface nve1 is creating a VXLAN (in this case 120020), you will note this matches the eVPN tunnel-ID that we configured back in step 3, we are binding this VXLAN segment to VRF Blue and using BGP for the reachability, we are also telling the VNI construct that we want to use VXLAN as our overlay, and source the tunnels from loopback0.

Lastly under L2VPN (which is traditionally used for different configurations) we build a bridge-group + domain and use the routed BVI interface and bind this again to VNI 120020

If you remember IOS-XR, without configuring BGP policies no routes are allowed ingress or egress, so first lets not forget to make a simple policy:

route-policy IN
  pass
end-policy

Lastly, we have the BGP configuration, this is a little longer than we have seen before, so i’ll try to comment on the relevant parts inline..

router bgp 3301
 bgp router-id 44.44.44.1
 address-family ipv4 unicast
  network 44.44.44.1/32                                  <--- Advertise our Lo0 IP via IP4 AF, this is needed as this is the source of the ASR9k TEP.
 !
 address-family vpnv4 unicast
  advertise best-external
 !
 address-family ipv6 unicast
 !
 address-family vpnv6 unicast
 !
 address-family l2vpn evpn
 !
 neighbor 192.168.102.240                                 <-- This is the T0 neighbour
  remote-as 65000                                         <--- The T0 BGP AS
  ebgp-multihop 10
  address-family ipv4 unicast
   route-policy IN in
   route-policy IN out                                    <--- Lets not forget those route-policies
  !
  address-family l2vpn evpn                               <--- Enabling L2VPN EvPN AF
   import stitching-rt re-originate                       <--- Import eVPN prefixes with Stitching RT
   send-community-ebgp
   route-policy IN in
   encapsulation-type vxlan                               <--- This is VERY important, without this line I was not able to see BGP prefixes being sent to the NSX T0
   route-policy IN out
   send-extended-community-ebgp
   advertise vpnv4 unicast re-originated stitching-rt     <--- 
   advertise vpnv6 unicast re-originated stitching-rt     <--- Advertise our V4 and V6 prefixes
   soft-reconfiguration inbound always
  !
 !
 vrf blue                                                 <--- Remember the VRF Blue configuration
  rd 65000:120                                            <--- Set the VRF RD
  address-family ipv4 unicast                             <--- Enable IPv4
   advertise best-external
   maximum-paths ebgp 4
   redistribute connected metric 1 route-policy IN
  !
  address-family ipv6 unicast                            <--- Enable IPv6
   redistribute connected route-policy IN
  !
 !
!

At this point we should be set and everything should be working … lets have a look at some troubleshooting commands on the ASR9K and the NSX-T Edge Node:

// Lets check the interface NVE and its configuration 

interface nve1
 member vni 120020
  vrf blue
  host-reachability protocol bgp
 !
 overlay-encapsulation vxlan
 source-interface Loopback0
!

// Lets also make sure its UP!

RP/0/RP0/CPU0:ASR9000-7.1.1#show int nve 1
Sun Apr 19 18:18:09.849 UTC
nve1 is up, line protocol is not ready
  Interface state transitions: 1
  Hardware is Overlay
  Internet address is Unknown
  MTU 1500 bytes, BW 0 Kbit
     reliability Unknown, txload Unknown, rxload Unknown
  Encapsulation Unknown(0),  loopback not set,
  Last link flapped 3d11h
  Last input Unknown, output Unknown
  Last clearing of "show interface" counters Unknown
  Input/output data rate is disabled.

// Lets check the VNI (VXLAN) mapping and make sure that its also UP

RP/0/RP0/CPU0:ASR9000-7.1.1#show nve vni
Sun Apr 19 18:19:20.450 UTC
Interface  VNI          MCAST        VNI State        Mode
nve1       120020       0.0.0.0      Up               L3 Control

// If everything is working you should also see a NVE peer to the NSX-T0

RP/0/RP0/CPU0:ASR9000-7.1.1#show nve peers
Sun Apr 19 18:19:48.438 UTC
Interface  Peer-IP        Local VNI  Output VNI Peer-MAC        Mode      Flags
nve1       192.168.102.240   120020     120020     0250.5600.0001  control   0xc

//We should also see a L2VPN BGP peering established with the NSX-T0, NOTE: here we only show one peer //because we only configured the T0 on Edge Node 01, if we had allocated an uplink / BGP Neighbour to Edge //Node 02 we would see two. !

RP/0/RP0/CPU0:ASR9000-7.1.1#show bgp l2vpn evpn summary
Sun Apr 19 18:20:24.705 UTC
BGP router identifier 44.44.44.1, local AS number 3301
BGP generic scan interval 60 secs
BGP table state: Active
Table ID: 0x0   RD version: 0
BGP main routing table version 44
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.


Process       RcvTblVer   bRIB/RIB   LabelVer  ImportVer  SendTblVer  StandbyVer
Speaker              44         44         44         44          44          44

Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
192.168.102.240   0 65000    5025    5094       44    0    0 02:49:55          2

// Here we also see we are receiving two prefixes, so lets take a look:

RP/0/RP0/CPU0:ASR9000-7.1.1#show bgp l2vpn evpn
Sun Apr 19 18:21:17.035 UTC
BGP router identifier 44.44.44.1, local AS number 3301
BGP generic scan interval 60 secs
BGP table state: Active
Table ID: 0x0   RD version: 0
BGP main routing table version 44
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65000:100
*> [5][0][32][11.11.11.11]/80
                      192.168.102.240          0             0 65000 65100 i
*> [5][0][128][6666:6666::1]/176
                      192.168.102.240          0             0 65000 65100 i

Processed 2 prefixes, 2 paths

//So, we see the two prefixes (one v4, one v6), the NH is 192.168.102.240 (the IPv4 peer address of the NSX //T0 on Edge Node 01.

//What you might find interesting is that at this point we are NOT seeing the local prefix of VRF Blue in the /eVPN table, but I can assure  you that it's being
//sent tothe NSX-T0. Even if you check the ASRS9K advertised prefixes it will show a count of 0! but lets check on NSX.

nsx-edge01>
nsx-edge01> get logical-routers
Logical Router
UUID                                   VRF    LR-ID  Name                              Type                        Ports
736a80e3-23f6-5a2d-81d6-bbefb2786666   0      0                                        TUNNEL                      3    
b2e75c32-5cb2-4cc6-91ac-55421daf1716   1      5      SR-Main T0 GW                     SERVICE_ROUTER_TIER0        10   
bb9b07dc-8f8d-43b2-a7e3-417492890676   5      31     SR-VRF-VRFBlue                    VRF_SERVICE_ROUTER_TIER0    6

// Lets switch to the main T0 SR construct (also called VRF - although this is an internal VRF construct no the RF we have just configured).

nsx-edge01> vrf 1

// Lets check BGP eVPN

nsx-edge01(tier0_sr)> get bgp evpn
BGP table version is 147, local router ID is 10.1.1.60
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65000:100
*> [5]:[0]:[32]:[11.11.11.11]
                    192.168.102.240          0             0 65100 i
*> [5]:[0]:[128]:[6666:6666::1]
                    192.168.102.240          0             0 65100 i
Route Distinguisher: 65000:120
*> [5]:[0]:[24]:[65.65.65.0]
                    44.44.44.1               1             0 3301 ?
*> [5]:[0]:[64]:[7777:7777::]
                    44.44.44.1               0             0 3301 ?

Displayed 4 prefixes (4 paths)

Here we’re seeing not 2 but 4 prefixes in the eVPN table, you can tell from the AS_PATH that two come from AS#65100, thats the BGP AS of our CSR-1K VM and two come from AS#3301, thats the loopback IP prefixes from the ASR9k.

Lets just have a little peek at the detail of some of these prefixes, its more descriptive getting this from the ASR than NSX-T, so lets have a look at a prefix that we receive from NSX-T.

RP/0/RP0/CPU0:ASR9000-7.1.1#show bgp l2vpn evpn rd 65000:100 [5][0][32][11.11.11.11]/80
Sun Apr 19 18:25:29.216 UTC
BGP routing table entry for [5][0][32][11.11.11.11]/80, Route Distinguisher: 65000:100
Versions:
  Process           bRIB/RIB  SendTblVer
  Speaker                 43          43
Last Modified: Apr 19 15:30:30.408 for 02:54:58
Paths: (1 available, best #1)
  Not advertised to any peer
  Path #1: Received by speaker 0
  Not advertised to any peer
  65000 65100, (received & used)
    192.168.102.240 from 192.168.102.240 (10.1.1.60)
      Received Label 120020
      Origin IGP, metric 0, localpref 100, valid, external, best, group-best, import-candidate, reoriginate, not-in-vrf
      Received Path ID 0, Local Path ID 1, version 43
      Extended community: Flags 0x6: Encapsulation Type:8 Router MAC:0250.5600.0001 RT:65000:120
      EVPN ESI: 0000.0000.0000.0000.0000, Gateway Address : 0.0.0.0

So, looking at this prefix (and referencing back to the RFC highlighted earlier), we can see the following:

• Received Label: 120020 – This is the VXLAN VNI that we configured on both the T0 and the ASR
• EVPN ESI is all zero’s indicating this is NOT an overlay index but a regular T5 prefix
• a GW IP of all zeros also indicating this is not an overlay index.
• RT’s are set as per advertisment from the NSX-T T0
• RMAC is that of the VXLAN construct on the NSX-T Edge node

nsx-edge01(tier0_sr)> get vrf vni
VRF                                   VNI        VxLAN IF             L3-SVI               State Rmac
VRF-30                                120020     vxlan-120020         br-120020            Up    02:50:56:00:00:

Note: Future models may set the ESI and GWIP, but thats a story for the future, the overlay index allows for a recursive lookup, that would be to use the GW-IP as the NH for the VXLAN TEP instead of the BGP NH (as used in this example).

Thats it ! Hopefully you found this interesting, please let me know!