Blog

Verifying a BGP signaled VPLS

January 21, 2016 12:40 pm

In a previous post, I walked through the configuration of a VPLS with an L2VPN stitched into it. What I did not do was verify the configuration. This post is about using the Junos OS cli to analyze and verify the BGP signalled VPLS operation.

In the previous post, I did not list any PE name or IP address. Since they matter for verification purposes, let’s briefly re-examine the scenario we previously created:

  verify-vpls-1

 

A total of 5 PE routers are involved in this scenario. LDP and IS-IS are used to establish the LSPs between the PE routers. All PE routers are running BGP with two route-reflectors. The customer locations have layer 2 connectivity between them. Locations c1-s1 and c1-s2 are standard VPLS locations. The location c1-s3 is a multi-homed location that is connected to two PE routers. The location c1-s4 is a location that is connected via an L2VPN. This L2VPN runs between pe52 and pe4. On the pe4 router, the L2VPN is stitched into the VPLS.

In this post, I’ll verify the VPLS mostly from the pe4 router point of view.  I’ll examine the BGP operation and the state of the VPLS connections. At the end of the article, I’ll zoom in on pe1 and pe2 and verify the multihomed site. But before that, let’s quickly verify that there is a full mesh of LSPs established between the PE routers.

 

Verifying the full mesh of LSPs.

 

Let’s not spend too much time verifying the IGP and label distribution. After all, the focus here is on verifying the BGP signalled VPLS. Regardless of the mixture of protocols used ( OSPF/IS-IS with LDP/RSVP), the PE routers will need a full mesh of established LSPs between each other to support the VPLS. The pe4 router will also require an LSP towards the endpoint of the L2VPN connection.

 

To quickly verify whether or not a PE router has a full mesh of LSPs towards all other PE routers, we can consult the ‘inet.3’ table. The inet.3 table, populated by your label distribution protocol of choice, contains routes towards all of the LSPs egress LSRs. By default, the BGP protocol can use this table to resolve the next-hop in any of the received routes.

 

Let’s verify the connectivity from the pe4 router point of view:

  verify-vpls-2

 

 

This output shows us that pe4 has MPLS routes towards all of the other PE routers. The routes are LDP here, but RSVP signalled routing information would be visible using the same command. And since the LSPs are unidirectional, a thorough check involves running the command on all routers.

Additionally, there are the route-reflectors to consider. They need to be able to resolve the next-hop that is advertised by the PE router. If they cannot, there will be very little route-reflection going on.

After making sure that the pe4 router will be able to resolve any reflected VPLS or L2VPN routes, we can move on and continue our verification by checking BGP.

 

Verifying BGP

 

In this scenario, the route-reflectors reflect the PE routers’ routes so that all PE routers have the necessary routing-information. To verify that everything is working as it is supposed to, we can look at a number of different things. Are the BGP sessions configured properly? Are the route-reflectors reflecting routes? Let’s again verify things from the perspective of pe4 and start off with a ‘show bgp summary’:

inetzero@pe4> show bgp summary group rr-cluster
Groups: 3 Peers: 4 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0
                      86         86          0          0          0          0
bgp.l2vpn.0
                       6          3          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
172.16.0.1            65000      52879      53156       0       0 2w2d 18:39:17 Establ
  bgp.l2vpn.0: 3/3/3/0
  c1-vpls.l2vpn.0: 2/2/2/0
  l2vpn-stitch.l2vpn.0: 1/1/1/0
172.16.0.2            65000      52864      53157       0       0 2w2d 18:39:21 Establ
  bgp.l2vpn.0: 0/3/3/0
  c1-vpls.l2vpn.0: 0/2/2/0
  l2vpn-stitch.l2vpn.0: 0/1/1/0

By displaying summary information for the ‘rr-cluster’ group, we are presented with a very compact view of the BGP sessions. This output is telling us several important things. We can see that both sessions are Established and we can see what routing-tables that the pe4 router is currently exchanging with the route-reflectors. By default, a Juniper router will attempt to exchange routing-information for the ‘inet.0 table’. This default behavior changes as soon as another address family is configured. When another address family is configured, the inet address family needs to be configured as well, otherwise, the router will not attempt to exchange routes for this address family.

Since this router is configured with ‘family l2vpn signaling’, only that address family is active.

Furthermore, we can see that the pe4 router is exchanging three different tables.  The ‘bgp.l2vpn.0’ table contains all l2vpn routing-information. The other xx.l2vpn tables contain routing information for that specific VPN only.

 

The command output also indicates how many routes are being received. The fields after the individual tables indicate the active, the received, the accepted, and damped routes. Active indicates how many of the routes the peer has advertised are in use. Received indicated how many routes the peer has send and accepted indicates how many routes have made it through the routing-policy. The last number is an indication as to how many routes are currently dampened.

 

Alternatively, we can look at the individual BGP sessions with the route-reflectors using the following command:

inetzero@pe4> show bgp neighbor 172.16.0.1
Peer: 172.16.0.1+179 AS 65000  Local: 192.168.1.4+49262 AS 65000
  Type: Internal    State: Established    Flags: 
  Last State: OpenConfirm   Last Event: RecvKeepAlive
  Last Error: Cease
  Options: 
  Authentication key is configured
  Address families configured:  l2vpn-signaling
  Local Address: 192.168.1.4 Holdtime: 90 Preference: 170
  Number of flaps: 1
  Last flap event: Stop
  Error: 'Cease' Sent: 1 Recv: 0
  Peer ID: 172.16.0.1      Local ID: 192.168.1.4       Active Holdtime: 90
  Keepalive Interval: 30         Group index: 3    Peer index: 1
  BFD: disabled, down
  NLRI for restart configured on peer: l2vpn
  NLRI advertised by peer: inet-unicast inet-vpn-unicast l2vpn inet6-vpn-unicast
  NLRI for this session: l2vpn
  Peer supports Refresh capability (2)
  Stale routes from peer are kept for: 300
  Peer does not support Restarter functionality
  NLRI that restart is negotiated for: l2vpn
  NLRI of received end-of-rib markers: l2vpn
  NLRI of all end-of-rib markers sent: l2vpn
  Peer supports 4 byte AS extension (peer-as 65000)
  Peer does not support Addpath
  Table bgp.l2vpn.0
    RIB State: BGP restart is complete
    RIB State: VPN restart is complete
    Send state: not advertising
    Active prefixes:              3
    Received prefixes:            3
    Accepted prefixes:            3
    Suppressed due to damping:    0
  Table c1-vpls.l2vpn.0 Bit: 70000
    RIB State: BGP restart is complete
    RIB State: VPN restart is complete
    Send state: in sync
    Active prefixes:              2
    Received prefixes:            2
    Accepted prefixes:            2
    Suppressed due to damping:    0
    Advertised prefixes:          1
  Table l2vpn-stitch.l2vpn.0 Bit: 80000
    RIB State: BGP restart is complete
    RIB State: VPN restart is complete
    Send state: in sync
    Active prefixes:              1
    Received prefixes:            1
    Accepted prefixes:            1
    Suppressed due to damping:    0
    Advertised prefixes:          1
  Last traffic (seconds): Received 22   Sent 15   Checked 41
  Input messages:  Total 68     Updates 7       Refreshes 0     Octets 1805
  Output messages: Total 65     Updates 2       Refreshes 0     Octets 1447
  Output Queue[6]: 0            (c1-vpls.l2vpn.0, l2vpn)
  Output Queue[7]: 0            (l2vpn-stitch.l2vpn.0, l2vpn)

 

If you only want to verify a VPLS, this command basically outputs the same with only 2 real additions: the address families that are configured on both sides and the number of prefixes advertised by the local router.

The advertised routes and received routes can also be examined in further detail. To see what routes a router is receiving from a BGP peer, we can issue the ‘show route receive-protocol bgp x.x.x.x’ command. This command will display what routes the peer has exported to the local router, but it will not show the effect that an import policy may have on the routes.

The command to see what routes are advertised from across a BGP session is the ‘show route advertising-protocol bgp x.x.x.x’ command. With this command, you can see the routes that were exported to a BGP peer after being filtered by the export policy.

Let’s look at some sample output and see what VPLS route the pe4 router is advertising towards one of the route-reflectors:

 

inetzero@pe4> show route advertising-protocol bgp 172.16.0.1 table c1-vpls.l2vpn.0 extensive

c1-vpls.l2vpn.0: 3 destinations, 5 routes (3 active, 0 holddown, 0 hidden)
*  65000:605:1:1/96 (1 entry, 1 announced)
 BGP group rr-cluster type Internal
     Route Distinguisher: 65000:605
     Label-base: 262153, range: 8, offset: 1
     Nexthop: Self
     Flags: Nexthop Change
     Localpref: 100
     AS path: [65000] I
     Communities: target:65000:605 Layer2-info: encaps: VPLS, control flags:[0x0] , mtu: 0, site preference: 100

 

Here we can see exactly what the pe4 router is advertising and we can see exactly what the properties of the route are. This may be a straightforward scenario, but when you are doing something more exotic like route-target normalization, you might need to examine this information as well.

When you are looking at thousands of routes and VPNs on the route-reflector, a nice addition to this command may be the option ‘community’ or ‘community-name’. A route-reflector will not have all the routing-instances configured. So in case you want to see the routes for a specific routing-instance received from a single PE, you could give the following command:

 

inetzero@rr-1> show route receive-protocol bgp 192.168.1.4 community target:65000:605

bgp.l2vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  65000:605:1:1/96
*                         192.168.1.4                  100        I

 

Verifying the state of the VPLS connections.

 

Let’s look at how we can verify the VPLS connection itself from the pe4 router:

 

inetzero@pe4> show vpls connections instance c1-vpls extensive
Layer-2 VPN connections:

Legend for connection status (St)
EI -- encapsulation invalid      NC -- interface encapsulation not CCC/TCC/VPLS
EM -- encapsulation mismatch     WE -- interface and instance encaps not same
VC-Dn -- Virtual circuit down    NP -- interface hardware not present
CM -- control-word mismatch      -> -- only outbound connection is up
CN -- circuit not provisioned    lt-0/2/0.0
    xe-0/3/0.3002
    lsi.84934685    2    Intf - vpls c1-vpls local site 1 remote site 2
    lsi.84934684    3    Intf - vpls c1-vpls local site 1 remote site 3
    Label-base        Offset     Size  Range     Preference
    262153            1          8      8         100
    connection-site           Type  St     Time last up          # Up trans
    2                         rmt   Up     Nov 26 08:05:54 2015           1
      Remote PE: 192.168.1.51, Negotiated control-word: No
      Incoming label: 262154, Outgoing label: 262153
      Local interface: lsi.84934685, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls c1-vpls local site 1 remote site 2
    Connection History:
        Nov 26 08:05:54 2015  status update timer
        Nov 26 08:05:54 2015  loc intf up                 lsi.84934685
        Nov 26 08:05:53 2015  PE route changed
        Nov 26 08:05:53 2015  Out lbl Update                    262153
        Nov 26 08:05:53 2015  In lbl Update                     262154
        Nov 26 08:05:53 2015  loc intf down
    3                         rmt   Up     Nov 26 08:05:54 2015           1
      Remote PE: 192.168.1.2, Negotiated control-word: No
      Incoming label: 262155, Outgoing label: 262177
      Local interface: lsi.84934684, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls c1-vpls local site 1 remote site 3
    Connection History:
        Nov 26 08:05:54 2015  status update timer
        Nov 26 08:05:54 2015  loc intf up                 lsi.84934684
        Nov 26 08:05:53 2015  PE route changed
        Nov 26 08:05:53 2015  Out lbl Update                    262177
        Nov 26 08:05:53 2015  In lbl Update                     262155
        Nov 26 08:05:53 2015  loc intf down


This command tells us quite a bit about the VPLS. For starters, we can see the local interfaces involved in the VPLS. The lt-0/2/0.0 interface is there to stitch in the l2vpn. The xe-0/3/0.3002 interface is an ‘actual’ interface that is used to connect the locally attached site to this PE. The lsi interfaces are a result of the ‘no-tunnel-services’ configuration command ( necessary due to the lack of tunnel services PIC). The lsi interfaces are created dynamically by the MX for every remote site that is active inside a VPLS and it is used to receive packets from remote PE routers. What LSI belongs to what remote site is also listed below the IP address of the remote PE’s involved in the VPLS. This could be relevant information when you are examining the MAC address table for this VPLS.

The output also lists the incoming and the outgoing label, preventing you from having to calculate it. If you really wanted to though, the formula would be as follows:  ‘VPN label = label-base-remote + local-site-id – label-offset-remote ’

Also relevant is the status of the VPLS, which is up in our case. The command also displays the history of every individual pseudowire, revealing the stability of towards the individual remote-PE routers.

On to MAC learning. Inside a VPLS, this is a hardware operation and it doesn’t have anything to do with BGP. The PE creates a separate MAC table for each VPLS. This MAC table can be consulted by issuing the ‘show vpls mac-table command’:

  verify-vpls-3

 

This command, issued on pe 4, is showing us where we learned the various MAC addresses. On pe4, there are 4 ports on which the MX is learning MAC addresses for this customers’ table. The lt-0/2/0.0 is the port on which the pe router is learning MAC addresses from devices that reside within the L2VPN. The xe-0/3/0.3002 port is a port that is locally active inside the routing-instance and the LSI ports are dynamically created ports for each remote VPLS endpoint.

 

Verifying the multihomed location.

 

One last thing I wanted to do is verify the multihomed location.

verify-vpls-4

 

The c1-s3 CPE is connected to both pe1 and pe2. The pe2 router is configured with ‘multi-homing’ and ‘site-preference primary’. Let’s check how things are after we configured both PE routers:

 

inetzero@pe2> show vpls connections instance c1-vpls

..

Instance: c1-vpls
Edge protection: Not-Primary
  Local site: c1-s3 (3)
    connection-site           Type  St     Time last up          # Up trans
    1                         rmt   Up     Nov 26 08:05:54 2015           1
      Remote PE: 192.168.1.4, Negotiated control-word: No
      Incoming label: 262177, Outgoing label: 262155
      Local interface: lsi.219152413, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls c1-vpls local site 3 remote site 1
    2                         rmt   Up     Nov 25 21:25:31 2015           1
      Remote PE: 192.168.1.51, Negotiated control-word: No
      Incoming label: 262178, Outgoing label: 262155
      Local interface: lsi.219152412, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls c1-vpls local site 3 remote site 2

inetzero@pe1> show vpls connections instance c1-vpls

..

Instance: c1-vpls
Edge protection: Not-Primary
  Local site: c1-s3 (3)
    connection-site           Type  St     Time last up          # Up trans
    1                         rmt   LN
    2                         rmt   LN
    3                         rmt   LN

 

This looks fine. The pe2 router sees the other endpoints of the VPLS as UP. The pe1 router lists connections to the other sites as LN – Local site not designated. Let’s look at the reason why the local site did not establish any pseudowires by examining the route-advertisements from both PE routers:

inetzero@pe2> show route advertising-protocol bgp 172.16.0.1 table c1-vpls.l2vpn.0 detail

c1-vpls.l2vpn.0: 3 destinations, 5 routes (3 active, 0 holddown, 0 hidden)
*  65000:605:3:1/96 (1 entry, 1 announced)
 BGP group rr-cluster type Internal
     Route Distinguisher: 65000:605
     Label-base: 262177, range: 8, offset: 1
     Nexthop: Self
     Flags: Nexthop Change
     Localpref: 65535
     AS path: [65000] I
     Communities: target:65000:605 Layer2-info: encaps: VPLS, control flags:[0x0] , mtu: 0, site preference: 65535

inetzero@pe1> show route advertising-protocol bgp 172.16.0.1 table c1-vpls.l2vpn.0 detail

{master}
inetzero@pe1>

 

We can see that the primary site advertises a route with a local preference of 65535 and we can see that pe1 is not advertising anything. Let’s see what happens to the route advertisements on pe1 when we configure ‘deactivate routing-instances c1-vpls’ on pe2:

 

inetzero@pe1> show route advertising-protocol bgp 172.16.0.1 table c1-vpls.l2vpn.0 detail

c1-vpls.l2vpn.0: 3 destinations, 5 routes (3 active, 0 holddown, 0 hidden)
*  65000:605:3:1/96 (1 entry, 1 announced)
 BGP group rr-cluster type Internal
     Route Distinguisher: 65000:605
     Label-base: 262209, range: 8, offset: 1
     Nexthop: Self
     Flags: Nexthop Change
     Localpref: 1
     AS path: [65000] I
     Communities: target:65000:605 Layer2-info: encaps: VPLS, control flags:[0x0] , mtu: 0, site preference: 1

inetzero@pe1> show vpls connections instance c1-vpls
Layer-2 VPN connections:

..

Instance: c1-vpls
Edge protection: Not-Primary
  Local site: c1-s3 (3)
    connection-site           Type  St     Time last up          # Up trans
    1                         rmt   Up     Nov 26 11:38:33 2015           1
      Remote PE: 192.168.1.4, Negotiated control-word: No
      Incoming label: 262209, Outgoing label: 262155
      Local interface: lsi.252706837, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls c1-vpls local site 3 remote site 1
    2                         rmt   Up     Nov 26 11:38:33 2015           1
      Remote PE: 192.168.1.51, Negotiated control-word: No
      Incoming label: 262210, Outgoing label: 262155
      Local interface: lsi.252706838, Status: Up, Encapsulation: VPLS
        Description: Intf - vpls c1-vpls local site 3 remote site 2

 

The route that was being advertised by the pe2 router is withdrawn. After this, the pe1 router starts advertising its own route. This route is exactly the same, except for 1 thing: the local preference. The backup site has a local-preference of 1.

There are plenty of other commands at your disposal and there are many ways to go about verifying whether or not a VPLS is working. Hopefully this walkthrough gave you some new insight or ideas related to the verification of a VPLS on a Juniper MX.

Said van de Klundert

Said van de Klundert

Netherlands based networking enthusiast and Juniper Networks ambassador. Has spent most of his career on the service-providers’ side of things. Known to lab up and write about whatever sparked his interest networking wise. Other than that, he is a father to his son, husband to his wife and he enjoys long dinners with friends.

More Posts - Website - Twitter

Categorised in:

  • NgocND says:

    Hi Said,

    Seemly you had a mistake here: “We can see that the primary site advertises a route with a local preference of 65535 and we can see that pe1 is not advertising anything.”. —> I verified in my lab, in normal case, both PRIMARY site and BACKUP site will be advertise the NLRI via BGP, just difference preference value (65535 vs 1).

    on PRIMARY site:
    juniper@HCM-UPE-01-re0# run show route advertising-protocol bgp 10.255.95.1 table VPLS-CUS-1501.l2vpn.0 extensive

    VPLS-CUS-1501.l2vpn.0: 6 destinations, 11 routes (6 active, 0 holddown, 0 hidden)
    * 10.255.64.1:1501:1:1/96 (1 entry, 1 announced)
    BGP group Access type Internal
    Route Distinguisher: 10.255.64.1:1501
    Label-base: 132920, range: 8, offset: 1
    Nexthop: Self
    Flags: Nexthop Change
    Localpref: 65535
    AS path: [18403] I
    Communities: target:18403:1501 Layer2-info: encaps: VPLS, control flags:[0x0] , mtu: 0, site preference: 65535

    [edit]
    juniper@HCM-UPE-01-re0#

    On BACKUP site:
    juniper@HCM-UPE-02-re0# run show route advertising-protocol bgp 10.255.95.1 table VPLS-CUS-1501.l2vpn.0 extensive

    VPLS-CUS-1501.l2vpn.0: 6 destinations, 11 routes (6 active, 0 holddown, 0 hidden)
    * 10.255.64.2:1501:1:1/96 (1 entry, 1 announced)
    BGP group Access type Internal
    Route Distinguisher: 10.255.64.2:1501
    Label-base: 264200, range: 8, offset: 1
    Nexthop: Self
    Flags: Nexthop Change
    Localpref: 1
    AS path: [18403] I
    Communities: target:18403:1501 Layer2-info: encaps: VPLS, control flags:[0x0] , mtu: 0, site preference: 1

    [edit]
    juniper@HCM-UPE-02-re0#

  • When the primary and secondary PE use different route-distinguisher values, the routes are made unique. In that case, the secondary PE router will advertise the route regardless of whether or not it has a copy of the route advertised by the primary PE.

    From your output, I can see that this is the case.

    In the article above, both primary and secondary PE routers are configured to use the same route-distinguisher value (65000:605). This leads to both routes being exactly the same (65000:605:3:1/96). In that case, the secondary PE router will not advertise the route for as long as it sees a superior copy that advertised by the primary PE. It will only start advertising its own route when that one is best.

    I should have emphasized this so it would have been more clear as to why the secondary PE is not advertising anything in the article above.

    Thanks for commenting!

  • MOHIT says:

    Thanks very much 🙂

  • Shreeram Pardhy says:

    hi,
    What is the meaning of the line: “Edge protection: Not-Primary” in the last output that you have shared?

    Regards,
    SP

Leave a Reply

Your email address will not be published. Required fields are marked *