Blog

Junos OS route-reflection considerations

April 19, 2016 10:17 am

So you’ve decided on route-reflection being your BGP scaling mechanism of choice. Everything is setup and the provide edge (pe) routers are exchanging VPN routing information with an out of path route-reflector (rr) like this:

inetzero-rr-1

In the above scenario, traffic is never forwarded through the rr. To be able to setup BGP sessions with all of the other pe routers, the rr is running OSPF with the rest of the network. Let’s check the inet.0 table on the rr:

 

inetzero@rr> show route 192.168.1.0/29

inet.0: 469 destinations, 776 routes (402 active, 0 holddown, 373 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.1.1/32     *[OSPF/10] 00:44:57, metric 2
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.2/32     *[OSPF/10] 00:44:57, metric 3
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.3/32     *[OSPF/10] 00:44:57, metric 3
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.4/32     *[OSPF/10] 00:44:57, metric 4
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.5/32     *[OSPF/10] 00:44:57, metric 2
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.6/32     *[OSPF/10] 00:44:57, metric 4
                    > to 172.16.1.10 via xe-0/2/0.1518

We can see that the rr has a route towards the loopback IP address all of the pe routers. BGP sessions between the rr and the pe routers have already been established:

 

inetzero@rr> show bgp summary | match Establ
192.168.1.1           65000       2675       3831       0       0    20:12:58 Establ
192.168.1.2           65000       2668       3833       0       0    20:12:55 Establ
192.168.1.3           65000       2872       3784       0       0    20:12:57 Establ
192.168.1.4           65000       2710       3771       0       0    20:12:48 Establ
192.168.1.5           65000       2910       3147       0       0    20:12:58 Establ
192.168.1.6           65000       2667       3800       0       0    20:12:54 Establ

The out of path rr is not involved in any label distribution leaving the inet.3 table completely empty:

inetzero@rr> show route table inet.3

inetzero@rr>

Looks like we’re all set. However, when we look at what happens when pe1 advertises a subnet for the c1 routing-instance, we can see there is not a lot of route-reflection going on:

inetzero-rr-2

 

inetzero@pe1> show route advertising-protocol bgp 172.16.0.1 table c1.inet.0 192.168.50.0/24

c1.inet.0: 10 destinations, 16 routes (10 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
* 192.168.50.0/24         Self                         100        I

 

In the output above, we can see that the pe1 router is advertising a route for a routing-instance named ‘c1’. The source of the routing update is its own loopback IP address. On the rr, we can observe the following:

inetzero@rr> show route receive-protocol bgp 192.168.1.1 192.168.50.0/24

bgp.l3vpn.0: 106 destinations, 106 routes (0 active, 0 holddown, 106 hidden)

inetzero@rr>

We verified that the pe1 router has send the routing information towards the rr. Why is it then that we aren’t seeing anything on the rr?

 

inetzero@rr> show route receive-protocol bgp 192.168.1.1 192.168.50.0/24 hidden

bgp.l3vpn.0: 106 destinations, 106 routes (0 active, 0 holddown, 106 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  192.168.1.1:176:192.168.50.0/24
                          192.168.1.1                  100        I

So the route is hidden.

The bgp.l3vpn.0 table contains all IPv4 VPN routes, and by default, these routes are resolved in the inet.3 table. Since our rr is not involved in any label distribution protocol, it is unable to resolve the received routing updates. Routes that cannot be resolved are hidden and not reflected by our rr. This leaves us with a problem.

What we want to accomplish is a situation wherein the route-reflector actually reflects the routes. The Junos OS offers us various options to make this happen. Let’s discuss and examine the following solutions to our problem: enabling a label distribution protocol, configuring static routes, changing the route resolution and using a rib-group.

Using a Label distribution protocol.

 

The fact that the rr is out of path means that the rr doesn’t need to be burdened with a label distribution protocol. This doesn’t mean that it cannot.

inetzero-rr-3

We can fill the inet.3 table by simply enabling RSVP signaled LSPs towards all the pe’s or by having the route-reflector participate in LDP.

Let’s go with LDP and do a quick ‘set protocols mpls interface all’ and ‘set protocols ldp interface all’. In addition to this, we also configure the mpls address family on the relevant interfaces. After doing so, we can verify that we learned all the necessary LDP routes like this:

 

inetzero@rr> show route protocol ldp 192.168.1.0/29

inet.0: 470 destinations, 777 routes (403 active, 0 holddown, 373 hidden)

inet.3: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.1.1/32     *[LDP/9] 00:00:21, metric 1
                    > to 172.16.1.10 via xe-0/2/0.1518, Push 299792
192.168.1.2/32     *[LDP/9] 00:00:21, metric 1
                    > to 172.16.1.10 via xe-0/2/0.1518, Push 299808
192.168.1.3/32     *[LDP/9] 00:00:21, metric 1
                    > to 172.16.1.10 via xe-0/2/0.1518, Push 299904
192.168.1.4/32     *[LDP/9] 00:00:21, metric 1
                    > to 172.16.1.10 via xe-0/2/0.1518, Push 299840
192.168.1.5/32     *[LDP/9] 00:00:21, metric 1
                    > to 172.16.1.10 via xe-0/2/0.1518, Push 299888
192.168.1.6/32     *[LDP/9] 00:00:21, metric 1
                    > to 172.16.1.10 via xe-0/2/0.1518, Push 299936

Since the routes towards all the pe routers are now present in the inet.3 table, the rr should have no problem reflecting the c1 route to pe6. Let’s verify this by issuing the following command:

 

inetzero@rr> show route advertising-protocol bgp 192.168.1.6 192.168.50.0/24

bgp.l3vpn.0: 106 destinations, 106 routes (106 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  192.168.1.1:176:192.168.50.0/24
*                         192.168.1.1                  100        I

Even though this will work just fine, this option does require the route-reflector to exchange additional routing information with the rest of the network. Let’s look at other ways to achieve our goal without introducing an additional burden like this on the route-reflector.

 

Using a static route to populate the inet.3 table.

 

inetzero-rr-4

Instead of turning on LDP, another quick and easy fix could be a static route. Instead of configuring a static route for the usual inet.0 table, we can configure one specifically for the inet.3 table like this:

set routing-options rib inet.3 static route 192.168.1.1/32 reject

After configuring this route, we can observe the following:

inetzero@rr> show route 192.168.1.0/29

inet.0: 469 destinations, 776 routes (402 active, 0 holddown, 373 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.1.1/32     *[OSPF/10] 00:49:30, metric 2
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.2/32     *[OSPF/10] 00:49:30, metric 3
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.3/32     *[OSPF/10] 00:49:30, metric 3
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.4/32     *[OSPF/10] 00:49:30, metric 4
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.5/32     *[OSPF/10] 00:49:30, metric 2
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.6/32     *[OSPF/10] 00:49:30, metric 4
                    > to 172.16.1.10 via xe-0/2/0.1518

inet.3: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.1.1/32     *[Static/5] 00:00:05
                      Reject

A static route is placed inside the inet.3 table. This enables the rr to resolve all the pe1 VPN routes and reflect them towards the pe6 router:

 

inetzero@rr> show route advertising-protocol bgp 192.168.1.6 192.168.50.0/24

bgp.l3vpn.0: 106 destinations, 106 routes (106 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  192.168.1.1:176:192.168.50.0/24
*                         192.168.1.1                  100        I

Since this would only make the rr resolve pe1, we could install a route for a subnet containing all the pe routers, like so:

set routing-options rib inet.3 static route 192.168.1.0/29 reject

In this case, a reject route was created. Discard or receive would have worked all the same. The difference is that a discard route will not elicit a response from the routing-engine and the receive option will install a route towards the routing-engine. Since the rr does not need to use the inet.3 route to forward traffic, the type of static does not really matter.

 

Altering the default route resolution.

 

After removing the static route from the rr, the initial problem is back. Let’s solve the problem by instructing the router to simply make use of the PE addresses it already learned. The rr is participating in a single area OSPF with the rest of the network. Wouldn’t it be easy if we could just tell the rr something like  ‘look at your inet.0 table, if you can resolve it there it’s OK to reflect it’.

inetzero-rr-5

Well, with Junos OS, you can!

Telling the rr to start using inet.0 for VPN routes could be done in the following way:

 

set routing-options resolution rib bgp.l3vpn.0 resolution-ribs inet.0

This statement tells the router that routing-information from the bgp.l3vpn.0 table is to be resolved in the inet.0 table. Now, even though our inet.3 table is empty, the VPN route is reflected by the rr:

inetzero@rr> show route table inet.3

inetzero@rr> show route receive-protocol bgp 192.168.1.1 192.168.50.0/24

bgp.l3vpn.0: 106 destinations, 106 routes (106 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  192.168.1.1:176:192.168.50.0/24
*                         192.168.1.1                  100        I

With the current configuration, the rr will use all information inside the inet.0 table. To narrow it down, we could contemplate the introduction of a policy that limits the resolution to a specific subnet, like this:

set policy-options policy-statement pe-lo0 term pe-loopbacks from route-filter 192.168.1.0/29 prefix-length-range /32-/32
set policy-options policy-statement pe-lo0 term pe-loopbacks then accept
set policy-options policy-statement pe-lo0 term reject-rest then reject

set routing-options resolution rib bgp.l3vpn.0 resolution-ribs inet.0
set routing-options resolution rib bgp.l3vpn.0 import pe-lo0

The end result can be verified by using the following command:

inetzero@rr> show route resolution table bgp.l3vpn.0
Tree Index: 6, Nodes 6, Reference Count 2
Contributing routing tables: inet.0
Policy: [ pe-lo0 ]
192.168.1.1/32 Originating RIB: inet.0
  Metric: 2 Node path count: 1
  Forwarding nexthops: 1
192.168.1.2/32 Originating RIB: inet.0
  Metric: 3 Node path count: 1
  Forwarding nexthops: 1
192.168.1.3/32 Originating RIB: inet.0
  Metric: 3 Node path count: 1
  Forwarding nexthops: 1
192.168.1.4/32 Originating RIB: inet.0
  Metric: 4 Node path count: 1
  Forwarding nexthops: 1
192.168.1.5/32 Originating RIB: inet.0
  Metric: 2 Node path count: 1
  Forwarding nexthops: 1
192.168.1.6/32 Originating RIB: inet.0
  Metric: 4 Node path count: 1
  Forwarding nexthops: 1

 

Copy information using a rib-group.

 

There is yet another option that Junos OS gives us. We can make our route reflector resolve routes by using rib-groups. Rib-groups can be used to place routing information from a single source into a group of tables. The use of rib-groups can quickly escalate the complexity of an infrastructure, but it’s a pretty powerful tool nevertheless.

In this specific scenario, by using rib-groups, we can make the Junos OS place OSPF routes into the inet.0 AND into the inet.3 table:

inetzero-rr-6

In order to make this happen, we have to alter the configuration in two separate places. First, we have to create the rib-group under the [ routing-options ] stanza. Second is the application of the rib-group under the [ protocols ospf ] stanza.

The configuration of the rib-group can be done as follows:

set routing-options rib-groups inet0-to-inet3 import-rib inet.0
set routing-options rib-groups inet0-to-inet3 import-rib inet.3

The rib-group will make sure that routes, normally placed in the inet.0 table, will be placed into both the inet.0 as well as the inet.3 table. For this reason, the inet.0 table is configured first.

The second part of the required configuration is the application of the rib-group. This is configured in the configuration stanza where the routes would normally be learned. In our case, this means that we will apply the rib-group under the OSPF stanza, like this:

set protocols ospf rib-group inet0-to-inet3

After committing this configuration, we can observe the result by consulting the inet.3 table:

inetzero@rr> show route table inet.3 terse

inet.3: 37 destinations, 37 routes (37 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

A V Destination        P Prf   Metric 1   Metric 2  Next hop        AS path
* ? 10.0.0.0/30        O  10          2            >172.16.1.10
* ? 10.0.0.4/30        O  10          2            >172.16.1.10
* ? 10.0.0.8/30        O  10          3            >172.16.1.10
* ? 10.0.0.12/30       O  10          3            >172.16.1.10
* ? 10.0.0.16/30       O  10          4            >172.16.1.10
* ? 10.0.0.20/30       O  10          4            >172.16.1.10
* ? 10.0.0.24/30       O  10          3            >172.16.1.10
* ? 10.1.0.0/30        O  10          2            >172.16.1.10
* ? 10.1.0.4/30        O  10          3            >172.16.1.10
* ? 10.1.0.8/30        O  10          4            >172.16.1.10
* ? 10.1.0.12/30       O  10          3            >172.16.1.10
* ? 10.1.0.16/30       O  10          3            >172.16.1.10
* ? 10.1.0.20/30       O  10          4            >172.16.1.10
* ? 10.1.0.24/30       O  10          4            >172.16.1.10
* ? 10.1.0.28/30       O  10          5            >172.16.1.10
* ? 10.1.0.32/30       O  10          5            >172.16.1.10
* ? 10.1.0.36/30       O  10          4            >172.16.1.10
* ? 10.1.0.40/30       O  10          2            >172.16.1.10
* ? 10.1.0.44/30       O  10          3            >172.16.1.10
* ? 10.1.0.48/30       O  10          3            >172.16.1.10
* ? 10.1.0.52/30       O  10          4            >172.16.1.10
* ? 172.16.0.1/32      O  10          0            >lo0.100
* ? 172.16.0.2/32      O  10          5            >172.16.1.10
* ? 172.16.1.0/30      O  10          4            >172.16.1.10
* ? 172.16.1.4/30      O  10          5            >172.16.1.10
* ? 192.168.0.1/32     O  10          1            >172.16.1.10
* ? 192.168.0.2/32     O  10          2            >172.16.1.10
* ? 192.168.0.3/32     O  10          3            >172.16.1.10
* ? 192.168.0.4/32     O  10          2            >172.16.1.10
* ? 192.168.0.5/32     O  10          3            >172.16.1.10
* ? 192.168.0.6/32     O  10          4            >172.16.1.10
* ? 192.168.1.1/32     O  10          2            >172.16.1.10
* ? 192.168.1.2/32     O  10          3            >172.16.1.10
* ? 192.168.1.3/32     O  10          3            >172.16.1.10
* ? 192.168.1.4/32     O  10          4            >172.16.1.10
* ? 192.168.1.5/32     O  10          2            >172.16.1.10
* ? 192.168.1.6/32     O  10          4            >172.16.1.10

Since all we really need in the inet.3 table are the loopback IP addresses of our pe routers, this may be a bit much. Luckily, we can also apply a policy to a rib-group. Let’s create the following policy:

set policy-options policy-statement pe-lo0 term pe-lo0 from route-filter 192.168.1.0/29 prefix-length-range /32-/32
set policy-options policy-statement pe-lo0 term pe-lo0 then accept
set policy-options policy-statement pe-lo0 term reject-rest then reject

 

And apply it to the rib-group like this:

set routing-options rib-groups inet0-to-inet3 import-policy pe-lo0

 

After applying the policy, the inet.3 table looks like this:

inetzero@rr> show route table inet.3

inet.3: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.1.1/32     *[OSPF/10] 00:00:14, metric 2
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.2/32     *[OSPF/10] 00:00:14, metric 3
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.3/32     *[OSPF/10] 00:00:14, metric 3
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.4/32     *[OSPF/10] 00:00:14, metric 4
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.5/32     *[OSPF/10] 00:00:14, metric 2
                    > to 172.16.1.10 via xe-0/2/0.1518
192.168.1.6/32     *[OSPF/10] 00:00:14, metric 4
                    > to 172.16.1.10 via xe-0/2/0.1518

And with the inet.3 table looking like that, we can see that the VPN prefix is reflected once more:

inetzero@rr> show route advertising-protocol bgp 192.168.1.6 192.168.50.0/24

bgp.l3vpn.0: 106 destinations, 106 routes (106 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  192.168.1.1:176:192.168.50.0/24
*                         192.168.1.1                  100        I

 

Nice!

 

The Junos OS has a lot of tools that can be used to manipulate routing and forwarding information. It’s these tools that make Junos a very versatile OS for routing. Hopefully, this article has shed some light on how these tools can be made of use in a scenario with an out of path rr.

Other scenarios challenging you to take it to the next level can be found in the iNET ZERO JNCIE-SP workbook.

Said van de Klundert

Said van de Klundert

Netherlands based networking enthusiast and Juniper Networks ambassador. Has spent most of his career on the service-providers’ side of things. Known to lab up and write about whatever sparked his interest networking wise. Other than that, he is a father to his son, husband to his wife and he enjoys long dinners with friends.

More Posts - Website - Twitter

Categorised in:

  • Blake says:

    Nice article, thanks.

    & if you’re fortunate enough to be using BGP-LU in your backbone, you can just use the “resolve-vpn” knob: http://www.juniper.net/documentation/en_US/junos15.1/topics/reference/configuration-statement/resolve-vpn-edit-protocols-bgp.html

  • Vigor says:

    in this link, I see a command “set routing-options resolution rib inet.0 import POLICY-limit-resolve-routes”. so, what’s different with command “set routing-options resolution rib bgp.l3vpn.0 import pe-lo0” ??
    http://www.juniper.net/documentation/en_US/junos15.1/topics/example/vpns-layer-3-route-resolution-route-reflector.html

  • Victor Boglea says:

    Great article!

    I am just wondering if the import policy name should not be “rib-import” instead of “pe-lo0” in the following line: set routing-options resolution rib bgp.l3vpn.0 import pe-lo0 ?

    Keep up the good work!

  • Victor,

    spot on! Changed the name of the policy-statement name to ‘pe-lo0’. Don’t know exactly what went wrong but my bad.

    Vigor,

    in that link, the things that’s described is the use of a policy to gain more fine grained control. Instead of just resolving the routes in the entire table, only routing information allowed in the policy is used. That’s exactly what I did. But what I did was reference a subnet that contains the loopback IP addresses of all PE routers. This as opposed to specifying a protocol.

    Hopefully things are also a little more clear now that I corrected the policy-name!

    Blake,

    excellent addition.

  • Peter says:

    Hi Said,

    Is there a way to copy routes from inet.0 to inet6.3 in case you don’t want to insert the routes inet.3? I have tried rib-groups but this mapped ipv4 translation is not happening.

    • Hi Peter,
      I do not know of any way to copy and translate routes from inet.0 to inet6.3, sorry.
      I do not think there is such a knob in Junos. The way to populate the inet6.3 table is through the use of ‘protocols mpls ipv6-tunneling’ (copying and translating the inet.3 routes into the inet6.3 table), by configuring static routes or by installing an entry using a static LSP. When you configure a static LSP, you can use the ‘ no-install-to-address’ to make sure that the IPv4 address is not installed into the inet.3 table and you can use the ‘install’ knob to install the (mapped) IPv6 address. I came across this possibility by accident when I was doing some labs. I also intend to leave it just there, in a lab.

    • xinhui jiang says:

      yes, there is example, use v6 route-filter match v4 addresses,it will translate to v6 mapping v4 addresses
      term 2 {
      from {
      route-filter ::/0 prefix-length-range /128-/128;
      }
      then accept;
      }

  • olivier says:

    Hi,

    I try to do the same thing but with VPLS !
    So two PE that want to do VPLS and we use a RR to import/export the route to each PE.
    I use the method :
    – set routing-options resolution rib bgp.rtarget.0 resolution-ribs inet.0
    – set routing-options rib inet.3 static route 1.1.1.1/32 reject

    In both case the RR changing the next-hop :

    The receive of the RR :

    o@RR-A# run show route receive-protocol bgp 1.1.1.1

    inet.0: 17 destinations, 17 routes (17 active, 0 holddown, 0 hidden)

    iso.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)

    mpls.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)

    bgp.rtarget.0: 2 destinations, 3 routes (2 active, 0 holddown, 0 hidden)
    Prefix Nexthop MED Lclpref AS path
    8218:8218:101/96
    * 1.1.1.1 100 I
    8218:8281:100/96
    * 1.1.1.1 100 I

    The advertise of the RR :

    [edit]
    o@RR-A# run show route advertising-protocol bgp 4.4.4.4

    bgp.rtarget.0: 2 destinations, 3 routes (2 active, 0 holddown, 0 hidden)
    Prefix Nexthop MED Lclpref AS path
    8218:8218:101/96
    * Self 100 I
    8218:8281:100/96
    * Self 100 I

    Can you help me on this one ?

    Thanks 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *