So you’ve decided on route-reflection being your BGP scaling mechanism of choice. Everything is setup and the provide edge (pe) routers are exchanging VPN routing information with an out of path route-reflector (rr) like this:
In the above scenario, traffic is never forwarded through the rr. To be able to setup BGP sessions with all of the other pe routers, the rr is running OSPF with the rest of the network. Let's check the inet.0 table on the rr:
inetzero@rr> show route 192.168.1.0/29
inet.0: 469 destinations, 776 routes (402 active, 0 holddown, 373 hidden)
+ = Active Route, - = Last Active, * = Both
192.168.1.1/32 *[OSPF/10] 00:44:57, metric 2
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.2/32 *[OSPF/10] 00:44:57, metric 3
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.3/32 *[OSPF/10] 00:44:57, metric 3
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.4/32 *[OSPF/10] 00:44:57, metric 4
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.5/32 *[OSPF/10] 00:44:57, metric 2
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.6/32 *[OSPF/10] 00:44:57, metric 4
> to 172.16.1.10 via xe-0/2/0.1518
We can see that the rr has a route towards the loopback IP address all of the pe routers. BGP sessions between the rr and the pe routers have already been established:
inetzero@rr> show bgp summary | match Establ 192.168.1.1 65000 2675 3831 0 0 20:12:58 Establ 192.168.1.2 65000 2668 3833 0 0 20:12:55 Establ 192.168.1.3 65000 2872 3784 0 0 20:12:57 Establ 192.168.1.4 65000 2710 3771 0 0 20:12:48 Establ 192.168.1.5 65000 2910 3147 0 0 20:12:58 Establ 192.168.1.6 65000 2667 3800 0 0 20:12:54 Establ
The out of path rr is not involved in any label distribution leaving the inet.3 table completely empty:
inetzero@rr> show route table inet.3 inetzero@rr>
Looks like we’re all set. However, when we look at what happens when pe1 advertises a subnet for the c1 routing-instance, we can see there is not a lot of route-reflection going on:
inetzero@pe1> show route advertising-protocol bgp 172.16.0.1 table c1.inet.0 192.168.50.0/24 c1.inet.0: 10 destinations, 16 routes (10 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path * 192.168.50.0/24 Self 100 I
In the output above, we can see that the pe1 router is advertising a route for a routing-instance named ‘c1’. The source of the routing update is its own loopback IP address. On the rr, we can observe the following:
inetzero@rr> show route receive-protocol bgp 192.168.1.1 192.168.50.0/24 bgp.l3vpn.0: 106 destinations, 106 routes (0 active, 0 holddown, 106 hidden) inetzero@rr>
We verified that the pe1 router has send the routing information towards the rr. Why is it then that we aren’t seeing anything on the rr?
inetzero@rr> show route receive-protocol bgp 192.168.1.1 192.168.50.0/24 hidden
bgp.l3vpn.0: 106 destinations, 106 routes (0 active, 0 holddown, 106 hidden)
Prefix Nexthop MED Lclpref AS path
192.168.1.1:176:192.168.50.0/24
192.168.1.1 100 I
So the route is hidden.
The bgp.l3vpn.0 table contains all IPv4 VPN routes, and by default, these routes are resolved in the inet.3 table. Since our rr is not involved in any label distribution protocol, it is unable to resolve the received routing updates. Routes that cannot be resolved are hidden and not reflected by our rr. This leaves us with a problem.
What we want to accomplish is a situation wherein the route-reflector actually reflects the routes. The Junos OS offers us various options to make this happen. Let’s discuss and examine the following solutions to our problem: enabling a label distribution protocol, configuring static routes, changing the route resolution and using a rib-group.
Using a Label distribution protocol.
The fact that the rr is out of path means that the rr doesn’t need to be burdened with a label distribution protocol. This doesn’t mean that it cannot.
We can fill the inet.3 table by simply enabling RSVP signaled LSPs towards all the pe’s or by having the route-reflector participate in LDP.
Let’s go with LDP and do a quick ‘set protocols mpls interface all’ and ‘set protocols ldp interface all’. In addition to this, we also configure the mpls address family on the relevant interfaces. After doing so, we can verify that we learned all the necessary LDP routes like this:
inetzero@rr> show route protocol ldp 192.168.1.0/29
inet.0: 470 destinations, 777 routes (403 active, 0 holddown, 373 hidden)
inet.3: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
192.168.1.1/32 *[LDP/9] 00:00:21, metric 1
> to 172.16.1.10 via xe-0/2/0.1518, Push 299792
192.168.1.2/32 *[LDP/9] 00:00:21, metric 1
> to 172.16.1.10 via xe-0/2/0.1518, Push 299808
192.168.1.3/32 *[LDP/9] 00:00:21, metric 1
> to 172.16.1.10 via xe-0/2/0.1518, Push 299904
192.168.1.4/32 *[LDP/9] 00:00:21, metric 1
> to 172.16.1.10 via xe-0/2/0.1518, Push 299840
192.168.1.5/32 *[LDP/9] 00:00:21, metric 1
> to 172.16.1.10 via xe-0/2/0.1518, Push 299888
192.168.1.6/32 *[LDP/9] 00:00:21, metric 1
> to 172.16.1.10 via xe-0/2/0.1518, Push 299936
Since the routes towards all the pe routers are now present in the inet.3 table, the rr should have no problem reflecting the c1 route to pe6. Let’s verify this by issuing the following command:
inetzero@rr> show route advertising-protocol bgp 192.168.1.6 192.168.50.0/24 bgp.l3vpn.0: 106 destinations, 106 routes (106 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 192.168.1.1:176:192.168.50.0/24 * 192.168.1.1 100 I
Even though this will work just fine, this option does require the route-reflector to exchange additional routing information with the rest of the network. Let’s look at other ways to achieve our goal without introducing an additional burden like this on the route-reflector.
Using a static route to populate the inet.3 table.
Instead of turning on LDP, another quick and easy fix could be a static route. Instead of configuring a static route for the usual inet.0 table, we can configure one specifically for the inet.3 table like this:
set routing-options rib inet.3 static route 192.168.1.1/32 reject
After configuring this route, we can observe the following:
inetzero@rr> show route 192.168.1.0/29
inet.0: 469 destinations, 776 routes (402 active, 0 holddown, 373 hidden)
+ = Active Route, - = Last Active, * = Both
192.168.1.1/32 *[OSPF/10] 00:49:30, metric 2
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.2/32 *[OSPF/10] 00:49:30, metric 3
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.3/32 *[OSPF/10] 00:49:30, metric 3
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.4/32 *[OSPF/10] 00:49:30, metric 4
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.5/32 *[OSPF/10] 00:49:30, metric 2
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.6/32 *[OSPF/10] 00:49:30, metric 4
> to 172.16.1.10 via xe-0/2/0.1518
inet.3: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
192.168.1.1/32 *[Static/5] 00:00:05
Reject
A static route is placed inside the inet.3 table. This enables the rr to resolve all the pe1 VPN routes and reflect them towards the pe6 router:
inetzero@rr> show route advertising-protocol bgp 192.168.1.6 192.168.50.0/24 bgp.l3vpn.0: 106 destinations, 106 routes (106 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 192.168.1.1:176:192.168.50.0/24 * 192.168.1.1 100 I
Since this would only make the rr resolve pe1, we could install a route for a subnet containing all the pe routers, like so:
set routing-options rib inet.3 static route 192.168.1.0/29 reject
In this case, a reject route was created. Discard or receive would have worked all the same. The difference is that a discard route will not elicit a response from the routing-engine and the receive option will install a route towards the routing-engine. Since the rr does not need to use the inet.3 route to forward traffic, the type of static does not really matter.
Altering the default route resolution.
After removing the static route from the rr, the initial problem is back. Let’s solve the problem by instructing the router to simply make use of the PE addresses it already learned. The rr is participating in a single area OSPF with the rest of the network. Wouldn’t it be easy if we could just tell the rr something like ‘look at your inet.0 table, if you can resolve it there it’s OK to reflect it’.
Well, with Junos OS, you can!
Telling the rr to start using inet.0 for VPN routes could be done in the following way:
set routing-options resolution rib bgp.l3vpn.0 resolution-ribs inet.0
This statement tells the router that routing-information from the bgp.l3vpn.0 table is to be resolved in the inet.0 table. Now, even though our inet.3 table is empty, the VPN route is reflected by the rr:
inetzero@rr> show route table inet.3 inetzero@rr> show route receive-protocol bgp 192.168.1.1 192.168.50.0/24 bgp.l3vpn.0: 106 destinations, 106 routes (106 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 192.168.1.1:176:192.168.50.0/24 * 192.168.1.1 100 I
With the current configuration, the rr will use all information inside the inet.0 table. To narrow it down, we could contemplate the introduction of a policy that limits the resolution to a specific subnet, like this:
set policy-options policy-statement pe-lo0 term pe-loopbacks from route-filter 192.168.1.0/29 prefix-length-range /32-/32 set policy-options policy-statement pe-lo0 term pe-loopbacks then accept set policy-options policy-statement pe-lo0 term reject-rest then reject set routing-options resolution rib bgp.l3vpn.0 resolution-ribs inet.0 set routing-options resolution rib bgp.l3vpn.0 import pe-lo0
The end result can be verified by using the following command:
inetzero@rr> show route resolution table bgp.l3vpn.0 Tree Index: 6, Nodes 6, Reference Count 2 Contributing routing tables: inet.0 Policy: [ pe-lo0 ] 192.168.1.1/32 Originating RIB: inet.0 Metric: 2 Node path count: 1 Forwarding nexthops: 1 192.168.1.2/32 Originating RIB: inet.0 Metric: 3 Node path count: 1 Forwarding nexthops: 1 192.168.1.3/32 Originating RIB: inet.0 Metric: 3 Node path count: 1 Forwarding nexthops: 1 192.168.1.4/32 Originating RIB: inet.0 Metric: 4 Node path count: 1 Forwarding nexthops: 1 192.168.1.5/32 Originating RIB: inet.0 Metric: 2 Node path count: 1 Forwarding nexthops: 1 192.168.1.6/32 Originating RIB: inet.0 Metric: 4 Node path count: 1 Forwarding nexthops: 1
Copy information using a rib-group.
There is yet another option that Junos OS gives us. We can make our route reflector resolve routes by using rib-groups. Rib-groups can be used to place routing information from a single source into a group of tables. The use of rib-groups can quickly escalate the complexity of an infrastructure, but it’s a pretty powerful tool nevertheless.
In this specific scenario, by using rib-groups, we can make the Junos OS place OSPF routes into the inet.0 AND into the inet.3 table:
In order to make this happen, we have to alter the configuration in two separate places. First, we have to create the rib-group under the [ routing-options ] stanza. Second is the application of the rib-group under the [ protocols ospf ] stanza.
The configuration of the rib-group can be done as follows:
set routing-options rib-groups inet0-to-inet3 import-rib inet.0 set routing-options rib-groups inet0-to-inet3 import-rib inet.3
The rib-group will make sure that routes, normally placed in the inet.0 table, will be placed into both the inet.0 as well as the inet.3 table. For this reason, the inet.0 table is configured first.
The second part of the required configuration is the application of the rib-group. This is configured in the configuration stanza where the routes would normally be learned. In our case, this means that we will apply the rib-group under the OSPF stanza, like this:
set protocols ospf rib-group inet0-to-inet3
After committing this configuration, we can observe the result by consulting the inet.3 table:
inetzero@rr> show route table inet.3 terse inet.3: 37 destinations, 37 routes (37 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both A V Destination P Prf Metric 1 Metric 2 Next hop AS path * ? 10.0.0.0/30 O 10 2 >172.16.1.10 * ? 10.0.0.4/30 O 10 2 >172.16.1.10 * ? 10.0.0.8/30 O 10 3 >172.16.1.10 * ? 10.0.0.12/30 O 10 3 >172.16.1.10 * ? 10.0.0.16/30 O 10 4 >172.16.1.10 * ? 10.0.0.20/30 O 10 4 >172.16.1.10 * ? 10.0.0.24/30 O 10 3 >172.16.1.10 * ? 10.1.0.0/30 O 10 2 >172.16.1.10 * ? 10.1.0.4/30 O 10 3 >172.16.1.10 * ? 10.1.0.8/30 O 10 4 >172.16.1.10 * ? 10.1.0.12/30 O 10 3 >172.16.1.10 * ? 10.1.0.16/30 O 10 3 >172.16.1.10 * ? 10.1.0.20/30 O 10 4 >172.16.1.10 * ? 10.1.0.24/30 O 10 4 >172.16.1.10 * ? 10.1.0.28/30 O 10 5 >172.16.1.10 * ? 10.1.0.32/30 O 10 5 >172.16.1.10 * ? 10.1.0.36/30 O 10 4 >172.16.1.10 * ? 10.1.0.40/30 O 10 2 >172.16.1.10 * ? 10.1.0.44/30 O 10 3 >172.16.1.10 * ? 10.1.0.48/30 O 10 3 >172.16.1.10 * ? 10.1.0.52/30 O 10 4 >172.16.1.10 * ? 172.16.0.1/32 O 10 0 >lo0.100 * ? 172.16.0.2/32 O 10 5 >172.16.1.10 * ? 172.16.1.0/30 O 10 4 >172.16.1.10 * ? 172.16.1.4/30 O 10 5 >172.16.1.10 * ? 192.168.0.1/32 O 10 1 >172.16.1.10 * ? 192.168.0.2/32 O 10 2 >172.16.1.10 * ? 192.168.0.3/32 O 10 3 >172.16.1.10 * ? 192.168.0.4/32 O 10 2 >172.16.1.10 * ? 192.168.0.5/32 O 10 3 >172.16.1.10 * ? 192.168.0.6/32 O 10 4 >172.16.1.10 * ? 192.168.1.1/32 O 10 2 >172.16.1.10 * ? 192.168.1.2/32 O 10 3 >172.16.1.10 * ? 192.168.1.3/32 O 10 3 >172.16.1.10 * ? 192.168.1.4/32 O 10 4 >172.16.1.10 * ? 192.168.1.5/32 O 10 2 >172.16.1.10 * ? 192.168.1.6/32 O 10 4 >172.16.1.10
Since all we really need in the inet.3 table are the loopback IP addresses of our pe routers, this may be a bit much. Luckily, we can also apply a policy to a rib-group. Let’s create the following policy:
set policy-options policy-statement pe-lo0 term pe-lo0 from route-filter 192.168.1.0/29 prefix-length-range /32-/32 set policy-options policy-statement pe-lo0 term pe-lo0 then accept set policy-options policy-statement pe-lo0 term reject-rest then reject
And apply it to the rib-group like this:
set routing-options rib-groups inet0-to-inet3 import-policy pe-lo0
After applying the policy, the inet.3 table looks like this:
inetzero@rr> show route table inet.3
inet.3: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
192.168.1.1/32 *[OSPF/10] 00:00:14, metric 2
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.2/32 *[OSPF/10] 00:00:14, metric 3
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.3/32 *[OSPF/10] 00:00:14, metric 3
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.4/32 *[OSPF/10] 00:00:14, metric 4
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.5/32 *[OSPF/10] 00:00:14, metric 2
> to 172.16.1.10 via xe-0/2/0.1518
192.168.1.6/32 *[OSPF/10] 00:00:14, metric 4
> to 172.16.1.10 via xe-0/2/0.1518
And with the inet.3 table looking like that, we can see that the VPN prefix is reflected once more:
inetzero@rr> show route advertising-protocol bgp 192.168.1.6 192.168.50.0/24 bgp.l3vpn.0: 106 destinations, 106 routes (106 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 192.168.1.1:176:192.168.50.0/24 * 192.168.1.1 100 I
Nice!
The Junos OS has a lot of tools that can be used to manipulate routing and forwarding information. It’s these tools that make Junos a very versatile OS for routing. Hopefully, this article has shed some light on how these tools can be made of use in a scenario with an out of path rr.
Other scenarios challenging you to take it to the next level can be found in the iNET ZERO JNCIE-SP workbook.
