Title here
Summary here
Open Source Routing: Practical Lab
June 15, 2015 in Systems11 minutes
Earlier, I wrote about some interesting open source routing software that I’ve been exploring lately. In this post, I’ll provide you with some tools to get this lab running on your lab, using Vagrant and Ansible.
In this post, I’ll be using VirtualBox, and also Ansible and Vagrant. For this purpose, I’m assuming you’re at least somewhat familiar with these tools.
Please checkout my GitHub repository for access to the latest versions of all of the files we’ll discuss below - and an easy way to spin all of this up yourself.
Topology
First, here’s the topology we’ll be working with.
All “circuits” are implemented using VirtualBox host networks, described in the Vagrantfile:
# -*- mode: ruby -*-
# vi: set ft=ruby :
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "trusty64"
config.vm.box_url = "http://cloud-images.ubuntu.com/vagrant/trusty/current/trusty-server-cloudimg-amd64-vagrant-disk1.box"
config.vm.define "r1" do |r1|
r1.vm.host_name = "r1"
r1.vm.network "private_network",
ip: "192.168.12.11",
virtualbox__intnet: "01-to-02"
r1.vm.network "private_network",
ip: "192.168.31.11",
virtualbox__intnet: "03-to-01"
r1.vm.network "private_network",
ip: "1.1.1.10",
virtualbox__intnet: "Network to Advertise"
r1.vm.provision "ansible" do |ansible|
ansible.playbook = "r1.yml"
end
end
config.vm.define "r2" do |r2|
r2.vm.host_name = "r2"
r2.vm.network "private_network",
ip: "192.168.23.12",
virtualbox__intnet: "02-to-03"
r2.vm.network "private_network",
ip: "192.168.12.12",
virtualbox__intnet: "01-to-02"
r2.vm.network "private_network",
ip: "2.2.2.10",
virtualbox__intnet: "Network to Advertise"
r2.vm.provision "ansible" do |ansible|
ansible.playbook = "r2.yml"
end
end
config.vm.define "r3" do |r3|
r3.vm.host_name = "r3"
r3.vm.network "private_network",
ip: "192.168.31.13",
virtualbox__intnet: "03-to-01"
r3.vm.network "private_network",
ip: "192.168.23.13",
virtualbox__intnet: "02-to-03"
r3.vm.network "private_network",
ip: "3.3.3.10",
virtualbox__intnet: "Network to Advertise"
r3.vm.provision "ansible" do |ansible|
ansible.playbook = "r3.yml"
end
end
end
Initial Setup
All configuration files and Ansible playbooks necessary for configuring all three virtual machines are provided in the GitHub repository. As you may have seen in the Vagrantfile listed above, all of these playbooks are linked to each virtual machine, so the steps required to clone this repo, and get the lab running are very simple:
~$ git clone https://github.com/Mierdin/ossrouting.git
~$ cd ossrouting
~$ vagrant up
In case you’re not familiar with Vagrant, the last step will create and start all virtual machines, as well as provision them using Ansible. If you wish to make changes to any of the files in this repo, you’ll have to run “vagrant provision” to push those changes into the VMs once they’ve been initially provisioned once.
At this point, you should have running VMs that have been configured to run BGP with each other using the software on each. I will now go into detail on how to play with each instance - in order to follow along, you should SSH into each VM using Vagrant. If I’m talking about R1 (Quagga) for instance, you’d type:
~$ vagrant ssh r1
The specific name wil be listed at the top of each section.
Quagga (r1)
After provisioning, this virtual machine is actively running zebra and bgpd:
vagrant@r1:~$ service quagga status
bgpd zebra
As mentioned in the previous article, Quagga comes with a handy tool called “vtysh” that unifies the configuration for all running daemons under one CLI context, making it feel very similar to something like Cisco IOS.
vagrant@r1:~$ sudo vtysh
Hello, this is Quagga (version 0.99.22.4).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
r1#
You may have to press “q” immediately after running vtysh to get to the Quagga CLI
Again - with a certain familiarity, you can display the running configuration, and notice a basic BGP configuration:
r1# show run
Building configuration...
Current configuration:
!
[omitted]
!
router bgp 111
bgp router-id 1.1.1.1
bgp log-neighbor-changes
network 1.1.1.0/24
neighbor 192.168.12.12 remote-as 121
neighbor 192.168.12.12 next-hop-self
neighbor 192.168.12.12 soft-reconfiguration inbound
neighbor 192.168.31.13 remote-as 131
neighbor 192.168.31.13 next-hop-self
neighbor 192.168.31.13 soft-reconfiguration inbound
You can also show the active BGP neighbors, as well as learned prefixes:
r1# show ip bgp sum
BGP router identifier 1.1.1.1, local AS number 111
RIB entries 5, using 560 bytes of memory
Peers 2, using 9120 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
192.168.12.12 4 121 18 22 0 0 0 00:16:16 1
192.168.31.13 4 131 43 40 0 0 0 00:34:59 1
Total number of neighbors 2
r1# show ip bgp
BGP table version is 0, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 1.1.1.0/24 0.0.0.0 0 32768 i
*> 2.2.2.0/24 192.168.12.12 0 121 i
*> 3.3.3.0/24 192.168.31.13 0 63000 63000 63000 131 i
Obviously, there are quite a few operational differences between Quagga and the “standard” network operating systems it mimics - if you want to make changes to Quagga configuration, your options are limited to either changing the configuration files behind the scenes and restarting the service(s), or making changes from vtysh. The former option is what I’m doing with Ansible, but this has obvious drawbacks in production.
If you exit out to the shell, you’ll notice that Quagga pushes these prefixes into the linux routing table proper:
vagrant@r1:~$ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.0.2.2 0.0.0.0 UG 0 0 0 eth0
1.1.1.0 * 255.255.255.0 U 0 0 0 eth3
2.2.2.0 192.168.12.12 255.255.255.0 UG 0 0 0 eth1
3.3.3.0 192.168.31.13 255.255.255.0 UG 0 0 0 eth2
10.0.2.0 * 255.255.255.0 U 0 0 0 eth0
192.168.12.0 * 255.255.255.0 U 0 0 0 eth1
192.168.31.0 * 255.255.255.0 U 0 0 0 eth2
ExaBGP (r2)
This virtual machine has not been configured to start ExaBGP for us automatically, so we’ll need to do that. First, a quick tour. Upon connecting to our ExaBGP instance via SSH, you’ll notice there are a few files in ~/exabgp:
vagrant@r2:~$ cd exabgp
vagrant@r2:~/exabgp$ ll
total 20
drwxr-xr-x 2 root root 4096 May 28 03:51 ./
drwxr-xr-x 6 vagrant vagrant 4096 May 24 21:43 ../
-rw-r--r-- 1 root root 285 May 28 03:51 advroutes.py
-rw-r--r-- 1 root root 289 May 24 08:32 conf.ini
-rw-r--r-- 1 root root 639 May 25 07:06 exabgp.env
“conf.ini” is our configuration file. If you look at this, you’ll notice a very basic, two-neighbor configuration:
vagrant@r2:~/exabgp$ cat conf.ini
group BGProuters {
router-id 2.2.2.2;
neighbor 192.168.12.11 {
local-address 192.168.12.12;
local-as 121;
peer-as 111;
graceful-restart;
process announce-routes {
run /usr/bin/python /home/vagrant/exabgp/advroutes.py;
}
}
neighbor 192.168.23.13 {
local-address 192.168.23.12;
local-as 121;
peer-as 131;
graceful-restart;
process announce-routes {
run /usr/bin/python /home/vagrant/exabgp/advroutes.py;
}
}
}
You might notice that each neighbor runs a small python script that outputs ExaBGP advertisements to stdout:
vagrant@r2:~/exabgp$ cat advroutes.py
#!/usr/bin/env python
import sys
import time
messages = [
'announce route 1.1.1.0/24 next-hop self',
'announce route 2.2.2.0/24 next-hop self',
'announce route 3.3.3.0/24 next-hop self',
]
time.sleep(2)
while messages:
message = messages.pop(0)
sys.stdout.write( message + '\n')
sys.stdout.flush()
time.sleep(1)
while True:
time.sleep(1)There is also an environment file in this directory that I used to ensure you could recreate the environment with my settings. Let’s start ExaBGP using the environment and configuration files in this directory.
vagrant@r2:~/exabgp$ exabgp --env exabgp.env conf.ini
I won’t bother posting the full output here, but the resulting output is fairly self-explanatory. It may take a few extra seconds to connect to our “r3” instance, running BIRD, but it should eventually connect.
BIRD (r3)
Very similar to Quagga, BIRD should already be running with the correct settings upon connection:
vagrant@r3:~$ service bird status
bird start/running, process 4252
The Ansible role for configuring BIRD is fairly simple - I really only needed to overwrite the default configuration file and make sure the BIRD service is started (after installing it of course).
The configuration file is interesting - it seems to allow us to perform some native scripting-esque logic right there in the file:
vagrant@r3:~$ sudo cat /etc/bird/bird.conf
router id 33.33.33.33;
protocol kernel {
persist;
scan time 20;
export all;
import all;
}
protocol device {
scan time 10;
}
protocol static {
}
protocol direct {
interface "eth3";
}
filter out_loopback1 {
if (net = 3.3.3.0/24) then
{
bgp_community.empty;
bgp_path.prepend(63000);
bgp_path.prepend(63000);
bgp_path.prepend(63000);
accept;
}
else reject;
}
protocol bgp ToQuagga {
description "Quagga";
debug { states, events };
local as 131;
neighbor 192.168.31.11 as 111;
next hop self;
route limit 50000;
default bgp_local_pref 300;
import all;
export filter out_loopback1;
source address 192.168.31.13;
}
protocol bgp ToExaBGP {
description "ExaBGP";
debug { states, events };
local as 131;
neighbor 192.168.23.12 as 121;
next hop self;
route limit 50000;
default bgp_local_pref 300;
import all;
export filter out_loopback1;
source address 192.168.23.13;
}
I have my familiar two-neighbor configuration, but each refers to an export filter called “out_loopback” which executes a basic conditional statement. If true, I advertise the route (and perform some AS path prepending).
What really sets BIRD apart from the others in my mind is the detached client, “birdc”. As mentioned in the previous article, this comes with the BIRD server itself, but it could communicate with the BIRD server through network RPCs, meaning it could be used remotely.
To start it in this environment, just run “sudo birdc” and start poking around:
vagrant@r3:~$ sudo birdc
BIRD 1.5.0 ready.
bird> show status
BIRD 1.5.0
Router ID is 33.33.33.33
Current server time is 2015-06-15 02:07:07
Last reboot on 2015-06-14 16:29:38
Last reconfiguration on 2015-06-14 16:29:38
Daemon is up and running
As you can see, this does seem to dump us into a CLI context of some kind, but it doesn’t seem to mimic any other style of CLI:
bird> ?
add roa ... Add ROA record
configure ... Reload configuration
debug ... Control protocol debugging via BIRD logs
delete roa ... Delete ROA record
disable <protocol> | "<pattern>" | all Disable protocol
down Shut the daemon down
dump ... Dump debugging information
echo ... Control echoing of log messages
enable <protocol> | "<pattern>" | all Enable protocol
eval <expr> Evaluate an expression
exit Exit the client
flush roa [table <name>] Removes all dynamic ROA records
help Description of the help system
mrtdump ... Control protocol debugging via MRTdump files
quit Quit the client
reload <protocol> | "<pattern>" | all Reload protocol
restart <protocol> | "<pattern>" | all Restart protocol
restrict Restrict current CLI session to safe commands
show ... Show status information
I recommend you poke around these options yourself, there are quite a few useful tools here. However, I would like to briefly mention that there seems to be some interesting configuration safety tools included in the daemon that the client is able to leverage. For instance - checking the validity of a configuration file:
bird> configure ?
configure ["<file>"] [timeout [<sec>]] Reload configuration
configure check ["<file>"] Parse configuration and check its validity
configure confirm Confirm last configuration change - deactivate undo timeout
configure soft ["<file>"] [timeout [<sec>]] Reload configuration and ignore changes in filters
configure timeout [<sec>] Reload configuration with undo timeout
configure undo Undo last configuration change
bird> configure check "/etc/bird/bird.good"
Reading configuration from /etc/bird/bird.good
Configuration OK
bird> configure check "/etc/bird/bird.bad"
Reading configuration from /etc/bird/bird.bad
/etc/bird/bird.bad, line 33: syntax error
This would be hugely useful for automating changes to this configuration file, since we don’t have to actually use a configuration file in order to check it (just need to point to it).
Of course, this section would be incomplete without checking on our neighbors. We can plainly see that our two intended BGP neighbor relationships are configured and working well, and we can also see that we’ve learned the prefixes we wanted to see, with the right number of paths:
bird> show protocols all ToExaBGP
name proto table state since info
ToExaBGP BGP master up 02:01:10 Established
(omitted)
bird> show protocols all ToQuagga
name proto table state since info
ToQuagga BGP master up 16:30:23 Established
(omitted)
bird> show route all
1.1.1.0/24 via 192.168.31.11 on eth1 [ToQuagga 16:30:22] * (100) [AS111i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 111
BGP.next_hop: 192.168.31.11
BGP.med: 0
BGP.local_pref: 300
via 192.168.23.12 on eth2 [ToExaBGP 02:01:10] (100) [AS121i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 121
BGP.next_hop: 192.168.23.12
BGP.local_pref: 300
2.2.2.0/24 via 192.168.23.12 on eth2 [ToExaBGP 02:01:10] * (100) [AS121i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 121
BGP.next_hop: 192.168.23.12
BGP.local_pref: 300
via 192.168.31.11 on eth1 [ToQuagga 02:00:59] (100) [AS121i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 111 121
BGP.next_hop: 192.168.31.11
BGP.local_pref: 300
3.3.3.0/24 dev eth3 [direct1 16:29:37] * (240)
Type: device unicast univ
via 192.168.23.12 on eth2 [ToExaBGP 02:01:10] (100) [AS121i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 121
BGP.next_hop: 192.168.23.12
BGP.local_pref: 300
via 192.168.31.11 on eth1 [ToQuagga 02:00:59] (100) [AS121i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 111 121
BGP.next_hop: 192.168.31.11
BGP.local_pref: 300
Conclusion
I hope you enjoyed this lab that I set up to help get your feet wet with open source routing! There’s much more where this came from - please don’t stop here, take this as far as it will let you. There are a lot of options involved that I didn’t have space to explore.
Please check out the Github project for access to all of the files you’ll need to get started. If you are so inclined, and you feel that I am missing a crucial piece of this lab, or maybe I have a typo, feel free to submit a PR!