In pursuit of some Friday Evening Fun I decided to try to get a Cisco CSR 1000V VM running on SmartOS. I was successful. It runs. It seems to run quite well.
First, a overview of the systems involved since the CSR1000V is fairly new and SmartOS is relatively uncommon.
The Cisco Cloud Services Router 1000V (CSR1000V) is Cisco’s entry in the virtualized router field. It runs IOS-XE and brings many of the handy features that you can find on the ASR1k platform into the cloud so you can do fun and exciting things with MPLS, VPLS, LISP, etc. This is a pleasant change from Vyatta which lacks any sort of MPLS support and Mikrotik which blackholes traffic for no good reason.
SmartOS is a hypervisor based on Illumos which is in turn a fork of OpenSolaris which was next-gen Solaris before Oracle borged Sun. SmartOS has all the wonderful features of OpenSolaris (ZFS, Zones, network virtualization) and adds support for KVM. There are some interesting design choices in SmartOS that turn out to be pretty cool once you get used to them, in particular: Netboot only (or USB if you are desperate) with all local disk used for VMs and VM Management by JSON config files.
I started by downloading the CSR1000V ISO (csr1000v-universalk9.03.10.00.S.153-3.S-ext.iso) from CCO and then uploading it to a server in the datacenter. I followed the instructions for creating a KVM VM on SmartOS except I copied the CSR1000V ISO over instead of a Debian ISO. Installation was smooth but after boot and initial configuration I wasn’t able to reach the VM. It turns out that SmartOS has some decent network security policies by default so I had to add "allow_ip_spoofing": true to the VM config before traffic started to flow.
After the CSR was installed and configured for basic connectivity I set it up as a route reflector and sent it the partial BGP feed that I run on our old Cisco 6500 series routers (~100,000 routes). sho proc cpu hist showed up to 40% CPU usage while it was loading. After that I added a table-map containing a deny statement to the BGP config to prevent it from wasting time trying to update the RIB. This reduced loading time to two seconds with no response lag on the terminal.
My feeling in general on the CSR is that it simply has too large of a footprint for general cloud routing & VPN duties. Vyatta gets the job done with a lot less than 4G of RAM and four CPU cores. If your needs go beyond simple routing & VPN then the CSR becomes more appealing. So far it seems well suited to the route reflector role but I have more testing to do before I commit to it.
If the CSR holds up to testing I hope to run a pair of them in place of the aging 7200s that are currently acting as my route reflectors. Cisco’s recommended route reflector platform is the ASR1k but as a small ISP I just can’t justify the cost on something that doesn’t generate revenue. If this works I can stay with Cisco instead of cobbling something together on a different platform.
Configs used are available as Github Gists: VM config json and CSR config.