NetBox scripts – now with clean error handling

Recently I wrote a scripts which aids in provisioning things inside our network, which will do some sanity checks and if all is good set up a number of things (prefixes, IPs, sub-interfaces, IPs on them, etc.). The reason to do this inside a script is to do this in an atomic operation, so either the full provision process is done, or nothing is changed at all.

If the sanity checks fail (invalid input, trying to create something with overlapping resources, etc.) the script should fail and ideally report a clear error message one what was wrong, which should be reported to the caller (via API).

Now I was looking into exiting the script on an error and only found the option to throw any Exception which will produce a stack trace of all internal Exceptions which had been caught and handled. There is a AbortTransaction Exception, which allows to terminate the script and thereby the DB transaction, but it was not designed to carry an error message.

Looking into the code it seemed like adding support to gracefully abort a would be rather straight forward to add, so I did (issue, PR). Today the PR got merged and NetBox v3.4.4 (and later) includes the AbortScript exception to elegantly abort scripts, which you can use like this:

from utilities.exceptions import AbortScript

if some_error:
     raise AbortScript("Some meaningful error message")

Insights into Network Automation with Go

A great new book on Network Automation with Go just dropped recently and if you want to get into automation parts of your network or wants to start doing so with Go, it’s definitely for you!

Network Automation with Go book

It contains a lot of background on the Golang programming language, its concepts and how to use them to build reliable, scalable, testable, and observable applications. The authors also discuss Network Automation and configuration management approaches in general, and dig into APIs and network monitoring.

Continue reading Insights into Network Automation with Go

This is the way: Holistic approach on network automation

In the past months and years I’ve had a number of great discussion with a lot of fantastic networking people on how to (not) do network automation (at scale), what worked for them and what didn’t. At a smaller scale I made quite some experiences myself at previous roles as well as consulting engagements and in particular by building and operating the Freifunk Hochstift community ISP network plus it’s SDN. This article is the distilled result of those discussions and experiences and might be somewhat opinionated.

Network automation done right

Continue reading This is the way: Holistic approach on network automation

DENOG12 Netbox workshop summary

One week ago DENOG12 took place virtually as an awesome venueless conference and I had the pleasure to hold a workshop about netbox and how to use it for automation. As promised here are the notes we collected while the workshop and projects that emerged since. This is provided as-is and as I don’t have used most of the external tools/scripts/reports/etc. linked below 🙂

Deployment

I forked the officiall netbox-docker GIT repository to set up netbox as a number of Docker containers and made some small changes, to run the PostgreSQL DB outside of Docker and restarte the container on Docker restarts / system reboots.

As the time of this writing this repo is configured for netbox version 2.9.8.

Device Types

The community Device Type library

Within the netbox-community organization on Github you can find the semi-official community Netbox Device Type library where a lot of Device Types are present and ready to be imported into your Netbox. Be aware that you have to create the Manufacturer first before you can import Device Types for that manufacturer. If you happen to create Device Types for devices which are no present in the library please open a PR – sharing is caring 🙂

Importing multiple Device Types at once

Someone mentioned this script/repository which offers the possibility to import multiple Device Types at once. (I didn’t test it yet :))

Reports and Scripts

This repository  hold a bunch of example Netbox Reports for various things within Netbox (circuits, Cabling, IPs/DNS, VMs, etc.) as well as Netbox Scripts to create a VM as well as a geolocator for a site.

My own netbox-scripts repository contains a script to populate a Freifunk Hochstift Backbone POP from one script.

This (archived) repository holds a bunch for tools to import/sync stuff into/with netbox.

Wikimedia seems to use Netbox, too and has open sourced some tooling including a zone file generator.

Other

Johannes wrote an article about adding own buttons within Netbox to open an SSH session into a router.

If you want to extent the authentication options of your Netbox there is a Plugin for SSO using SAML2.

A thread from the Google Group on setting custom fields via the API.

Deploying a Freifunk Hochstift backbone POP with Netbox Scripts

Some weeks ago Network to Code held the first (virtual) Netbox Day (YouTube playlist, Slides repo on github). John Anderson gave a great NetBox Extensibility Overview and introduced me to Netbox Scripts (Video, Slide deck, Slide 28) which allow to add custom Python code to add own procedures to netbox. I was hooked. About three to four hours of fiddling, digging through the docs, and some hundred lines of Python later I had put together a procedure to provision a complete Freifunk Hochstift Backbone POP within Netbox according to our design. I’m going to share my proof of concept code here and  walk you through the key parts of the script.

Netbox scripts provide a great and really simple interface to codify procedures and design principles which apply to your infrastructure and fire up complex network setups within netbox by just entering a set of config parameters in a form like the following and a click of one button.

Provision Backbone POP form
Provision Backbone POP form

Continue reading Deploying a Freifunk Hochstift backbone POP with Netbox Scripts

Seriously predictable interface names – An introduction to systemd .link files

Predictable interface names are a new thing. The most common argument made is that they are not really predictable though, depending on the point of view. How about making interface names predictable and meaningful in the same time?

Most admins will probably think of udev right now, which previously was heavily used to achieve exactly that. In times of systemd the new hotness are .link files which provide similar capabilities and allow even more options to be set for interfaces.

Continue reading Seriously predictable interface names – An introduction to systemd .link files

Anycasted services with Debian, bird, anycast-healthchecker and Cisco Nexus 7000

A while ago we started getting alerts, that one of our Kerberos KDCs had problem with the Kerberos database replication. A little digging revealed, that the problems are caused by load spikes on the KDC which were the result of a burst of legitimate queries fired by some systems we didn’t have much control over. Additionally we found that the MIT Kerberos implementation queries all KDCs provided in the configuration file in sequential order, so the first KDC get’s nearly all queries. While thinking about load balancing solutions, quickly anycast came to mind, so we decided to set it up. Anycast leverages the Equal Cost Multipath Routing (ECMP)  capability of common routers to distribute traffic to multiple next-hops for the same destination.

The solution consists of three corner stones:

  1. anycast-healtchecker as a means to check service availability
  2. bird as a BGP speaker on the KDCs and route reflectors
  3. Data center routers (Cisco Nexus 7010) speaking BGP to the route reflectors

The topology is as follows:

Topology KDCs

Continue reading Anycasted services with Debian, bird, anycast-healthchecker and Cisco Nexus 7000

Building your own Software Defined Network with Linux and Open Source Tools

Nearly two years ago, I started thinking about a next generation design for the Freifunk Hochstift backbone infrastructure, motivated by the limits and design choices made before (we were young and didn’t know better.. or didn’t listen..).

This post is the starting point of a series of posts about building a software defined wireless ISP network with Linux, a fistful of Open Source tools and low cost hardware.

Continue reading Building your own Software Defined Network with Linux and Open Source Tools