Taming Netlink

Posted by Dave on 12 January 2015

Netlink is a Linux Kernel interface that is used to talk Networking between Kernel processes, or from userland to Kernel processes. This post documents how I tamed Netlink and make working with it in Golang easier.

If you want to create an interface, set an MTU, set an IP address etc... on a Linux machine then you have to talk Netlink to do it. For Golang, there are a number of libraries, but this one seems to be gaining a lot of traction recently. Honorable mentions go to Tenus and Docker's libcontainer.

Now, this library is good, but it isn't complete and some of the behavior I need is missing. So how does one go about implementing it you might ask?

Here's my process:

## 1. Browse the iproute2 source code

The iproute2 utility is insanely useful - it replaces ifconfig and route for those with memories long enough. In my case, the feature I wanted was here and I could take a look at the exact message structure required :)

Once I'd written my tests and code, I wasn't surprised to see that things were not working. strace will get you part of the way, but it's not easy to read the encoded Netlink messages.

As it transpires, my friend Thomas Graf happened to have written a Netlink library in C called libnl-3. It also just so happens that somebody wrote a Netlink debugger using this library called nltrace.

To install on my Debian workstation I had to:

:::bash
apt-get install libnl-3-dev libnl-genl-3-dev libnl-nf-3-dev libnl-route-3-dev
git clone https://github.com/socketpair/nltrace
make

To use it, simply just nltrace <your command>and the output will look a little something like this.

dave@daves-linux-box:~$ nltrace ip route get 8.8.8.8
netlink send(3):
Setting msg proto to 0
--------------------------   BEGIN NETLINK MESSAGE ---------------------------
  [NETLINK HEADER] 16 octets
    .nlmsg_len = 36
    .type = 26 <route/route::get>
    .flags = 1 <REQUEST>
    .seq = 1421005121
    .port = 0
  [PAYLOAD] 12 octets
    02 20 00 00 00 00 00 00 00 00 00 00             . ..........
  [ATTR 01] 4 octets
    08 08 08 08                                     ....
---------------------------  END NETLINK MESSAGE   ---------------------------
netlink recv(3):
Setting msg proto to 0
--------------------------   BEGIN NETLINK MESSAGE ---------------------------
  [NETLINK HEADER] 16 octets
    .nlmsg_len = 104
    .type = 24 <route/route::new>
    .flags = 0 <>
    .seq = 1421005121
    .port = 12871
  [PAYLOAD] 12 octets
    02 20 00 00 fe 00 00 01 00 02 00 00             . ..........
  [ATTR 15] 4 octets
    fe 00 00 00                                     ....
  [ATTR 01] 4 octets
    08 08 08 08                                     ....
  [ATTR 04] 4 octets
    02 00 00 00                                     ....
  [ATTR 07] 4 octets
    0a 00 02 0f                                     ....
  [ATTR 05] 4 octets
    0a 00 02 02                                     ....
  [ATTR 12] 32 octets
    01 00 00 00 8e 2c 03 00 00 00 00 00 00 00 00 00 .....,..........
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
---------------------------  END NETLINK MESSAGE   ---------------------------
Destroying descriptor fd 4
8.8.8.8 via 10.0.2.2 dev eth0  src 10.0.2.15
cache
Destroying descriptor fd 3

In the case of my Golang code, I had to write a small test program as running nltrace go run main.go didn't work out so well. The end result though was a really good way of inspecting the Netlink messages that are being sent/received by a process.

3. Other Helpful Resources

Conclusion

Having now got to grips with using Netlink, it's not as scary as it first seemed. It's likely I'll be digging in deeper in the near future, but for now, I'm pleased with how far I've come

Do you have any tips for working with Netlink? Let me know in the comments below

@dave_tucker