Building a Custom Device Integration

This tutorial will demonstrate how to build a new custom device integration. Say we want to allow MacBooks to register with CloudVision and stream out interface data. This example is simple and doesn’t stream all the kinds of data that CloudVision can use, but should serve to illustrate the features of the SDK. The full device and provider implementations can be found in device/devices/darwin.go and provider/darwin/darwin.go, respectively.

darwin Provider

First we’ll build a provider that can pull OpenConfig-modeled interface statistics from the MacBook and stream them out as gNMI updates. The interface we want to implement is

type GNMIProvider interface {
    Provider
    InitGNMI(client gnmi.GNMIClient)
    OpenConfig() bool
    Origin() string
}

which embeds the Provider interface:

type Provider interface {
    Run(ctx context.Context) error
}

The InitGNMI method passes a gNMI client to the provider, which is what provider will use to stream out updates. Run is what starts the provider. First we’ll define a struct to hold the variables we’ll need:

type darwin struct {
    client       gnmi.GNMIClient
    errc         chan error
    pollInterval time.Duration
}

InitGNMI just stores the client in our struct:

func (d *darwin) InitGNMI(client gnmi.GNMIClient) {
    d.client = client
}

We’ll set OpenConfig() to always return true, indicating that this provider expects to produce OpenConfig-modeled data and wants type-checking against the set of supported OpenConfig models.

func (d *darwin) OpenConfig() bool {
    return true
}

Let's have the Origin() function always return "openconfig". This function should return the origin of the YANG model being used. Valid values are "openconfig" and "arista". This origin is automatically set for all gNMI requests sent out by the Collector. In case a provider needs to emit updates for two or models with different origins, it can also override this value by explicitly setting the origin value in the gNMI SetRequest.

func (d *darwin) Origin() string {
    return "openconfig"
}

The gnmi package in the provider/ directory has a handy function, PollForever, that takes a context, a GNMI client, a polling interval, an error channel, and a function that returns a slice of gNMI SetRequests to give to the gNMI client. It will run this function at the specified polling interval forever, or until its context says it’s done. If it encounters an error, it will put the error on the error channel provided. So our Run method is simple:

func (d *darwin) Run(ctx context.Context) error {
    go pgnmi.PollForever(ctx, d.client, d.pollInterval,
         d.updateInterfaces, d.errc)
    return d.handleErrors(ctx)
}

Then all that’s left is to write the function that creates a new darwin provider:

func NewDarwinProvider(pollInterval time.Duration) provider.GNMIProvider {
    return &darwin{
        errc:         make(chan error),
        pollInterval: pollInterval,
    }
}

Just kidding! We still have to write the actual polling logic. That’s the bulk of the code here. The approach we’ll take for this provider is to use the BSD netstat command to get the laptop’s interface statistics. It will produce output like the following:

Name  Mtu   Network       Address            Ipkts Ierrs     Ibytes    Opkts Oerrs     Obytes  Coll
lo0   16384 <Link#1>                       1071838     0  201427255  1071838     0  201427255     0
lo0   16384 127           127.0.0.1        1071838     -  201427255  1071838     -  201427255     -
lo0   16384 ::1/128     ::1                1071838     -  201427255  1071838     -  201427255     -
lo0   16384 fe80::1%lo0 fe80:1::1          1071838     -  201427255  1071838     -  201427255     -
en0   1500  <Link#8>    8c:85:90:ba:17:3a 54120416     0 58229015732 33337146     0 6320538923     0
en0   1500  fe80::c14:a fe80:8::c14:abfe: 54120416     - 58229015732 33337146     - 6320538923     -
en0   1500  10.81.20/23   10.81.20.116    54120416     - 58229015732 33337146     - 6320538923     -
p2p0  2304  <Link#9>    0e:85:90:ba:17:3a        0     0          0        0     0          0     0
en1   1500  <Link#11>   c6:00:28:50:11:01        0     0          0        0     0          0     0
en2   1500  <Link#12>   c6:00:28:50:11:00        0     0          0        0     0          0     0
en3   1500  <Link#13>   c6:00:28:50:11:05        0     0          0        0     0          0     0
en4   1500  <Link#14>   c6:00:28:50:11:04        0     0          0        0     0          0     0
bridg 1500  <Link#15>   c6:00:28:50:11:01        0     0          0        1     0        342     0
utun0 2000  <Link#16>                            0     0          0        2     0        200     0

and so on.

From this information, for each interface of interest we can get the following paths from the OpenConfig interfaces model:

interfaces/interface[name]/name
interfaces/interface[name]/config/name
interfaces/interface[name]/state/name
interfaces/interface[name]/state/type
interfaces/interface[name]/state/mtu
interfaces/interface[name]/state/admin-status
interfaces/interface[name]/state/oper-status
interfaces/interface[name]/state/counters/in-octets
interfaces/interface[name]/state/counters/in-unicast-pkts
interfaces/interface[name]/state/counters/in-errors
interfaces/interface[name]/state/counters/out-octets
interfaces/interface[name]/state/counters/out-unicast-pkts
interfaces/interface[name]/state/counters/out-errors

One tricky point to note here is that, while in general all streamed device state is state data, config/name is included in the update because OpenConfig has a constraint that the value of name must also be present in config/name, and failing to satisfy this constraint will produce an error from the OpenConfig typechecker.

The updateInterfaces method looks something like this:

func (d *darwin) updateInterfaces() (*gnmi.SetRequest, error) {
    ns := exec.Command("netstat", "-ibn")
    out, err := ns.CombinedOutput()
    if err != nil {
        return nil, err
    }

    updates := make([]*gnmi.Update, 0)
    for _, line := range strings.Split(string(out), "\n") {
        // If this line is an ethernet interface, create an update and add
        // it to updates.
        u, err := updatesFromNetstatLine(line)
        if err != nil {
            return nil, err
        }
        if update != nil {
            updates = append(updates, u...)
        }
    }

    setRequest := new(gnmi.SetRequest)
    setRequest.Delete = []*gnmi.Path{pgnmi.Path("interfaces")}
    setRequest.Replace = updates
    return setRequest, nil
}

The updatesFromNetstatLine function is what builds the updates. For brevity’s sake it’s not included in full here, but a version of it that only included an update of the in-octets counter, say, would look like this:

func updatesFromNetstatLine(fields []string) ([]*gnmi.Update, error) {
    intfName := fields[0]
    inBytes, err := strconv.ParseUint(fields[6], 10, 64)
    if err != nil {
        return nil, err
    }
    return []*gnmi.Update{
        pgnmi.Update(pgnmi.IntfStateCountersPath(intfName, "in-octets"),
            pgnmi.Uintval(inBytes)),
    }, nil
}

pgnmi.Update, pgnmi.IntfStateCountersPath, and pgnmi.Uintval are helper functions in the gnmi library in provider/gnmi.

And that’s it! Now we have a provider that collects device information and streams out OpenConfig-modeled data via a gNMI client.

darwin Device

Next we have to build a device implementation to model a MacBook. The interface to implement is as follows:

type Device interface {
    Alive() (bool, error)
    DeviceID() (string, error)
    Providers() ([]provider.Provider, error)
    Type() string
    IPAddr() string
}

Our new implementation will live in device/devices/darwin.go. We’ll start with a struct called darwin:

type darwin struct {
    deviceID string
    provider provider.GNMIProvider
}

Most of its methods are very simple. The Alive() implementation just returns true. Since we’re running this device on the target device itself, if the code is running, the device is alive:

func (d *darwin) Alive() (bool, error) {
    return true, nil
}

The darwin device uses just the darwin provider, so its Providers() implementation is also short and sweet:

func (d *darwin) Providers() ([]provider.Provider, error) {
    return []provider.Provider{d.provider}, nil
}

The DeviceID() method is a little more involved since it has to get the MacBook’s serial number using the system_profiler command:

func (d *darwin) DeviceID() (string, error) {
    if d.deviceID != "" {
        return d.deviceID, nil
    }

    profiler := exec.Command("system_profiler", "SPHardwareDataType")
    awk := exec.Command("awk", "/Serial/ {print $4}")

    // etc., etc.

    out, err := awk.Output()
    if err != nil {
         return "", err
    }
    d.deviceID = strings.TrimSuffix(string(out), "\n")
    return d.deviceID, nil
}

The Type() method is required to return a string representing a device type defined in the device metadata model. We can return the empty string to let the Collector use a default value for our device.

func (d *darwin) Type() string {
    return ""
}

The last method, IPAddr(), should return the management IP address of the device as a string, if known. Again, we use the MacOS command line utilities to find it.

func (d *darwin) IPAddr() string {
    // we recompute this every time since it can potentially change.
    shCmd := `ifconfig $(route -n get 0.0.0.0 2>/dev/null | awk '/interface: / {print $2}') | ` +
        `grep "inet " | grep -v 127.0.0.1 | awk '{print $2}'`
    out, _ := exec.Command("/bin/sh", "-c", shCmd).Output()
    return strings.TrimSpace(string(out))
}

We also need to register this device so that the Collector knows it exists. Otherwise there’ll be no way to run the device. This is done in the init() function, which is run implicitly when the Collector starts up, as it imports the devices package.

func init() {
    options := map[string]device.Option{
        "pollInterval": device.Option{
            Description: "Polling interval, with unit suffix (s/m/h)",
            Default:     "20s",
        },
    }
    device.Register("darwin", NewDarwinDevice, options)
}

First we define a device option, pollInterval, which makes the interval at which the provider gathers interface statistics configurable. Device options are passed to the Collector binary with the -deviceoption <option>=<value> pattern. Here we make the option non-required and set a default of 20 seconds.

The device.Register call sets the device’s name as darwin, so it can be invoked with

./Collector -device darwin -deviceoption pollInterval=1m ...

It associates with the name darwin a function, NewDarwinDevice, that takes in an option map and returns a new device:

func NewDarwinDevice(options map[string]string) (device.Device, error) {
    pollInterval, err := device.GetDurationOption("pollInterval", options)
    if err != nil {
        return nil, err
    }

    device := darwin{}
    if _, err := device.DeviceID(); err != nil { // Set device ID
        return nil, fmt.Errorf("Failure getting device ID: %v", err)
    }

    device.provider = pdarwin.NewDarwinProvider(pollInterval)
    return &device, nil
}

And that’s it! To check that the device is registered, rebuild the Collector and run:

$ ./Collector -help

The darwin device name should show up as an available device in the -device option’s help string.

Running the Collector

To run this device, first download the GNMIAdapter and the supported YANG models from arista.com and run the GNMIAdapter:

$ ./GNMIAdapter -ingestauth=none -ingestgrpcurl=<cvp-address>:9900 -yangmodules <path-to-modules> &

Then run the Collector with the darwin device:

$ ./Collector -device darwin -deviceoption pollInterval=15s

The MacBook should show up in the CVP Device Inventory, and interface data should appear in the Telemetry Browser under /OpenConfig/interfaces. Of course, the device will not display its associated model, software version, topology data, or other usual features, because we didn’t implement them!