Using Golang for Creating Linux System Monitoring Daemons

When I first created my current Raspberry Pi 4 NTP server, I created a monitoring daemon in Java using Pi4j. I’ve got decades of experience in Java programming, so it was pretty straight-forward. The biggest problem for the Java application was writing the device driver for the Adafruit I2C/SPI backpack.

I wanted to learn Golang and this seemed like a great project. It was re-implementing something I’d already done, and it was also non-trivial, requiring some real-world skills including:

  • Network Socket Communication
  • Processing JSON messages
  • Hardware device interfaces
  • Executing external programs and reading output.
  • Multi-processing/multi-threading.
  • Processing configuration files.

What I found is that golang is really well suited for writing daemons. This isn’t surprising when you consider some of the software that is powered by golang. Things like docker, helm, kubernetes, etc. It also threads the needle between interpreted code, and compiled code with the use of a garbage collector.

Go Routines

The first really cool thing about Go is how simple it is to write a background process. In Java, you would create a class that implements Runnable, and create a new thread with it. In golang, you declare a function and execute using the go statement.

func monitor_something() {
    for {
        // Perform the monitoring process here
        time.Sleep(time.Second)
    }
}

func main() {
  go monitor_something()
  go monitor_something_else()
  for {
    time.Sleep(10 * time.Second)
  }
}

You can also create go routines as closures:

func main() {
  i:=0
  go func() {
     for {
         // Some background process code here.
         i = i + 1
         time.Sleep(time.Second)
     }
  }()

  for {
     // do other things.
     log.Print("The value of i is: ", i)
     time.Sleep(time.Second)
  }
}

the anonymous go routine runs as a background process but has scoped access to the variables in the enclosing function.

Channels

The next really cool feature of golang that made this project easier was channels. In golang, a channel is a communications pipe that you can send and receive data on. In Java and other languages, communications between threads is done using memory variables, and access is controlled via semaphores/mutexes. In golang, channels are the preferred mechanism. Here’s a really simple example:

// Create the channel
ch := make(chan int, 16)
// Write the value 32 to it.
ch<- 32
// Read the next value from the channel and assign the value to chValue
chValue:=<-ch

Channels can also handle more complex data types:

type Alarm struct {
	Key         string
	Description string
	Raised      bool
}
alarmChannel = make(chan Alarm, alarmChannelSize)

Here’s an example of a rotary switch implementation using channels. It shows how the switch class generates messages on a channel that consumers can read from.

type RotarySwitchValue int

func (sw *RotarySwitch) Position() int {
	return sw.position
}
// Construct a new Rotary Switch by passing in the gpio.PinIO.
// 
// maxPosition can be used to  keep track of the relative position of the switch.
//
// To receive events, read the Channel returned by RotarySwitch.Channel(). The
// value returned will be one of the constants.
//
// If you're using a Rotary Switch that doesn't include a button switch, pass
// gpio.INVALID for buttonPin.
func NewRotarySwitch(statePin, dataPin, buttonPin gpio.PinIO, maxPositions int)
			 *RotarySwitch {
	statePin.In(gpio.PullUp, gpio.BothEdges)
	dataPin.In(gpio.PullUp, gpio.NoEdge) 
	
	channelSize := 16
	sw := RotarySwitch{ch: make(chan RotarySwitchValue, channelSize),
		st_pin:        statePin,
		data_pin:      dataPin,
		button_pin:    buttonPin,
		max_positions: maxPositions,
		channelSize:   channelSize,
		last_state:    gpio.High,
		last_button:   gpio.Low}
	// Start the go routine that will put events on the channel.
	go rotaryHandler(&sw)
    if buttonPin!=gpio.INVALID {
        buttonPin.In(gpio.PullUp, gpio.BothEdges)
	// Start the go routine that will put button presses on the channel.
        go buttonHandler(&sw)
    }
	log.Print("Rotary switch provisioned.")
	return &sw
}

func main() {
	sw := switches.NewRotarySwitch(
        	gpioreg.ByName("GPIO21"),
	        gpioreg.ByName("GPIO20"),
		gpioreg.ByName("GPIO19"),
		6)

	defer sw.Close()
	for {
		f := functions[functionIndex]
		mr := f()
                scroller.SetLines(mr.Data)
		select {
		case swAction := <-sw.Channel():
		switch swAction {
                case switches.ButtonPress:
                	// TODO: Turn the display off on a button press.
			log.Println("Button Pressed")
		case switches.ButtonRelease:
			log.Println("Button released")
	         default:
        	        functionIndex = sw.Position()
		}
		case <-time.After(10 * time.Second):
	}
}

In this case, the select switch will block until a rotary switch event happens, putting data on the channel, or until a timeout occurs. The really clever thing about this is that when a rotary switch event happens, the select statement unblocks immediately. I don’t have to do anything tricky to make the sleep routine exit prematurely. If I were doing this in Java, I would have a listener function that would receive the event notification and it would change the variable. I would then use a short timeout and update if the value changed.

In a future post, I’ll discuss the alarming system of the monitoring daemon and how it uses channels.

Resources

The two best resources for learning Golang on this project were:

The Go Programming Language by Alan Donovan and Brian Kernighan

The Go Programming Language is the authoritative resource for any programmer who wants to learn Go. It shows how to write clear and idiomatic Go to solve real-world problems. The book does not assume prior knowledge of Go nor experience with any specific language, so you’ll find it accessible whether you’re most comfortable with JavaScript, Ruby, Python, Java, or C++.

Concurrency in Go by Ketherine Cox-Buday

Concurrency can be notoriously difficult to get right, but fortunately, the Go open source programming language makes working with concurrency tractable and even easy. If you’re a developer familiar with Go, this practical book demonstrates best practices and patterns to help you incorporate concurrency into your systems.