In the previous posts, I covered the basics of connecting to NATS in Go and the different ways subscribers can request information is sent to them.

In this post, I’d like to build on those concepts by exploring how to structure your NATS-powered Go code so that things are clean and DRY. I’ll also show that trying to make things too DRY can be problematic; as with everything, moderation is a good idea.

Getting a Little Too DRY

I have seen a lot of projects use messaging systems like NATS, where each service implements its own connectivity logic from scratch, which is not only unnecessarily repetitive, it also creates a bit of a management nightmare.

Rather, it seems sensible to centralize communications-related functionality in a separate package of your Go applications, rather than have function calls sprinkled throughout your application that call a system like NATS. This will allow you to make changes to the communications layer much more cleanly in the future.

To facilitate this, I created a package called comms where I created a type for holding not only the NATS connection objects, but also all channels that a microservice might want to use.

// Comms centralizes connection objects and channels so we have one place to
// go in order to send or receive messages with NATS.
type Comms struct {

	// NATS connection types
	Nc *nats.Conn
	Ec *nats.EncodedConn

	// Send/Receive Channels
	RequestChanSend chan *Request
	RequestChanRecv chan *Request
}

I also created a function that can be called by any of my services which returns a pre-populated instance of Comms.

// NewComms returns an instance of Comms with a running connection to the NATS server
// and channels pre-bound to NATS subjects, ready to send/receive messages
func NewComms() (*Comms, error) {

	nc, err := nats.Connect(nats.DefaultURL)
	if err != nil {
		return nil, err
	}

	ec, err := nats.NewEncodedConn(nc, nats.JSON_ENCODER)
	if err != nil {
		return nil, err
	}

	cc := Comms{}

	// Bind recieve channel using Queues
	cc.RequestChanRecv = make(chan *Request)
	ec.BindRecvQueueChan(SUBJECT_HELLO, "hello_queue", cc.RequestChanRecv)

	// Bind send channel
	cc.RequestChanSend = make(chan *Request)
	ec.BindSendChan(SUBJECT_HELLO, cc.RequestChanSend)

	return &cc, nil
}

The idea here was that I would just need to call this function once from each service, and it will return a pointer to a Comms instance with channels ready to send and receive data.

This definitely did have a positive effect on the code at the publisher and subscriber - instead of having to do the above code for each service, I just had to call the NewComms() function.

// A single type for all comms means all our communication channels are
// properties of c.
c, err := comms.NewComms()
if err != nil {
  panic(err)
}
defer c.Ec.Close()

However, when I actually ran these, I saw some interesting behavior. It appeared as though some of the messages being sent from the publisher were being dropped along the way. Since I was putting a counter variable in each of the messages, this was easy to see:

Since I’ve only ever heard that NATS is super battle-tested and stable, I knew it was something I was doing wrong, and not NATS going on the fritz. So ensued several hours of me looking through the source of the Go client - here’s what I found.

The BindRecvChan() function we have been calling in our code is located in netchan.go and is ultimately satisfied by an unexported function that declares a handler for incoming messages, and passes it off to the subscribe function of the connection object. If you look at how that function handles subscriptions when such a handler is defined, you’ll notice that it spins up a goroutine to handle all future incoming messages.

What this means is that simply by binding our Go channel to the NATS client, we’re already subscribing to the indicated channel. This is easy to miss if you’re not even intending on using the subscription logic - if you centralize your code like I did above, even if you only care to use the publishing channel, on binding a receive channel, you’re a subscriber. What was happening in the terminal output above was that NATS wasn’t dropping any messages; rather, I had one more subscriber than I intended, and NATS was load-balancing messages to both.

This seems a bit obvious in retrospect, but this kind of thing can happen often if you get over-zealous about code DRY-ness, neglecting deep understanding about what your code actually does.

Events, Personas, and Constructors

Okay now that my silly mistake (which totally didn’t take hours out of my evening) is out of the way - how can we centralize the communication logic to get as DRY as possible without going overboard?

As with anything in programming, there’s probably no single “right” answer. For me, though, I still wanted to use the experience of binding to Go channels, which meant I needed to find a way to allocate the right channels for the right use cases while consolidating as much as possible into a single package and a few functions so that we retain the cleanliness of having this logic centralized.

For communications-related functionality like this I like to think about the various microservices that will be using NATS and breaking them down by use case. When building a proper comms package, we really need three things:

• Events - we need to define the various messages or “events” that will be sent to NATS. This is mostly just a bunch of Go structs, which represent the messages that will be flowing “on the wire”. I especially like having this in its own file, so anyone can at any point know the kind of things that could be taking place across the whole application.
• Personas - next, we need to define the use cases, or “personas” that will be using NATS. In our hyper-simplistic example, we have a “publisher”, and a “subscriber”. This file should define each as its own struct where we can store all of the connection objects and channels we will actually use for communication.
• Constructors - finally, we need functions that are designed to set up a new instance of each persona struct, and return it to the caller that it can start using it. The idea is that each persona gets it’s own function for doing this, so that the caller has one place to go to get their comms set up and ready to go.

I like this layout because:

• We have still defined everything centrally in the comms package, so all services can stay simple.
• Each file contains only one aspect of the communication paradigm, so finding things is easy.
• Extending this by adding a new struct or function to each file is intuitive.
• Each persona gets its own setup function, so we have the ability to only include the channels that are needed by that particular use case, which avoids our earlier difficulties.

I’m sure it’s not the only way to do this, but it’s a way that makes sense to me. If you know of a better way, especially one that’s more Go-idiomatic - feel free to let me know in the comments below, I’m still learning!

Stay tuned for the next post in this series - I have a bit more NATS work to do. :)