github.com/thediveo/noleak

noleak

This module has been integrated into Gomega and become github.com/onsi/gomega/gleak. It has been officially released as of Gomega v1.20.0. Please see the Gomega documentation for gleak: Finding Leaked Goroutines on usage.

noleak complements Gomega with goroutine discovery and leak matchers.

Basic Usage

In your project (with a go.mod) run go get github.com/thediveo/noleak to get and install the latest stable release.

AfterEach(func() {
    // Notice: Goroutines, not: Goroutines()
    Eventually(Goroutines).ShouldNot(HaveLeaked())
})

In case there are "background" goroutines from database drivers, container engine clients, et cetera, you can take a "snapshot" of good goroutines before each test and then afterwards filter out the known good goroutines:

var ignoreGood []goroutine.Goroutine

BeforeEach(func() {
    ignoreGood = Goroutines()
})

AfterEach(func() {
    Eventually(Goroutines).ShouldNot(HaveLeaked(ignoreGood))
})

For more details, please refer to the noleak package documentation.

Credits

noleak has been heavily inspired by Uber's goleak goroutine leak detector. That's definitely a fine piece of work!

But then why another goroutine leak package? After a deep analysis of Uber's goleak we decided against crunching goleak somehow half-assed into the Gomega TDD matcher ecosystem. In particular, reusing and wrapping of the existing Uber implementation would have become very awkward: goleak.Find combines all the different elements of getting actual goroutines information, filtering them, arriving at a leak conclusion, and even retrying multiple times all in just one single exported function. Unfortunately, goleak makes gathering information about all goroutines an internal matter, so we cannot reuse such functionality elsewhere.

Users of the Gomega ecosystem are already experienced in arriving at conclusions and retrying temporarily failing expectations: Gomega does it in form of Eventually().ShouldNot(), and (without the trying aspect) with Expect().NotTo(). So what is missing is only a goroutine leak detector in form of the HaveLeaked matcher, as well as the ability to specify goroutine filters in order to sort out the non-leaking (and therefore expected) goroutines, using a few filter criteria. That is, a few new goroutine-related matchers. In this architecture, even existing Gomega matchers can optionally be (re)used as the need arises.

In the end, we now can fluently write in typical Gomega style and when dot-importing noleak:

// goleak: Expect(goleak.Find()).NotTo(HaveOccured())
Eventually(Goroutines).ShouldNot(HaveLeaked())

Or when ignoring "non-standard" background goroutines:

ignoreGood := Goroutine()
DoSomething()
Eventually(Goroutines).ShouldNot(HaveLeaked(ignoreGood))
// goleak:
//   opt := goleak.IgnoreCurrent()
//   DoSomething()
//   Expect(goleak.Find(opt)).NotTo(HaveOccured())

Notes

Go Version Support

noleak supports versions of Go that are noted by the Go release policy, that is, major versions N and N-1 (where N is the current major version).

Leakiee the Gopher

Credit, where credit is due: our mascot Leakiee clearly has been inspired by Renee French's (he of the Go gopher) work of art. But there's not only a father, but also a mother: Leakiee is the sibling to Morby, the "Incontinentainer" whale; which in turn has been inspired by Laurel's beautiful Moby Dock and friends artwork for Docker Inc.

Goroutine IDs

In order to detect goroutine identities, we use what is termed "goroutine IDs". These IDs appear in runtime stack dumps ("backtraces"). But … are these goroutine IDs even unambiguous? What are their "guarantees" (if there are any at all)?

First, Go's runtime code uses the identifier (and thus term) goid for Goroutine IDs. Good to know in case you need to find your way around Go's runtime code.

Now, based on Go's goid runtime allocation code (links to v1.18 branch), goroutine IDs never get reused – unless you manage to make the 64bit "master counter" of the Go runtime scheduler to wrap around. However, not all goroutine IDs up to the largest one currently seen might ever be used, because as an optimization goroutine IDs are always assigned to Go's "P" processors for assignment to newly created "G" goroutines in batches of 16. In consequence, there may be gaps and later goroutines might have lower goroutine IDs if they get created from a different P.

Finally, Scott Mansfield on Goroutine IDs. To sum up Scott's point of view: don't use goroutine IDs. He spells out good reasons for why you should not use them. However, obviously logging, debugging and testing looks like a valid exemption from his rule, not least runtime.Stack includes the goids for some reason.

⚖️ Copyright and License

noleak is Copyright 2022 Harald Albrecht, and licensed under the Apache License, Version 2.0.