Office Hours: ACL-aware filtering in your own database with SpiceDB and AuthZed Materialize

We picked Go for the Ecosystem

/assets/team/jimmy-zelinskie.jpg
October 20, 2021|5 min read

At Authzed, we're building a platform to manage the permissions built into applications. While services like Okta and PingFederate tackle the question of who you are, Authzed tackles what you can do. The foundation of solving this problem is SpiceDB, our open source graph database that builds upon the design expertise used to solve this problem by companies like Google (via Zanzibar) and Facebook (via Social Graph). After building an initial prototype of SpiceDB in Python, it became clear that we needed to address some architectural concerns and we took the opportunity to consider rebuilding with a different language. Despite the common sentiment on Python's performance characteristics, our prototype was surprisingly fast, as we heavily used functional programming patterns to compose and perform computations lazily and efficiently.

Our requirements were to mostly maintain the clarity of implementation, but also have more control over concurrency, performance, and memory efficiency. SpiceDB requires reasonably low latency: roughly ~20ms p99 for our permissions checking API requests. This is not quite the threshold where we'd have to use a non-garbage collected language, but regardless we did take a moment to consider Rust anyway. Our team is mostly comprised of ex-CoreOS engineers, so we were extremely familiar with Go. I'll try to avoid rehashing all of the commonly discussed advantages of Go, since those have been well covered for years.

While gophers love to gush over the fast compiler toolchain and standard tools, you'd be leaving something on the table by ignoring the third-party tools outside of what the Go team has built. Linting tools like golangci-lint and release management tools like goreleaser, once setup, are a phenomenal boon to contributor workflow. This attention to a good experience spans not only developer tools, but also the libraries that build experiences for our own users. For example, the excellent spf13/cobra makes it trivial to create intuitive command-line interfaces with tab-completion and a unified way to pass flags as environment variables. For our power-users that are looking for good experiences with deployment and operational tooling, the ease at which we could expose our Go software to the industry's best observability tooling, was second to none. With almost no additional development overhead, we've been able to instrument our programs with metrics via Prometheus, distributed tracing via OpenTelemetry, structured logging via zerolog, and profiling via pprof.

You might be seeing a theme here: the largest impact on our decision was not the language properties, but actually the Go ecosystem. There are many databases implemented in Go, so we have the option to avoid reimplementing many algorithms and libraries that we needed to build SpiceDB. Notably, we're huge fans of dgraph's Ristretto and InfluxDB's tdigest. Because of the ubiquity of Go idioms and standard workflows throughout the Go open source community, once adopted, it was quite easy for us to contribute back changes to the code we were sharing, benefiting everyone. As SpiceDB requires client libraries for various languages, we are quite sensitive to the maturity of the gRPC ecosystem in our primary language. In this respect, Go shines: all gRPC and Protobuf libraries are natively implemented in Go and are extremely mature -- better than any other language I've used. The availability of idiomatic, native Go libraries for software that is typically only made available via C is a breath of fresh air coming from our previous language choice.

But, in the end, we cannot forget that it's always about people and not code. The Go community has always been extremely welcoming and caring to their fellow gophers. As we've worked with Go over the years, we've met some of the most phenomenally smart, thoughtful, and kind people. These are the people that we'd be honored to have represent and extend our company culture as we continue to grow. It's been almost a year since we made our decision to invest in Go as our primary development language. We're so appreciative of all the value we've gained from doing so and plan to ensure that our work will always reciprocate value back to the wonderful people that have made all of this possible.

If this post resonates with you and you have a passion for open source, Go, databases, distributed systems, we invite you to explore the community and code for SpiceDB and, if you're looking for a workplace with those same values, you can take a look at openings on our Jobs page.

Additional Reading

If you’re interested in learning more about Authorization and Google Zanzibar, we recommend reading the following posts:

Get started for free

Join 1000s of companies doing authorization the right way.