Validation, Testing, Debugging SpiceDB Schemas

Whether you're designing the first iteration of your schema or you're running SpiceDB in production, you'll want tools to build confidence in performance, correctness, and design choices. Tools for validation, testing, and debugging are often overlooked by those building bespoke systems because they can only dedicate enough engineering resources to solving their problem, rather than creating the proper foundation they'll need to continue to be successful in the future.

SpiceDB has been designed with an eye towards being a foundation for authorization and subsequently provides various tools for working.

SpiceDB

Integration Test Server

In order for applications to more easily perform integration tests against SpiceDB, there is a command in SpiceDB for running an integration test server. The integration test server provides an isolated, empty datastore for each unique preshared key used to authenticate an API request. The result of this design is that applications can run integration tests in parallel all against a single SpiceDB so long as they provide a unique credential per test. By default, the server runs on port 50051 and also runs an additional read-only server on port 50052.

You can run the integration test server by executing spicedb serve-testing or by using our GitHub Action that runs the same command.

CheckPermission Tracing Header

While it is recommended that SpiceDB schema be validated and tested before production deployment, there are many scenarios where being able to see the actual paths taken against production data is incredibly important.

To support this, SpiceDB's v1 CheckPermission API supports a debug header that will cause SpiceDB to trace the full set of relations and permission traversed while computing the check.

Configuring this header is done by setting the header io.spicedb.requestdebuginfo to the string true.

The response will include a trailer, io.spicedb.respmeta.debuginfo, with a JSON-encoded tree.

Playground

Assertions

In order to ensure that particular invariants are maintained in a schema, assertions about permissionship can be made.

Assertions come in two flavors: positive and negative. Assertions are written as a YAML list containing zero or more relationships.

assertTrue:
  - "document:validation-testing-debugging#reader@user:you"
assertFalse: []

Caveat Context In Assertions

You can provide caveat context as part of an assertion:

assertTrue:
  - 'document:validation-testing-debugging#reader@user:you with {"somecondition": 42, "anothercondition": "hello world"}'
assertFalse: []

You can also assert that a caveat context is required for a particular expression using assertCaveated:

assertTrue: []
assertCaveated:
  - "document:validation-testing-debugging#reader@user:you"
assertFalse: []

Check Watches

Check Watches are type of assertion that updates in real-time with changes in the Playground. This enables an even tighter feedback-loop when developing a schema.

Below is an example of configuring a Check Watch:

Watches can show any of the following states:

• ✅ Permission Allowed
• ❔ Permission Caveated
• ❌ Permission Denied
• ⚠️ Invalid Check

Expected Relations

Expected Relations are a type of assertion that can be used to enumerate access to a specific relation. This is useful when you want to exhaustively determine all the possible ways that one might acquire access.

Expected Relations are written as YAML lists for each relation:

document:validation-testing-debugging#reader:
  - "[user:you] is <document:validation-testing-debugging#reader>"

Because access can be transitive, Expected Relations include how they achieved access. For example, if a schema is modeled hierarchically with a platform, organization, and project, Expected Relations for projects will include subjects from all points of the hierarchy that have access:

project:docs#admin:
  - "[organization:authzed] is <project:docs#owner>"
  - "[user:rauchg] is <platform:vercel#admin>"

Caveats in Expected Relations

When caveats are involved, and due to the unbounded nature of it, the Playground will focus on enumerating expected relations with "maybe" semantics. You can't specify an expected relation with a specific caveat context, because the Playground supports inferring those for you, and that would lead potentially to an infinite number of possible caveat context values.

What you'll see is an expected relation with the caveat context denoted as [...] right after the resource. This reads as user:rauchg may have admin permission over platform vercel.

project:docs#admin:
  - "[user:rauchg[...]] is <platform:vercel#admin>"

Exceptions in Expected Relations

There are also scenarios where an expected relation is described with an exception, which indicates that a permission holds for a specific resource and subject pair, but with a potential exception.

The following example reads like: user:rauchg has admin permission over platform vercel, unless user:rauchg is banned.

project:docs#admin:
  - "[user:rauchg[...]] is <platform:vercel#admin>/<platform:vercel#banned>"

Check Tracing

SpiceDB supports tracing of check requests to view the path(s) taken to compute the result, as well as timing information.

Request tracing information by setting with_tracing: true in the request message and the information will be found in the response message.

Zed

Zed Validate

The zed binary provides a means of validating and testing a schema locally and in CI:

zed validate my-schema.zed

It will load and validate the schema using the same parsing logic that the SpiceDB binary uses, ensuring that a schema that passes validation will be considered a valid schema by your SpiceDB instance.

You can validate the functionality of your schema using validation yaml files, such as those exported by the Playground:

zed validate schema-and-validations.yaml

Validation files take this form:

schema: >-
  // schema goes here
# -- OR --
schemaFile: "./path/to/schema.zed"
 
# Note that relations are a single heredoc string rather than a yaml list
relationships: >-
  object:foo#relation@subject:bar
  object:baz#relation@subject:qux
 
assertions:
  assertTrue:
    - object:foo#relation@subject:bar
  assertFalse:
    - object:foo#relation@subject:qux
validation:
  object:foo#relation:
    - "[subject:bar] is <object:foo#user>"

As of version v0.25.0, zed validate command can take multiple files as arguments:

zed validate some-validations.yaml some-other-validations.yaml

This means you can validate a folder full of files using shell globbing:

zed validate validations/*

There's an example of this available in the examples repository (opens in a new tab).

If you're using GitHub, there's a GitHub Action for running this validation.

Explain Flag

The zed permission check command has an optional flag, --explain, that will cause SpiceDB to collect the actual paths taken against the live system to compute a permission check. If you're interested in learning more about this functionality in SpiceDB, you can read about the tracing header above.

Here's an example using --explain:

$ zed permission check --explain document:firstdoc view user:fred
true
✓ document:firstdoc view (66.333µs)
├── ⨉ document:firstdoc writer (12.375µs)
└── ✓ document:firstdoc reader (20.667µs)
   └── user:fred 

This command will also highlight which parts of the traversal were cached and if a cycle is detected.

SpiceDB GitHub Actions

authzed/action-spicedb (opens in a new tab)

This GitHub Action runs the SpiceDB Integration Test Server for your workflows with the ability to configure different versions of SpiceDB.

Here's an example snippet of a GitHub Workflow:

steps:
  - uses: "authzed/action-spicedb@v1"
    with:
      version: "latest"

authzed/action-spicedb-validate (opens in a new tab)

The Playground offers a variety of tools that are useful for validating a design, but running the playground isn't designed for operating within a typical CI/CD environment.

Zed provides a command for validation of files exported from the playground which is a perfect fit for being executed within a typical CI/CD environment.

This GitHub Action runs the zed validation command on a provided file for your workflows.

Here's an example snippet of a GitHub Workflow:

steps:
  - uses: "actions/checkout@v4"
  - uses: "authzed/action-spicedb-validate@v1"
    with:
      validationfile: "your-schema.yaml"