Selecting a Datastore
SpiceDB ships with a number of datastores -- drivers used to store and retrieve the schema and relationship data for computing permissions.
There are a few available datastores with various design goals:
- CockroachDB - Recommended for multi-region deployments
- Cloud Spanner - Suitable for cloud multi-region deployments
- MySQL - Recommended for single-region deployments and those familiar with traditional RDBMS operations
- PostgreSQL - Recommended for single-region deployments and those familiar with traditional RDBMS operations
- memdb - Recommended for local development and integration testing with applications written to be SpiceDB-native
Migrations​
Before a datastore can be used by SpiceDB or before running a new version of SpiceDB, you must execute all available migrations. The only exception is the memdb datastore because it does not persist any data.
In order to migrate a datastore, run the following command with your desired values:
spicedb migrate head --datastore-engine $DESIRED_ENGINE --datastore-conn-uri $CONNECTION_STRING
CockroachDB​
note
If you want to use the Watch API with CockroachDB, an additional capability must be enabled within the database. See Enabling Watch API for CRDB
Usage Notes​
- Recommended for multi-region deployments, with configurable region awareness
- Enables horizonally scalablity by adding more SpiceDB and CockroachDB instances
- Resiliency to individual CockroachDB instance failures
- Query and data balanced across the CockroachDB
- Setup and operational complexity of running CockroachDB
Developer Notes​
- Code can be found here
- Documentation can be found here
- Implemented using pgx for a SQL driver and connection pooling
- Has a native changefeed
- Stores migration revisions using the same strategy as Alembic
Configuration​
danger
In distributed systems, you can trade-off consistency for performance.
Users that are willing to rely on subtle guarantees to mitigate the New Enemy Problem can configure --datastore-tx-overlap-strategy.
The available strategies are:
static(default) - A single key (--datastore-tx-overlap-key) is used in all writes to ensure proper consistencyprefix(unsafe if misused) - A key with the prefix from the object type is used to protect writes with the same prefixinsecure(unsafe) - Disables the overlap strategy entirely leaving queries vulnerable to the New Enemy problem
Required Parameters​
| Parameter | Description | Example |
|---|---|---|
datastore-engine | the datastore engine | --datastore-engine=cockroachdb |
datastore-conn-uri | connection string used to connect to CRDB | --datastore-conn-uri="postgres://user:password@localhost:26257/spicedb?sslmode=disable" |
Optional Parameters​
| Parameter | Description | Example |
|---|---|---|
datastore-max-tx-retries | Maximum number of times to retry a query before raising an error | --datastore-max-tx-retries=50 |
datastore-tx-overlap-strategy | The overlap strategy to prevent New Enemy on CRDB (see below) | --datastore-tx-overlap-strategy=static |
datastore-tx-overlap-key | The key to use for the overlap strategy (see below) | --datastore-tx-overlap-key="foo" |
datastore-conn-max-idletime | Maximum idle time for a connection before it is recycled | --datastore-conn-max-idletime=60s |
datastore-conn-max-lifetime | Maximum lifetime for a connection before it is recycled | --datastore-conn-max-lifetime=300s |
datastore-conn-max-open | Maximum number of concurrent connections to open | --datastore-conn-max-open=10 |
datastore-conn-min-open | Minimum number of concurrent connections to open | --datastore-conn-min-open=1 |
datastore-query-split-size | The (estimated) query size at which to split a query into multiple queries | --datastore-query-split-size=5kb |
datastore-gc-window | Sets the window outside of which overwritten relationships are no longer accessible | --datastore-gc-window=1s |
datastore-revision-fuzzing-duration | Sets a fuzzing window on all zookies/zedtokens | --datastore-revision-fuzzing-duration=50ms |
datastore-readonly | Places the datastore into readonly mode | --datastore-readonly=true |
datastore-follower-read-delay-duration | Amount of time to subtract from non-sync revision timestamps to ensure follower reads | -datastore-follower-read-delay-duration=4.8s |
Cloud Spanner​
Usage Notes​
- Requires a Google Cloud Account with an active Cloud Spanner instance
- Take advantage of Google's TrueTime. The Spanner driver assumes the database is linearizable and skips the transaction overlap strategy required by CockroachDB.
Developer Notes​
- Code can be found here
- Documentation can be found here
- Starts a background GC worker to clean up old entries from the manually-generated changelog table
Configuration​
- The Cloud Spanner docs outline how to set up an instance
- Authentication via service accounts: The service account that runs migrations must have
Cloud Spanner Database Admin; SpiceDB (non-migrations) must haveCloud Spanner Database User.
Required Parameters​
| Parameter | Description | Example |
|---|---|---|
datastore-engine | the datastore engine | --datastore-engine=spanner |
datastore-conn-uri | the cloud spanner database identifier | --datastore-conn-uri="projects/project-id/instances/instance-id/databases/database-id" |
Optional Parameters​
| Parameter | Description | Example |
|---|---|---|
datastore-spanner-credentials | JSON service account token (omit to use application default credentials) | --datastore-spanner-credentials=./spanner.json |
datastore-gc-interval | Amount of time to wait between garbage collection passes | --datastore-gc-interval=3m |
datastore-gc-window | Sets the window outside of which overwritten relationships are no longer accessible | --datastore-gc-window=1s |
datastore-revision-fuzzing-duration | Sets a fuzzing window on all zookies/zedtokens | --datastore-revision-fuzzing-duration=50ms |
datastore-readonly | Places the datastore into readonly mode | --datastore-readonly=true |
datastore-follower-read-delay-duration | Amount of time to subtract from non-sync revision timestamps to ensure stale reads | --datastore-follower-read-delay-duration=4.8s |
MySQL​
Usage Notes​
- Recommended for single-region deployments
- Setup and operational complexity of running MySQL
- Does not rely on any non-standard MySQL extensions
- Compatible with managed MySQL services
Developer Notes​
- Code can be found here
- Documentation can be found here
- Implemented using Go-MySQL-Driver for a SQL driver
- Query optimizations are documented here
- Implements its own MVCC model by storing its data with transaction IDs
Configuration​
Required Parameters​
| Parameter | Description | Example | |
|---|---|---|---|
datastore-engine | the datastore engine | --datastore-engine=mysql | |
datastore-conn-uri | connection string used to connect to MySQL | --datastore-conn-uri="mysql:password@localhost:5432/spicedb?parseTime=True&sslmode=disable" |
MySQL Time​
note
--datastore-conn-uri must contain parseTime=true as a query parameter
in order support time-based operations such as Garbage Collection in SpiceDB.
Example: --datastore-conn-uri="mysql:password@localhost:5432/spicedb?sslmode=disable&parseTime=True"
Optional Parameters​
| Parameter | Description | Example |
|---|---|---|
datastore-conn-max-idletime | Maximum idle time for a connection before it is recycled | --datastore-conn-max-idletime=60s |
datastore-conn-max-lifetime | Maximum lifetime for a connection before it is recycled | --datastore-conn-max-lifetime=300s |
datastore-conn-max-open | Maximum number of concurrent connections to open | --datastore-conn-max-open=10 |
datastore-query-split-size | The (estimated) query size at which to split a query into multiple queries | --datastore-query-split-size=5kb |
datastore-gc-window | Sets the window outside of which overwritten relationships are no longer accessible | --datastore-gc-window=1s |
datastore-revision-fuzzing-duration | Sets a fuzzing window on all zookies/zedtokens | --datastore-revision-fuzzing-duration=50ms |
datastore-mysql-table-prefix string | Prefix to add to the name of all SpiceDB database tables | --datastore-mysql-table-prefix=spicedb |
datastore-readonly | Places the datastore into readonly mode | --datastore-readonly=true |
PostgreSQL​
note
If you want to use the Watch API with PostgreSQL, an additional capability must be enabled within the database. See Enabling Watch API for PostgreSQL
Usage Notes​
- Recommended for single-region deployments
- Resiliency to failures only when PostgreSQL is operating with a follower and proper failover
- Setup and operational complexity of running PostgreSQL
- Does not rely on any non-standard PostgreSQL extensions
- Compatible with managed PostgreSQL services
Developer Notes​
- Code can be found here
- Documentation can be found here
- Implemented using pgx for a SQL driver and connection pooling
- Stores migration revisions using the same strategy as Alembic
- Implements its own MVCC model by storing its data with transaction IDs
Configuration​
Required Parameters​
| Parameter | Description | Example |
|---|---|---|
datastore-engine | the datastore engine | --datastore-engine=postgres |
datastore-conn-uri | connection string used to connect to PostgreSQL | --datastore-conn-uri="postgres://postgres:password@localhost:5432/spicedb?sslmode=disable" |
Optional Parameters​
| Parameter | Description | Example |
|---|---|---|
datastore-conn-max-idletime | Maximum idle time for a connection before it is recycled | --datastore-conn-max-idletime=60s |
datastore-conn-max-lifetime | Maximum lifetime for a connection before it is recycled | --datastore-conn-max-lifetime=300s |
datastore-conn-max-open | Maximum number of concurrent connections to open | --datastore-conn-max-open=10 |
datastore-conn-min-open | Minimum number of concurrent connections to open | --datastore-conn-min-open=1 |
datastore-query-split-size | The (estimated) query size at which to split a query into multiple queries | --datastore-query-split-size=5kb |
datastore-gc-window | Sets the window outside of which overwritten relationships are no longer accessible | --datastore-gc-window=1s |
datastore-revision-fuzzing-duration | Sets a fuzzing window on all zookies/zedtokens | --datastore-revision-fuzzing-duration=50ms |
datastore-readonly | Places the datastore into readonly mode | --datastore-readonly=true |
memdb​
Usage Notes​
- Fully ephemeral; all data is lost when the process is terminated
- Intended for usage with SpiceDB itself and testing application integrations
- Cannot be ran highly-available as multiple instances will not share the same in-memory data
note
If you need an ephemeral datastore designed for validation or testing, see the test server system in Validating and Testing
Developer Notes​
- Code can be found here
- Documentation can be found here
- Implements its own MVCC model by storing its data with transaction IDs
Configuration​
Required Parameters​
| Parameter | Description | Example |
|---|---|---|
datastore-engine | the datastore engine | --datastore-engine memory |
Optional Parameters​
| Parameter | Description | Example |
|---|---|---|
datastore-revision-fuzzing-duration | Sets a fuzzing window on all zookies/zedtokens | --datastore-revision-fuzzing-duration=50ms |
datastore-gc-window | Sets the window outside of which overwritten relationships are no longer accessible | --datastore-gc-window=1s |
datastore-readonly | Places the datastore into readonly mode | --datastore-readonly=true |