Deploying SpiceDB on AWS EKS

AWS EKS (opens in a new tab) is the managed Kubernetes service provided by Amazon and is the best way to run SpiceDB on AWS.

This guide walks through creating a highly-available SpiceDB deployment on an existing EKS cluster. TLS configuration is managed by cert-manager (opens in a new tab) and AWS Route53 (opens in a new tab).

Steps

Confirming the Prerequisites

The rest of this guide assumes kubectl (opens in a new tab) is configured to use an operational EKS cluster along with a Route53 External Hosted Zone.

Creating an IAM Policy

In order for the cluster to dynamically configure DNS, the first step is grant access via the creation of an IAM Policy and attach it to the role used for the pods in EKS:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "route53:GetChange",
      "Resource": "arn:aws:route53:::change/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "route53:ChangeResourceRecordSets",
        "route53:ListResourceRecordSets"
      ],
      "Resource": "arn:aws:route53:::hostedzone/*" 
    {
      "Effect": "Allow",
      "Action": "route53:ListHostedZonesByName",
      "Resource": "*"
    }
  ]
}

Deploying cert-manager

Before modifying any cluster, we recommend double-checking that your current context is configured for the target cluster:

kubectl config current-context

Now you're ready to apply the manifests that install cert-manager:

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/latest/download/cert-manager.yaml

If you'd like to use another installation method for cert-manager, you can find more in the cert-manager documentation (opens in a new tab).

When cert-manager is operational, all the pods should be healthy provided the following command:

kubectl -n cert-manager get pods

Creating a Namespace

Next up, we'll create a new Namespace (opens in a new tab) for deploying SpiceDB.

This guide uses one named spicedb, but feel free to use replace this throughout the guide with whatever you prefer.

kubectl apply --server-side -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: spicedb
EOF

Creating an Issuer & Certificate

In order to setup an ACME (opens in a new tab) flow with Let's Encrypt (opens in a new tab), we'll be configuring cert-manager to use a DNS challenge.

After changing the commented fields below, apply the following manifests:

kubectl apply --server-side -f - <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: spicedb-tls-issuer
  namespace: spicedb
spec:
  acme:
    email: example@email.com                                # Change this
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-production
    solvers:
    - selector:
        dnsZones:
          - "example.com"                                  # Change this
      dns01:
        route53:
          region: us-east-1                                # Change this
          hostedZoneID: ABC123ABC123                       # Change this
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: spicedb-le-certificate
  namespace: spicedb
spec:
  secretName: spicedb-le-tls
  issuerRef:
    name: spicedb-tls-issuer
    kind: ClusterIssuer
  commonName: demo.example.com                            # Change this
  dnsNames:
  - demo.example.com                                      # Change this
EOF

After about one minute, cert-manager should have detected the creation of these resources and created a new Secret (opens in a new tab):

kubectl -n spicedb get secrets

If the spicedb-le-tls secret is not present, follow this troubleshooting guide (opens in a new tab).

Configuring Dispatch TLS

SpiceDB [dispatches] work across instances in the same deployment via [gRPC]. We'll be securing this pod-to-pod communication with its own internal-only certificate:

[gRPC] [dispatches]:

kubectl apply --server-side -f - <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: dispatch-selfsigned-issuer
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: dispatch-ca
  namespace: spicedb
spec:
  isCA: true
  commonName: dev.spicedb   # Change optional: in this example, "dev" is the name of the SpiceDBCluster object (defined at the "Configure SpiceDB settings" step below). If you don't wan't use "dev", change "dev" to what you will use.
  dnsNames:
  - dev.spicedb             # Change optional: (see above)
  secretName: dispatch-root-secret
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: dispatch-selfsigned-issuer
    kind: ClusterIssuer
    group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: my-ca-issuer
  namespace: cert-manager
spec:
  ca:
    secretName: dispatch-root-secret
EOF

Apply the above configuration with: kubectl apply -f internal-dispatch.yaml

Deploying a Datastore

Deploy a Postgres RDS datastore that is network reachable from SpiceDB’s EKS cluster. SpiceDB will connect to your Postgres DB with a connection string.

Configuring and Deploy SpiceDB

Initialize the Operator

Install a release of the operator with: kubectl apply --server-side -f https://github.com/authzed/spicedb-operator/releases/latest/download/bundle.yaml

Configure SpiceDB settings

cat << EOF > spicedb-config.yaml
apiVersion: authzed.com/v1alpha1
kind: SpiceDBCluster
metadata:
  name: dev                                #Change optional: you can change this name, but be mindful of your dispatch TLS certificate URL and service selector
spec:
  config:
    datastoreEngine: postgres                          
    replicas:  2                           #Change optional: at least two replicas are required for HA           
    tlsSecretName: spicedb-le-tls
    dispatchUpstreamCASecretName: dispatch-root-secret
    dispatchClusterTLSCertPath: "/etc/dispatch/tls.crt"
    dispatchClusterTLSKeyPath: "/etc/dispatch/tls.key"
  secretName: dev-spicedb-config
  patches:                                
  - kind: Deployment
     patch:  
       spec:
         template:
           spec:
             containers:
             - name: spicedb
               volumeMounts:
               - name: custom-dispatch-tls
                 readOnly: true
                 mountPath: "/etc/dispatch"
             volumes:
             - name: custom-dispatch-tls
               secret:
                 secretName: dispatch-root-secret
---
apiVersion: v1
kind: Secret
metadata:
  name: dev-spicedb-config
stringData:
  preshared_key: "averysecretpresharedkey" #Change: this is your API token definition and should be kept secure. 
  datastore_uri: "postgresql://user:password@postgres.com:5432" #Change: this is a Postgres connection string
EOF

Apply the above configuration with: kubectl apply -f spicedb-config.yaml -n spicedb

Deploy Cloud Load Balancer Service

This step will deploy a service of type load balancer, which will deploy an external AWS load balancer to route traffic to our SpiceDB pods.

cat << EOF > spicedb-lb.yaml
apiVersion: v1
kind: Service
metadata:
  name: spicedb-external-lb
  namespace: spicedb
spec:
  ports:
  - name: grpc
    port: 50051
    protocol: TCP
    targetPort: 50051
  - name: gateway
    port: 8443
    protocol: TCP
    targetPort: 8443
  - name: metrics
    port: 9090
    protocol: TCP
    targetPort: 9090
  selector:
    app.kubernetes.io/instance: dev-spicedb    #Change optional: in this example, "dev" is the name of the SpiceDBCluster object. If you didn't use "dev", change "dev" to what you used. 
  sessionAffinity: None
  type: LoadBalancer
EOF

Apply the above configuration with: kubectl apply -f spicedb-lb.yaml
Run the following command to get the External-IP of the load balancer: kubectl get services spicedb-external-lb -o json | jq '.status.loadBalancer.ingress[0].hostname'
Take the output of the command and add it as a C-Name record in your route 53 hosted zone

Test

You can use the Zed CLI tool to make sure everything works as expected:

zed context set eks-guide demo.example.com averysecretpresharedkey

Write a schema

zed schema write <(cat << EOF
definition user {}
 
definition doc {
  relation owner: user
  permission view = owner
}
EOF
)

Write a relationship:

zed relationship create doc:1 writer user:emilia

Check a permission:

zed permission check doc:1 view user:emilia

Deploying the SpiceDB Operator Load Testing