CockroachDB

Deploy CockroachDB on Control Plane using the Template Catalog. Covers locations and replicas, PgBouncer connection pooling, region survivability, backups to S3 or GCS, the two working restore paths, and migrating from template version 1.x.

Template version 2.0.0 is a breaking change, and one of the changes is a data-loss hazard.

• The template no longer creates a GVC. It deploys into a GVC you already have. gvc.name is gone, and gvc.locations moved to a top-level locations list.
• Never upgrade a 1.x release onto 2.0.0 in place. A 1.x release owns the GVC it created, and Helm deletes what a chart stops declaring — the upgrade destroys that GVC and everything inside it. Install a new release instead: Migrating from 1.x.
• The restore procedure earlier versions documented could never work. The backup is a full-cluster backup, and CockroachDB refuses to restore one onto a cluster that already has databases — which this template always creates. Both working paths are in Restoring a Backup.
• The default is now a single location, so surviving the loss of a region is off until you configure three. See Multi-Region Survivability.

Overview

CockroachDB is a distributed SQL database that provides automatic replication, horizontal scalability, and built-in fault tolerance. This template deploys a CockroachDB cluster on Control Plane as a stateful workload with replica-direct load balancing, a PgBouncer connection pooler in front of it, and optional scheduled backups to S3 or GCS.

Each location runs a configurable number of nodes that discover and join one another using Control Plane’s internal DNS. On first deployment the cluster initializes itself and creates a database and user. With three or more locations it also registers every location as a CockroachDB region and sets the survival goal to SURVIVE REGION FAILURE.

What Gets Created

• Stateful CockroachDB Workload — CockroachDB (v25.4.0) with per-location replica counts and replica-direct load balancing. Each node drains gracefully on shutdown.
• Standard PgBouncer Workload (enabled by default) — A connection pooler that multiplexes application connections across every CockroachDB node in every location. When enabled, this becomes the primary connection endpoint.
• Volume Set — Persistent ext4 storage (general-purpose SSD), one volume per node, with a final snapshot on deletion and 7-day retention.
• Identity & Two Policies — An identity bound to the workloads, with reveal on this release’s secrets, view on the one GVC you install into so each node can confirm at boot that the GVC really has every location you listed, and cloud storage access when backup is enabled.
• Secrets — The CockroachDB startup script, the PgBouncer startup script, and a small database-configuration secret.
• Backup Cron Workload (optional) — A scheduled job that issues a CockroachDB BACKUP INTO statement, unsuspended in exactly one location.

This template does not create a GVC. It deploys into a GVC you already have — every resource lands in the GVC you install into, so cpln workload exec, cpln logs and uninstalling all work against that GVC, and uninstalling can never delete it. Every location you list in locations must already be on that GVC, and a GVC location you did not list simply runs nothing.

Architecture

CockroachDB uses the Raft consensus protocol to replicate data across nodes. Each Control Plane location maps to a CockroachDB locality region, and nodes advertise their address via internal DNS (replica-N.WORKLOAD.LOCATION.GVC.cpln.local).

With three or more locations and the SURVIVE REGION FAILURE survival goal, the cluster tolerates the complete loss of one location without interrupting reads or writes.

Prerequisites

A GVC must already exist, and it must contain every location you list in locations. The requirement is one-directional — the GVC may have more locations than you list, and nothing CockroachDB-related runs in those. Check what a GVC has before installing:

cpln gvc get GVC_NAME -o json

The locations are under spec.staticPlacement.locationLinks. To add a missing one:

cpln gvc add-location GVC_NAME --location aws-us-east-2

Every workload in a GVC runs in every location that GVC has, so add locations to a shared GVC deliberately.

A location the GVC does not have is not caught at install time. The install succeeds — the platform does not validate it — and the CockroachDB containers then refuse to initialize, restarting with this in the logs:

[cockroach] FATAL: locations declared in values are not in GVC 'my-gvc': aws-us-west-2
[cockroach] GVC 'my-gvc' has: aws-us-east-1 aws-us-east-2

A node that already holds data logs a WARNING instead and keeps serving, so this check can never take down a running cluster. Read it with a server-side filter:

cpln logs '{gvc="GVC_NAME", workload="RELEASE_NAME-cockroach"} |= "[cockroach]"' --limit 100 --since 15m

Nothing else is required for a default install. Backups additionally need a storage bucket and a Control Plane cloud account — see Backup.

Installation

To install, follow the instructions for your preferred method:

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Migrating from 1.x

Template versions through 1.5.0 created their own GVC, so that GVC is part of the 1.x release’s manifest. Version 2.0.0 does not declare it — and Helm deletes what a chart stops declaring.

An in-place upgrade from 1.x to 2.0.0 destroys the cluster. Measured on a sibling template with the guard removed: the upgrade deleted the GVC and every workload, volume set and identity inside it in about six seconds, while printing upgraded successfully. Reading the GVC back afterwards returned 404. The volume sets hold your data.

The chart ships a render-time refusal so this cannot happen by accident: any leftover gvc key in your values aborts the upgrade before a single API call is made, leaving your cluster untouched and running.

Error: execution error at (cockroach/templates/identity.yaml:2:4): cockroach 2.0.0: the `gvc` values
key was REMOVED. This chart no longer creates a GVC -- it deploys into the GVC you install into, and
`gvc.locations` moved to the top-level `locations`. DO NOT `helm upgrade` a 1.x release onto 2.0.0:
the upgrade drops `kind: gvc` from the manifest and Helm deletes what a chart no longer declares,
which DESTROYS that GVC and every workload, volumeset and identity inside it.

The guard cannot cover one case: an upgrade run with no values file at all. A 1.x release installed on pure defaults has no gvc key for the chart to see, so nothing fires and the deletion proceeds. Do not run an upgrade of a 1.x release against the 2.0.0 chart under any circumstances — install a new release and move the data across.

1. Back up the old cluster

   Enable backup on the 1.x release, or issue a BACKUP INTO by hand against a node. See Backup for the bucket and cloud account setup.

2. Choose the GVC for the new release

   Create or pick a GVC and make sure it has exactly the locations you intend to list in locations. See Prerequisites.

3. Install 2.0.0 as a NEW release into that GVC

   Use a different release name: secret names are org-wide, so a same-named release collides with the 1.x one even in another GVC.

4. Restore and cut over

   Restore into the new cluster (see Restoring a Backup), then point your applications at the new PgBouncer endpoint.

5. Uninstall the old release against the GVC you installed it into

   Not the GVC it created — the GVC you passed at install time is where Helm tracks the release, and uninstalling from there takes the created GVC with it.

Two other values keys changed in 2.0.0:

• gvc.locations is now the top-level locations, and gvc.name is gone entirely.
• replicas: 0 on a location is refused at render. It used to suspend that location silently while still counting it as a region the database had been told about, which no node ever joined. Remove the location from locations instead.

Configuration

The default values.yaml for this template:

# ─── Locations ────────────────────────────────────────────────────────────────
# This chart deploys into the GVC you install into — it does NOT create one.
# Every location listed here MUST already exist in that GVC, or the CockroachDB
# nodes refuse to initialise with a named error (see Prerequisites in the README).
# Extra locations in the GVC are fine: nothing CockroachDB-related runs in them.
#
# Surviving the loss of a whole location needs at least THREE locations. One
# location with three replicas survives a node loss, not a region loss.
locations:
  - name: aws-us-east-1
    replicas: 3

image: cockroachdb/cockroach:v25.4.0

# Confirm every location in your GVC supports multi-zone before enabling this --
# a location that does not will wedge with no error. See the README.
multiZone: false

resources:
  cpu: 2
  memory: 4Gi

database:
  name: mydb
  user: myuser

volumeset:
  capacity: 10 # Initial capacity in GiB (minimum is 10)
  autoscaling:
    enabled: false # Set to true to enable autoscaling
    maxCapacity: 100 # Maximum capacity in GiB when autoscaling is enabled
    minFreePercentage: 10 # Minimum free percentage to trigger scaling when autoscaling is enabled
    scalingFactor: 1.2 # Scaling factor to determine how much to scale up when autoscaling is triggered

internal_access:
  type: same-gvc # options: same-gvc, same-org, workload-list; used for CockroachDB when pgbouncer is disabled
  workloads:  # Note: can only be used if type is workload-list
    #- //gvc/GVC_NAME/workload/WORKLOAD_NAME

pgbouncer:
  enabled: true
  image: edoburu/pgbouncer:v1.25.1-p0
  poolMode: transaction  # options: session, transaction, statement
  defaultPoolSize: 25    # number of real CockroachDB connections PgBouncer maintains per pod
  maxClientConn: 250     # maximum number of client connections PgBouncer accepts per pod
  maxDbConnections: 100  # hard cap on total CockroachDB connections regardless of how many PgBouncer pods are running
  minReplicas: 2         # per location
  maxReplicas: 4         # per location
  serverCheckDelay: 30      # seconds between idle server connection health checks (default 30)
  serverConnectTimeout: 2   # seconds before giving up on a new server connection (default 15)
  serverLoginRetry: 0       # seconds before retrying a failed server login; 0 = no caching of failures (default 15)
  clientLoginTimeout: 10    # seconds before rejecting a client waiting for login (default 60)
  queryWaitTimeout: 10      # seconds before rejecting a logged-in client waiting for a server connection (default 120)
  internal_access:
    type: same-gvc # options: same-gvc, same-org, workload-list; controls who can connect to PgBouncer
    workloads:  # Note: can only be used if type is workload-list
      #- //gvc/GVC_NAME/workload/WORKLOAD_NAME
  resources:
    maxCpu: 200m
    minCpu: 100m
    maxMemory: 1Gi
    minMemory: 128Mi

backup:
  enabled: false
  image: ghcr.io/controlplane-com/backup-images/cockroach-backup:1.1
  schedule: "0 2 * * *"
  activeDeadlineSeconds: 14400   # hard kill after 4 hours if backup hangs
  location: aws-us-east-1        # MUST be one of `locations` above; put it near your bucket
  resources:
    cpu: 500m
    memory: 512Mi
  provider: aws  # options: aws, gcp
  aws:
    bucket: my-backup-bucket
    region: us-east-1
    cloudAccountName: my-backup-cloudaccount
    policyName: my-backup-policy
    prefix: cockroach/backups
  gcp:
    bucket: my-backup-bucket
    cloudAccountName: my-backup-cloudaccount
    prefix: cockroach/backups

Locations and Replicas

Each entry in locations pairs a location with a node count. Every location listed must already exist in the GVC you install into; extra GVC locations run nothing. Listing the same location twice is refused at render, as is an empty list, an entry with no name, and replicas: 0.

The default is one location with three nodes. That survives the loss of a node; it does not survive the loss of a location — see Multi-Region Survivability.

A one-node deployment is refused at render. Each node joins every other node, so a single node gets an empty --join list and the binary rejects it. The smallest supported shape is 3 replicas in one location, matching CockroachDB’s default replication factor of 3.

Error: execution error at (cockroach/templates/identity.yaml:2:4): cockroach: a one-node deployment
cannot start. Each node joins every OTHER replica, so a single node gets an empty --join list ...
Use at least 3 replicas in total; 3 replicas in one location is the smallest supported shape.

GVC locations you did not list show as red in the console, with This workload location is deactivated because maxScale is set to 0. That is the mechanism that keeps a shared GVC safe — it is what a healthy install looks like, not a fault.

Multi-Zone Placement

Set multiZone: true to spread nodes across availability zones within each location.

Confirm every location in your GVC supports multi-zone placement before enabling this. A location that does not support it accepts the setting and then wedges — the workload never becomes ready and nothing in the logs explains why, because no container ever starts. The only place the reason appears is status.versions[].message:

cpln workload get-deployments RELEASE_NAME-cockroach --gvc GVC_NAME -o yaml

Workload is misconfigured (multizone enabled in an incompatible cluster).
Please contact support or disable multizone.

The incompatibility was measured on a stateful workload with a block volume set — which is exactly the shape every CockroachDB node has — in aws-us-west-2, while aws-us-east-1 and aws-us-east-2 were unaffected. Leave it false if you are unsure.

Database Initialization

The database section specifies a database and user to create automatically when the cluster first initializes:

database:
  name: mydb
  user: myuser

The created user is granted ALL on that database.

These values are only applied on the first initialization. If the cluster has already been initialized, they are skipped on restart or upgrade. To change the user or the database name on an existing cluster, use CockroachDB’s native commands (ALTER USER, RENAME DATABASE).

In CockroachDB v25, ALL on a database does not confer SELECT on tables owned by another user, so the application user cannot read tables root created. Tables the application user creates itself work normally.

Resources and Storage

• resources.cpu and resources.memory set the CPU and memory allocated to each CockroachDB node.
• volumeset.capacity sets the initial persistent volume size in GiB (minimum 10).
• volumeset.autoscaling.enabled — Enable automatic volume expansion as data grows.
• volumeset.autoscaling.maxCapacity — Maximum volume size in GiB.
• volumeset.autoscaling.minFreePercentage — Triggers a scale-up when free space falls below this percentage.
• volumeset.autoscaling.scalingFactor — Multiplier applied to the current capacity on each scale-up.

Internal Access

The internal_access section controls which workloads can reach CockroachDB directly, and pgbouncer.internal_access controls who can connect through the pooler. Neither workload is exposed publicly, and the template has no public access option.

Type	Description
same-gvc	Allow access from all workloads in the same GVC
same-org	Allow access from all workloads in the same organization
workload-list	Allow access only from specific workloads listed in workloads

When using workload-list, specify each workload using its full link format:

internal_access:
  type: workload-list
  workloads:
    - //gvc/GVC_NAME/workload/WORKLOAD_NAME

List only your clients. CockroachDB is peer-to-peer — node-to-node Raft traffic is inbound traffic to the same workload — so the chart adds this release’s own workloads (the cluster, PgBouncer and the backup cron) to every list it renders. Without that, a workload-list naming only your applications would blackhole replication while every replica still reported ready: true.

With PgBouncer enabled, the CockroachDB workload’s own list is the release’s workloads only, and internal_access.workloads is not applied to it — clients are expected to go through the pooler. Set pgbouncer.internal_access to control who reaches the pooler, and disable PgBouncer if you want clients on port 26257 directly.

An access-knob change takes time to propagate. Measurements on this template ranged from about 2 minutes to just over 9 minutes for the new rule to be fully in force, so keep re-testing rather than concluding the knob did not work.

PgBouncer Connection Pooling

PgBouncer is enabled by default. It multiplexes application connections into a smaller pool of real database connections, reducing overhead and protecting CockroachDB from connection exhaustion under high concurrency. It is configured with every CockroachDB node across every location as a backend, so failover and load distribution are handled transparently.

From 2.0.0 that backend list is built by PgBouncer’s own startup script from the same topology the CockroachDB nodes use to build their --join list, rather than being rendered separately by Helm — so the two tiers cannot disagree about which nodes exist.

When enabled, PgBouncer becomes the primary connection endpoint — connect to it instead of CockroachDB directly:

RELEASE_NAME-cockroach-pgbouncer.GVC_NAME.cpln.local:5432

Pool modes:

Mode	Behavior
transaction	Connection held only for the duration of a transaction. Best for most web and API workloads. Not compatible with SET variables, temporary tables, or advisory locks.
session	Connection held for the entire client session. Compatible with all features but provides less connection reuse.
statement	Connection returned after every statement. Transactions are not supported. Rarely used.

• pgbouncer.defaultPoolSize — Real CockroachDB connections PgBouncer maintains per pod.
• pgbouncer.maxClientConn — Maximum client connections PgBouncer accepts per pod. A client beyond the limit is rejected with FATAL: no more connections allowed (max_client_conn).
• pgbouncer.maxDbConnections — Hard cap on total CockroachDB connections shared across all PgBouncer pods. Set this to a value your cluster can safely handle regardless of how many pods are running.
• pgbouncer.minReplicas / pgbouncer.maxReplicas — PgBouncer autoscales on RPS between these bounds in each configured location.
• pgbouncer.serverCheckDelay, serverConnectTimeout, serverLoginRetry, clientLoginTimeout, queryWaitTimeout — PgBouncer’s own timeouts, in seconds.
• pgbouncer.resources — minCpu / maxCpu / minMemory / maxMemory per PgBouncer pod. These were named cpu and memory before 2.0.0; the old names are refused at render with a message naming the replacement.
• pgbouncer.internal_access — Controls which workloads can connect to PgBouncer (same options as internal_access).

Connecting to CockroachDB

The cluster runs in --insecure mode, so there are no SQL credentials — access is governed by internal_access and the GVC boundary and nothing else. Control Plane provides mTLS for all inter-workload communication.

What	Where
PgBouncer (the default endpoint)	RELEASE_NAME-cockroach-pgbouncer.GVC_NAME.cpln.local:5432
CockroachDB SQL, direct	RELEASE_NAME-cockroach.GVC_NAME.cpln.local:26257
One specific node	replica-N.RELEASE_NAME-cockroach.LOCATION.GVC_NAME.cpln.local:26257
DB Console (HTTP UI)	port 8080, not exposed — reach it with cpln port-forward
Credentials	none; the cluster is insecure-mode and network-isolated

From another workload in the same GVC:

cockroach sql --insecure \
  --host=RELEASE_NAME-cockroach-pgbouncer.GVC_NAME.cpln.local:5432 --database=mydb

Always use the fully qualified .GVC_NAME.cpln.local form. The bare workload name is not reliable on this platform — whether it resolves depends on the workload type.

The DB Console shows cluster health, query performance and node status. It is not exposed externally; forward port 8080 of the CockroachDB workload and open http://localhost:8080:

cpln port-forward RELEASE_NAME-cockroach 8080:8080 --gvc GVC_NAME

cpln port-forward is a top-level command, not a cpln workload subcommand.

Application Retry Logic

Your application must implement retry logic on database connections. PgBouncer routes around failed CockroachDB nodes, but transient errors are still surfaced to the application during failover events — such as a location outage or rolling restarts — while PgBouncer cycles through backends and Raft leader elections complete. Without retries, these transient errors will propagate directly to the client.

Multi-Region Survivability

Surviving the loss of an entire location requires at least three locations, and the template’s default is one. With fewer than three, that step is skipped entirely — no CockroachDB configuration can make a one- or two-location cluster survive the loss of a region.

With three or more locations, the first deploy registers every configured location as a CockroachDB region and sets the database’s survival goal to REGION. Verify it on the cluster itself:

SHOW SURVIVAL GOAL FROM DATABASE mydb;
SHOW REGIONS FROM CLUSTER;

Earlier versions could fail this step silently. The SURVIVE REGION FAILURE statement ran in a way that swallowed its own error, so the install reported success while the database was left at the default zone survival goal. From 2.0.0 a failure is reported explicitly and prints SHOW REGIONS FROM CLUSTER. On a three-location cluster the goal is applied and verified. If you are on 1.x, check the survival goal directly rather than trusting the log’s silence.

A production cluster survives a location outage cleanly — a measured three-location, nine-node cluster served reads and writes with zero failures throughout the loss of one whole location, and recovered by itself. Rolling out or restarting nodes in the remaining locations during an outage exceeds the cluster’s fault tolerance and causes a brief period of downtime for ranges on those nodes.

Backup

Backup is disabled by default. When enabled, a cron workload issues a CockroachDB BACKUP INTO statement on the configured schedule. CockroachDB nodes stream the backup data to cloud storage themselves using the workload identity — the backup job only triggers the SQL command and does not transfer data itself.

• backup.enabled — Enable scheduled backups.
• backup.schedule — Cron expression for backup frequency (default: daily at 2am UTC).
• backup.provider — aws or gcp. Any other value is refused at render.
• backup.location — The Control Plane location where the backup cron runs. It must be one of your locations, and the chart refuses to render otherwise: the cron is suspended everywhere else, so a mismatch would mean the backup never ran anywhere with no failed run to observe. Set it to the location nearest your bucket to avoid cross-region egress.
• backup.activeDeadlineSeconds — Hard timeout for the backup job in seconds (default: 14400 / 4 hours). The job is killed if it exceeds this limit.
• backup.resources.cpu / backup.resources.memory — Resources for the backup cron container.

AWS S3

Before enabling backup with provider: aws, complete the following in your AWS account:

1. Create an S3 bucket. Set backup.aws.bucket to its name and backup.aws.region to its region.
2. If you do not have a Cloud Account set up, refer to the docs to Create a Cloud Account. Set backup.aws.cloudAccountName to its name.
3. Create an IAM policy with the following JSON, replacing YOUR_BUCKET_NAME:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetObjectVersion",
                "s3:DeleteObjectVersion",
                "s3:GetBucketLocation",
                "s3:AbortMultipartUpload",
                "s3:ListBucketMultipartUploads",
                "s3:ListMultipartUploadParts"
            ],
            "Resource": [
                "arn:aws:s3:::YOUR_BUCKET_NAME",
                "arn:aws:s3:::YOUR_BUCKET_NAME/*"
            ]
        }
    ]
}

4. Set backup.aws.policyName to the name of the policy created in step 3.
5. Set backup.aws.prefix to the folder path where backups will be stored.

GCS

Before enabling backup with provider: gcp, complete the following in your GCP account:

1. Create a GCS bucket. Set backup.gcp.bucket to its name.
2. If you do not have a Cloud Account set up, refer to the docs to Create a Cloud Account. Set backup.gcp.cloudAccountName to its name.
3. Add the Storage Admin role to the GCP service account associated with the Cloud Account.
4. Set backup.gcp.prefix to the folder path where backups will be stored.

Switch providers with a fresh install, not an upgrade. An identity’s cloud binding is never removed once set — the API merges rather than replaces — so a release switched from aws to gcp keeps the old provider’s binding attached even though the chart no longer renders it.

Restoring a Backup

Backups are written as a full-cluster backup collection at BUCKET/PREFIX/. Run the restore from a workload inside the GVC, or through a forwarded port to a CockroachDB node — the cluster is not reachable from outside.

Which statement you use depends on what you are restoring into, and getting this wrong is the common failure.

Into the cluster you already have

Restore the database under a new name. A full-cluster restore cannot be used here: CockroachDB refuses one on any cluster that already has user databases, and this template always creates mydb and myuser on first deploy.

cockroach sql --insecure --host="RELEASE_NAME-cockroach.GVC_NAME.cpln.local:26257" \
  --execute="RESTORE DATABASE mydb FROM LATEST IN 's3://BUCKET_NAME/PREFIX?AUTH=implicit&AWS_REGION=BUCKET_REGION' WITH new_db_name = 'mydb_restored';"

On GCS, swap the URI for 'gs://BUCKET_NAME/PREFIX?AUTH=implicit'. Drop WITH new_db_name only if you have already dropped mydb.

Into an empty cluster

A full-cluster restore also brings back users and cluster settings, but it requires the target to have no user databases at all — so drop the ones this template created first:

cockroach sql --insecure --host="RELEASE_NAME-cockroach.GVC_NAME.cpln.local:26257" \
  --execute="DROP DATABASE mydb CASCADE; DROP USER myuser;"

cockroach sql --insecure --host="RELEASE_NAME-cockroach.GVC_NAME.cpln.local:26257" \
  --execute="RESTORE FROM LATEST IN 's3://BUCKET_NAME/PREFIX?AUTH=implicit&AWS_REGION=BUCKET_REGION';"

Running the second statement without the first fails with full cluster restore can only be run on a cluster with no tables or databases.

A full-cluster restore looks like it failed, and succeeds anyway. Because it restores the system tables, the client returns

ERROR: job ...: node liveness error: restarting in background

one node restarts and reports is_live=false for roughly two minutes, and defaultdb/system briefly show is offline: restoring. The restore nevertheless completes — measured at about 40 seconds to the restored database being visible and about two and a half minutes to every node live again. Check SHOW JOBS rather than believing the error. Restoring into the cluster you already have has none of this behavior.

External References

CockroachDB Documentation

Official CockroachDB documentation

Multi-Region Overview

Learn about multi-region deployments

Survival Goals

Configure region failure survival goals

RESTORE

The RESTORE statement reference

Backup Image Source

Source code for the CockroachDB backup container image

CockroachDB Template

View the source files, default values, and chart definition