OpenSearch

Overview

OpenSearch is an open-source distributed search and analytics engine. This template deploys a production-ready OpenSearch cluster with automated node configuration, an optional Dashboards visualization UI, an optional demo log pipeline, and optional scheduled snapshot backups to AWS S3 or GCS.

What Gets Created

Stateful Workload — An OpenSearch cluster with a configurable number of replicas (must be odd). Installs S3 or GCS repository plugins at startup when backup is enabled.
Volume Set — Persistent storage per replica with optional autoscaling.
Secret — An opaque startup script secret that generates the OpenSearch node configuration and starts the process.
Identity & Policy — An identity bound to the OpenSearch workload with reveal access to the startup script secret. When backup is enabled, the identity also grants cloud storage access to the cluster.
OpenSearch Dashboards Workload (optional) — A web-based visualization UI for logs and data. Enabled when dashboard.enabled: true. Not exposed externally — access via cpln port-forward.
Demo Logs Workload (optional) — A Python log generator with a Fluent Bit sidecar that ships sample logs to OpenSearch, plus a one-time setup job that creates index templates and dashboard patterns. Creates an additional Fluent Bit config secret with its own identity and policy. Enabled when demoLogs.enabled: true.
Backup Setup Workload (optional) — A one-time job that registers the snapshot repository and creates an automated snapshot policy via the OpenSearch API. Enabled when backup.enabled: true.

This template does not create a GVC. You must deploy it into an existing GVC.

Upgrade to 1.0.2 — versions 1.0.0 and 1.0.1 do not work outside a GVC named test-gvc, and one option granted org-wide secret access. See Upgrading From 1.0.x. Both defects have been present since 1.0.0.

Prerequisites

Prerequisites are only required if you plan to enable automated backups (backup.enabled: true). Skip this section if backups are not needed.

AWS S3

Create an S3 bucket. Set backup.aws.bucket and backup.aws.region in your values file.
If you do not have a Control Plane Cloud Account set up, follow the Create a Cloud Account guide. Set backup.aws.cloudAccountName to the name of your Cloud Account.
Create an IAM policy with the following JSON, replacing YOUR_BUCKET_NAME:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetObjectVersion",
                "s3:DeleteObjectVersion"
            ],
            "Resource": [
                "arn:aws:s3:::YOUR_BUCKET_NAME/*"
            ]
        }
    ]
}

Set backup.aws.policyName to the name of the policy created in step 3.

GCS

Create a GCS bucket. Set backup.gcp.bucket in your values file.
If you do not have a Control Plane Cloud Account set up, follow the Create a Cloud Account guide. Set backup.gcp.cloudAccountName to the name of your Cloud Account.
Add the Storage Admin role to the GCP service account associated with the Cloud Account.

Installation

To install, follow the instructions for your preferred method:

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Upgrading From 1.0.x

Version 1.0.2 fixes five defects, all present since 1.0.0. Two of them matter regardless of how you configured the template.

The identity links were hardcoded to `test-gvc`

Both workloads’ identityLink and both policies’ principalLinks pointed at //gvc/test-gvc/identity/... rather than at your GVC. An install into any GVC not literally named test-gvc created its identity in your GVC while every reference pointed into one you do not have. There is nothing to migrate — upgrade to 1.0.2 and the links resolve correctly.

The demo-logs policy granted `reveal` on every secret in your org

When demoLogs.enabled: true, the demo-logs policy carried a targetQuery of match: all with an empty terms list alongside its targetLinks. That query resolves to every secret in the organization, not just the one the demo pipeline needs — so the demo-logs identity was granted reveal on all of them.

If you ran 1.0.0 or 1.0.1 with demoLogs.enabled: true, treat every secret in that organization as having been readable by that identity. Upgrading to 1.0.2 narrows the policy to the single Fluent Bit config secret, but it does not undo past access — review what that organization holds and rotate anything sensitive.

The remaining three

Backup provider conditions never compared anything. A GCP backup also emitted an AWS binding pointing at a nonexistent cloud account.
aws::ReadOnlyAccess is no longer attached to the identity. It granted read on every bucket in the account and carried none of the write actions a snapshot repository needs; backups work without it.
An unsupported backup.provider now fails at render instead of installing cleanly and silently configuring no snapshot repository at all. An odd replicas count is enforced too.

Configuration

The default values.yaml for this template:

image: opensearchproject/opensearch:3.4.0

replicas: 3 # Must be odd

clusterName: my-opensearch-cluster

resources:
  minCpu: 500m
  minMemory: 2Gi
  maxCpu: 1
  maxMemory: 4Gi

volumeset:
  capacity: 10 # initial capacity in GiB (minimum is 10)
  autoscaling:
    enabled: false
    maxCapacity: 100 # Maximum capacity in GiB
    minFreePercentage: 10 # Trigger scaling when free space drops below this percentage
    scalingFactor: 1.2 # Multiply current capacity by this factor when scaling up

internal_access:
  type: same-gvc # options: same-gvc, same-org, workload-list
  workloads: # Note: can only be used if type is same-gvc or workload-list
    #- //gvc/GVC_NAME/workload/WORKLOAD_NAME

# Dashboard Configuration (recommended)
dashboard:
  enabled: true
  image: opensearchproject/opensearch-dashboards:3.4.0
  resources:
    cpu: 100m
    memory: 512Mi

# Demo Logs Configuration (optional)
demoLogs:
  enabled: false
  remove_setup_workload: false # Set to true after setup completes to reduce resource usage

# Backup Configuration (optional)
backup:
  enabled: false
  remove_setup_workload: false # Set to true after setup completes to reduce resource usage

  provider: aws # Options: aws or gcp

  schedule: 0 2 * * * # Daily at 2am UTC

  retention:
    maxAge: 30d     # Delete snapshots older than 30 days
    maxCount: 30    # Keep maximum 30 snapshots

  aws:
    bucket: my-s3-bucket
    region: us-east-1
    prefix: opensearch-snapshots
    cloudAccountName: my-cloud-account
    policyName: my-backup-policy

  gcp:
    bucket: my-gcs-bucket
    prefix: opensearch-snapshots
    cloudAccountName: my-cloud-account

Cluster

replicas — Number of OpenSearch nodes. Must be an odd number (3, 5, 7) to ensure quorum.
clusterName — Name used for internal cluster coordination.

Use a minimum of 3 replicas for high availability. For production workloads, plan resources based on log volume: 1 CPU / 4Gi handles 10–50 GB/day; scale to 2 CPU / 8Gi for 50–100 GB/day.

Resources

resources.minCpu / resources.minMemory — Minimum CPU and memory guaranteed per node.
resources.maxCpu / resources.maxMemory — Maximum CPU and memory per node.

Storage

volumeset.capacity — Initial volume size per node in GiB (minimum 10).
volumeset.autoscaling.enabled — Automatically expand each volume as it fills. When enabled:
- maxCapacity — Maximum volume size in GiB.
- minFreePercentage — Trigger a scale-up when free space drops below this percentage.
- scalingFactor — Multiply the current capacity by this factor when scaling up.

Internal Access

internal_access.type — Controls which workloads can connect to OpenSearch on port 9200:

Type	Description
`same-gvc`	Allow access from all workloads in the same GVC
`same-org`	Allow access from all workloads in the same organization
`workload-list`	Allow access only from specific workloads listed in `workloads`

External access to OpenSearch is blocked by default. The Dashboards UI is also not exposed externally — use cpln port-forward to access it.

OpenSearch Dashboards

dashboard.enabled — When true, deploys an OpenSearch Dashboards workload for log visualization and search. Recommended for most deployments.
dashboard.resources.cpu / dashboard.resources.memory — CPU and memory for the Dashboards workload.

To access the Dashboards UI, use port-forwarding:

cpln port-forward WORKLOAD_NAME --location LOCATION --org ORG_NAME 5601:5601

Then open http://localhost:5601 in your browser.

Demo Logs

demoLogs.enabled — When true, deploys a sample log generator with a Fluent Bit sidecar that ships logs to OpenSearch, along with a setup job that creates the index template and dashboard pattern. Useful for testing the pipeline end-to-end.
demoLogs.remove_setup_workload — Set to true after the setup job completes (~1–2 minutes) to remove the one-time setup workload and reduce resource usage. The log pipeline continues running.

Backup

Set backup.enabled: true to enable automated OpenSearch snapshot backups. The backup setup workload registers the snapshot repository and creates the snapshot policy via the OpenSearch API. Once it completes successfully, set backup.remove_setup_workload: true and run cpln helm upgrade to remove it — snapshots will continue on schedule.

backup.provider — aws or gcp.
backup.schedule — Cron schedule for automated snapshots (UTC).
backup.retention.maxAge — Delete snapshots older than this duration.
backup.retention.maxCount — Maximum number of snapshots to retain.

Set backup.provider to aws or gcp and fill in the corresponding provider section.

Restoring a Snapshot

Restore snapshots from any workload that can reach the cluster on port 9200. Restore to an empty cluster:

curl -X POST "http://WORKLOAD_NAME:9200/_snapshot/backup-repo/SNAPSHOT_NAME/_restore" \
  -H 'Content-Type: application/json' \
  -d '{
    "indices": "*",
    "ignore_unavailable": true,
    "include_global_state": false
  }'

Restore to an existing cluster (close indices first):

curl -X POST "http://WORKLOAD_NAME:9200/_all/_close"

curl -X POST "http://WORKLOAD_NAME:9200/_snapshot/backup-repo/SNAPSHOT_NAME/_restore" \
  -H 'Content-Type: application/json' \
  -d '{
    "indices": "*",
    "ignore_unavailable": true,
    "include_global_state": false
  }'

curl -X POST "http://WORKLOAD_NAME:9200/_all/_open"

List available snapshots:

curl "http://WORKLOAD_NAME:9200/_snapshot/backup-repo/_all?pretty"

Connecting to OpenSearch

Once deployed, connect to the cluster from within the same GVC using:

http://RELEASE_NAME-opensearch.GVC_NAME.cpln.local:9200

Important Notes

The security plugin is disabled — there is no authentication. Any workload permitted by internal_access has full read/write admin access to every index, and there are no credentials anywhere in this template. Keep internal_access.type as narrow as your deployment allows, and prefer workload-list over same-org for anything sensitive.
replicas must be odd. An even count cannot form a quorum and is rejected at render from 1.0.2.
Enabling backups on a running cluster fails for a few minutes before it succeeds. The snapshot repository plugin installs at node startup, so every node must roll before the repository can be registered — meanwhile the setup job logs repository type [s3] does not exist and restarts. It retries until all nodes carry the plugin, about 4–5 minutes for three nodes, and then succeeds. Enabling backups at install time avoids this entirely.
Switching backup.provider on an existing release leaves the old cloud binding attached. The API merges identity updates and never removes a provider block once set, so an AWS-then-GCP switch leaves the identity holding both. Uninstall and reinstall if you need the old binding gone.
Set the snapshot IAM policy to both bucket ARNs — arn:aws:s3:::BUCKET and arn:aws:s3:::BUCKET/*. s3:ListBucket authorizes against the bucket itself, so a policy carrying only /* registers the repository and then fails when OpenSearch enumerates it.
Use the fully-qualified internal hostname — RELEASE_NAME-opensearch.GVC_NAME.cpln.local:9200. The bare workload name is not reliably resolvable.
Uninstall deletes the volume sets. Enable snapshots if the data matters.

External References

OpenSearch Documentation

Official OpenSearch documentation

OpenSearch Dashboards

OpenSearch Dashboards documentation

Snapshot Management

OpenSearch snapshot and backup guide

OpenSearch Template

View the source files, default values, and chart definition

Template Catalog

Templates

Overview

What Gets Created

Prerequisites

AWS S3

GCS

Installation

UI

CLI

Terraform

Pulumi

Upgrading From 1.0.x

The identity links were hardcoded to `test-gvc`

The demo-logs policy granted `reveal` on every secret in your org

The remaining three

Configuration

Cluster

Resources

Storage

Internal Access

OpenSearch Dashboards

Demo Logs

Backup

Restoring a Snapshot

Connecting to OpenSearch

Important Notes

External References

OpenSearch Documentation

OpenSearch Dashboards

Snapshot Management

OpenSearch Template

​Overview

​What Gets Created

​Prerequisites

​AWS S3

​GCS

​Installation

UI

CLI

Terraform

Pulumi

​Upgrading From 1.0.x

​The identity links were hardcoded to test-gvc

​The demo-logs policy granted reveal on every secret in your org

​The remaining three

​Configuration

​Cluster

​Resources

​Storage

​Internal Access

​OpenSearch Dashboards

​Demo Logs

​Backup

​Restoring a Snapshot

​Connecting to OpenSearch

​Important Notes

​External References

OpenSearch Documentation

OpenSearch Dashboards

Snapshot Management

OpenSearch Template

Overview

What Gets Created

Prerequisites

AWS S3

GCS

Installation

Upgrading From 1.0.x

The identity links were hardcoded to `test-gvc`

The demo-logs policy granted `reveal` on every secret in your org

The remaining three

Configuration

Cluster

Resources

Storage

Internal Access

OpenSearch Dashboards

Demo Logs

Backup

Restoring a Snapshot

Connecting to OpenSearch

Important Notes

External References