> ## Documentation Index > Fetch the complete documentation index at: https://controlplanecorporation-majid-docs-content-expansion.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Weaviate > Deploy a Weaviate vector database cluster on Control Plane. Covers the prerequisite API key secret, Raft clustering and its restart limitation, AI provider modules, internal-only access, and scheduled backups to S3 or GCS. ## Overview **Upgrading from 1.0.1 or earlier?** The Weaviate API key and every AI provider key are now prerequisite secrets you create, and the old values keys stop the render — see [Upgrading From 1.0.1 or Earlier](#upgrading-from-1-0-1-or-earlier). 1.0.1 and earlier shipped a **working 64-character API key** as a values default, published in this repository and shared by every install that did not override it. It is the only authentication Weaviate has, so anyone holding it can read and write every collection. Treat the key on any earlier install as **compromised**: rotate it rather than simply upgrading. Weaviate is an AI-native vector database for storing, indexing, and querying vector embeddings alongside structured object data. This template deploys a Weaviate 1.38 cluster of `replicas` nodes in a single location, using Raft consensus for schema and cluster state, with one persistent volume per node, optional AI provider modules, and optional scheduled backups to AWS S3 or GCS. The cluster has **no public endpoint by design** — it is reachable only from inside Control Plane, scoped by `internalAccess.type`. ### What Gets Created * **Weaviate Workload** — A `stateful` workload of `replicas` nodes forming a Raft cluster, serving REST and GraphQL on port `8080` and gRPC on `50051`. * **Volume Set** — One volume per replica holding that node's objects and vector indexes, with autoscaling. * **Credentials Secret** — Template-managed `dictionary` secret holding the non-sensitive `api-user` value, plus the backup bucket coordinates when backups are enabled. No credential passes through it. * **Start Script Secret** — The boot script mounted into each container. * **Identity & Policy** — An identity bound to the workloads, and a policy granting it `reveal` on exactly four things at most: the two template secrets, your API key secret, and each AI provider secret you name. Cloud storage access is added to the identity only when backups are enabled. * **Backup Cron Workload** *(optional)* — Calls Weaviate's backup API on a schedule to write a full snapshot to cloud storage. This template does not create a GVC. You must deploy it into an existing GVC. ## Prerequisites ### API Key Weaviate has no anonymous access in this template. The API key is a **required prerequisite `opaque` secret that must exist before you install** — the key never enters the Helm release, and the template only ever refers to it by name. ```bash theme={null} printf '%s' "$(openssl rand -hex 32)" | \ cpln secret create-opaque --name my-weaviate-api-key --encoding plain -f - ``` Set `apiKeySecretName` to that name, and set `apiUser` to the username the key maps to. Keep your own copy of the key: the platform is the only place it is stored, and every client authenticates with it. **Installing without the secret looks like a platform fault, not a missing step.** `cpln helm install` reports complete success and creates every resource, then the workload simply never becomes ready. `cpln logs` returns **zero lines** — there is no container to produce output — and the only diagnostic is in `status.versions[].message`: ```text theme={null} The secret my-weaviate-api-key no longer exists. Workload updates are paused until the secret is added or the reference to the secret removed. ``` Read it with `cpln workload get-deployments {release}-weaviate --gvc {gvc} -o yaml`. Creating the secret afterwards **does** release it, without further action, but slowly — measured between 5.5 and 10.5 minutes across five templates, so poll rather than time-box it. To skip the wait, run `cpln workload force-redeployment {release}-weaviate --gvc {gvc}`, which cleared it in about 90 seconds. ### AI Provider Keys Only needed if you want Weaviate to call a provider for embeddings or generative search. Create one `opaque` secret per provider, holding just that provider's key: ```bash theme={null} printf '%s' "sk-..." | \ cpln secret create-opaque --name my-weaviate-openai-key --encoding plain -f - ``` Set `modules.openai.apiKeySecretName` (or the `anthropic`, `cohere`, `huggingface` equivalent) to that name. Leaving a provider's `apiKeySecretName` empty means the provider is genuinely off: no environment variable, no `reveal` grant, and no outbound internet access on the workload. All three were confirmed against the live resources on a default install, where the container could not reach `api.openai.com` at all. **Naming a provider secret gives you a live, billable integration.** It is not inert configuration held in reserve — see [Provider Modules Load Regardless of the Module List](#provider-modules-load-regardless-of-the-module-list). Only supply a provider key when you intend that provider to be used. ### Cloud Storage Only needed if you set `backup.enabled: true`. See [Backing Up](#backing-up) for the bucket, Cloud Account, and IAM steps. ## Installation To install, follow the instructions for your preferred method: Browse, install, and manage templates visually Manage templates from your terminal }> Declare templates in your Terraform configurations Pulumi Icon Streamline Icon: https://streamlinehq.com } > Declare templates in your Pulumi programs ## Configuration The default `values.yaml` for this template: ```yaml theme={null} # ─── Cluster ────────────────────────────────────────────────────────────────── replicas: 3 image: semitechnologies/weaviate:1.38.0 # ─── Authentication ─────────────────────────────────────────────────────────── # REQUIRED PREREQUISITE SECRET — CREATE IT BEFORE YOU INSTALL. # An `opaque` secret (encoding: plain) whose entire payload is the API key: # printf '%s' "$(openssl rand -hex 32)" | \ # cpln secret create-opaque --name my-weaviate-api-key --encoding plain -f - # If it does not exist at install time the deployment WEDGES waiting on it and # looks broken. See Prerequisites in the README. apiKeySecretName: my-weaviate-api-key # Username the API key maps to. Not a secret — it is the admin-list identity. apiUser: admin@example.com # ─── Query Behavior ─────────────────────────────────────────────────────────── # Default result limit for queries queryDefaultsLimit: 25 # Default vectorizer module applied to new collections # Options: none, text2vec-openai, text2vec-cohere, text2vec-huggingface, text2vec-aws defaultVectorizerModule: none # ─── AI Modules ─────────────────────────────────────────────────────────────── modules: # Every module you intend to use must be listed here — supplying a key alone # does not activate it. # Common options: text2vec-openai, text2vec-cohere, text2vec-huggingface, # generative-openai, generative-cohere, generative-anthropic, # qna-openai, ref2vec-centroid enabled: [] # OPTIONAL PREREQUISITE SECRETS — one `opaque` secret (encoding: plain) per # provider, holding only that provider's key. Empty = provider off: no env # var, no reveal grant, no outbound egress opened. # printf '%s' "sk-..." | \ # cpln secret create-opaque --name my-weaviate-openai-key --encoding plain -f - openai: apiKeySecretName: "" # e.g. my-weaviate-openai-key anthropic: apiKeySecretName: "" # e.g. my-weaviate-anthropic-key cohere: apiKeySecretName: "" # e.g. my-weaviate-cohere-key huggingface: apiKeySecretName: "" # e.g. my-weaviate-huggingface-key # ─── Resources ──────────────────────────────────────────────────────────────── # HNSW vector indexes are RAM-resident. Rough sizing: # memory ≈ vectors × dimensions × 4 bytes × 1.5 cpu: 2 memory: 4Gi # ─── Storage ────────────────────────────────────────────────────────────────── volumes: data: initialCapacity: 20 # GiB per replica (platform minimum 10) autoscaling: maxCapacity: 200 minFreePercentage: 20 scalingFactor: 1.5 # ─── Placement ──────────────────────────────────────────────────────────────── multiZone: enabled: false # true = spread replicas across AZs (location must support it) # ─── Access ─────────────────────────────────────────────────────────────────── # Weaviate is reachable only from inside Control Plane — there is no public # endpoint, by design. See Important Notes in the README. internalAccess: type: same-gvc # none | same-gvc | same-org | workload-list workloads: [] # used only with workload-list # workloads: # - //gvc/GVC_NAME/workload/WORKLOAD_NAME # ─── Backup ─────────────────────────────────────────────────────────────────── backup: enabled: false provider: aws # aws | gcp schedule: "0 2 * * *" # cron in UTC — daily at 02:00 resources: cpu: 250m memory: 256Mi aws: bucket: my-weaviate-backup-bucket region: us-east-1 cloudAccountName: my-s3-cloud-account policyName: my-weaviate-backup-policy path: weaviate/backups gcp: bucket: my-weaviate-backup-bucket cloudAccountName: my-gcs-cloud-account path: weaviate/backups ``` ### Cluster * `replicas` — Number of Weaviate nodes. Three is the default and the practical minimum for Raft, which needs a quorum (2 of 3) to elect a leader and accept schema changes. Read [Rolling Restarts Can Split a Replica Out of the Cluster](#rolling-restarts-can-split-a-replica-out-of-the-cluster) before running a multi-replica cluster in production. * `image` — The Weaviate image and tag. The template is tested on `semitechnologies/weaviate:1.38.0`. A first install of three replicas took about **165 seconds** to reach all-ready in testing: the first replica is ready at roughly 60 seconds and the ordered rollout paces the remaining two. ### Authentication * `apiKeySecretName` — Name of the pre-created `opaque` secret whose entire payload is the API key. Required; see [Prerequisites](#prerequisites). The key is read straight from the secret into `AUTHENTICATION_APIKEY_ALLOWED_KEYS` and never appears in the Helm release, the rendered manifest, or the stored workload spec. * `apiUser` — The username the API key maps to, and the entry in Weaviate's admin list. This is an identity, not a credential, so it is an ordinary value and is stored in the template's own dictionary secret. Anonymous access is disabled unconditionally. In testing, an unauthenticated request and a request with a wrong key both returned `401 Unauthorized`, and only the exact key from the secret returned `200`. ### Query Behavior * `queryDefaultsLimit` — Default result limit applied to queries that do not specify one. * `defaultVectorizerModule` — The vectorizer applied to new collections. Leave it at `none` when your application supplies its own vectors; set it to a provider module to have Weaviate call that provider's embedding API on insert and query. ### AI Modules * `modules.enabled` — List of modules written to Weaviate's `ENABLE_MODULES` variable. When backups are enabled the template appends `backup-s3` or `backup-gcs` to this list for you. * `modules.openai.apiKeySecretName`, `modules.anthropic.apiKeySecretName`, `modules.cohere.apiKeySecretName`, `modules.huggingface.apiKeySecretName` — Name of the `opaque` secret holding that provider's key. Empty means the provider is off. | Module | Type | Provider | Secret knob | | ---------------------------------------------------- | ----------------------- | ------------ | -------------------------------------- | | `text2vec-openai`, `generative-openai`, `qna-openai` | Vectorizer / Generative | OpenAI | `modules.openai.apiKeySecretName` | | `generative-anthropic` | Generative | Anthropic | `modules.anthropic.apiKeySecretName` | | `text2vec-cohere`, `generative-cohere` | Vectorizer / Generative | Cohere | `modules.cohere.apiKeySecretName` | | `text2vec-huggingface` | Vectorizer | Hugging Face | `modules.huggingface.apiKeySecretName` | Naming any provider secret, or enabling backups, opens outbound internet access on the Weaviate workload so it can reach that API. With neither, the workload has no egress at all. The comment in `values.yaml` — "supplying a key alone does not activate it" — **overstates what `modules.enabled` does** on Weaviate 1.38. See [Provider Modules Load Regardless of the Module List](#provider-modules-load-regardless-of-the-module-list). ### Resources * `cpu` / `memory` — Limits applied to each Weaviate replica. Memory is the sizing constraint, not CPU: HNSW vector indexes are held in RAM. Size roughly as `vectors × dimensions × 4 bytes × 1.5`. Under-provisioning surfaces as an out-of-memory restart rather than slow queries. ### Storage * `volumes.data.initialCapacity` — Initial volume size in GiB per replica. The platform minimum is 10. * `volumes.data.autoscaling.maxCapacity` — Maximum volume size in GiB. * `volumes.data.autoscaling.minFreePercentage` — Free-space percentage that triggers a scale-up. * `volumes.data.autoscaling.scalingFactor` — Multiplier applied to current capacity when scaling up. Volume data survives a redeploy. After a full restart in testing, all objects written beforehand were still present and byte-identical, including their original write timestamps. ### Placement Set `multiZone.enabled: true` to spread replicas across availability zones within the location. Verify the location supports multi-zone before enabling. ### Access * `internalAccess.type` — Which workloads may reach Weaviate: `none`, `same-gvc` (default), `same-org`, or `workload-list`. * `internalAccess.workloads` — Used only with `workload-list`. Full workload links, for example `//gvc/GVC_NAME/workload/WORKLOAD_NAME`. All four settings were exercised against a live cluster, including the negative case: a workload absent from `workload-list` was refused, and adding it to the list let it through. **A firewall change takes up to a couple of minutes to take effect.** Measured propagation was 45–95 seconds. A denial surfaces to the caller as `HTTP 503 upstream connect error`, which is indistinguishable from an unhealthy upstream — re-test after a couple of minutes before concluding a knob did not work. ## Upgrading From 1.0.1 or Earlier `1.1.0` is a security release. Every credential the chart used to accept as a value is now a secret you create and reference by name, and carrying a `1.0.1` values file forward **fails at render** with a message naming its replacement — nothing silently falls back to a default. | `1.0.1` and earlier | `1.1.0` | Why | | -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `apiKey` (a working 64-character key) | `apiKeySecretName` → required prerequisite `opaque` secret | The default key was published in this repository and shared by every install that did not override it. It is the only authentication Weaviate has | | `modules.openai.apiKey` and the `anthropic`, `cohere`, `huggingface` equivalents | `modules.{provider}.apiKeySecretName` → optional prerequisite `opaque` secrets | Your own provider billing keys were landing in the Helm release in plaintext | | `internal_access` | `internalAccess` | Renamed only; the fields (`type`, `workloads`) are unchanged | | `clusterName` | Removed | It was never rendered into any resource. Nothing replaces it | The render guard for the API key reads: ```text theme={null} weaviate: `apiKey` was removed in 1.1.0 — the API key is now a REQUIRED prerequisite `opaque` secret (encoding: plain). Create it, then set `apiKeySecretName` to its name. ``` Generate a **new** key and create the secret as shown in [Prerequisites](#prerequisites). The old default was public, so an install that used it should be treated as having had its data readable and writable by anyone who found it. If you overrode the default with your own value, that value still travelled in the Helm release and is worth rotating too. Delete `apiKey`, `clusterName`, and any `modules.{provider}.apiKey` entries from your values. Set `apiKeySecretName`, rename `internal_access` to `internalAccess`, and create a provider secret for each provider you actually use. The API key changes, so every application, notebook, and client library holding the old bearer token must be updated. A `helm upgrade` restarts the cluster one replica at a time. Read [Rolling Restarts Can Split a Replica Out of the Cluster](#rolling-restarts-can-split-a-replica-out-of-the-cluster) first and verify cluster health afterwards — the platform's ready status will not tell you if a node failed to rejoin. ## Cluster Behavior and Known Limits Both limitations below are **pre-existing** — `1.1.0` changed no clustering configuration — and both were confirmed on a live three-replica cluster. ### Rolling Restarts Can Split a Replica Out of the Cluster A single redeployment of a three-replica cluster left one node **permanently split out of the Raft cluster**. The restarted node came up as a leaderless `Candidate` — it had Raft log entries on disk but had applied none of them, so its schema state was empty. The other two replicas elected a leader without it, and it **never rejoined**: it was still in that state 31 minutes later, and it did not recover on its own. The part that makes this dangerous is what the platform reports: * **All three replicas report `ready`.** The readiness probe checks `/v1/.well-known/live`, which only proves the HTTP server is up, so the split node stays in the service-DNS rotation. * **Roughly a third of client requests fail silently.** A 60-sample probe against the load-balanced service DNS measured `200: 42 (70%)` and `404: 18 (30%)` — exactly one node in three. The failures are `404 Not Found`, an answered, routed response rather than a connection error, so a client sees "no such collection", not "cluster unhealthy". This is a **rejoin** defect, not a rollout-pacing one: the restart of the other two replicas was seamless, with 270 of 270 probe samples returning `200` while they cycled one at a time. **`replicas: 3` is not safe across an upgrade**, and `helm upgrade` is how you will meet this. A restart is routine — an upgrade, an image bump, a platform reschedule — so **verify cluster health after any restart instead of trusting the platform's ready status.** Check membership from inside any replica. A healthy cluster reports one `Leader` and two `Follower` nodes on the same term, and `/v1/nodes` lists every replica as `HEALTHY`: ```bash theme={null} cpln workload exec {release}-weaviate --gvc {gvc} --container weaviate -- \ /bin/sh -c "wget -qO- --header='Authorization: Bearer YOUR_API_KEY' \ http://localhost:8080/v1/cluster/statistics" ``` A split node reports `"state":"Candidate"` with an empty `leaderId` and `"lastContact":"never"`, and `/v1/nodes` queried from a healthy replica lists fewer nodes than you deployed. A single-replica install (`replicas: 1`) has no Raft membership to lose and is not exposed to this. ### Provider Modules Load Regardless of the Module List `modules.enabled` does not gate the API-based vectorizer and generative modules. With `modules.enabled: []` and only a provider secret named, `text2vec-openai` was still loaded, a collection could be created with `"vectorizer":"text2vec-openai"`, and inserting an object with **no vector** made Weaviate call the provider and store the embedding it returned. `/v1/meta` reports the same 41 modules loaded whether `ENABLE_MODULES` is set or not — on 1.38 these modules are compiled in and always available. The risk is the **inverse** of what the values comment suggests. Supplying a provider key "just to have it configured" gives you a **live, billable** provider integration that will be used the moment any collection specifies that vectorizer. Only name a provider secret when you intend that provider to be used, and remove the secret name when you stop using it. `modules.enabled` still has a job — it is the documented upstream knob and it is how the backup modules are turned on, which the template does for you when `backup.enabled: true`. It is simply not a safety gate on provider access. ## Connecting Weaviate is reachable from workloads inside Control Plane, subject to `internalAccess.type`. There is no public endpoint. | What | Where | | ----------------------- | --------------------------------------------------------------------- | | Cluster (load balanced) | `{release}-weaviate.{gvc}.cpln.local` | | A specific replica | `{release}-weaviate-{n}.{gvc}.cpln.local` | | REST and GraphQL port | `8080` | | gRPC port | `50051` | | Credentials | The key in the secret named by `apiKeySecretName`; username `apiUser` | | Public endpoint | None | Authenticate with the key as a bearer token: ```bash theme={null} curl -H "Authorization: Bearer YOUR_API_KEY" \ http://{release}-weaviate.{gvc}.cpln.local:8080/v1/meta ``` The workload is assigned a canonical `*.cpln.app` hostname like any other, but `inboundAllowCIDR` is empty unconditionally, so every request from the internet is refused at the platform edge with `403 RBAC: access denied` — **a valid API key does not help**, because the request never reaches Weaviate. If you need browser or off-platform access, put your own authenticating proxy in front of it inside the GVC. ## Backing Up With `backup.enabled: true`, a cron workload calls Weaviate's backup API on `backup.schedule` and writes a full snapshot of every collection to `{path}/{backup-id}/` in your bucket. **The backup path has not been exercised end to end.** Rendering, the chart's validation of the backup settings, and the backup job's discovery of the cluster endpoint were all checked live, but no snapshot has been written to S3 or GCS from this template — a real run needs cloud resources outside the test environment. Treat backups as unverified: run one on your own bucket and confirm the objects land before you rely on it. A misconfigured cloud binding can report a successful install while the identity is unusable, so check the objects, not the install. **With `internalAccess.type: workload-list`, the backup job is denied.** The cron job is a separate workload and the template does not add it to the allow-list, so its calls to Weaviate are refused like any other unlisted workload. Add `//gvc/{gvc}/workload/{release}-weaviate-backup` to `internalAccess.workloads` yourself, or use `same-gvc` or `same-org`. ### AWS S3 Create an S3 bucket. Set `backup.aws.bucket` and `backup.aws.region` to match, and `backup.aws.path` to the prefix you want snapshots written under. If you do not have one, [create a Cloud Account](https://docs.controlplane.com/guides/create-cloud-account) for the AWS account holding the bucket. Set `backup.aws.cloudAccountName` to its name. Create an IAM policy with the following JSON, replacing `YOUR_BUCKET_NAME`, and set `backup.aws.policyName` to its name: **Version 1.1.1 narrows AWS backup permissions.** This version removes `aws::ReadOnlyAccess` from the backup identity. That AWS managed policy granted read access to **every bucket in your AWS account** and contains no write actions at all, so it was never carrying the backup itself — but it *was* silently supplying any read action your own bucket-scoped policy happened to omit. **Update your IAM policy to the full action list below before upgrading.** If it already matches, no action is needed. The identity now carries `cpln-connector` and your bucket-scoped policy only, which is strictly narrower than before. Nothing else changes. ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket", "s3:GetObjectVersion", "s3:DeleteObjectVersion", "s3:GetBucketLocation", "s3:AbortMultipartUpload", "s3:ListBucketMultipartUploads", "s3:ListMultipartUploadParts" ], "Resource": [ "arn:aws:s3:::YOUR_BUCKET_NAME", "arn:aws:s3:::YOUR_BUCKET_NAME/*" ] } ] } ``` ### GCS Create a GCS bucket. Set `backup.gcp.bucket` to its name and `backup.gcp.path` to the prefix. If you do not have one, [create a Cloud Account](https://docs.controlplane.com/guides/create-cloud-account) for the GCP project. Set `backup.gcp.cloudAccountName` to its name. Grant the Cloud Account's service account `roles/storage.objectAdmin` on that bucket. The template requests exactly that role on exactly that bucket, and nothing more. ## Restoring a Backup Run the restore from inside any Weaviate replica. Use `gcs` in place of `s3` for GCP backups, and replace `BACKUP_ID` with the backup name from your bucket (they are written as `weaviate-backup-YYYYMMDD-HHMMSS`): ```bash theme={null} # Trigger the restore wget -qO- \ --header='Authorization: Bearer YOUR_API_KEY' \ --header='Content-Type: application/json' \ --post-data='{}' \ 'http://localhost:8080/v1/backups/s3/BACKUP_ID/restore' # Poll for completion wget -qO- \ --header='Authorization: Bearer YOUR_API_KEY' \ 'http://localhost:8080/v1/backups/s3/BACKUP_ID/restore' ``` A restore is not a merge. It fails if a collection from the backup already exists on the cluster — drop the collection first, or restore into a fresh deployment. ## Important Notes * **Create the API key secret before you install.** Without it the install reports success and the deployment silently wedges, with no container and no logs. See [Prerequisites](#prerequisites). * **Verify cluster health after every restart.** A replica that fails to rejoin the Raft cluster still reports `ready` and still receives traffic. See [Rolling Restarts Can Split a Replica Out of the Cluster](#rolling-restarts-can-split-a-replica-out-of-the-cluster). * **A named provider secret is a live billable integration**, whether or not the module is listed in `modules.enabled`. * **Keep your own copy of the API key.** Rotating it means updating the secret and restarting the cluster; losing it locks you out of every collection. * **There is no public endpoint**, and that is not configurable in this template. Reachability is `internalAccess` only, and an access change takes up to a couple of minutes to settle. * **Size memory, not CPU.** Vector indexes are RAM-resident; an undersized cluster fails with out-of-memory restarts. * **Backups are unverified.** Confirm a snapshot lands in your bucket before depending on the schedule. ## External References Official Weaviate documentation REST API reference, including the backup and restore endpoints Vectorizer and generative module configuration How Weaviate API key authentication and the admin list work Upstream documentation for the backup and restore API View the source files, default values, and chart definition