Update to Control Plane

{
  "name": "stateful-storage",
  "description": "Creates persistent storage for stateful workloads on Control Plane. Use when the user asks about volumes, volume sets, disks, mounting storage, snapshots, volume expansion, filesystems, shared storage, or backups.",
  "included_files": [],
  "skill_md_contents": "---\nname: stateful-storage\ndescription: \"Creates persistent storage for stateful workloads on Control Plane. Use when the user asks about volumes, volume sets, disks, mounting storage, snapshots, volume expansion, filesystems, shared storage, or backups.\"\n---\n\n# Stateful Storage & VolumeSets\n\nA **VolumeSet** is GVC-scoped persistent storage for workloads. The `workload` skill covers the basics (stateful type, reserved mount paths, the 15-volume limit, create-then-verify); this skill is the full volume-set detail. The one trap that drives most rework: **`fileSystemType` and `performanceClass` are immutable** (a PATCH that changes either returns HTTP 400) — to change either you must create a new volumeset, and the old data does not carry over. Choose both at creation.\n\n**Most databases don't need this skill:** `template-catalog` installs Postgres, Redis, MySQL, MongoDB, and more with the volumeset, snapshots, and credentials already wired — hand-build only for a custom app or an unsupported engine.\n\n## Filesystem types and performance classes\n\n| Filesystem | Access | Workloads | Volumes provisioned | Snapshots / shrink / delete-volume |\n|---|---|---|---|---|\n| `ext4` | read-write-once | one stateful/vm workload | one per replica, per location | yes |\n| `xfs` | read-write-once | one stateful/vm workload | one per replica, per location | yes |\n| `shared` | read-write-many | any workload type, many at once | one per location (shared by all replicas) | no — expand only |\n\n| Performance class | Min | Max | Filesystems |\n|---|---|---|---|\n| `general-purpose-ssd` | 10 GB | 65536 GB | ext4, xfs |\n| `high-throughput-ssd` | 200 GB | 65536 GB | ext4, xfs |\n| `shared` | 10 GB | 65536 GB | shared (auto-set) |\n\nWhen `fileSystemType: shared`, `performanceClass` is **auto-set to `shared`** — do not specify another. Data is **per-location** and never replicated across locations; for cross-location redundancy, replicate at the application layer (e.g. WAL streaming).\n\n## Create a volumeset\n\nUse `create_volumeset` (MCP create/mount tools default `fileSystemType` to **xfs** and `performanceClass` to `general-purpose-ssd`; the raw API/`cpln apply` default is **ext4**). YAML for IaC / CLI fallback:\n\n```yaml\nkind: volumeset\nname: pg-data\ngvc: GVC\nspec:\n  fileSystemType: ext4\n  performanceClass: general-purpose-ssd\n  initialCapacity: 20          # GB; within the class min/max and <= autoscaling.maxCapacity\n  autoscaling:\n    maxCapacity: 100\n    minFreePercentage: 20      # 1-100\n    scalingFactor: 1.5         # >= 1.1\n  snapshots:\n    schedule: \"0 2 * * *\"      # cron; no more than once per hour\n    retentionDuration: 7d      # float + d/h/m; tool default 7d\n```\n\nApply with `cpln apply -f volumeset.yaml --gvc GVC`. Update mutable fields (capacity, autoscaling, snapshot policy, tags) with `update_volumeset`.\n\n### Autoscaling\n\n**Reactive**: a background job checks volumes about once a minute; when free space falls below `minFreePercentage` it resizes the volume to hold current usage at that margin, scaled up: `new_capacity = ceil(usedGB / (1 - minFreePercentage/100) * scalingFactor)`, capped at `maxCapacity`. Both fields are required, or autoscaling does nothing.\n\n**Predictive** runs the same formula on *projected* usage (from the recent growth rate) to expand ahead of demand; the larger of the reactive and predictive targets wins. Requires `minFreePercentage > 0` and `scalingFactor >= 1.1`:\n\n```yaml\n  autoscaling:\n    maxCapacity: 200\n    minFreePercentage: 20\n    scalingFactor: 1.5\n    predictive:\n      enabled: true            # default false\n      lookbackHours: 24        # 1-168\n      projectionHours: 6       # 1-72\n      minDataPoints: 10        # 2-100\n      minGrowthRateGBPerHour: 0.01\n      scalingFactor: 1.2       # >= 1.1; defaults to the parent scalingFactor\n```\n\n## Mount to a workload\n\nMount with `mount_volumeset_to_workload` — it attaches to the **first container** and creates the volumeset if missing (create-only defaults, ignored when the volumeset already exists: path `/mnt/{volumesetName}`, filesystem `xfs`, class `general-purpose-ssd`). Volume URI is `cpln://volumeset/VOLUMESET`.\n\n- **ext4/xfs require a `stateful` or `vm` workload** (mounting on serverless/standard returns HTTP 400); `shared` mounts on any type. Workload type is immutable — see \"Migrating to stateful\" below.\n- Up to **15 volumes** per container. **Reserved mount paths** (rejected): `/dev`, `/dev/log`, `/tmp`, `/var`, `/var/log`.\n- `recoveryPolicy`: `retain` (default — reuse an existing volume's data on a new replica) or `recycle` (start fresh).\n- `path` is required for non-vm workloads and rejected for `vm` (VM disks use `name`/`bus`/`bootOrder` instead).\n- Stateful workloads give each replica a stable index and its own volume; `spec.loadBalancer.replicaDirect` (stateful-only) exposes per-replica endpoints — see the `workload` skill.\n\n```yaml\nkind: workload\nname: pg\ngvc: GVC\nspec:\n  type: stateful\n  containers:\n    - name: postgres\n      image: //image/postgres:16\n      ports:\n        - number: 5432\n          protocol: tcp        # http | http2 | grpc | tcp — a DB is tcp, not http\n      volumes:\n        - uri: cpln://volumeset/pg-data\n          path: /var/lib/postgresql/data\n```\n\n## Snapshots\n\nSnapshots are **ext4/xfs only — never `shared`**. Automatic policy lives in `spec.snapshots`: `createFinalSnapshot` (default `true` — a snapshot is taken before any volume in the set is deleted), `retentionDuration`, and `schedule` (cron whose minute field must be a single concrete value, so no more than once per hour). Manual: `create_volumeset_snapshot`, `list_volumeset_snapshots`, `restore_volumeset_snapshot`, `delete_volumeset_snapshot`. A restore creates a **new volume** and discards everything written since the snapshot.\n\n## Resize and delete volumes\n\n- **Expand** — live, no downtime, all filesystems. Throttled to **4 expansions per volume per rolling 24 hours**; the 5th returns **HTTP 429** and a brief wait does not help (the oldest expansion must age out of the window). `expand_volumeset`.\n- **Shrink** (ext4/xfs only) — data is migrated to the new smaller volume via an online presync + final delta sync, and the replica restarts during the swap. The platform **rejects the shrink with HTTP 400 when known used bytes (+5% metadata headroom) would not fit**; data is only lost if used bytes genuinely exceed the new capacity. Floor is the class minimum (10 / 200 GB). `shrink_volumeset`.\n- **Delete a volume** (ext4/xfs only) — permanent loss of that volume's data. `delete_volumeset_volume`.\n\nShrink, volume-delete, snapshot-delete, and restore are destructive: **snapshot first** as the recovery net, then confirm the blast radius (the destructive-ops guardrail returns an impact preview before executing).\n\n## Shared filesystem\n\nA `shared` volumeset is mounted read-write by many workloads at once but supports only expand — no snapshots, shrink, or volume-delete. Each mount point is provisioned its own CPU/memory; tune with `mountOptions.resources` (defaults `minCpu 500m`, `maxCpu 2000m`, `minMemory 1Gi`, `maxMemory 2Gi`; max/min at most 4000m and 4096Mi apart, ratio at most 4:1):\n\n```yaml\nspec:\n  fileSystemType: shared       # performanceClass auto-set to \"shared\"\n  initialCapacity: 50\n  mountOptions:\n    resources: { minCpu: 500m, maxCpu: 2000m, minMemory: 1Gi, maxMemory: 2Gi }\n```\n\n## Custom encryption (AWS only)\n\nVolumes are encrypted by default. To use your own AWS KMS keys on ext4/xfs volumes (not `shared`, not BYOK):\n\n```yaml\nspec:\n  customEncryption:\n    regions:\n      aws-us-east-1:           # format: {cloud-provider}-{region}\n        keyId: \"arn:aws:kms:us-east-1:123456789:key/KEY_ID\"\n```\n\nThe `keyId` is injected as the EBS storage-class `kmsKeyId`. The KMS key policy **must grant Control Plane's AWS account `arn:aws:iam::957753459089:root`** the volume-encryption permissions (`Decrypt`, `Encrypt`, `GenerateDataKey`, `CreateGrant`, etc.); the key is immutable once a volume exists.\n\n## BYOK storage classes\n\nOn self-hosted clusters, volumes use the storage class `{performanceClass}-{fileSystemType}` (e.g. `general-purpose-ssd-ext4`) and the cluster needs a CSI-compatible driver. `spec.storageClassSuffix` selects an alternative `{performanceClass}-{fileSystemType}-{suffix}`, falling back to the unsuffixed class if it is not found.\n\n## Migrating a workload to stateful\n\nWorkload type is immutable, so adding an ext4/xfs volume to a serverless/standard workload means **delete + recreate as `stateful`** — destructive. Before deleting, confirm with the user: the public URL that 5xx's during the cutover, any internal callers that fail until recreate, runtime/in-memory state lost at delete, and that the recreate typically takes 2-5 min. Sequence: capture the spec (`cpln workload get WORKLOAD --gvc GVC -o yaml-slim > bak.yaml`) as a rollback artifact; apply the volumeset; delete the old workload; apply the new manifest with `spec.type: stateful` + the volume mount, **keeping the same name** to preserve URL/DNS/policy/identity links. For the deploy-wait pattern, see the `workload` skill.\n\n## Verify\n\n- `get_resource` (kind `volumeset`): `status.locations[].volumes[]` show per-volume `currentSize`, `currentBytesUsed`, `lifecycle` (expect `bound`), and snapshot counts; `status.usedByWorkload` names the bound workload.\n- After mounting, poll `list_deployments` until ready and confirm the container's volume is mounted at the expected path.\n\n## Troubleshooting\n\n| Symptom | Cause | Fix |\n|---|---|---|\n| 400 mounting a volumeset | ext4/xfs on a serverless/standard workload | Use a `stateful` or `vm` workload (recreate to change type) |\n| 400 \"performanceClass / fileSystemType is immutable\" | Tried to change either on update | Create a new volumeset; migrate data via snapshot/restore |\n| 400 on mount with a path | Path is reserved (`/dev`, `/tmp`, `/var`, ...) | Mount elsewhere (e.g. `/data`, `/mnt/...`) |\n| HTTP 429 on expand | 4 expansions on that volume in the last 24 h | Wait for the oldest to age out; plan larger steps |\n| 400 on shrink | New size cannot hold used bytes (+5%) | Shrink less, or free space / snapshot then rebuild |\n| Snapshot fields rejected | Volumeset is `shared` | Snapshots need ext4/xfs |\n| Deployment stuck after mount | Volume provisioning (2-5 min on first deploy) | Poll `list_deployments`; check `get_workload_logs` if it stays unready |\n\n## MCP tools quick reference\n\n| Tool | Purpose | Tier |\n|---|---|---|\n| `create_volumeset` | Create a volumeset | core |\n| `update_volumeset` | Update mutable fields (capacity, autoscaling, snapshots, tags) | core |\n| `mount_volumeset_to_workload` | Mount to a workload (creates the volumeset if missing) | core |\n| `expand_volumeset` | Grow a volume (4 / 24 h limit) | core |\n| `shrink_volumeset` | Shrink a volume (ext4/xfs) | full |\n| `delete_volumeset_volume` | Delete one volume (ext4/xfs) | full |\n| `create_volumeset_snapshot` | Point-in-time snapshot | full |\n| `list_volumeset_snapshots` | List snapshots | full |\n| `restore_volumeset_snapshot` | Restore a snapshot to a new volume | full |\n| `delete_volumeset_snapshot` | Delete a snapshot | full |\n| `get_resource` / `list_resources` / `delete_resource` (kind `volumeset`) | Read / list / delete a volumeset | core |\n\nCLI fallback (CI/CD via a service-account `CPLN_TOKEN`): `cpln volumeset create|get|update|delete|expand|shrink`, `cpln volumeset snapshot create|get|restore|delete`, `cpln volumeset volume get|delete`; `expand`/`shrink` need `--new-size` (`--location`/`--volume-index` optional), and `cpln apply -f` for YAML.\n\n## Related skills\n\n| Skill | For |\n|---|---|\n| `workload` | Workload types, the deploy-and-verify flow, load-balancer/`replicaDirect` config |\n| `template-catalog` | Postgres, Redis, and other databases that provision volumesets for you |\n| `firewall-networking` | Outbound rules for cloud-bucket volumes (`s3://`, `gs://`, `azureblob://`) |\n\n## Documentation\n\n- [Volume Set Reference](https://docs.controlplane.com/reference/volumeset.md)\n- [Workload Volumes](https://docs.controlplane.com/reference/workload/volumes.md)\n- [CLI volumeset Commands](https://docs.controlplane.com/cli-reference/commands/volumeset.md)\n"
}