Update to Flyte

{
  "description": "Deploy Flyte on a kind cluster onto any host — your local machine, or a fresh cloud VM (DigitalOcean, AWS EC2, or GCP Compute Engine). Provisions the VM (firewall scoped to your IP), installs Docker/kind/kubectl/helm, then runs the kind Flyte deploy on that host and tunnels access back to your machine. Use when the user wants Flyte-on-kind but hasn't decided where to run it, or wants it on a cloud VM rather than locally. For evaluation only (single-node kind, static credentials). Delegates the Flyte install itself to the deploy-flyte-kind skill.",
  "included_files": [],
  "name": "deploy-flyte-kind-vm",
  "skill_md_contents": "---\nname: deploy-flyte-kind-vm\ndescription: Deploy Flyte on a kind cluster onto any host — your local machine, or a fresh cloud VM (DigitalOcean, AWS EC2, or GCP Compute Engine). Provisions the VM (firewall scoped to your IP), installs Docker/kind/kubectl/helm, then runs the kind Flyte deploy on that host and tunnels access back to your machine. Use when the user wants Flyte-on-kind but hasn't decided where to run it, or wants it on a cloud VM rather than locally. For evaluation only (single-node kind, static credentials). Delegates the Flyte install itself to the deploy-flyte-kind skill.\n---\n\n# Deploy Flyte on kind — local machine or a cloud VM\n\nkind runs anywhere Docker runs, so the same Flyte-on-kind deploy works on your\nown machine **or** on a cloud VM. This skill picks the **host**, provisions it if\nit's a cloud VM, installs the prerequisites, and then runs the actual Flyte\ndeploy — the cluster, the hosted PostgreSQL + object store, the helm install, and\noptional auth — by handing off to the **`deploy-flyte-kind`** skill. The only\nthings that change per host are *where the commands run* and *how you reach the\nAPI afterward*.\n\n> **For evaluation only.** Single-node kind, static credentials, no workload\n> identity. On a cloud VM the stack is reachable from the public internet, so the\n> provisioning steps below **restrict inbound 80/443 (and 22) to your own IP**.\n> For production, use the `flyte-deploy-aws` skill instead.\n\nThis skill is guided: several steps need **human-in-the-loop** input — which host,\nconfirming billable VM creation, SSH keys, and (in the deploy hand-off) the\nSupabase / R2 / S3 credentials. Ask; never invent account IDs, IPs, keys, or\npasswords.\n\n## Step 0: Choose the host\n\nAsk the user where to run the cluster (use `AskUserQuestion`):\n\n- **Local machine** — kind runs in your local Docker. Simplest; nothing to\n  provision.\n- **DigitalOcean** — a Droplet, provisioned with `doctl`.\n- **AWS EC2** — an instance, provisioned with the `aws` CLI.\n- **GCP Compute Engine** — an instance, provisioned with `gcloud`.\n\nThen:\n\n- **Local machine** → skip to **Step 3** (nothing to provision). The whole deploy\n  runs locally.\n- **A cloud VM** → do **Step 1** (provision) and **Step 2** (define how remote\n  commands run), then Step 3.\n\nkind needs a few GB of headroom, so any VM should be **at least 4 vCPU / 8 GB**.\n\n## Step 1: Provision the cloud VM (skip for local)\n\nFirst confirm the provider CLI is installed and authenticated locally, since these\ncommands run on **your** machine:\n\n```bash\n# DigitalOcean\ncommand -v doctl >/dev/null && doctl account get >/dev/null 2>&1 || echo \"doctl: install and run 'doctl auth init'\"\n# AWS\ncommand -v aws   >/dev/null && aws sts get-caller-identity >/dev/null 2>&1 || echo \"aws: install and configure credentials\"\n# GCP\ncommand -v gcloud >/dev/null && gcloud auth list >/dev/null 2>&1 || echo \"gcloud: install and run 'gcloud auth login'\"\n```\n\nIf the CLI is missing or unauthenticated, stop and have the user set it up (an\ninteractive login like `gcloud auth login` is easiest run by the user — suggest\nthey type `! gcloud auth login` so it runs in this session). **Creating a VM\nincurs cost — confirm with the user before running any `create`/`run-instances`\ncommand,** and ask for the values it needs (SSH key ID / key-pair name / zone).\n\nPick the tab for the chosen provider. Each: create a firewall/security rule\nscoped to the user's IP, create the VM, SSH in, install Docker + kind + kubectl +\nhelm.\n\n### DigitalOcean\n\nSetting up `doctl` if it's missing or unauthenticated (`brew install doctl` on\nmacOS): auth is interactive by default, but if the user pastes an API token it\ncan be done non-interactively with `doctl auth init -t <token>`. The SSH key must\nbe **imported to DigitalOcean** before it can be passed to `droplet create`:\n\n```bash\ndoctl compute ssh-key list                 # already imported? use its ID below\ndoctl compute ssh-key import <name> --public-key-file ~/.ssh/id_ed25519.pub   # prints the ID\n```\n\n```bash\n# Create the Droplet (ask the user for their SSH key ID: `doctl compute ssh-key list`)\ndoctl compute droplet create flyte-kind \\\n  --image ubuntu-24-04-x64 --size s-4vcpu-8gb --region nyc1 \\\n  --ssh-keys <your-ssh-key-id>\n```\n\nScope inbound 22/80/443 to your own IP with a cloud firewall (do this before\nexposing anything — see the evaluation-only note). Then SSH in and install the\ntools (DigitalOcean's Ubuntu image logs in as `root`, so no `sudo` needed):\n\n```bash\nssh root@<droplet-ip> 'bash -s' <<'EOF'\ncurl -fsSL https://get.docker.com | sh\ncurl -Lo /usr/local/bin/kind https://github.com/kubernetes-sigs/kind/releases/latest/download/kind-linux-amd64 && chmod +x /usr/local/bin/kind\ncurl -Lo /usr/local/bin/kubectl \"https://dl.k8s.io/release/$(curl -Ls https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl\" && chmod +x /usr/local/bin/kubectl\ncurl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash\nEOF\n```\n\n### AWS EC2\n\nCreate a security group that admits **only your IP** on 22/80/443, then launch the\ninstance with the current Ubuntu 24.04 AMI:\n\n```bash\nMY_IP=$(curl -s https://checkip.amazonaws.com)\naws ec2 create-security-group --group-name flyte-kind --description \"Flyte kind evaluation\"\nfor port in 22 80 443; do\n  aws ec2 authorize-security-group-ingress --group-name flyte-kind \\\n    --protocol tcp --port $port --cidr ${MY_IP}/32\ndone\n\naws ec2 run-instances \\\n  --image-id \"$(aws ssm get-parameters \\\n      --names /aws/service/canonical/ubuntu/server/24.04/stable/current/amd64/hvm/ebs-gp3/ami-id \\\n      --query 'Parameters[0].Value' --output text)\" \\\n  --instance-type t3.xlarge \\\n  --key-name <your-key-pair> \\\n  --security-groups flyte-kind\n```\n\nSSH in as `ubuntu` (installs need `sudo`) and install the tools:\n\n```bash\nssh -i <your-key.pem> ubuntu@<instance-public-ip> 'bash -s' <<'EOF'\ncurl -fsSL https://get.docker.com | sudo sh\nsudo usermod -aG docker $USER\nsudo curl -Lo /usr/local/bin/kind https://github.com/kubernetes-sigs/kind/releases/latest/download/kind-linux-amd64 && sudo chmod +x /usr/local/bin/kind\nsudo curl -Lo /usr/local/bin/kubectl \"https://dl.k8s.io/release/$(curl -Ls https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl\" && sudo chmod +x /usr/local/bin/kubectl\ncurl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | sudo bash\nEOF\n```\n\nThe `usermod -aG docker` takes effect on the **next** login, so the Step 2 SSH\nsessions can run `docker`/`kind` without `sudo`. (In a single interactive session\nyou'd `newgrp docker`; over one-shot `ssh` commands, just reconnect.)\n\n### GCP Compute Engine\n\n```bash\ngcloud compute instances create flyte-kind \\\n  --machine-type e2-standard-4 --zone <your-zone> \\\n  --image-family ubuntu-2404-lts-amd64 --image-project ubuntu-os-cloud \\\n  --tags flyte-kind\n\n# GCP blocks inbound 80/443 until a rule allows them — scope to the tag + your IP:\nMY_IP=$(curl -s https://checkip.amazonaws.com)\ngcloud compute firewall-rules create flyte-kind-web \\\n  --allow tcp:80,tcp:443 --target-tags flyte-kind --source-ranges ${MY_IP}/32\n```\n\nSSH in (installs need `sudo`) and install the tools:\n\n```bash\ngcloud compute ssh flyte-kind --zone <your-zone> --command='bash -s' <<'EOF'\ncurl -fsSL https://get.docker.com | sudo sh\nsudo usermod -aG docker $USER\nsudo curl -Lo /usr/local/bin/kind https://github.com/kubernetes-sigs/kind/releases/latest/download/kind-linux-amd64 && sudo chmod +x /usr/local/bin/kind\nsudo curl -Lo /usr/local/bin/kubectl \"https://dl.k8s.io/release/$(curl -Ls https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl\" && sudo chmod +x /usr/local/bin/kubectl\ncurl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | sudo bash\nEOF\n```\n\n## Step 2: Define how the deploy commands run on the VM (skip for local)\n\nEvery `kind`, `kubectl`, and `helm` command from the deploy runs **on the VM**,\nnot your machine. Fix the SSH target once and reuse it:\n\n```bash\nVM=\"root@<droplet-ip>\"                 # DigitalOcean\n# VM=\"ubuntu@<instance-public-ip>\"     # AWS EC2 (add -i <your-key.pem> to ssh below)\n# GCP: use `gcloud compute ssh flyte-kind --zone <zone> --command='...'` instead of ssh $VM\n```\n\nThen, when the `deploy-flyte-kind` skill (Step 3) says to run a command, run it on\nthe VM instead of locally:\n\n- **Single command:** `ssh $VM '<command>'`\n- **A block or a heredoc** (e.g. `kind create cluster --config - <<'EOF' … EOF`):\n  pipe it through SSH — `ssh $VM 'bash -s' <<'EOF' … EOF`.\n\nAlternatively, open one interactive session (`ssh $VM`) and run the deploy\ncommands there. Either way, **only the SDK/CLI and browser run on your machine**;\neverything cluster-side runs on the VM. The access tunnel in Step 4 bridges the\ntwo.\n\n## Step 3: Run the Flyte deploy (hand off to `deploy-flyte-kind`)\n\nNow do the actual install by following the **`deploy-flyte-kind`** skill. It\ncovers, in order:\n\n1. **Prereqs + existing-cluster check** — you've already installed the tools; on a\n   VM the `kind get clusters` / prereq checks run via `ssh $VM '...'`.\n2. **Create the kind cluster** — with the two host-port mappings (`30080→80`,\n   `30443→443`). On a VM these bind to the VM's public IP; `flyte.local` will point\n   at that IP (not `127.0.0.1`) if you enable auth.\n3. **Choose + configure dependencies** — this is the main **human-in-the-loop**\n   part: PostgreSQL (**Supabase** or external) and object store (**AWS S3** or\n   **Cloudflare R2**). The user creates these in their consoles and supplies the\n   connection details (they can paste a screenshot). Follow that skill's guidance\n   exactly — especially Supabase's **session pooler** requirement (kind is IPv4-only).\n4. **Write `values-local.yaml`** and **`helm install`** the flyte-binary chart.\n   Don't skip that skill's **inline block** (task-pod `FLYTE_AWS_*` credentials +\n   `runs.storagePrefix`) — without it the API works but every task fails.\n5. **Web console access** (Step 6 there) — Traefik + the two unified-origin\n   routes. See Step 4 below for the VM-specific ways to reach it.\n6. **Optional OIDC auth** via Traefik + oauth2-proxy (and the `start-dex-local`\n   skill for an in-cluster Dex IdP).\n7. **Optional app serving** (Knative + Kourier) — that skill's \"Enable app\n   serving\" section. On a cloud VM the apps base domain is `<vm-ip>.sslip.io`,\n   and apps are served directly on port 80 (firewall-scoped to the user's IP).\n\n**When on a cloud VM, apply the Step 2 wrapper to every command in that skill** —\ncreate the values/config files on the VM (write them via the piped heredoc, or\nedit them in the interactive session), since helm reads them there. The logic is\nidentical; only the execution location changes.\n\n## Step 4: Access Flyte\n\nThe API is reached at `localhost:8090` on the machine where you run the SDK/CLI.\n\n**Local machine** — just port-forward:\n\n```bash\nkubectl -n flyte port-forward service/flyte-http 8090:8090\n```\n\n**Cloud VM** — the port-forward runs on the VM, so tunnel it back over SSH. One\ncommand starts the forward on the VM and exposes it at `localhost:8090` locally:\n\n```bash\n# DigitalOcean\nssh -L 8090:localhost:8090 root@<droplet-ip> \\\n  kubectl -n flyte port-forward service/flyte-http 8090:8090\n# AWS EC2\nssh -i <your-key.pem> -L 8090:localhost:8090 ubuntu@<instance-public-ip> \\\n  kubectl -n flyte port-forward service/flyte-http 8090:8090\n# GCP\ngcloud compute ssh flyte-kind --zone <your-zone> \\\n  --ssh-flag=\"-L 8090:localhost:8090\" \\\n  --command=\"kubectl -n flyte port-forward service/flyte-http 8090:8090\"\n```\n\n> [!WARNING] Do NOT add `-N` to these tunnel commands\n> These tunnels run `kubectl port-forward` as the SSH **remote command**, and\n> `-N` tells SSH to skip the remote command — the forward never starts and every\n> connection gets `connection refused`. `-N` is right only for a pure tunnel with\n> no remote command (like the console tunnel below).\n\nKeep it running. Verify from your machine (a JSON response, not a connection\nerror, confirms Flyte is up and talking to its database):\n\n```bash\ncurl -s -X POST http://localhost:8090/flyteidl2.project.ProjectService/ListProjects \\\n  -H 'Content-Type: application/json' -d '{}'\n```\n\nThen point the SDK at it — the `deploy-flyte-kind` \"Verify access\" step has the\n`~/.flyte/config.yaml` block (`endpoint: dns:///localhost:8090`, `insecure: True`).\nThe tunnel makes the VM deploy behave exactly like a local one for the SDK. The\ncode-bundle upload needs no second tunnel — the S3/R2 endpoint is publicly\nresolvable, so the SDK uploads to the presigned URL directly. Note `helm upgrade`\nrolls the flyte pod and **drops this tunnel's port-forward** — restart it after\neach upgrade.\n\n### Console + tunnel-free SDK access from a VM (no auth)\n\nOnce Traefik + the two unified-origin routes from `deploy-flyte-kind` **Step 6**\nare installed (run those commands on the VM), the kind host-port mapping\n(`30080 → 80`) binds Traefik to the VM's **public IP**, and both browser and SDK\ncan skip the tunnels entirely — the Step 1 firewall already scopes port 80 to the\nuser's IP:\n\n- **Console** — directly at **`http://<vm-ip>/v2`**, no tunnel. Or tunnel it\n  (note this pure tunnel **does** want `-N` — there's no remote command):\n  ```bash\n  ssh -N -L 8080:localhost:80 root@<vm-ip>     # then open http://localhost:8080/v2\n  ```\n- **SDK, tunnel-free** — gRPC rides Traefik's `web` entrypoint over h2c on\n  port 80, so the SDK can point straight at the VM:\n  ```yaml\n  # .flyte/config.yaml\n  admin:\n    endpoint: dns:///<vm-ip>:80\n    insecure: true\n  ```\n  Lower-friction than the Step 4 tunnel (nothing to keep running, survives helm\n  upgrades), with the caveat that it's **plain HTTP over the public internet** —\n  acceptable for evaluation only because the firewall admits just the user's IP.\n- **Run-URL host mismatch** — `flyte run` prints\n  `URL: http://localhost:8080/v2/...` regardless of where Flyte runs. On a VM\n  those links are dead as printed: either keep the `-N` console tunnel above on\n  port 8080 so they work as-is, or swap `localhost:8080` for `<vm-ip>`.\n\n> If you enable **auth** (Step 3.6), the SDK reaches Flyte at `https://flyte.local`\n> over the `443` mapping, not this port-forward. On a cloud VM, `flyte.local` must\n> resolve to the **VM's public IP** in your local `/etc/hosts` (not `127.0.0.1`),\n> and inbound 443 must be open to your IP (Step 1). Otherwise the auth flow is\n> identical to `deploy-flyte-kind`.\n\n## Optional: load a local image into kind\n\nkind nodes can't pull from a host Docker daemon, so a custom task/Flyte image must\nbe loaded into the cluster:\n\n```bash\nkind load docker-image <your-image>:<tag> --name flyte     # local\n```\n\nOn a cloud VM the image must be in the **VM's** Docker daemon first — either build\nit on the VM, or ship it from your machine:\n\n```bash\ndocker save <your-image>:<tag> | ssh $VM docker load\nssh $VM 'kind load docker-image <your-image>:<tag> --name flyte'\n```\n\nReference that exact `<image>:<tag>` in task config; the `IfNotPresent` pull policy\nthen uses the loaded image.\n\n## Tear down\n\n```bash\nkind delete cluster --name flyte          # local\nssh $VM 'kind delete cluster --name flyte'   # cloud VM\n```\n\nOn a cloud VM, also delete the instance so it stops billing (confirm with the\nuser), and remove the firewall/security group you created:\n\n```bash\ndoctl compute droplet delete flyte-kind                        # DigitalOcean\naws ec2 terminate-instances --instance-ids <instance-id>       # AWS EC2 — also: aws ec2 delete-security-group --group-name flyte-kind\ngcloud compute instances delete flyte-kind --zone <your-zone>  # GCP — also: gcloud compute firewall-rules delete flyte-kind-web\n```\n\nThe hosted PostgreSQL and S3/R2 bucket are untouched — clean those up in their own\nconsoles.\n"
}