← Plugin catalog
Developer Tools
Flyte
SAMHITA ALLA v1.0.1
Publisher description
From the marketplace listing
Author, run, and deploy Flyte 2 workflows, apps, and agents with skills grounded in the Flyte SDK and documentation. In ChatGPT web, search Flyte docs and SDK examples. In a local ChatGPT environment with the Flyte CLI configured, run and manage your authenticated Flyte cluster.
Language: English · Automatically detected from descriptions.
Files & skills
File archives
Plugin package22 files · 108 KBBrowse files →
Skill instructions
deploy-flyte-kind45.1 KB
---
name: deploy-flyte-kind
description: Deploy a complete Flyte stack (flyte-binary + a hosted PostgreSQL + an object store) onto a kind cluster, running on the user's own machine or a DigitalOcean VM (droplet). PostgreSQL is hosted (Supabase or external); the object store is AWS S3 or Cloudflare R2. Use when the user wants to run Flyte on kind — either reusing an existing kind cluster or creating a new one. For evaluation only (no TLS/auth on the base deployment).
---
# Deploy Flyte to a kind cluster
Stand up Flyte on a [kind](https://kind.sigs.k8s.io/) cluster: the flyte-binary
plus a hosted PostgreSQL and an S3-compatible object store. For **evaluation
only** — no TLS, no auth, static credentials.
kind runs anywhere Docker runs, so the cluster can live on the user's **own
machine** (default) or a **DigitalOcean VM** (droplet) — the host is a choice
made in Step 0.
The PostgreSQL and object store are independent choices the user makes in Step 2:
- **PostgreSQL** — **Supabase** or another external/self-hosted PostgreSQL.
- **Object store** — **AWS S3** or **Cloudflare R2**.
Both are hosted; kind runs only the flyte-binary. The user supplies connection
details for each.
## Step 0: Choose the host, check prerequisites, and check for an existing cluster
**First, ask the user where kind should run** (use `AskUserQuestion`):
- **the user's own machine** (default), or
- a **DigitalOcean VM** (droplet) — the only cloud-VM host this skill supports.
Do **not** offer or hand-roll AWS EC2 or GCP VM setups; for a real cloud
deployment, point the user at the AWS deployment skill instead.
If the user picks the droplet, **every `kind`, `kubectl`, and `helm` command
below runs on the droplet** (over SSH) — only the SDK/CLI and browser run on the
user's own machine. Provision it and install the tools there first (needs a few
GB of headroom for kind, so ≥ 4 vCPU / 8 GB):
```bash
# create the droplet (dashboard or doctl)
doctl compute droplet create flyte-kind \
--image ubuntu-24-04-x64 --size s-4vcpu-8gb --region nyc1 \
--ssh-keys <your-ssh-key-id>
# SSH in and install Docker, kind, kubectl, helm ON the droplet
ssh root@<droplet-ip>
curl -fsSL https://get.docker.com | sh
curl -Lo /usr/local/bin/kind \
https://github.com/kubernetes-sigs/kind/releases/latest/download/kind-linux-amd64 \
&& chmod +x /usr/local/bin/kind
curl -Lo /usr/local/bin/kubectl \
"https://dl.k8s.io/release/$(curl -Ls https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" \
&& chmod +x /usr/local/bin/kubectl
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
```
> [!WARNING] A cloud VM is exposed to the internet
> On a droplet the stack is reachable from the public internet. Restrict ports
> `80`, `443`, and `22` to the user's own IP with a
> [cloud firewall](https://docs.digitalocean.com/products/networking/firewalls/)
> while they evaluate.
Then verify the required tools **on whichever host runs kind** (run the check on
the droplet over SSH if that's the host), and decide whether to create a cluster
or reuse one:
```bash
for t in docker kind kubectl helm; do command -v $t >/dev/null || echo "MISSING: $t"; done
kind get clusters
```
- If any tool is `MISSING`, stop and tell the user to install it
([docker](https://docs.docker.com/get-docker/),
[kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation),
[kubectl](https://kubernetes.io/docs/tasks/tools/),
[helm](https://helm.sh/docs/intro/install/)).
Read the `kind get clusters` output literally — it is the exact list of existing
cluster names, one per line. Do **not** assume a `flyte` cluster exists because
this guide uses that name; only treat it as present if `flyte` appears verbatim
in the output. A line like `kind` is a cluster named `kind`, not `flyte`.
- **`flyte` is in the list** → reuse it: skip Step 1, use `--context kind-flyte`
below.
- **`flyte` is not in the list** → create it in Step 1. (Don't silently reuse a
differently-named cluster; if the user wants to reuse one, confirm its name
and substitute it into `--context kind-<name>` everywhere below.)
- **No clusters at all** → create one in Step 1.
## Step 1: Create the kind cluster (skip if reusing)
Create the cluster with **two** host-port mappings. kind fixes a cluster's port
mappings **at creation time** — they can't be added later — so map them both now
regardless of whether auth is added:
- **`30080 → 80`** lets the **browser** reach the Traefik ingress (plain HTTP) used
by the optional auth section at `http://flyte.local`.
- **`30443 → 443`** lets the **SDK/CLI** reach the Traefik ingress over **TLS** at
`https://flyte.local`. The SDK only authenticates over HTTPS (see the SDK-auth
part of the auth section), so this is required if the user enables auth *and*
wants to submit runs from the SDK. Harmless otherwise.
```bash
kind create cluster --name flyte --config - <<'EOF'
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 30080 # Traefik web (HTTP) nodePort (auth section)
hostPort: 80 # reach the ingress at http://flyte.local
protocol: TCP
- containerPort: 30443 # Traefik websecure (HTTPS) nodePort — SDK auth
hostPort: 443 # reach the TLS ingress at https://flyte.local
protocol: TCP
EOF
```
Map both even if the user isn't sure about auth — they're harmless if auth is never
added. **If a plain `kind create cluster --name flyte` was already made without
these, delete it (`kind delete cluster --name flyte`) and recreate with the config
above** — they can't be added in place.
On a **DigitalOcean droplet** these mappings bind to the droplet's **public IP** —
so in the auth section `flyte.local` points at that IP instead of `127.0.0.1`, and
the two ports are open to the internet unless restricted by the cloud firewall (see
the warning in Step 0).
```bash
kubectl cluster-info --context kind-flyte
```
## Step 2: Choose and deploy dependencies (PostgreSQL + object store)
kind runs only the flyte-binary; the database and object store are hosted. **Before
writing the values file, ask the user two questions** (use the `AskUserQuestion`
tool — one question per choice, or one multi-part prompt):
1. **PostgreSQL** — *Supabase* or *another external/self-hosted PostgreSQL*?
2. **Object store** — *AWS S3* or *Cloudflare R2*?
Collect the connection details for each — the user can **type them or paste a
screenshot** of the relevant console page (Supabase Project Settings → Database; the
AWS S3 / Cloudflare R2 credentials page). Read the values out of the screenshot; if
anything required is missing or unreadable, ask for just that field. **Never invent
or guess a host, bucket name, key, or password** — if it isn't provided, stop and
ask. Required fields per choice are listed below.
Create the namespace:
```bash
kubectl create namespace flyte
```
### PostgreSQL
**Supabase** — nothing to install in the cluster. **Must use the session pooler, not
the direct connection** (see the warning below). Have the user open **Project Settings
→ Database → Connection string**, switch the tab to **Session pooler**, and type or
screenshot that string. Collect from it:
- host — `aws-<n>-<region>.pooler.supabase.com` (the pooler host, *not* `db.<ref>.supabase.co`)
- database name (Supabase default: `postgres`)
- username — `postgres.<project-ref>` (pooler username carries the project ref)
- password
Supabase requires TLS, so use `sslmode=require` in Step 3.
> [!WARNING] Why the session pooler, not the direct connection
> The direct host `db.<ref>.supabase.co` resolves to **IPv6 only**; kind is IPv4-only,
> so the Flyte pod can't reach it. `wait-for-db` still passes (it only probes the
> port via `pg_isready`), then Flyte crash-loops on `failed to connect`. The session
> pooler host has IPv4. Use port **`5432` (session)**, not `6543` (transaction) —
> Flyte's migrations need session semantics. **Read the host and username straight
> from the Session pooler tab; never reconstruct them** — a wrong region connects but
> is rejected with `tenant/user not found`, and the pooler username must be
> `postgres.<ref>`, not bare `postgres`.
**Reusing your own PostgreSQL** is also fine: take the same fields as Supabase. For
a DB on the host machine use `host.docker.internal` as the host. The database must
already exist.
### Object store
**AWS S3** — nothing to install. The user creates the bucket and an access key in
their AWS account; collect (type or screenshot):
- bucket name
- region (e.g. `us-east-1`)
- access key ID
- secret access key
**Cloudflare R2** — nothing to install. The user creates an R2 bucket and an R2 API
token in the Cloudflare dashboard; collect (type or screenshot):
- bucket name
- account endpoint (`https://<account-id>.r2.cloudflarestorage.com`)
- access key ID
- secret access key
Both endpoints are publicly resolvable, so no `signedURL` override is needed — the
SDK uploads code bundles straight to the bucket.
## Step 3: Write the values file
Assemble `values-local.yaml` from the `database` and `storage` blocks matching the
Step 2 choices. The skeleton:
```yaml
# values-local.yaml — local kind deployment
fullnameOverride: flyte
configuration:
# << database block — Supabase/external below >>
# << storage block — S3 or R2 below >>
# << inline block — task-pod storage credentials + storagePrefix, REQUIRED (below) >>
serviceAccount:
create: true
annotations: {}
ingress:
create: false
```
### Database block
**Supabase (or other external PostgreSQL)** — fill in the collected values. For
Supabase, host and username come from the **Session pooler** connection string:
```yaml
database:
postgres:
host: aws-<n>-<region>.pooler.supabase.com # session pooler host (has IPv4)
port: 5432 # session mode (not 6543 transaction mode)
dbname: postgres # Supabase default
username: postgres.<project-ref> # pooler username carries the project ref
password: <supabase-db-password>
options: "sslmode=require" # Supabase requires TLS
```
### Storage block
**AWS S3** — fill in the collected values:
```yaml
storage:
metadataContainer: <s3-bucket>
userDataContainer: <s3-bucket>
provider: s3
providerConfig:
s3:
region: <bucket-region>
authType: accesskey
accessKey: <aws-access-key-id>
secretKey: <aws-secret-access-key>
```
**Cloudflare R2** — fill in the collected values:
```yaml
storage:
metadataContainer: <r2-bucket>
userDataContainer: <r2-bucket>
provider: s3
providerConfig:
s3:
endpoint: https://<account-id>.r2.cloudflarestorage.com
region: auto # R2 ignores region; "auto" is conventional
authType: accesskey
accessKey: <r2-access-key-id>
secretKey: <r2-secret-access-key>
v2Signing: false
```
### Inline block — task-pod storage credentials + storagePrefix (required)
The `storage` block above configures only the **control plane**. Two more settings
are required for tasks to actually run — without them the API works but **every task
fails**:
- **Task pods get no object-store credentials.** The task-side SDK reads static
credentials from the `FLYTE_AWS_ENDPOINT` / `FLYTE_AWS_ACCESS_KEY_ID` /
`FLYTE_AWS_SECRET_ACCESS_KEY` env vars (`flyte/storage/_config.py`); with none set
it falls back to the default AWS credential chain and probes the EC2 metadata
endpoint, so tasks fail with
`OSError: Generic S3 error: Error performing PUT http://169.254.169.254/latest/api/token`.
The chart's `storage.*` values do **not** propagate to task pods — inject the vars
via `plugins.k8s.default-env-vars`.
- **Task I/O goes to a nonexistent bucket.** `runs.storagePrefix` defaults to
`s3://flyte-data`, so task input/output/`error.pb` writes fail with
`403 Forbidden AccessDenied`. It is **distinct from**
`metadataContainer`/`userDataContainer` (those only configure the control plane's
dataproxy) — point it at the real bucket.
Add this under `configuration:`. **The `default-env-vars` list replaces the chart
default outright**, so the three `_U_*` control-plane vars must be repeated — dropping
them breaks task→control-plane callbacks:
```yaml
inline:
runs:
storagePrefix: s3://<bucket> # the SAME bucket as the storage block
plugins:
k8s:
default-env-vars: # replaces the chart default — keep all three _U_* vars
- _U_EP_OVERRIDE: "flyte-http.flyte:8090"
- _U_INSECURE: "true"
- _U_USE_ACTIONS: "1"
- FLYTE_AWS_ACCESS_KEY_ID: "<access-key-id>"
- FLYTE_AWS_SECRET_ACCESS_KEY: "<secret-access-key>"
# Cloudflare R2 only — task pods must also be told the endpoint:
- FLYTE_AWS_ENDPOINT: "https://<account-id>.r2.cloudflarestorage.com"
```
For **AWS S3**, omit `FLYTE_AWS_ENDPOINT` and add the standard
`- AWS_REGION: "<bucket-region>"` instead (the SDK's object store reads the standard
AWS env vars for anything the `FLYTE_AWS_*` overrides don't cover).
## Step 4: Install Flyte
```bash
helm repo add flyteorg https://flyteorg.github.io/flyte
helm repo update
helm install flyte flyteorg/flyte-binary -n flyte -f values-local.yaml
kubectl -n flyte rollout status deploy/flyte
kubectl -n flyte get pods
```
If a pod is stuck in `Init`, the `wait-for-db` init container is blocking on
PostgreSQL — the DB isn't up yet, or the host/credentials are wrong. Check
`kubectl -n flyte logs <pod> -c wait-for-db`.
## Step 5: Verify access
Make the API reachable at `localhost:8090` on the machine where the SDK/CLI runs:
```bash
kubectl -n flyte port-forward service/flyte-http 8090:8090
```
**On a DigitalOcean droplet** the port-forward runs on the droplet, so tunnel it
back to the user's own machine over SSH — this one command starts the port-forward
on the droplet *and* exposes it at `localhost:8090` locally:
```bash
ssh -L 8090:localhost:8090 root@<droplet-ip> \
kubectl -n flyte port-forward service/flyte-http 8090:8090
```
> [!NOTE] `helm upgrade` kills this port-forward
> Every `helm upgrade` rolls the flyte pod, which drops the `flyte-http`
> port-forward — the SDK then reports "Flyte system is currently unavailable."
> Restart the port-forward (and the SSH tunnel, on a droplet) after each upgrade.
In another terminal:
```bash
curl -s -X POST \
http://localhost:8090/flyteidl2.project.ProjectService/ListProjects \
-H 'Content-Type: application/json' -d '{}'
```
A JSON response (not a connection error) confirms Flyte is up and talking to
its database. The base deployment is done.
**To submit runs from the SDK**, point it at the API forward. **Ask the user where
their SDK config lives** — the SDK reads the project-local `.flyte/config.yaml` (the
run directory) before `~/.flyte/config.yaml` — **and whether to edit it for them or
just give them the block to apply themselves.** Use this config:
```yaml
admin:
endpoint: dns:///localhost:8090 # the port-forwarded API — 8090, NOT 8080
insecure: True # plain HTTP, no TLS
task:
org: local
domain: development
project: flytesnacks
```
The code-bundle upload needs no second port-forward — the S3/R2 endpoint is
publicly resolvable, so the SDK uploads straight to the bucket.
**Two different ports are in play — don't conflate them.** The SDK talks to the
API on **`:8090`** (this port-forward), while the browser console lives on
**`:8080`** (Step 6). The run URL that `flyte run` prints
(`http://localhost:8080/v2/...`) is a **console** link — it only works once Step 6
is done; it is not the API endpoint, and pointing `admin.endpoint` at `:8080`
does not work.
## Step 6: Access the web console (no auth)
The base deployment leaves the console unreachable: port-forwarding
`flyte-console` directly serves only the SPA, whose frontend calls the API at the
**same origin** it was served from (`NEXT_PUBLIC_ADMIN_API_URL` is unset, so the
API base URL defaults to `/`) — those calls 404 and run pages load blank. The fix
is to put console + API behind **one origin** with Traefik.
Install Traefik (identical to step 1 of the auth section — if it's already
installed, skip this command):
```bash
helm repo add traefik https://traefik.github.io/charts
helm repo update
helm install traefik traefik/traefik -n traefik --create-namespace \
--kube-context kind-flyte \
--set "service.type=NodePort" \
--set "ports.web.nodePort=30080" \
--set "ports.websecure.nodePort=30443"
```
Route the two path groups to one origin — `flyteidl2.*` (the Connect API, over
h2c) to `flyte-http`, everything else to the console:
```bash
kubectl --context kind-flyte apply -f - <<'EOF'
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: flyte-api-noauth
namespace: flyte
spec:
entryPoints: [web]
routes:
- kind: Rule
priority: 100
match: PathPrefix(`/flyteidl2.`)
services:
- name: flyte-http
port: 8090
scheme: h2c # gRPC/Connect over cleartext HTTP/2
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: flyte-console-noauth
namespace: flyte
spec:
entryPoints: [web]
routes:
- kind: Rule
priority: 10
match: PathPrefix(`/`)
services:
- name: flyte-console
port: 80
EOF
```
Then open the console:
- **Local machine** — forward Traefik to **8080** (this makes the
`http://localhost:8080/v2/...` run URLs the SDK prints work as-is):
```bash
kubectl -n traefik --context kind-flyte port-forward service/traefik 8080:80
```
Open `http://localhost:8080/v2`. (Alternatively, the Step 1 host-port mapping
already exposes Traefik at `http://localhost/v2` with no port-forward — but the
SDK's printed run URLs still say `:8080`.)
- **DigitalOcean droplet** — the Step 1 mapping binds host port 80 on the
droplet's public IP, so the console is directly at **`http://<droplet-ip>/v2`**
(the Step 0 firewall scopes it to the user's IP). Or tunnel it:
`ssh -N -L 8080:localhost:80 root@<droplet-ip>` → `http://localhost:8080/v2`.
Note the SDK still prints run URLs as `http://localhost:8080/...` — swap
`localhost:8080` for `<droplet-ip>` unless the tunnel is up.
These routes carry **no auth** — they're the evaluation-mode front door. If the
user later enables the auth section, **delete them first**
(`kubectl -n flyte delete ingressroute flyte-api-noauth flyte-console-noauth`);
they match any host at low priority and would otherwise bypass the OIDC gate.
Now **ask the user whether they want to add OIDC authentication.** The base
deployment has no auth — anyone with network access can reach the API. If they
say yes, do the "Add OIDC authentication via an ingress controller" section
below. If no, stop here.
## Optional extras
Only do these if the user asks.
- **Load a local image into kind** (custom task/Flyte image, no registry):
```bash
kind load docker-image <your-image>:<tag> --name flyte
```
Reference that exact `<image>:<tag>` in task config; `IfNotPresent` pull
policy then uses the loaded image. On a **DigitalOcean droplet** the image must
be in the droplet's Docker daemon first — build it there, or ship it from the
user's machine with `docker save <image> | ssh root@<droplet-ip> docker load`.
## Add OIDC authentication via an ingress controller
Do this when the user opts in at the Step 6 prompt (or asks later). This adds
OIDC single sign-on at the edge, the kind equivalent of gating the cloud
console behind an ALB.
The pattern: run [Traefik](https://doc.traefik.io/traefik/) as the ingress
controller and delegate auth to
[oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/). Traefik
intercepts each request through a `ForwardAuth` middleware, asks oauth2-proxy
whether the caller is logged in, and redirects to the IdP if not. oauth2-proxy
is the auth proxy at the edge.
### First: choose the OIDC provider
oauth2-proxy needs an OIDC provider to validate against. **Ask the user which
they want** before installing anything:
- **External IdP** (Okta, Google, Auth0, …) — for a setup close to production.
Requires a registered app with redirect URI `http://flyte.local/oauth2/callback`,
and its **client ID** and **client secret**. If the user picks this but
doesn't have those ready, stop — the rest won't work.
- **Dex (local, in-cluster)** — an IdP stand-in for testing, no cloud account
or real users. If the user picks this, you'll deploy Dex via the
**`start-dex-local` skill** after Traefik is up (step 2 below).
Steps 1 and 3–4 are the same either way; only step 2 (the provider) differs.
### 1. Install Traefik
**First confirm the cluster has the `hostPort: 80 → 30080` mapping from Step 1**
(`docker ps --filter name=flyte-control-plane --format '{{.Ports}}'` should show
`0.0.0.0:80->30080/tcp`). If it doesn't — e.g. the user reused a plain cluster —
`http://flyte.local` can't reach Traefik, and the mapping can't be added in
place. Stop and have the user recreate the cluster with the auth-ready config in
Step 1 (`kind delete cluster --name flyte`, then recreate). Warn that this wipes
all data. If the user also wants SDK auth, the cluster needs `hostPort: 443 →
30443` too (also from Step 1) — same recreate-if-missing rule.
Expose both entrypoints: `web` (HTTP, for the browser) and `websecure` (HTTPS,
for the SDK — the SDK only authenticates over TLS):
```bash
helm repo add traefik https://traefik.github.io/charts
helm repo update
helm install traefik traefik/traefik -n traefik --create-namespace \
--kube-context kind-flyte \
--set "service.type=NodePort" \
--set "ports.web.nodePort=30080" \
--set "ports.websecure.nodePort=30443"
```
Installs the `Middleware` CRD, registers a `traefik` IngressClass, and serves a
default self-signed cert on `websecure` — fine for the browser.
**If Traefik is already installed from Step 6**, skip the install but **delete
the no-auth routes** — they match any host at low priority and would bypass the
OIDC gate added below:
```bash
kubectl -n flyte --context kind-flyte delete ingressroute flyte-api-noauth flyte-console-noauth
```
#### Replace the default cert with one for `flyte.local` (only if SDK auth)
Skip this if the user only needs the browser console. The SDK rejects Traefik's
default cert for two reasons, hit in sequence if you only set `insecureSkipVerify`:
- Its SAN is `*.traefik.default`, so the hostname check fails with
`certificate not valid for name "flyte.local"`. The SDK validates the SAN **even
with `insecureSkipVerify`** (that flag relaxes CA trust, not the hostname).
- The SDK implements `insecureSkipVerify` by fetching the server's chain and
**pinning it as the CA**. A bare self-signed leaf then fails with
`CaUsedAsEndEntity` — rustls won't use a leaf as a CA.
The fix is a **two-tier chain**: a self-signed root CA signs a leaf carrying
`SAN=flyte.local`. Traefik serves `leaf + CA`; the SDK pins the root as CA.
```bash
# 1. Root CA
openssl req -x509 -nodes -newkey rsa:2048 -days 3650 \
-keyout ca.key -out ca.crt -subj "/CN=flyte-local-ca" \
-addext "basicConstraints=critical,CA:TRUE" \
-addext "keyUsage=critical,keyCertSign,cRLSign"
# 2. Leaf key + CSR
openssl req -nodes -newkey rsa:2048 -keyout leaf.key -out leaf.csr -subj "/CN=flyte.local"
# 3. CA signs the leaf (CA:FALSE, SAN=flyte.local, server auth)
openssl x509 -req -in leaf.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 3650 -out leaf.crt \
-extfile <(printf "subjectAltName=DNS:flyte.local\nbasicConstraints=critical,CA:FALSE\nkeyUsage=critical,digitalSignature,keyEncipherment\nextendedKeyUsage=serverAuth")
# 4. Secret holds the full chain so Traefik serves both
cat leaf.crt ca.crt > fullchain.crt
kubectl --context kind-flyte -n traefik create secret tls flyte-local-tls \
--cert=fullchain.crt --key=leaf.key
```
Point Traefik's cluster-wide default cert at it with a `TLSStore` named `default`
(the only name Traefik honours), then restart Traefik:
```bash
kubectl --context kind-flyte apply -f - <<'EOF'
apiVersion: traefik.io/v1alpha1
kind: TLSStore
metadata:
name: default
namespace: traefik
spec:
defaultCertificate:
secretName: flyte-local-tls
EOF
kubectl --context kind-flyte -n traefik rollout restart deploy/traefik
```
The cert still chains to a self-signed root the SDK doesn't trust, so the SDK
config in step 5 **still** sets `insecureSkipVerify`. To drop that entirely you'd
install `ca.crt` into each client's trust store, or use a publicly-resolvable
domain + a publicly-trusted cert (Traefik ACME / Let's Encrypt) — impossible for a
purely-local `flyte.local`.
### 2. Set up the provider + oauth2-proxy
**If the user chose Dex:** invoke the **`start-dex-local` skill** now (Traefik
is up, which it requires). That skill deploys Dex, routes its issuer through
Traefik, and installs oauth2-proxy already pointed at Dex
(`oidc-issuer-url=http://flyte.local/dex`). When it finishes, skip to step 3 —
the middlewares. (If the user also wants SDK auth, the three SDK Bearer flags
below must be added to that oauth2-proxy install too — `helm upgrade` it with
`--reuse-values` and the three `--set extraArgs.*` lines.)
**If the user chose an external IdP:** install oauth2-proxy yourself.
`set-xauthrequest` emits the `X-Auth-Request-*` headers Traefik forwards
downstream (these feed Flyte's `executed_by` run attribution); `reverse-proxy`
trusts the forwarded host/proto from Traefik. The last three flags let the
**SDK/CLI** authenticate too (not just the browser) — include them now if the
user wants SDK auth, so you don't have to upgrade later. Substitute the user's
IdP values for the `<...>` placeholders.
```bash
# Cookie secret MUST decode to 16/24/32 bytes — head -c 32 trims the base64
# string; a raw 44-char value fails with "cookie_secret must be 16, 24, or 32 bytes".
COOKIE_SECRET=$(openssl rand -base64 32 | head -c 32)
helm repo add oauth2-proxy https://oauth2-proxy.github.io/manifests
helm repo update
helm install oauth2-proxy oauth2-proxy/oauth2-proxy -n flyte \
--kube-context kind-flyte \
--set config.clientID='<oidc-client-id>' \
--set config.clientSecret='<oidc-client-secret>' \
--set config.cookieSecret="$COOKIE_SECRET" \
--set extraArgs.provider=oidc \
--set extraArgs.oidc-issuer-url='https://<your-idp>/oauth2/default' \
--set extraArgs.upstream='static://202' \
--set extraArgs.reverse-proxy='true' \
--set extraArgs.set-xauthrequest='true' \
--set extraArgs.email-domain='*' \
--set extraArgs.cookie-secure='false' \ # local HTTP, not HTTPS
--set extraArgs.skip-jwt-bearer-tokens='true' \ # accept SDK Bearer JWTs
--set extraArgs.oidc-extra-audience='<public-client-id>' \ # SDK client's audience
--set extraArgs.bearer-token-login-fallback='false' # invalid token → 403, not HTML
```
The browser uses the session cookie; the SDK sends an `Authorization: Bearer`
JWT. `skip-jwt-bearer-tokens` verifies that JWT against the IdP's JWKS and passes
it through; `oidc-extra-audience` must be the **public client ID** the SDK uses
(the `flyteClient.clientId` advertised in `authMetadata`) — its tokens carry that
audience. The flag is **singular** (`oidc-extra-audience`); the plural form is not
a valid flag and crash-loops oauth2-proxy with `unknown flag`. Without these flags
the SDK is rejected and `flyte.run` fails the upload with `Unauthorized`.
### 3. Create the ForwardAuth middlewares
Two Traefik `Middleware` objects: one sends each request to oauth2-proxy for a
verdict and forwards the identity headers; the other catches the `401` an
unauthenticated request gets and redirects to the sign-in page. Apply with
`kubectl --context kind-flyte apply -f -`:
```yaml
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: oauth2-auth
namespace: flyte
spec:
forwardAuth:
address: http://oauth2-proxy.flyte.svc.cluster.local/oauth2/auth
trustForwardHeader: true
authResponseHeaders: # forwarded to Flyte; feed executed_by attribution
- X-Auth-Request-User
- X-Auth-Request-Email
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: oauth2-signin
namespace: flyte
spec:
errors:
status:
- "401"
service:
name: oauth2-proxy
port: 80
query: "/oauth2/sign_in?rd={url}"
```
### 4. Enable the Flyte ingress with the middlewares
Replace the `ingress.create: false` block in `values-local.yaml` with this. The
`router.middlewares` annotation chains both middlewares onto every route
(reference format `<namespace>-<name>@kubernetescrd`):
```yaml
ingress:
create: true
host: flyte.local # add "127.0.0.1 flyte.local" to /etc/hosts
ingressClassName: traefik
httpAnnotations:
traefik.ingress.kubernetes.io/router.middlewares: flyte-oauth2-signin@kubernetescrd,flyte-oauth2-auth@kubernetescrd
```
Also apply a route that sends `/oauth2` to oauth2-proxy itself so the sign-in
redirect resolves:
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: oauth2-proxy
namespace: flyte
spec:
ingressClassName: traefik
rules:
- host: flyte.local
http:
paths:
- path: /oauth2
pathType: Prefix
backend:
service:
name: oauth2-proxy
port:
number: 80
```
For `executed_by` run attribution, add `identityHeaders` to `values-local.yaml` so
Flyte reads the headers oauth2-proxy forwards (`X-Auth-Request-*`), not ALB's
`X-Amzn-Oidc-*` defaults — otherwise `executed_by` is left unset:
```yaml
flyte-core-components:
runs:
identityHeaders:
claimsJwtHeader: ""
subjectHeader: X-Auth-Request-User
emailHeader: X-Auth-Request-Email
```
Re-render Flyte:
```bash
helm upgrade flyte flyteorg/flyte-binary -n flyte --kube-context kind-flyte \
-f values-local.yaml
```
Then add a hosts entry so the browser can resolve `flyte.local` to the local
Traefik node port. Editing `/etc/hosts` needs sudo, so **have the user run it**
rather than running it yourself. First check if it's already there:
```bash
grep -q "flyte.local" /etc/hosts && echo "present" || echo "absent"
```
- **`present`** → continue.
- **`absent`** → tell the user to run it (suggest `! echo "127.0.0.1
flyte.local" | sudo tee -a /etc/hosts` so it runs in this session), then **ask
whether they've added it or want to skip.** If added, re-run the `grep` to
confirm, then continue. If skip, stop here — the deployment is complete, but
browser login won't work until the entry exists. (Opening the console by raw
IP is not a substitute: Traefik has no route for that host, and the OIDC issuer
is `flyte.local`, so login fails on an issuer mismatch.)
**On a DigitalOcean droplet**, Traefik's node ports are bound to the droplet's
**public IP**, so point `flyte.local` there in the user's **own machine's**
`/etc/hosts` (not the droplet's) — `echo "<droplet-ip> flyte.local" | sudo tee -a
/etc/hosts`. Every other `flyte.local` reference (Dex issuer, redirect URIs, cert
SAN, ingress host) stays the same; only this mapping differs. Alternatively, point
a real DNS A record at the droplet and substitute that hostname everywhere.
Once present, open `http://flyte.local/v2` — Traefik bounces you through the IdP
and back into the console.
This gates the **browser** only.
#### Split the API and discovery paths off the browser middleware (required for SDK auth)
The same `oauth2-signin` redirect on **every** path breaks the SDK, so do this before
SDK auth. Two path groups need different handling (the cloud equivalent is the
three-ingress `ingress`/`apiJwtIngress`/`wellknownIngress` split):
- **Auth-discovery** (`AuthMetadataService`, `IdentityService`) — the SDK reads these
*before* it has a token, so they must **bypass auth**. Gated, they return a
`text/plain` 401 that ConnectRPC reports as `UNAVAILABLE` (`flyte.run` fails with
"Service is unavailable"), and the SDK never starts login.
- **The `flyteidl2.*` API** — needs `oauth2-auth` (Bearer validation) but **not**
`oauth2-signin`, so an unauthenticated call gets a clean gRPC 401 the SDK retries
after login, not sign-in HTML.
Two higher-priority `IngressRoute`s (Traefik matches highest `priority` first):
```bash
kubectl --context kind-flyte apply -f - <<'EOF'
# Discovery — highest priority, NO middleware (= wellknownIngress).
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: flyte-auth-discovery
namespace: flyte
spec:
entryPoints: [web, websecure]
routes:
- kind: Rule
priority: 300
match: Host(`flyte.local`) && (PathPrefix(`/flyteidl2.auth.AuthMetadataService`) || PathPrefix(`/flyteidl2.auth.IdentityService`))
services:
- name: flyte-http
port: 8090
scheme: h2c # gRPC over cleartext HTTP/2
---
# API — oauth2-auth only, no oauth2-signin (= apiJwtIngress).
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: flyte-api-bearer
namespace: flyte
spec:
entryPoints: [web, websecure]
routes:
- kind: Rule
priority: 100
match: Host(`flyte.local`) && PathPrefix(`/flyteidl2.`)
middlewares:
- name: oauth2-auth
services:
- name: flyte-http
port: 8090
scheme: h2c
EOF
```
Verify discovery returns JSON, not oauth2-proxy's 401:
```bash
curl -s -X POST --resolve flyte.local:443:127.0.0.1 -k \
https://flyte.local/flyteidl2.auth.AuthMetadataService/GetPublicClientConfig \
-H 'Content-Type: application/json' -d '{}' | head -c 120
# → {"clientId":"flytectl", ...} (JSON, not "Unauthorized")
```
The `--resolve flyte.local:<port>:127.0.0.1` flags here (and in every other `curl`
below) assume the command runs on the host running kind. **On a DigitalOcean
droplet** that means running them in the SSH session, where `127.0.0.1` works
as-is; to run them from the user's own machine instead, substitute the droplet's
public IP for `127.0.0.1`.
### 5. Let the SDK/CLI authenticate (only if the user wants to submit runs)
The browser flow works over plain HTTP, but **the SDK does not**: it attaches its
auth interceptors only over **TLS**. With `insecure: True` it assumes "plaintext ⇒
no auth" and sends no token, so `flyte.run` fails the upload with `Unauthorized`
and no browser opens. Getting the SDK through auth needs the TLS pieces from above
(websecure on `30443`, the `443` mapping, and the three oauth2-proxy Bearer flags)
plus the SDK config below.
**Point the SDK at HTTPS.** The SDK reads the **project-local** `.flyte/config.yaml`
(the directory the run command is invoked from) *before* `~/.flyte/config.yaml`, so the
right file isn't always the home one. **Ask the user which config file applies, and
whether to edit it for them or just hand them the block to apply themselves** — don't
assume `~/.flyte/config.yaml`. Use this config:
```yaml
admin:
endpoint: dns:///flyte.local # must match SelectCluster's clusterEndpoint (no :443)
insecure: False # TLS — the SDK only authenticates over TLS
insecureSkipVerify: True # accept the self-signed CA (camelCase! see below)
authType: Pkce
task:
org: local
domain: development
project: flytesnacks
```
**The key is camelCase `insecureSkipVerify`** — the SDK reads `admin.insecureSkipVerify`;
snake_case `insecure_skip_verify` is silently ignored, so the SDK keeps full verification
and fails on the self-signed cert.
The `endpoint` must match what `SelectCluster` returns, or the SDK builds a
separate per-cluster session for the upload that may skip auth. Check it:
```bash
curl -s -X POST --resolve flyte.local:443:127.0.0.1 -k \
https://flyte.local/flyteidl2.cluster.ClusterService/SelectCluster \
-H 'Content-Type: application/json' \
-d '{"operation":"OPERATION_CREATE_UPLOAD_LOCATION","project":"flytesnacks","domain":"development","org":"local"}'
# → {"clusterEndpoint":"https://flyte.local"} ⇒ endpoint: dns:///flyte.local (no :443)
```
**If the IdP runs in-cluster (Dex):** an external IdP needs nothing more, but Dex
needs two fixes so Flyte's `GetOAuth2Metadata` (which fetches the IdP's discovery
doc to tell the SDK where to log in) succeeds:
- **DNS** — Flyte fetches `http://flyte.local/dex/...`, unresolvable in-cluster.
Add `flyte.local → Traefik ClusterIP` to the Flyte pod:
```bash
TRAEFIK_IP=$(kubectl -n traefik --context kind-flyte get svc traefik -o jsonpath='{.spec.clusterIP}')
helm upgrade flyte flyteorg/flyte-binary -n flyte --kube-context kind-flyte -f values-local.yaml \
--set "deployment.extraPodSpec.hostAliases[0].ip=$TRAEFIK_IP" \
--set "deployment.extraPodSpec.hostAliases[0].hostnames[0]=flyte.local"
```
- **Discovery path** — Flyte fetches `/.well-known/oauth-authorization-server`
(RFC 8414), but Dex only serves `/.well-known/openid-configuration` (→ 404).
Same endpoints; rewrite at Traefik:
```bash
kubectl --context kind-flyte apply -f - <<'EOF'
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: dex-wellknown-rewrite
namespace: flyte
spec:
replacePathRegex:
regex: ^/dex/\.well-known/oauth-authorization-server$
replacement: /dex/.well-known/openid-configuration
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: dex-oauth-metadata
namespace: flyte
spec:
entryPoints: [web, websecure]
routes:
- kind: Rule
priority: 200
match: Host(`flyte.local`) && Path(`/dex/.well-known/oauth-authorization-server`)
middlewares:
- name: dex-wellknown-rewrite
services:
- name: dex
port: 5556
EOF
```
Verify metadata resolves (should return JSON, not 404/timeout), then run an
example — `flyte.run` opens a browser to log in, then submits with the token:
```bash
curl -s -X POST --resolve flyte.local:443:127.0.0.1 -k \
https://flyte.local/flyteidl2.auth.AuthMetadataService/GetOAuth2Metadata \
-H 'Content-Type: application/json' -d '{}' | head -c 200
```
**First clear any stale SDK token from a previous cluster.** The SDK caches OAuth
tokens in the keyring (macOS Keychain), keyed by endpoint host — `kind delete
cluster` doesn't wipe them. Dex's `storage: memory` mints new signing keys on every
restart, so an old token fails signature check with `403 Forbidden` on
`SelectCluster` and **no browser opens**. Clear it after any cluster/Dex recreate:
```bash
# macOS; "not found" is fine. Linux: keyring del flyte.local access_token / refresh_token
for k in access_token refresh_token; do security delete-generic-password -s flyte.local -a "$k" 2>/dev/null; done
```
**SDK-auth troubleshooting:**
- Upload `Unauthorized`, **no browser** → SDK on plain HTTP. Use `insecure: False` + `https://flyte.local`.
- `Connection refused` to `https://flyte.local` → no TLS listener (websecure not exposed, or no `30443 → 443` mapping).
- 401 *after* a successful browser login → oauth2-proxy rejects the Bearer token; confirm `skip-jwt-bearer-tokens` + `oidc-extra-audience=<client-id>`; check its logs for `audience ... does not match`.
- `403 Forbidden` on `SelectCluster`, **no browser** (oauth2-proxy logs `failed to verify id token signature`) → stale cached token; Dex's in-memory keys changed on restart. Clear the keyring tokens (block above) and rerun.
- `GetOAuth2Metadata` 404 `oauth-authorization-server` → in-cluster IdP: well-known rewrite missing.
- `GetOAuth2Metadata` times out → in-cluster IdP: `hostAliases` missing on the Flyte pod.
## Enable app serving (optional — Knative + Kourier)
Flyte can host long-running **apps** (web services, dashboards, model servers —
deployed via the SDK), each published at `{name}-{project}-{domain}.<base-domain>`.
It's **off by default**: the binary always exposes `AppService`, but with no
controller behind it the console's Apps tab and any `flyteidl2.app.AppService/List`
call return `{"code":"unimplemented","message":"404 Not Found"}` until enabled.
Apps run as **Knative Services**, so Knative Serving + a Knative networking layer
(Kourier) must be installed first. Skip this section unless the user wants apps.
Official doc: https://www.union.ai/docs/v2/flyte/oss-deployment/app-serving/.
On kind there's no cloud load balancer and no real DNS, so this recipe uses
**[sslip.io](https://sslip.io) wildcard DNS** (any `X.127.0.0.1.sslip.io` resolves
to `127.0.0.1`; any `X.<droplet-ip>.sslip.io` to the droplet — `/etc/hosts` can't
do wildcards, and every app gets its own hostname) and routes app traffic through
**Traefik** on the existing host-port-80 mapping. It requires Traefik from Step 6
(or the auth section).
**1. Install Knative Serving + Kourier.** Pick a Knative release that supports the
cluster's k8s version (Knative supports only the most recent k8s minors — the
pinned version below may need bumping), and use the **same version** for serving
and net-kourier:
```bash
KV=knative-v1.22.1 # must support your k8s version; serving + net-kourier must match
kubectl --context kind-flyte apply -f https://github.com/knative/serving/releases/download/$KV/serving-crds.yaml
kubectl --context kind-flyte apply -f https://github.com/knative/serving/releases/download/$KV/serving-core.yaml
kubectl --context kind-flyte apply -f https://github.com/knative-extensions/net-kourier/releases/download/$KV/kourier.yaml
kubectl --context kind-flyte patch configmap/config-network -n knative-serving --type merge \
-p '{"data":{"ingress-class":"kourier.ingress.networking.knative.dev"}}'
kubectl --context kind-flyte wait --for=condition=Available deploy --all -n knative-serving --timeout=180s
kubectl --context kind-flyte wait --for=condition=Available deploy --all -n kourier-system --timeout=180s
```
If `kubectl apply` rejects the manifests, the Knative release is newer than the
cluster's k8s version supports — install an older one (serving + net-kourier
matched).
**2. Configure the apps domain.** Base domain = `127.0.0.1.sslip.io` locally, or
`<droplet-ip>.sslip.io` on a droplet. Drop the namespace from Knative's hostname
template so each app is a **single label** under the base domain (the default
`{{.Name}}.{{.Namespace}}.{{.Domain}}` is two labels):
```bash
BASE=127.0.0.1.sslip.io # droplet: <droplet-ip>.sslip.io
kubectl --context kind-flyte patch configmap/config-domain -n knative-serving --type merge \
-p "{\"data\":{\"$BASE\":\"\"}}"
kubectl --context kind-flyte patch configmap/config-network -n knative-serving --type merge \
-p '{"data":{"domain-template":"{{.Name}}.{{.Domain}}"}}'
```
**3. Route app hostnames through Traefik.** Kourier's `kourier` Service is
`LoadBalancer` type, which stays `<pending>` forever on kind — switch it to
`ClusterIP` and front it with an IngressRoute that matches any sslip.io app host.
The higher priority (150) wins over the Step 6 no-auth routes for app hosts;
`localhost`/`flyte.local` traffic is untouched:
```bash
kubectl --context kind-flyte patch svc kourier -n kourier-system --type merge -p '{"spec":{"type":"ClusterIP"}}'
kubectl --context kind-flyte apply -f - <<EOF
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: kourier-apps
namespace: kourier-system
spec:
entryPoints: [web]
routes:
- kind: Rule
priority: 150
match: HostRegexp(\`^.+\.${BASE//./\\.}$\`)
services:
- name: kourier
port: 80
EOF
```
**4. Enable the app controller in Flyte.** Add to `values-local.yaml` under the
existing `configuration.inline` block — `baseDomain` MUST equal the
`config-domain` from step 2:
```yaml
internalApps:
enabled: true
baseDomain: 127.0.0.1.sslip.io # droplet: <droplet-ip>.sslip.io
scheme: http # plain HTTP through Traefik (evaluation)
ingressAppsPort: 0 # apps ride host port 80; omit the port
```
```bash
helm upgrade flyte flyteorg/flyte-binary -n flyte --kube-context kind-flyte -f values-local.yaml
kubectl -n flyte --context kind-flyte rollout status deploy/flyte
```
The chart auto-grants the `serving.knative.dev` RBAC when `internalApps.enabled`.
The upgrade rolls the flyte pod, so **restart the `flyte-http` port-forward**
(Step 5) afterward.
**5. Verify.**
```bash
kubectl --context kind-flyte auth can-i create services.serving.knative.dev \
--as=system:serviceaccount:flyte:flyte -n flyte # => yes
# AppService now answers 200 + {} (NOT 404/unimplemented) — needs the Step 5 port-forward:
curl -s -o /dev/null -w '%{http_code}\n' -X POST \
http://localhost:8090/flyteidl2.app.AppService/List \
-H 'Content-Type: application/json' -d '{}' # => 200
```
The console's Apps tab now loads. Deploy an app with the SDK and open
`http://<name>-<project>-<domain>.<base-domain>/` (sslip.io needs internet DNS;
on a droplet the Step 0 firewall scopes port 80 to the user's IP).
**Gotchas:** (a) Knative version too new for the k8s version → manifests rejected
on apply. (b) two-label app hostnames → confirm the single-label
`domain-template`. (c) `baseDomain` ≠ `config-domain` → the URLs Flyte advertises
don't match what Knative serves. (d) `List` still 404s after enabling → the
binary didn't roll; `kubectl -n flyte rollout restart deploy/flyte`. (e) apps are
**unauthenticated** — anyone who can reach port 80 can open them (locally that's
just the machine; on a droplet, whatever the firewall admits).
## Tear down
```bash
kind delete cluster --name flyte
```
Deletes the cluster and Flyte. **On a DigitalOcean droplet**, also destroy the
droplet so it stops billing:
```bash
doctl compute droplet delete flyte-kind
```
The hosted PostgreSQL and S3/R2 bucket are untouched — clean those up in their own
consoles.
deploy-flyte-kind-vm15.5 KB
---
name: deploy-flyte-kind-vm
description: Deploy Flyte on a kind cluster onto any host — your local machine, or a fresh cloud VM (DigitalOcean, AWS EC2, or GCP Compute Engine). Provisions the VM (firewall scoped to your IP), installs Docker/kind/kubectl/helm, then runs the kind Flyte deploy on that host and tunnels access back to your machine. Use when the user wants Flyte-on-kind but hasn't decided where to run it, or wants it on a cloud VM rather than locally. For evaluation only (single-node kind, static credentials). Delegates the Flyte install itself to the deploy-flyte-kind skill.
---
# Deploy Flyte on kind — local machine or a cloud VM
kind runs anywhere Docker runs, so the same Flyte-on-kind deploy works on your
own machine **or** on a cloud VM. This skill picks the **host**, provisions it if
it's a cloud VM, installs the prerequisites, and then runs the actual Flyte
deploy — the cluster, the hosted PostgreSQL + object store, the helm install, and
optional auth — by handing off to the **`deploy-flyte-kind`** skill. The only
things that change per host are *where the commands run* and *how you reach the
API afterward*.
> **For evaluation only.** Single-node kind, static credentials, no workload
> identity. On a cloud VM the stack is reachable from the public internet, so the
> provisioning steps below **restrict inbound 80/443 (and 22) to your own IP**.
> For production, use the `flyte-deploy-aws` skill instead.
This skill is guided: several steps need **human-in-the-loop** input — which host,
confirming billable VM creation, SSH keys, and (in the deploy hand-off) the
Supabase / R2 / S3 credentials. Ask; never invent account IDs, IPs, keys, or
passwords.
## Step 0: Choose the host
Ask the user where to run the cluster (use `AskUserQuestion`):
- **Local machine** — kind runs in your local Docker. Simplest; nothing to
provision.
- **DigitalOcean** — a Droplet, provisioned with `doctl`.
- **AWS EC2** — an instance, provisioned with the `aws` CLI.
- **GCP Compute Engine** — an instance, provisioned with `gcloud`.
Then:
- **Local machine** → skip to **Step 3** (nothing to provision). The whole deploy
runs locally.
- **A cloud VM** → do **Step 1** (provision) and **Step 2** (define how remote
commands run), then Step 3.
kind needs a few GB of headroom, so any VM should be **at least 4 vCPU / 8 GB**.
## Step 1: Provision the cloud VM (skip for local)
First confirm the provider CLI is installed and authenticated locally, since these
commands run on **your** machine:
```bash
# DigitalOcean
command -v doctl >/dev/null && doctl account get >/dev/null 2>&1 || echo "doctl: install and run 'doctl auth init'"
# AWS
command -v aws >/dev/null && aws sts get-caller-identity >/dev/null 2>&1 || echo "aws: install and configure credentials"
# GCP
command -v gcloud >/dev/null && gcloud auth list >/dev/null 2>&1 || echo "gcloud: install and run 'gcloud auth login'"
```
If the CLI is missing or unauthenticated, stop and have the user set it up (an
interactive login like `gcloud auth login` is easiest run by the user — suggest
they type `! gcloud auth login` so it runs in this session). **Creating a VM
incurs cost — confirm with the user before running any `create`/`run-instances`
command,** and ask for the values it needs (SSH key ID / key-pair name / zone).
Pick the tab for the chosen provider. Each: create a firewall/security rule
scoped to the user's IP, create the VM, SSH in, install Docker + kind + kubectl +
helm.
### DigitalOcean
Setting up `doctl` if it's missing or unauthenticated (`brew install doctl` on
macOS): auth is interactive by default, but if the user pastes an API token it
can be done non-interactively with `doctl auth init -t <token>`. The SSH key must
be **imported to DigitalOcean** before it can be passed to `droplet create`:
```bash
doctl compute ssh-key list # already imported? use its ID below
doctl compute ssh-key import <name> --public-key-file ~/.ssh/id_ed25519.pub # prints the ID
```
```bash
# Create the Droplet (ask the user for their SSH key ID: `doctl compute ssh-key list`)
doctl compute droplet create flyte-kind \
--image ubuntu-24-04-x64 --size s-4vcpu-8gb --region nyc1 \
--ssh-keys <your-ssh-key-id>
```
Scope inbound 22/80/443 to your own IP with a cloud firewall (do this before
exposing anything — see the evaluation-only note). Then SSH in and install the
tools (DigitalOcean's Ubuntu image logs in as `root`, so no `sudo` needed):
```bash
ssh root@<droplet-ip> 'bash -s' <<'EOF'
curl -fsSL https://get.docker.com | sh
curl -Lo /usr/local/bin/kind https://github.com/kubernetes-sigs/kind/releases/latest/download/kind-linux-amd64 && chmod +x /usr/local/bin/kind
curl -Lo /usr/local/bin/kubectl "https://dl.k8s.io/release/$(curl -Ls https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && chmod +x /usr/local/bin/kubectl
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
EOF
```
### AWS EC2
Create a security group that admits **only your IP** on 22/80/443, then launch the
instance with the current Ubuntu 24.04 AMI:
```bash
MY_IP=$(curl -s https://checkip.amazonaws.com)
aws ec2 create-security-group --group-name flyte-kind --description "Flyte kind evaluation"
for port in 22 80 443; do
aws ec2 authorize-security-group-ingress --group-name flyte-kind \
--protocol tcp --port $port --cidr ${MY_IP}/32
done
aws ec2 run-instances \
--image-id "$(aws ssm get-parameters \
--names /aws/service/canonical/ubuntu/server/24.04/stable/current/amd64/hvm/ebs-gp3/ami-id \
--query 'Parameters[0].Value' --output text)" \
--instance-type t3.xlarge \
--key-name <your-key-pair> \
--security-groups flyte-kind
```
SSH in as `ubuntu` (installs need `sudo`) and install the tools:
```bash
ssh -i <your-key.pem> ubuntu@<instance-public-ip> 'bash -s' <<'EOF'
curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker $USER
sudo curl -Lo /usr/local/bin/kind https://github.com/kubernetes-sigs/kind/releases/latest/download/kind-linux-amd64 && sudo chmod +x /usr/local/bin/kind
sudo curl -Lo /usr/local/bin/kubectl "https://dl.k8s.io/release/$(curl -Ls https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && sudo chmod +x /usr/local/bin/kubectl
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | sudo bash
EOF
```
The `usermod -aG docker` takes effect on the **next** login, so the Step 2 SSH
sessions can run `docker`/`kind` without `sudo`. (In a single interactive session
you'd `newgrp docker`; over one-shot `ssh` commands, just reconnect.)
### GCP Compute Engine
```bash
gcloud compute instances create flyte-kind \
--machine-type e2-standard-4 --zone <your-zone> \
--image-family ubuntu-2404-lts-amd64 --image-project ubuntu-os-cloud \
--tags flyte-kind
# GCP blocks inbound 80/443 until a rule allows them — scope to the tag + your IP:
MY_IP=$(curl -s https://checkip.amazonaws.com)
gcloud compute firewall-rules create flyte-kind-web \
--allow tcp:80,tcp:443 --target-tags flyte-kind --source-ranges ${MY_IP}/32
```
SSH in (installs need `sudo`) and install the tools:
```bash
gcloud compute ssh flyte-kind --zone <your-zone> --command='bash -s' <<'EOF'
curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker $USER
sudo curl -Lo /usr/local/bin/kind https://github.com/kubernetes-sigs/kind/releases/latest/download/kind-linux-amd64 && sudo chmod +x /usr/local/bin/kind
sudo curl -Lo /usr/local/bin/kubectl "https://dl.k8s.io/release/$(curl -Ls https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && sudo chmod +x /usr/local/bin/kubectl
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | sudo bash
EOF
```
## Step 2: Define how the deploy commands run on the VM (skip for local)
Every `kind`, `kubectl`, and `helm` command from the deploy runs **on the VM**,
not your machine. Fix the SSH target once and reuse it:
```bash
VM="root@<droplet-ip>" # DigitalOcean
# VM="ubuntu@<instance-public-ip>" # AWS EC2 (add -i <your-key.pem> to ssh below)
# GCP: use `gcloud compute ssh flyte-kind --zone <zone> --command='...'` instead of ssh $VM
```
Then, when the `deploy-flyte-kind` skill (Step 3) says to run a command, run it on
the VM instead of locally:
- **Single command:** `ssh $VM '<command>'`
- **A block or a heredoc** (e.g. `kind create cluster --config - <<'EOF' … EOF`):
pipe it through SSH — `ssh $VM 'bash -s' <<'EOF' … EOF`.
Alternatively, open one interactive session (`ssh $VM`) and run the deploy
commands there. Either way, **only the SDK/CLI and browser run on your machine**;
everything cluster-side runs on the VM. The access tunnel in Step 4 bridges the
two.
## Step 3: Run the Flyte deploy (hand off to `deploy-flyte-kind`)
Now do the actual install by following the **`deploy-flyte-kind`** skill. It
covers, in order:
1. **Prereqs + existing-cluster check** — you've already installed the tools; on a
VM the `kind get clusters` / prereq checks run via `ssh $VM '...'`.
2. **Create the kind cluster** — with the two host-port mappings (`30080→80`,
`30443→443`). On a VM these bind to the VM's public IP; `flyte.local` will point
at that IP (not `127.0.0.1`) if you enable auth.
3. **Choose + configure dependencies** — this is the main **human-in-the-loop**
part: PostgreSQL (**Supabase** or external) and object store (**AWS S3** or
**Cloudflare R2**). The user creates these in their consoles and supplies the
connection details (they can paste a screenshot). Follow that skill's guidance
exactly — especially Supabase's **session pooler** requirement (kind is IPv4-only).
4. **Write `values-local.yaml`** and **`helm install`** the flyte-binary chart.
Don't skip that skill's **inline block** (task-pod `FLYTE_AWS_*` credentials +
`runs.storagePrefix`) — without it the API works but every task fails.
5. **Web console access** (Step 6 there) — Traefik + the two unified-origin
routes. See Step 4 below for the VM-specific ways to reach it.
6. **Optional OIDC auth** via Traefik + oauth2-proxy (and the `start-dex-local`
skill for an in-cluster Dex IdP).
7. **Optional app serving** (Knative + Kourier) — that skill's "Enable app
serving" section. On a cloud VM the apps base domain is `<vm-ip>.sslip.io`,
and apps are served directly on port 80 (firewall-scoped to the user's IP).
**When on a cloud VM, apply the Step 2 wrapper to every command in that skill** —
create the values/config files on the VM (write them via the piped heredoc, or
edit them in the interactive session), since helm reads them there. The logic is
identical; only the execution location changes.
## Step 4: Access Flyte
The API is reached at `localhost:8090` on the machine where you run the SDK/CLI.
**Local machine** — just port-forward:
```bash
kubectl -n flyte port-forward service/flyte-http 8090:8090
```
**Cloud VM** — the port-forward runs on the VM, so tunnel it back over SSH. One
command starts the forward on the VM and exposes it at `localhost:8090` locally:
```bash
# DigitalOcean
ssh -L 8090:localhost:8090 root@<droplet-ip> \
kubectl -n flyte port-forward service/flyte-http 8090:8090
# AWS EC2
ssh -i <your-key.pem> -L 8090:localhost:8090 ubuntu@<instance-public-ip> \
kubectl -n flyte port-forward service/flyte-http 8090:8090
# GCP
gcloud compute ssh flyte-kind --zone <your-zone> \
--ssh-flag="-L 8090:localhost:8090" \
--command="kubectl -n flyte port-forward service/flyte-http 8090:8090"
```
> [!WARNING] Do NOT add `-N` to these tunnel commands
> These tunnels run `kubectl port-forward` as the SSH **remote command**, and
> `-N` tells SSH to skip the remote command — the forward never starts and every
> connection gets `connection refused`. `-N` is right only for a pure tunnel with
> no remote command (like the console tunnel below).
Keep it running. Verify from your machine (a JSON response, not a connection
error, confirms Flyte is up and talking to its database):
```bash
curl -s -X POST http://localhost:8090/flyteidl2.project.ProjectService/ListProjects \
-H 'Content-Type: application/json' -d '{}'
```
Then point the SDK at it — the `deploy-flyte-kind` "Verify access" step has the
`~/.flyte/config.yaml` block (`endpoint: dns:///localhost:8090`, `insecure: True`).
The tunnel makes the VM deploy behave exactly like a local one for the SDK. The
code-bundle upload needs no second tunnel — the S3/R2 endpoint is publicly
resolvable, so the SDK uploads to the presigned URL directly. Note `helm upgrade`
rolls the flyte pod and **drops this tunnel's port-forward** — restart it after
each upgrade.
### Console + tunnel-free SDK access from a VM (no auth)
Once Traefik + the two unified-origin routes from `deploy-flyte-kind` **Step 6**
are installed (run those commands on the VM), the kind host-port mapping
(`30080 → 80`) binds Traefik to the VM's **public IP**, and both browser and SDK
can skip the tunnels entirely — the Step 1 firewall already scopes port 80 to the
user's IP:
- **Console** — directly at **`http://<vm-ip>/v2`**, no tunnel. Or tunnel it
(note this pure tunnel **does** want `-N` — there's no remote command):
```bash
ssh -N -L 8080:localhost:80 root@<vm-ip> # then open http://localhost:8080/v2
```
- **SDK, tunnel-free** — gRPC rides Traefik's `web` entrypoint over h2c on
port 80, so the SDK can point straight at the VM:
```yaml
# .flyte/config.yaml
admin:
endpoint: dns:///<vm-ip>:80
insecure: true
```
Lower-friction than the Step 4 tunnel (nothing to keep running, survives helm
upgrades), with the caveat that it's **plain HTTP over the public internet** —
acceptable for evaluation only because the firewall admits just the user's IP.
- **Run-URL host mismatch** — `flyte run` prints
`URL: http://localhost:8080/v2/...` regardless of where Flyte runs. On a VM
those links are dead as printed: either keep the `-N` console tunnel above on
port 8080 so they work as-is, or swap `localhost:8080` for `<vm-ip>`.
> If you enable **auth** (Step 3.6), the SDK reaches Flyte at `https://flyte.local`
> over the `443` mapping, not this port-forward. On a cloud VM, `flyte.local` must
> resolve to the **VM's public IP** in your local `/etc/hosts` (not `127.0.0.1`),
> and inbound 443 must be open to your IP (Step 1). Otherwise the auth flow is
> identical to `deploy-flyte-kind`.
## Optional: load a local image into kind
kind nodes can't pull from a host Docker daemon, so a custom task/Flyte image must
be loaded into the cluster:
```bash
kind load docker-image <your-image>:<tag> --name flyte # local
```
On a cloud VM the image must be in the **VM's** Docker daemon first — either build
it on the VM, or ship it from your machine:
```bash
docker save <your-image>:<tag> | ssh $VM docker load
ssh $VM 'kind load docker-image <your-image>:<tag> --name flyte'
```
Reference that exact `<image>:<tag>` in task config; the `IfNotPresent` pull policy
then uses the loaded image.
## Tear down
```bash
kind delete cluster --name flyte # local
ssh $VM 'kind delete cluster --name flyte' # cloud VM
```
On a cloud VM, also delete the instance so it stops billing (confirm with the
user), and remove the firewall/security group you created:
```bash
doctl compute droplet delete flyte-kind # DigitalOcean
aws ec2 terminate-instances --instance-ids <instance-id> # AWS EC2 — also: aws ec2 delete-security-group --group-name flyte-kind
gcloud compute instances delete flyte-kind --zone <your-zone> # GCP — also: gcloud compute firewall-rules delete flyte-kind-web
```
The hosted PostgreSQL and S3/R2 bucket are untouched — clean those up in their own
consoles.
flyte-deploy-aws57.2 KB
---
name: flyte-deploy-aws
description: 'Use when deploying a Flyte v2 (flyte-binary / flyte2) cluster on AWS from scratch — provisions EKS + S3 + RDS PostgreSQL + AWS Load Balancer Controller, then helm-installs the flyte-binary chart behind an ALB, with optional TLS and Okta SSO. Trigger words: "deploy flyte", "flyte v2 on AWS", "flyte EKS".'
---
# Deploying Flyte v2 on AWS (EKS + RDS + S3 + ALB)
Flyte v2 ships as a single unified binary (`flyte-binary-v2`) plus a separate console
image. One HTTP ingress serves the console (`/v2`), the `flyteidl2.*` Connect API, and
auth-discovery — there is no separate gRPC port. You scale it vertically.
The chart does NOT provision infrastructure. Stand up four things first:
**EKS cluster, S3 bucket, PostgreSQL (RDS), and (for external access) an ingress
controller.** This skill does all four with `eksctl` + `aws` + `helm`, then installs the
`flyte-binary` chart (the v2 chart; defaults to `flyte-binary-v2` + `flyteconsole-v2`).
**Get the chart first.** Install from the published Helm repo — this is what the official
docs do, and the released chart pins the Flyte image tag to the chart version (see Image
selection in Step 5). Also `helm pull --untar` a local copy so the TaskAction CRD file is on
disk for the Step 5 idempotency check:
```bash
helm repo add flyteorg https://flyteorg.github.io/flyte && helm repo update
helm pull flyteorg/flyte-binary --untar # ./flyte-binary/templates/crds/flyte.org_taskactions.yaml now resolves
```
(Alternatively clone the repo — `git clone https://github.com/flyteorg/flyte` — and install
from the local `./charts/flyte-binary` path for the bleeding-edge chart; its default image tag
is a floating `:latest`, so pair it with `pullPolicy: Always` — see Image selection.) Official
docs: https://www.union.ai/docs/v2/flyte/oss-deployment/aws-deployment/. Validated end-to-end
on EKS.
> Replace every placeholder in angle brackets and the example hostnames/IDs with your own.
## Prerequisites & decisions
- CLIs: `aws` v2, **`eksctl` ≥ 0.227** (older caps out at k8s 1.29 — see gotcha), `kubectl`, `helm`, `jq`.
- Admin (or EKS+RDS+IAM+S3+EC2) creds. STS/SSO works — export the 3 env vars + region.
- eksctl writes the kubeconfig context (e.g. `<user>@flyte-v2.<region>.eksctl.io`). Pass
`kubectl --context <ctx>` (and `helm --kube-context <ctx>`) per command rather than
`kubectl config use-context` — that way you don't mutate the operator's current context.
- Decide: region, name prefix, and **exposure**: ALB+TLS needs a Route53 zone + ACM cert;
**ALB HTTP-only needs neither** (reached at the auto `*.elb.amazonaws.com` name) — the
simplest default when you own no domain. (This skill provisions **RDS PostgreSQL** for the DB;
an in-cluster Postgres is out of scope here — RDS is assumed by Steps 3–5.)
**Persist your variables.** This deploy spans many commands and derives values you can't
recover later — most critically the **random `DBPW`** (Step 3), plus `ACCT`, `BUCKET`,
`RDS_HOST`, `IRSA_ARN`, etc. If your shell state resets between steps (or your AWS session
token expires and you re-auth in a fresh shell), these are gone. Keep them in a file you
re-`source` at the start of every step, and append each derived value as you compute it:
```bash
export AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... AWS_SESSION_TOKEN=...
export AWS_DEFAULT_REGION=us-west-2
ENVF=~/flyte-deploy.env # source this at every step
{ echo "export PREFIX=flyte-v2 REGION=us-west-2 CLUSTER=flyte-v2"
echo "export ACCT=$(aws sts get-caller-identity --query Account --output text)" # confirm the RIGHT account
} >> $ENVF && source $ENVF
# As you create infra, append its outputs, e.g.: echo "export DBPW='$DBPW' RDS_HOST=$RDS_HOST" >> $ENVF
# Check for an existing domain/cert (empty => go ALB HTTP-only):
aws route53 list-hosted-zones --query 'HostedZones[].Name' --output text
aws acm list-certificates --region $REGION --query 'CertificateSummaryList[].DomainName' --output text
```
## Step 0 — Reuse an existing cluster?
Before creating anything, list the EKS clusters already in the account/region and **ask the
user whether to deploy onto one of them or stand up a fresh cluster**. Reusing skips Step 1
(~15-20 min + the EKS control-plane + node cost).
```bash
aws eks list-clusters --region $REGION --query 'clusters' --output text
```
Present the list and let the user pick one (or choose "create new"). If they reuse one:
```bash
CLUSTER=<chosen-cluster>
aws eks update-kubeconfig --region $REGION --name $CLUSTER --alias $CLUSTER # writes + selects context
kubectl --context $CLUSTER get nodes # confirm reachable + Ready
# Confirm IRSA is possible (the chart needs an OIDC provider on the cluster):
aws eks describe-cluster --region $REGION --name $CLUSTER \
--query 'cluster.identity.oidc.issuer' --output text # empty => run: eksctl utils associate-iam-oidc-provider --cluster $CLUSTER --approve
```
Then **skip Step 1** and continue from Step 2. S3 (Step 2), RDS (Step 3), and the ALB
controller (Step 4) may already exist on a reused cluster — check before recreating
(`aws s3 ls`, `aws rds describe-db-instances`, `kubectl --context $CLUSTER -n kube-system get deploy aws-load-balancer-controller`)
and reuse what's there. Otherwise proceed normally. Pass `--context $CLUSTER` /
`--kube-context $CLUSTER` on the later kubectl/helm commands.
## Step 0.5 — Confirm deployment parameters (ASK up front, never assume)
**Before provisioning or installing anything, gather the deploy parameters by ASKING the
user — do NOT silently reuse values you happen to find.** A previous deploy leaves identifiers
lying around (an old `values-eks.yaml` with `HOST=`/`certificate-arn`/`password`, a live
`flyte*-console-oidc` k8s Secret, `authMetadata.flyteClient.clientId`, a memory of the last
run). These are **suggestions to confirm, not defaults.** Silently reusing the prior
hostname, OIDC client ID/secret, or cert is the #1 way this skill does the wrong thing.
For each parameter below, **discover any prior value, then present it as a choice** — e.g.
"reuse previous (`test.uniondemo.run`, loaded from the old values file / the in-cluster
Secret), enter a new one, or pick a different existing one" — and let the user decide. Restate
the final set back to them before `helm install`.
| Parameter | Where a prior value hides | Notes |
|---|---|---|
| Region / name prefix / cluster | Step 0, current kube-context | |
| Exposure (HTTP-only / TLS / TLS+SSO) | — | drives which params below apply |
| Hostname | `HOST=` in old `values-eks.yaml`; existing Route53 record | drives cert, OIDC redirect URI, DNS |
| ACM cert ARN | old values `certificate-arn`; `aws acm list-certificates` | must match the chosen hostname |
| OIDC issuer / client ID / **client secret** | `authMetadata` in old values; `flyte*-console-oidc` Secret; the IdP app | **never echo/ask for the secret in chat** — have the user create the Secret themselves (see ALB edge SSO) |
| OIDC CLI/PKCE client ID | `authMetadata.flyteClient.clientId` | |
| S3 bucket / RDS host+password | Step 2/3 outputs; old values | reuse the live infra's real values |
Only after the user confirms each value do you write `values-eks.yaml` (Step 5). If reusing a
secret/credential, confirm the user still wants *that* IdP app — switching IdP is just a new
Secret + issuer refs (no ALB/DNS churn).
## Step 1 — EKS cluster (eksctl)
`cluster.yaml` — `iam.withOIDC: true` is what makes IRSA possible:
```yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata: { name: flyte-v2, region: us-west-2, version: "1.33" }
iam: { withOIDC: true }
managedNodeGroups:
- name: ng-default
instanceType: m5.large
desiredCapacity: 2
minSize: 2
maxSize: 3
volumeSize: 50
iam: { withAddonPolicies: { ebs: true } }
addons: [{name: vpc-cni},{name: coredns},{name: kube-proxy},{name: aws-ebs-csi-driver}]
```
```bash
eksctl create cluster -f cluster.yaml # ~15-20 min; writes kubeconfig + sets context
kubectl get nodes # expect Ready
```
The VPC + private subnets exist within ~2 min (before the control plane finishes), so you
can start RDS (step 3) in parallel.
## Step 2 — S3 bucket + IRSA role
```bash
BUCKET=$PREFIX-data-$ACCT # account-id suffix => globally unique
aws s3api create-bucket --bucket $BUCKET --region $REGION \
--create-bucket-configuration LocationConstraint=$REGION
aws s3api put-public-access-block --bucket $BUCKET --public-access-block-configuration \
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
aws s3api put-bucket-encryption --bucket $BUCKET --server-side-encryption-configuration \
'{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'
# Scoped S3 policy: ListBucket on the bucket, Get/Put/Delete on its objects.
cat > s3-policy.json <<EOF
{"Version":"2012-10-17","Statement":[
{"Effect":"Allow","Action":["s3:ListBucket"],"Resource":"arn:aws:s3:::$BUCKET"},
{"Effect":"Allow","Action":["s3:GetObject","s3:PutObject","s3:DeleteObject"],"Resource":"arn:aws:s3:::$BUCKET/*"}]}
EOF
POLICY_ARN=$(aws iam create-policy --policy-name $PREFIX-s3-access \
--policy-document file://s3-policy.json --query Policy.Arn --output text)
# --role-only: create the IAM role with OIDC trust for system:serviceaccount:flyte:flyte,
# but NOT the k8s SA (the chart creates+annotates it). Works before the ns exists.
eksctl create iamserviceaccount --cluster $CLUSTER --region $REGION \
--namespace flyte --name flyte --role-name $PREFIX-irsa \
--attach-policy-arn "$POLICY_ARN" --role-only --approve
IRSA_ARN=$(aws iam get-role --role-name $PREFIX-irsa --query Role.Arn --output text)
```
## Step 3 — RDS PostgreSQL (can run in parallel with step 1)
```bash
VPC=$(aws ec2 describe-vpcs --region $REGION \
--filters "Name=tag:alpha.eksctl.io/cluster-name,Values=$CLUSTER" --query 'Vpcs[0].VpcId' --output text)
# Private subnets (internal-elb role tag):
SUBNETS=$(aws ec2 describe-subnets --region $REGION --filters "Name=vpc-id,Values=$VPC" \
"Name=tag:kubernetes.io/role/internal-elb,Values=1" --query 'Subnets[].SubnetId' --output text)
# CRITICAL: source SG must be the EKS-managed cluster SG actually on the NODES,
# NOT ClusterSharedNodeSecurityGroup. Pod egress uses the node primary-ENI SG.
aws rds create-db-subnet-group --region $REGION --db-subnet-group-name $PREFIX-db-subnets \
--db-subnet-group-description "Flyte v2 private DB subnets" --subnet-ids $SUBNETS
RDSSG=$(aws ec2 create-security-group --region $REGION --group-name $PREFIX-rds-sg \
--description "Flyte v2 RDS 5432 from cluster nodes" --vpc-id $VPC --query GroupId --output text)
DBPW=$(LC_ALL=C tr -dc 'A-Za-z0-9' </dev/urandom | head -c 28)
echo "export DBPW='$DBPW' RDSSG=$RDSSG VPC=$VPC" >> $ENVF # persist (DBPW is unrecoverable)
aws rds create-db-instance --region $REGION --db-instance-identifier $PREFIX-db \
--engine postgres --db-instance-class db.t3.micro --allocated-storage 20 --storage-type gp3 \
--master-username flyte --master-user-password "$DBPW" --db-name flyte \
--vpc-security-group-ids $RDSSG --db-subnet-group-name $PREFIX-db-subnets \
--no-publicly-accessible --backup-retention-period 1
# Endpoint (when status=available):
RDS_HOST=$(aws rds describe-db-instances --region $REGION --db-instance-identifier $PREFIX-db \
--query 'DBInstances[0].Endpoint.Address' --output text)
echo "export RDS_HOST=$RDS_HOST" >> $ENVF
```
**Open 5432 from the nodes — do this once the nodegroup is up** (`kubectl get nodes` Ready),
not before: pod egress uses the **EKS-managed cluster SG on the nodes** (`eks-cluster-sg-*`),
NOT `ClusterSharedNodeSecurityGroup` (gotcha 2). If you started RDS in parallel with Step 1,
the nodes may not exist yet — that's why this is its own step. The DB just needs this one rule:
```bash
NODESG=$(aws ec2 describe-instances --region $REGION \
--filters "Name=tag:eks:cluster-name,Values=$CLUSTER" "Name=instance-state-name,Values=running" \
--query 'Reservations[0].Instances[0].SecurityGroups[?contains(GroupName,`eks-cluster-sg`)].GroupId' --output text)
[ -n "$NODESG" ] || { echo "no running nodes yet — wait for the nodegroup, then re-run"; }
aws ec2 authorize-security-group-ingress --region $REGION --group-id $RDSSG \
--protocol tcp --port 5432 --source-group $NODESG # init container retries until this lands
```
## Step 4 — AWS Load Balancer Controller (for ALB ingress)
```bash
# Use the policy matching the controller version the chart installs — currently v3.x.
curl -sL https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v3.4.0/docs/install/iam_policy.json -o alb-iam-policy.json
ALB_POLICY_ARN=$(aws iam create-policy --policy-name AWSLoadBalancerControllerIAMPolicy \
--policy-document file://alb-iam-policy.json --query Policy.Arn --output text)
eksctl create iamserviceaccount --cluster $CLUSTER --region $REGION \
--namespace kube-system --name aws-load-balancer-controller \
--role-name $PREFIX-alb-controller --attach-policy-arn "$ALB_POLICY_ARN" --approve
helm repo add eks https://aws.github.io/eks-charts && helm repo update eks
helm upgrade --install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system \
--set clusterName=$CLUSTER --set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller --set region=$REGION --set vpcId=$VPC
kubectl -n kube-system rollout status deploy/aws-load-balancer-controller
```
If the controller image is newer than the policy you fetched, you'll see `AccessDenied`
on actions like `elasticloadbalancing:DescribeListenerAttributes`. Fix WITHOUT reinstalling:
```bash
curl -sL .../aws-load-balancer-controller/v<INSTALLED>/docs/install/iam_policy.json -o p.json
aws iam create-policy-version --policy-arn $ALB_POLICY_ARN --policy-document file://p.json --set-as-default
```
(Check version: `kubectl -n kube-system get deploy aws-load-balancer-controller -o jsonpath='{..image}'`.)
## Step 5 — helm install flyte-binary
`values-eks.yaml` (ALB HTTP-only variant). The UPPERCASE tokens (`BUCKET`, `RDS_HOST`, `DBPW`,
`IRSA_ARN`) and `region:` are placeholders — substitute your real values before installing, e.g.
`sed -i "s/BUCKET/$BUCKET/g; s/RDS_HOST/$RDS_HOST/; s/DBPW/$DBPW/; s#IRSA_ARN#$IRSA_ARN#; s/us-west-2/$REGION/g" values-eks.yaml`
(or hand-edit). Note this chart uses `metadataContainer` (no `userDataContainer`) and its run
output prefix defaults to a nonexistent `s3://flyte-data` — override `storagePrefix` to your bucket:
```yaml
fullnameOverride: flyte
flyte-core-components:
runs: { storagePrefix: "s3://BUCKET" } # under `runs`, NOT `runs.server` (else ignored)
# no image override needed: the repo chart pins its image tag to the chart version
# (only the git-main chart floats `:latest` — see Image selection below)
configuration:
database:
postgres:
host: RDS_HOST
port: 5432
dbname: flyte
username: flyte
password: "DBPW"
options: "sslmode=require"
storage:
metadataContainer: BUCKET
provider: s3
providerConfig: { s3: { region: us-west-2, authType: iam } } # set to your $REGION
inline: { executor: { defaultK8sServiceAccount: flyte } } # task pods inherit S3 via IRSA
serviceAccount:
create: true
name: flyte
annotations: { eks.amazonaws.com/role-arn: IRSA_ARN }
ingress:
create: true
host: "" # empty => rule matches any host => reach by ALB DNS name
ingressClassName: alb
httpAnnotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]'
alb.ingress.kubernetes.io/healthcheck-path: /healthz # binary serves /healthz on :8090
alb.ingress.kubernetes.io/healthcheck-port: "8090"
```
For **TLS**: add `certificate-arn`, `listen-ports: '[{"HTTP":80},{"HTTPS":443}]'`,
`ssl-redirect: "443"`, and set `ingress.host` to the cert hostname + a Route53 record.
See the TLS section below — works even when DNS lives in a different AWS account.
```bash
helm install flyte flyteorg/flyte-binary -n flyte --create-namespace -f values-eks.yaml --dry-run # check
helm install flyte flyteorg/flyte-binary -n flyte --create-namespace -f values-eks.yaml
kubectl -n flyte get pods # flyte stuck Init:0/1 => wait-for-db can't reach RDS (see gotchas)
```
**ALWAYS confirm the TaskAction CRD is present after install** — the chart ships it as a
plain template, so in a shared cluster it's easily deleted out-of-band, and the binary then
loops `Failed to watch ... taskactions.flyte.org` and every run sticks at "queued" (gotcha 8).
Make it idempotent at the end of every deploy:
```bash
kubectl --context <ctx> get crd taskactions.flyte.org >/dev/null 2>&1 \
|| kubectl --context <ctx> apply -f ./flyte-binary/templates/crds/flyte.org_taskactions.yaml # from `helm pull --untar`
kubectl --context <ctx> get crd taskactions.flyte.org -o jsonpath='{.status.conditions[?(@.type=="Established")].status}' # => True
# Pre-existing CRD blocks helm adopt? patch ownership, then (re)install:
# kubectl annotate crd taskactions.flyte.org meta.helm.sh/release-name=flyte meta.helm.sh/release-namespace=flyte --overwrite
# kubectl label crd taskactions.flyte.org app.kubernetes.io/managed-by=Helm --overwrite
```
Only `rollout restart` if you applied the CRD onto an **already-running** binary that was
missing it (the watch won't retry a resource that 404'd at boot). On a normal install the chart
creates the CRD before the pod is Ready, so the watch establishes on first boot — don't restart
reflexively, it's a wasted second rollout (+ image re-pull on a floating tag).
**Image selection.** The published repo chart (what the official docs install) **pins the
Flyte image tag to the chart version** — e.g. chart `v2.0.27` runs
`cr.flyte.org/flyteorg/flyte-binary-v2:v2.0.27` — plus console
`ghcr.io/unionai-oss/flyteconsole-v2:latest`, all with the default `pullPolicy: IfNotPresent`.
No image override is needed: the binary and the DB migrations it runs ship as a matched pair,
and upgrading is `helm repo update && helm upgrade` (a new chart brings its new pinned image).
Only the **git-main chart** (local `./charts/flyte-binary` from a clone) still defaults to a
floating `:latest`, which CI pushes on every merge to `main`. If you deploy that one, set:
```yaml
deployment:
image:
pullPolicy: Always # :latest is floating + kept current by CI; re-pull on (re)start
console:
image:
pullPolicy: Always
```
With a floating tag, `kubectl rollout restart deploy/flyte -n flyte` forces a fresh pull (the
wait-for-db init container uses a fixed `postgres` tag — leave it `IfNotPresent`). **For a timed
demo, a digest pin** (`repository@sha256:…`) + `pullPolicy: IfNotPresent` is stricter still —
the layer already on the node is reused and the build can't shift under you. The image and the
schema are a matched pair (the binary owns its migrations), so running an *older* image against
a DB a *newer* one migrated (rolled back, or switched from `:latest` to the pinned repo chart)
can hit the schema-mismatch error in gotcha 9 — a non-issue on a fresh DB.
## Step 6 — Verify
The HTTP ingress is named `<fullname>-http` — `flyte-http` with `fullnameOverride: flyte` above.
```bash
ALB=$(kubectl -n flyte get ingress flyte-http -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
curl -s -X POST "http://$ALB/flyteidl2.project.ProjectService/ListProjects" \
-H 'Content-Type: application/json' -d '{}' # => JSON listing flytesnacks (seeded on first boot)
curl -s -o /dev/null -w "%{http_code}\n" "http://$ALB/v2" # => 200 (console)
```
ALB takes ~2-3 min after the address appears to pass health checks and serve 200.
## Step 7 — Cost dashboard (SHOW THIS after a successful deploy)
Once the deploy verifies, present this breakdown so the user knows the steady-state
spend. Figures are **us-west-2 on-demand list prices at ~730 hrs/month** for the default
sizing in this skill (2× `m5.large`, `db.t3.micro`, 1 ALB); scale for your instance
types / region. Excludes data-transfer/egress, which depends on usage.
All amounts are **USD** (avoid bare `$` so GitHub doesn't render it as math).
| AWS service | Sizing | Rate | ~ USD/mo |
|---|---|---|---:|
| EKS control plane | 1 cluster | 0.10/hr | 73 |
| EC2 worker nodes | 2 × m5.large | 0.096/hr each | 140 |
| EBS (node disks) | 2 × 50 GB gp3 | 0.08/GB-mo | 8 |
| NAT gateway ⚠️ | 1 (eksctl default) | 0.045/hr + 0.045/GB | 33 + data |
| RDS PostgreSQL | db.t3.micro single-AZ | 0.018/hr | 13 |
| RDS storage | 20 GB gp3 | ~0.115/GB-mo | 2 |
| Application Load Balancer | 1 ALB | 0.0225/hr + ~1 LCU | 16 + ~6 |
| S3 | metadata + task I/O | 0.023/GB-mo + requests | <1 |
| ACM / IAM / OIDC / IRSA / Secrets | — | free | 0 |
| Route53 hosted zone (if added) | per zone | 0.50/zone-mo | 0.50 |
| **Total (default sizing)** | | | **≈ 290–310** |
Call-outs to make when showing it:
- **Nodes + control plane ≈ 70%** of the bill.
- **The NAT gateway is easy to miss** — eksctl's default VPC creates one (~33 USD/mo + data)
just so private-subnet nodes reach the internet.
- **Actual spend so far** is only ~0.40–0.60 USD/hr × hours-up, not the monthly figure.
- Levers: spot nodes (~−65% of node cost), 1 node / smaller instances, drop the NAT gateway
(public-subnet nodes, dev only), or **tear down when idle** (see Teardown) to stop the meter.
- For real numbers, offer to pull month-to-date from Cost Explorer (`aws ce get-cost-and-usage`,
needs `ce:GetCostAndUsage`).
## TLS (ACM + ALB), including cross-account DNS
**ASK the user which hostname to use** before requesting the cert or setting `ingress.host` —
do NOT default to `flyte.example.com`, and do NOT silently reuse a value left over from a
prior deploy (e.g. a `HOST=` in an old `values-eks.yaml`). The hostname drives the ACM cert,
the OIDC redirect URI, and the DNS record, so it must be the user's choice. If you find a
leftover value, surface it as a *suggestion* to confirm, not a default. Then set `HOST` below
to their answer.
The **cert and ALB must be in the Flyte account + the ALB's region**; the **DNS zone can
live in another account** — you just add two records there. No domain transfer/delegation.
TLS is a prerequisite for browser SSO (ALB `authenticate-oidc` only runs on HTTPS listeners;
IdPs reject non-`https` redirect URIs for non-localhost hosts).
```bash
HOST=flyte.example.com; ZONE=<YOUR_ROUTE53_ZONE_ID> # zone is in whichever account owns DNS
# 1. Flyte account: request a DNS-validated cert (same region as the ALB)
CERT=$(aws acm request-certificate --region $REGION --domain-name $HOST \
--validation-method DNS --query CertificateArn --output text)
aws acm describe-certificate --region $REGION --certificate-arn $CERT \
--query 'Certificate.DomainValidationOptions[0].ResourceRecord' # -> {Name,Type,Value}
# 2. DNS account: UPSERT that validation CNAME into the zone (change-resource-record-sets)
# 3. Flyte account: wait until status == ISSUED (aws acm describe-certificate ...)
# 4. Flyte account: helm upgrade with ingress.host=$HOST and these httpAnnotations:
# listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
# ssl-redirect: "443"
# certificate-arn: <CERT>
# 5. DNS account: UPSERT a CNAME $HOST -> <ALB DNS name> (subdomain => CNAME is fine; no
# cross-account alias-target dance needed)
# 6. Verify: curl https://$HOST/v2 -> 200; http -> 301; openssl s_client shows CN=$HOST
```
Two AWS credential sets may be in play (Flyte acct + DNS acct) — keep them in separate env
files and `source` the right one per command. STS/SSO session tokens expire mid-deploy;
when a call returns `ExpiredTokenException`, refresh that account's creds (kubectl/helm to
the cluster also need the Flyte account's creds, via `aws eks get-token`).
## ALB edge SSO (Okta / OIDC)
Gates the console at the load balancer via ALB `authenticate-oidc` — the binary is
unchanged. **Requires HTTPS** (see TLS above). Tradeoff: the action applies to ALL paths on
the ingress, so the browser console works (same-origin API calls carry the ALB session
cookie) but **CLI/SDK clients get 302'd** — add a higher-precedence `ingress.apiJwtIngress`
that matches `Authorization: Bearer*` and JWT-validates it at the ALB (see CLI Bearer-bypass
below).
> **The Flyte binary does NOT validate tokens.** Its server wires no auth interceptor — it
> trusts whatever reaches it (`TrustForwardedIdentityHeaders`). So auth has to be enforced at
> the edge. A Bearer-match ingress with *no* validation action just forwards the token blindly,
> leaving the API **wide open** (any `Authorization: Bearer anything` returns 200). To actually
> lock it down you need ALB-native JWT validation on that ingress (the `jwt-validation`
> annotation below) — not just a Bearer-match condition.
1. Add the redirect URI **`https://<host>/oauth2/idpresponse`** to the OIDC app (fixed ALB
callback path) — login fails without it.
2. Create the OIDC Secret in the **ingress namespace** (keys exactly `clientID`/`clientSecret`).
**Have the USER run this command themselves** — do NOT ask them to paste the client secret
into the chat. Give them the command and ask them to run it (inline with a leading `! ` or
in their own terminal) so the secret goes straight into kubectl and never reaches the assistant:
```bash
kubectl -n flyte create secret generic flyte-console-oidc \
--from-literal=clientID=<oidc-client-id> --from-literal=clientSecret=<oidc-secret>
```
3. Add these `ingress.httpAnnotations` and `helm upgrade`:
```yaml
alb.ingress.kubernetes.io/auth-type: oidc
alb.ingress.kubernetes.io/auth-scope: openid email profile # profile => given/family name in x-amzn-oidc-data
alb.ingress.kubernetes.io/auth-on-unauthenticated-request: authenticate
alb.ingress.kubernetes.io/auth-session-timeout: "604800"
alb.ingress.kubernetes.io/auth-idp-oidc: '{"issuer":"https://<idp>/oauth2/default","authorizationEndpoint":".../v1/authorize","tokenEndpoint":".../v1/token","userInfoEndpoint":".../v1/userinfo","secretName":"flyte-console-oidc"}'
```
4. **Grant the controller RBAC to read that Secret** (REQUIRED — otherwise the rule silently
stays plain `forward` and you keep getting HTTP 200 instead of 302). The controller SA
`kube-system:aws-load-balancer-controller` has no secret access in app namespaces by default:
```bash
kubectl -n flyte create role alb-oidc-secret-reader --verb=get,list,watch --resource=secrets
kubectl -n flyte create rolebinding alb-oidc-secret-reader --role=alb-oidc-secret-reader \
--serviceaccount=kube-system:aws-load-balancer-controller
```
Symptom if missing: controller logs `secrets "flyte-console-oidc" is forbidden`.
- **Okta issuer host:** use the **non-admin** org domain (`https://<org>.okta.com/oauth2/default`),
NOT the `-admin` console host — tokens carry the non-admin host as `iss`, so jwt-validation
fails if you use `-admin`. Confirm via `…/oauth2/default/.well-known/openid-configuration`.
Switching IdPs is pure config (Secret + the issuer refs + `flyteClient.clientId`); no ALB/DNS churn.
5. Verify: `curl -s -o /dev/null -w '%{http_code} %{redirect_url}' https://<host>/v2`
→ `302 https://<idp>/oauth2/default/v1/authorize?client_id=...&redirect_uri=https://<host>/oauth2/idpresponse`.
6. **IdP-side (can't be fixed from the cluster):** the user must be **assigned to the app**,
and (on Okta) the `default` auth server's **Access Policy** must permit the app + the
requested scopes (`openid email profile`). Okta error *"Bad Request — Policy evaluation
failed"* after the redirect = the access-policy rule is missing the app or restricts scopes
→ Security → API → Authorization Servers → default → Access Policies, allow the app with
"Any scopes" (or add `email`/`profile`).
### CLI Bearer-bypass (dual-auth: keep CLI/SDK working alongside console SSO)
Edge SSO alone 302s CLI clients. To let token clients through, add two more ingresses in
the SAME ALB group plus `authMetadata` so the CLI knows to fetch a token. Three ingresses,
ordered by `group.order` (lower = higher precedence), all sharing one `group.name`:
| Ingress | order | matches | auth |
|---|---|---|---|
| `wellknownIngress` | -150 | `/.well-known/*`, `AuthMetadataService` | none (discovery before token) |
| `apiJwtIngress` | -140 | `Authorization: Bearer*` (via `conditions.<fullname>-http`) | ALB `jwt-validation` vs IdP JWKS |
| http (main) | -100 | everything else | `authenticate-oidc` (cookie) |
1. Tell the binary to advertise the IdP + the PKCE CLI client — under **`runs`, NOT `runs.server`**:
```yaml
flyte-core-components:
runs:
storagePrefix: "s3://<bucket>" # also belongs under runs, not runs.server
authMetadata:
externalAuthServerBaseUrl: "https://<idp>/oauth2/default"
flyteClient: { clientId: <native-PKCE-app>, redirectUri: http://localhost:53593/callback, scopes: [openid, profile, offline_access] }
```
2. Add `group.name: <group>` + `group.order: "-100"` to the main `httpAnnotations`. Then add
`ingress.apiJwtIngress` (enabled, order -140) and `ingress.wellknownIngress` (enabled, order
-150, no auth). All three need cert-arn + listen-ports + ssl-redirect. The apiJwtIngress
carries **two** annotations that together do the lock-down — the Bearer-match condition AND
ALB-native JWT validation (the `conditions.*` key targets the rendered backend service name,
`<fullname>-http`, e.g. `flyte-http`):
```yaml
ingress:
apiJwtIngress:
enabled: true
annotations:
alb.ingress.kubernetes.io/group.name: <group>
alb.ingress.kubernetes.io/group.order: "-140"
alb.ingress.kubernetes.io/conditions.<fullname>-http: '[{"field":"http-header","httpHeaderConfig":{"httpHeaderName":"Authorization","values":["Bearer*"]}}]'
# ALB checks signature + iss/exp against the IdP JWKS; bad/expired token -> 401 at the edge
alb.ingress.kubernetes.io/jwt-validation: '{"jwksEndpoint":"https://<idp>/oauth2/default/v1/keys","issuer":"https://<idp>/oauth2/default"}'
# plus the shared cert-arn / listen-ports / ssl-redirect / target-type / healthcheck-* annotations
```
**`jwt-validation` needs a recent controller** — ALB native JWT verification shipped Nov 2025,
exposed by aws-load-balancer-controller as `alb.ingress.kubernetes.io/jwt-validation`. On an
older controller the annotation is silently ignored and the API stays open — confirm the
controller is new enough (check its release date, not just a high-looking version number) and
verify the 401 below. The `conditions.*` match alone (no `jwt-validation`) does NOT validate
anything; it only routes Bearer requests past the cookie flow.
3. `helm upgrade`. **Adding group.name recreates the ALB under a new name** (`k8s-<group>-*`)
and deletes the old standalone one — **re-point the DNS CNAME to the new ALB DNS name.**
4. Verify (use `curl --connect-to host:443:<alb>:443` before DNS propagates):
- `POST .../AuthMetadataService/GetOAuth2Metadata` (with `Content-Type: application/json`) → 200
- API `+ Authorization: Bearer fake` → **401, not 302** (matched the JWT ingress, validated)
- `/v2` and API without Bearer → 302 (cookie path)
Gotchas: (a) `authMetadata`/`storagePrefix` go under `runs`, not `runs.server` — misplaced,
they're silently ignored and `GetOAuth2Metadata` returns `unimplemented`. (b) config-only helm
changes may not roll the pod — `kubectl rollout restart deploy/flyte` to be sure. (c) the
`conditions.*` key must match the rendered backend service name (`<fullname>-http`).
## App serving (optional — Knative + Kourier)
Flyte v2 can host long-running **apps** (deployed via the SDK), each published at
`{name}-{project}-{domain}.<base-domain>`. It's **off by default**: the binary always exposes
`AppService`, but with no controller behind it the console's Apps tab and any
`flyteidl2.app.AppService/List` call return **`{"code":"unimplemented","message":"404 Not Found"}`**
until you enable it. Apps run as **Knative Services**, so this needs Knative Serving + a Knative
networking layer (Kourier) installed first — your cloud's ALB controller can't be Knative's
networking layer. Skip this section unless the user wants apps. Official doc:
https://www.union.ai/docs/v2/flyte/oss-deployment/app-serving/.
**1. Install Knative Serving + Kourier.** Pick a Knative release that supports your cluster's
k8s version — Knative only supports the most recent k8s minors, so the upstream doc's pinned
version is often too old (e.g. on k8s 1.34, v1.17 is rejected; v1.22 works). Check the latest
that matches, and use the **same version** for serving and net-kourier:
```bash
KV=knative-v1.22.1 # must support your k8s version; serving + net-kourier must match
kubectl --context <ctx> apply -f https://github.com/knative/serving/releases/download/$KV/serving-crds.yaml
kubectl --context <ctx> apply -f https://github.com/knative/serving/releases/download/$KV/serving-core.yaml
kubectl --context <ctx> apply -f https://github.com/knative-extensions/net-kourier/releases/download/$KV/kourier.yaml
kubectl --context <ctx> patch configmap/config-network -n knative-serving --type merge \
-p '{"data":{"ingress-class":"kourier.ingress.networking.knative.dev"}}'
kubectl --context <ctx> wait --for=condition=Available deploy --all -n knative-serving --timeout=180s
kubectl --context <ctx> wait --for=condition=Available deploy --all -n kourier-system --timeout=180s
```
**2. Configure the apps domain — single-label so one wildcard cert covers every app.** Set
`config-domain` to your base domain and drop the namespace from the hostname template (Knative's
default `{{.Name}}.{{.Namespace}}.{{.Domain}}` is two labels, which `*.<base-domain>` can't match):
```bash
kubectl --context <ctx> patch configmap/config-domain -n knative-serving --type merge \
-p '{"data":{"<base-domain>":""}}' # e.g. flyte-v2.example.com
kubectl --context <ctx> patch configmap/config-network -n knative-serving --type merge \
-p '{"data":{"domain-template":"{{.Name}}.{{.Domain}}"}}'
```
**3. Expose Kourier behind the existing ALB** (reuse the same `group.name` so apps share the
Flyte load balancer — no second LB, no extra cost). Switch the Kourier Service to `ClusterIP`
and add an Ingress in `kourier-system` joined to that group, with a **wildcard cert** and the
`*.<base-domain>` host. Match the group-level annotations (scheme / target-type / listen-ports /
ssl-redirect) to the Flyte ingresses or the controller errors on conflicting group config:
```bash
kubectl --context <ctx> patch svc kourier -n kourier-system --type merge -p '{"spec":{"type":"ClusterIP"}}'
```
```yaml
# kourier-alb-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: kourier-alb
namespace: kourier-system
annotations:
alb.ingress.kubernetes.io/group.name: flyte # SAME group as the Flyte ingresses
alb.ingress.kubernetes.io/group.order: "-50"
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
alb.ingress.kubernetes.io/ssl-redirect: "443"
alb.ingress.kubernetes.io/certificate-arn: <WILDCARD_CERT_ARN>
alb.ingress.kubernetes.io/healthcheck-path: /
alb.ingress.kubernetes.io/success-codes: "200,404" # Kourier 404s an unmatched host while healthy
spec:
ingressClassName: alb
rules:
- host: "*.<base-domain>"
http:
paths:
- { path: /, pathType: Prefix, backend: { service: { name: kourier, port: { number: 80 } } } }
```
- **Wildcard cert:** request `*.<base-domain>` in ACM. ACM derives the DNS-validation CNAME from
the base name, so if you already validated the console's `<base-domain>` cert, the wildcard
reuses the **same** validation record and auto-issues — often no new DNS needed.
- **Wildcard DNS:** add `*.<base-domain>` → the ALB DNS name (cross-account zones: a CNAME in the
DNS account, same as the console record).
- (Alternative: leave the Kourier Service as `LoadBalancer` and point `*.<base-domain>` at that NLB
— simpler, but a second load balancer + you manage its TLS separately.)
**3b. Require authentication for apps (optional but recommended).** By default any app URL is
**public** — anyone who can reach the ALB opens it. To gate every app behind the same OIDC login
as the console, add `authenticate-oidc` to the **`kourier-alb`** Ingress (same mechanism as the
console's edge SSO). Three parts, all required:
- **Annotations** on the `kourier-alb` Ingress (alongside the step-3 annotations):
```yaml
alb.ingress.kubernetes.io/auth-type: oidc
alb.ingress.kubernetes.io/auth-on-unauthenticated-request: authenticate
alb.ingress.kubernetes.io/auth-scope: openid email profile
alb.ingress.kubernetes.io/auth-session-timeout: "604800"
alb.ingress.kubernetes.io/auth-idp-oidc: '{"issuer":"https://<idp>/oauth2/default","authorizationEndpoint":"https://<idp>/oauth2/default/v1/authorize","tokenEndpoint":"https://<idp>/oauth2/default/v1/token","userInfoEndpoint":"https://<idp>/oauth2/default/v1/userinfo","secretName":"flyte-console-oidc"}'
```
- **The OIDC Secret must exist in `kourier-system`** (the controller reads it from the Ingress's
own namespace). Reuse the console's client by copying the existing Secret over:
```bash
kubectl --context <ctx> get secret flyte-console-oidc -n flyte -o json \
| jq '.metadata={name:"flyte-console-oidc",namespace:"kourier-system"}' \
| kubectl --context <ctx> apply -f -
```
- **RBAC for the controller to read it in `kourier-system`** — its Secret access is per-namespace,
so without a Role here the Ingress fails with `secrets "…" is forbidden`, which **stalls
reconciliation of the whole ALB group** (not just this Ingress — it can take the console down):
```bash
kubectl --context <ctx> -n kourier-system create role alb-oidc-secret-reader \
--verb=get,list,watch --resource=secrets
kubectl --context <ctx> -n kourier-system create rolebinding alb-oidc-secret-reader \
--role=alb-oidc-secret-reader --serviceaccount=kube-system:aws-load-balancer-controller
```
- **IdP redirect URI:** ALB's callback is `https://<app-host>/oauth2/idpresponse` and every app
has a different hostname, so register the **wildcard** `https://*.<base-domain>/oauth2/idpresponse`
as a sign-in redirect URI on the OIDC app (your IdP must allow wildcard redirect URIs). Without
it, login dead-ends after the redirect with a `redirect_uri` error.
Verify: `curl -s -o /dev/null -w '%{http_code}'` an app host → **302** to the IdP (was 200/404).
Note this gates apps with the **cookie** flow (browser) — app-to-app/API calls would need the same
Bearer-bypass treatment as the main API if you want programmatic access.
**4. Enable the controller in Flyte values** (under `configuration.inline`), then upgrade.
`baseDomain` MUST equal the `config-domain` from step 2 so advertised URLs match what Knative serves:
```yaml
configuration:
inline:
internalApps:
enabled: true
baseDomain: <base-domain>
scheme: https
ingressAppsPort: 0 # apps sit behind the ALB on 443; omit the port
defaultServiceAccount: flyte # app pods run under the IRSA'd SA (S3 access)
```
```bash
helm upgrade flyte flyteorg/flyte-binary -n flyte -f values-eks.yaml --kube-context <ctx>
kubectl --context <ctx> -n flyte rollout restart deploy/flyte # config-only change may not roll the pod
```
The chart auto-grants the `serving.knative.dev` RBAC when `internalApps.enabled` (flyte#7557).
**5. Verify.**
```bash
kubectl --context <ctx> auth can-i create services.serving.knative.dev \
--as=system:serviceaccount:flyte:flyte -n flyte # => yes
# AppService in-cluster (bypasses the ALB auth gate) — 200 + {} (NOT 404/unimplemented):
kubectl --context <ctx> -n flyte run c --rm -i --image=curlimages/curl:8.10.1 --restart=Never -- \
curl -s -o /dev/null -w '%{http_code}\n' -X POST \
http://flyte-http.flyte:8090/flyteidl2.app.AppService/List -H 'Content-Type: application/json' -d '{}'
# App path via ALB (before DNS, use --connect-to): wildcard TLS served + 404 from Kourier = healthy, no app yet:
curl -s -o /dev/null -w '%{http_code}\n' --connect-to "noapp.<base-domain>:443:<alb>:443" https://noapp.<base-domain>/
```
The console's Apps tab now loads; deploy an app with the SDK and open
`https://<name>-<project>-<domain>.<base-domain>`.
**Gotchas:** (a) Knative version too new for your k8s → `kubectl apply` rejects the manifests;
install an older Knative (serving + net-kourier matched). (b) Two-label app hostname → wildcard
TLS error; confirm the single-label `domain-template`. (c) `baseDomain` ≠ `config-domain` → URLs
Flyte advertises don't match what Knative serves. (d) **Apps are unauthenticated at the edge** —
the Kourier ingress carries no OIDC/JWT, so app URLs are public once DNS resolves (the console
Apps *tab* is still SSO-gated) — gate them with step 3b. (e) `List` still 404s after enabling →
the binary didn't roll; `rollout restart`. (f) Enabling app auth without the `kourier-system`
Secret RBAC (step 3b) → `secrets "…" is forbidden` stalls the **whole** ALB group, which can take
the console down too — add the Role/RoleBinding before (or with) the auth annotations.
## Optional `configuration.inline` tuning
Anything under `configuration.inline` is merged into the rendered Flyte config — it's how you
set options the top-level values don't expose. All of the below go in `values-eks.yaml`; apply
with `helm upgrade flyte ... -f values-eks.yaml` (config-only changes may not roll the pod —
`kubectl rollout restart deploy/flyte -n flyte` if it doesn't pick them up).
**Default task resources.** CPU/memory requests for task pods that don't set their own:
```yaml
configuration:
inline:
plugins:
k8s:
default-cpus: 500m
default-memory: 1Gi
```
**Default task scheduling.** Tolerations / affinity / node selectors / injected env on every
task pod (same `plugins.k8s` block — `default-env-vars` is also where gotcha 7's callback vars
go on older charts):
```yaml
configuration:
inline:
plugins:
k8s:
default-tolerations:
- { key: flyte.org/node-role, operator: Equal, value: worker, effect: NoSchedule }
default-affinity: {} # a standard core/v1 Affinity
default-env-vars:
- MY_ENV_VAR: value # injected into every task pod
```
**OpenTelemetry.** Off by default (`otel.type: noop`). Point it at an OTLP collector — prefer
`otlpgrpc` (the `otlphttp` metric exporter reuses the trace endpoint path):
```yaml
configuration:
inline:
otel:
type: otlpgrpc # noop | file | jaeger | otlpgrpc | otlphttp
otlpgrpc: { endpoint: http://otel-collector.flyte.svc.cluster.local:4317 }
sampler: { parentSampler: traceid, traceIdRatio: 0.01 } # keep 1% of traces in prod
```
**DB password (and S3 keys) from a Secret.** Setting `configuration.database.postgres.password`
already writes it into a mounted k8s Secret (not the plaintext ConfigMap); same for S3 access
keys when `authType: accesskey`. To keep the password out of the values file entirely, leave
`password` empty and either reference an existing Secret with
`configuration.extraInlineSecretRefs`, or mount it as a file and point
`configuration.database.postgres.passwordPath` at it (`password` and `passwordPath` are mutually
exclusive). This is the better choice than the plaintext `password:` shown in Step 5 when the
values file is committed or shared.
## Stable ALB across redeploys (anchor ingress)
By default the ALB is owned by Flyte's ingresses, so `helm uninstall` deletes it and the next
install mints a **new ALB with a new DNS name** — forcing a DNS re-point every cycle. To keep
one stable endpoint, exploit the controller's rule that it keeps exactly **one ALB per
`group.name` as long as ≥1 ingress in that group exists**: add a permanent "anchor" ingress in
the group, applied **out-of-band (NOT in the helm release)**, so the ALB survives uninstall.
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: flyte-alb-anchor
namespace: flyte
annotations:
alb.ingress.kubernetes.io/group.name: flyte # SAME group as the flyte ingresses
alb.ingress.kubernetes.io/group.order: "100" # lowest precedence; never shadows flyte rules
alb.ingress.kubernetes.io/scheme: internet-facing # group-level annotations MUST match the
alb.ingress.kubernetes.io/target-type: ip # flyte ingress group, else the controller
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]' # errors on conflicting config
alb.ingress.kubernetes.io/certificate-arn: <CERT_ARN>
# fixed-response backend => the anchor needs NO real service, so it stands alone when flyte is uninstalled:
alb.ingress.kubernetes.io/actions.anchor-ok: '{"type":"fixed-response","fixedResponseConfig":{"contentType":"text/plain","statusCode":"200","messageBody":"flyte-alb-anchor"}}'
spec:
ingressClassName: alb
rules:
- http:
paths:
- { path: /__alb_anchor, pathType: ImplementationSpecific, backend: { service: { name: anchor-ok, port: { name: use-annotation } } } }
```
```bash
kubectl --context <ctx> apply -f alb-anchor.yaml # provisions the ALB once
# point DNS at THIS ALB's name one time; it never changes again:
kubectl --context <ctx> -n flyte get ingress flyte-alb-anchor -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
```
Now `helm uninstall flyte` removes Flyte's rules but the anchor keeps the ALB (same DNS name)
alive; `helm install` re-adds Flyte's listener rules (including the `authenticate-oidc` SSO
rule) onto the surviving ALB. Verify the anchor still answers between deploys:
`curl http://<alb>/__alb_anchor` → `200 flyte-alb-anchor`. (To intentionally delete the ALB,
remove the anchor too.) Keeps the annotation-driven SSO config — the alternative, a fully
pre-provisioned BYO ALB via the controller's `TargetGroupBinding` CRD + `ingress.create:
false`, is more stable still but makes you hand-manage every listener/SSO rule yourself.
## Pruning run data from the DB
To wipe run history without reinstalling, prune the DB directly. The v2 run data lives in
just two tables (Postgres `flyte` DB): **`actions`** (one row per run/action) and
**`action_events`** (per-attempt events). They're linked by `(project, domain, run_name,
name)` — there's no FK, so delete events first, then actions. **Keep** `projects` (seeded
`flytesnacks`), `schema_migrations` (migration state), and `task_specs` (registered tasks).
RDS is private (no public access), so run psql from an **ephemeral in-cluster pod** rather
than your laptop. Prune only **finished** runs by keying on `ended_at IS NOT NULL` — that
skips anything still in-flight (an open run has `ended_at` null):
```bash
DBHOST=<rds-endpoint>; DBPW=<db-password> # from your values-eks.yaml
kubectl --context <ctx> run pgcli --rm -i --restart=Never -n flyte \
--image=postgres:16 --env PGPASSWORD="$DBPW" --command -- \
psql "host=$DBHOST user=flyte dbname=flyte sslmode=require" -P pager=off -v ON_ERROR_STOP=1 \
-c "begin;
delete from action_events ae using actions a
where ae.project=a.project and ae.domain=a.domain
and ae.run_name=a.run_name and ae.name=a.name and a.ended_at is not null;
delete from actions where ended_at is not null;
commit;"
```
Inspect first with `select relname,n_live_tup from pg_stat_user_tables order by 2 desc;`.
Note a run **stuck "queued"** (e.g. from the missing-CRD gotcha 8) has `ended_at` null, so
this leaves it untouched — delete those explicitly by `run_name` once you've confirmed no
task pod / TaskAction CR backs them. To wipe **everything** instead, `truncate actions,
action_events;` (projects/migrations survive). To reset the whole DB, see Teardown +
reinstall, or `drop database flyte; create database flyte;` and rollout-restart the binary.
## Teardown
```bash
helm uninstall flyte -n flyte # deletes the ingress => controller removes the ALB
# helm uninstall leaves the run/task pods behind (the controller created them, not Helm) —
# delete them explicitly so the namespace is clean for a redeploy:
kubectl --context <ctx> -n flyte delete pods --all
helm uninstall aws-load-balancer-controller -n kube-system
aws rds delete-db-instance --region $REGION --db-instance-identifier $PREFIX-db --skip-final-snapshot --delete-automated-backups
aws rds delete-db-subnet-group --region $REGION --db-subnet-group-name $PREFIX-db-subnets
aws s3 rb s3://$PREFIX-data-$ACCT --force
eksctl delete cluster -f cluster.yaml # tears down VPC, nodegroup, OIDC, IRSA stacks
# Delete the standalone IAM policies (detach first if needed):
aws iam delete-policy --policy-arn arn:aws:iam::$ACCT:policy/$PREFIX-s3-access
aws iam delete-policy --policy-arn arn:aws:iam::$ACCT:policy/AWSLoadBalancerControllerIAMPolicy
```
## Gotchas (each one bit during a real run)
1. **eksctl too old → "unsupported Kubernetes version".** eksctl 0.175 only offers up to
1.29, but EKS has dropped 1.29 from standard support → CFN `ControlPlane` fails ~30s in
and rolls back. Use eksctl ≥ 0.227 (defaults to a current version); pin a supported one
(1.33 worked). After a failed create, delete the `ROLLBACK_COMPLETE` stack before retrying.
2. **RDS unreachable: wrong source SG.** The pod stays `Init:0/1` (`wait-for-db ... no
response`). EKS managed-nodegroup nodes run with the **EKS-managed cluster SG**
(`eks-cluster-sg-<cluster>-*`), NOT `ClusterSharedNodeSecurityGroup`. Pod egress (VPC CNI
secondary IPs on the primary ENI) uses the node-ENI SG. Authorize 5432 on the RDS SG from
the actual node SG (`describe-instances ... SecurityGroups`), not the shared one. Init
container retries on its own once the rule lands.
3. **ALB controller IAM lag.** The eks chart installs the latest controller (v3.x), which
needs newer IAM actions (e.g. `DescribeListenerAttributes`) than older policy JSON.
Match `iam_policy.json` to the installed controller version (create-policy-version
--set-as-default; no reinstall needed).
4. **Default storagePrefix is fake.** `flyte-core-components.runs.storagePrefix` (under `runs`,
NOT `runs.server`) defaults to `s3://flyte-data` — override to your real bucket or run I/O
fails. Misplaced under `runs.server` it's silently ignored.
5. **ALB by DNS name:** leave `ingress.host: ""` so the rule matches any host; the binary
serves `/healthz` on `:8090` for the ALB health check. Add ACM cert + Route53 for TLS.
6. Postgres default major from RDS is fine (chart needs ≥12).
7. **Task pods loop/recreate every ~75s — missing control-plane callback env vars.** A run's
task pod (image `ghcr.io/flyteorg/flyte:py3.x-vX`) calls back to the backend to enqueue
child actions / watch state. Without config it uses the **devbox default
`host.docker.internal:8090`** → `dns error: Name or service not known` → retries exhaust →
controller recreates the pod, forever. Recent `flyte-binary` chart versions inject these by
default; on older charts add them via `configuration.inline.plugins.k8s.default-env-vars`:
```yaml
configuration:
inline:
plugins:
k8s:
default-env-vars:
- _U_EP_OVERRIDE: flyte-http.flyte:8090 # in-cluster HTTP svc = <fullname>-http.<ns>:8090
- _U_INSECURE: "true" # svc is plain HTTP on :8090; without this the
# SDK uses https:// → "received corrupt message
# of type InvalidContentType"
- _U_USE_ACTIONS: "1" # enable the QueueService/actions path
```
Verify a task pod: `kubectl -n flyte get pod <run>-a0-0 -o jsonpath='{..env[*].name}'` shows
`_U_EP_OVERRIDE`, and its logs no longer mention `host.docker.internal` or `InvalidContentType`.
8. **Runs stuck "queued" — missing TaskAction CRD.** The chart ships `taskactions.flyte.org`
under `templates/crds/` (NOT Helm's delete-protected `crds/` dir), so it's an ordinary,
release-owned template. Two consequences bite:
- `helm uninstall` DELETES it (and all TaskAction CRs); a later `helm install` doesn't
reliably re-establish it, and Helm won't recreate it while the release sits at
`deployed` even though `helm get manifest` still lists it.
- It's **cluster-scoped but owned by a namespaced release** — so in a **shared cluster**,
uninstalling *any* Flyte release (or a stray `kubectl delete crd`) wipes it for
everyone, and it can vanish *after* a successful install with the binary still running.
Symptom, two variants by binary version: older builds keep *running* and log `Failed to
watch ... could not find the requested resource (get taskactions.flyte.org)` every few
seconds (runs sit at "queued"); the **current `flyte-binary-v2` hard-fails at startup** —
`Error: setup failed: actions: failed to start TaskAction watcher: ... no matches for kind
"TaskAction" in version "flyte.org/v1"`, exit 1 → **CrashLoopBackOff** (the console still
serves and the ALB still 302s, so check the *binary* pod, not just the URL). Same root cause,
same fix:
```bash
kubectl --context <ctx> apply -f ./flyte-binary/templates/crds/flyte.org_taskactions.yaml # from `helm pull --untar`
kubectl --context <ctx> -n flyte rollout restart deploy/flyte # re-establish the watch
```
A normal `helm uninstall` → `helm install` cycle mostly self-heals: uninstall deletes the CRD,
install recreates it as a template. **But CRD registration and the pod start race.** If the
CRD wins, the binary boots clean on the first try (verified once: rollout Ready ~41s, single
rollout). If the pod wins, the current binary **crashloops** until the CRD is discoverable,
then comes up after a restart or two (~30–60s; `--wait` rides it out) — also verified in the
same cluster minutes later, so treat the crashloop as expected, not a failure. Either way
**verify the binary pod is `1/1 Running` and the CRD is `Established` after every deploy**
(see Step 5) — a green `helm install` and a `302` from the URL do NOT prove the binary is up
(the ALB 302s and the console serves even while the binary crashloops).
**Truly protecting it from `helm uninstall` requires removing it from the release manifest**
— stripping the live CRD's Helm ownership *labels* does NOT work, because uninstall deletes
by manifest membership, not by label (observed: labels stripped, uninstall still deleted it).
The ownership strip only avoids the *install-time* adopt conflict (Step 5). To survive
uninstall, manage the CRD entirely out-of-band: delete it from the chart's `templates/crds/`
(so no release ever lists it) and `kubectl apply` it yourself once. Otherwise just rely on the
self-healing install above — simpler, and fine for demos/redeploys.
9. **Listing runs/tasks fails: `missing destination name <col> in *[]*models.Action`.** The
install is green, the console loads, but opening a project's runs (or any
`RunService/ListActions` / `ListRuns` call) returns
`{"code":"internal","message":"failed to list actions: missing destination name created_by in *[]*models.Action"}`
(the column varies — `created_by`, `executed_by`, …). **Root cause: the binary image and the DB
schema disagree** — the `actions` table has a column the running `models.Action` struct can't
scan, i.e. the schema was migrated by a *different* image than the one running. A deploy on a
fresh DB won't hit this (any image — the pinned repo-chart tag or `:latest` — migrates its own
schema). It shows up when you **reuse a DB that a newer/feature-branch image migrated** and then
run an older image against it (e.g. rolled back, pinned a digest, or switched from a `:latest`
deploy to the older pinned repo-chart image). Diagnose, then make the schema match the image —
on a fresh/disposable DB just let the running image re-migrate from scratch:
```bash
# see the extra columns the DB has, and check there's no run data worth keeping:
kubectl --context <ctx> -n flyte run pg --rm -i --restart=Never --image=postgres:16 \
--env PGPASSWORD=<pw> --command -- psql "host=<rds> user=flyte dbname=flyte sslmode=require" -A -t \
-c "select column_name from information_schema.columns where table_name='actions' and column_name like '%_by%';" \
-c "select count(*) from actions;"
# if count is 0 (or disposable), reset the schema and let the image re-migrate on boot:
kubectl --context <ctx> -n flyte scale deploy/flyte --replicas=0
kubectl --context <ctx> -n flyte run pg --rm -i --restart=Never --image=postgres:16 \
--env PGPASSWORD=<pw> --command -- psql "host=<rds> user=flyte dbname=flyte sslmode=require" \
-c "DROP SCHEMA public CASCADE; CREATE SCHEMA public; GRANT ALL ON SCHEMA public TO flyte; GRANT ALL ON SCHEMA public TO public;"
kubectl --context <ctx> -n flyte scale deploy/flyte --replicas=1 # re-migrates clean
```
Confirm in-cluster (bypasses the ALB JWT gate):
`kubectl -n flyte run c --rm -i --image=curlimages/curl --restart=Never -- curl -s -XPOST
http://flyte-http.flyte:8090/flyteidl2.workflow.RunService/ListActions -H 'Content-Type: application/json'
-d '{"project_id":{"domain":"development","name":"flytesnacks"}}'` → `{}` (not the error).
A green `helm install` and a `302` from `/v2` do NOT prove run-listing works (the binary serves
the console and auth-redirects even when this query is broken), so exercise `ListActions` after a
deploy that pins/reuses an image or DB.
flyte-migrate17.9 KB
---
name: flyte-migrate
description: Entry-point orchestrator for porting Flyte 1 (flytekit) code to Flyte 2 (flyte). Explains the v1 to v2 shift, the terminology mapping, a recommended migration strategy, hybrid v1/v2 pipelines, and routes to sibling migration skills. Use when the user wants to migrate, port, or upgrade Flyte 1 (flytekit) code to Flyte 2. Trigger words are migrate, flytekit, v1 to v2, port, upgrade, convert workflow.
---
# Flyte 1 to 2 Migration Skill
This is the entry point for migrating a Flyte 1 (`flytekit`) codebase to Flyte 2 (`flyte`). Flyte 2 is a fundamental shift: there is no `@workflow` decorator, everything is a `@env.task`, orchestration runs as real Python at runtime, and parallelism is expressed with `asyncio`. This skill explains the overall shift, gives a recommended migration strategy, covers hybrid v1/v2 pipelines during the transition, and routes to the sibling skills that handle each theme in depth.
## Grounding References
| Resource | URL |
|---|---|
| Migration guide | https://www.union.ai/docs/v2/flyte/user-guide/migration/flyte-2/ |
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via `flyte-mcp` server |
## The overall v1 to v2 shift
Two conceptual shifts motivate almost every change — **pure Python execution** and the **asynchronous model** — after which most migrations come down to a couple of mechanical moves.
- **`flytekit` (package) becomes `flyte`.** Imports change from `import flytekit` to `import flyte`.
- **`pyflyte` (CLI) becomes `flyte`.** The command-line tool was renamed.
- **Everything is a task.** In Flyte 1, `@workflow` functions were constrained to a DSL subset of Python that compiled to a static DAG. In Flyte 2 there is **no `@workflow` decorator**: everything is a `@env.task`, and a "workflow" is simply a task that calls other tasks. Loops, conditionals, and `try`/`except` work anywhere.
- **Async is the parallelism model.** Flyte 2 is built on `asyncio`, with the Flyte orchestrator acting as the event loop, scheduling awaited tasks across distributed infrastructure. `await` signals where a task can be scheduled in parallel, and `asyncio.gather` tells the orchestrator that a set of tasks are independent.
### Simplified API mapping
| Use case | Flyte 1 | Flyte 2 |
| --- | --- | --- |
| Environment management | `N/A` | `TaskEnvironment` |
| Perform basic computation | `@task` | `@env.task` |
| Combine tasks into a workflow | `@workflow` | `@env.task` |
| Create dynamic workflows | `@dynamic` | `@env.task` |
| Fanout parallelism | `flytekit.map` | Python `for` loop with `asyncio.gather` |
| Conditional execution | `flytekit.conditional` | Python `if-elif-else` |
| Catching workflow failures | `@workflow(on_failure=...)` | Python `try-except` |
## Terminology and concept mapping
Several Flyte 1 concepts were renamed or reshaped in Flyte 2. The table below maps the ones you'll meet most often.
| Flyte 1 | Flyte 2 | Notes |
|---|---|---|
| `flytekit` (package) | `flyte` (package) | The Python SDK was renamed; imports change from `import flytekit` to `import flyte`. |
| `pyflyte` (CLI) | `flyte` (CLI) | The command-line tool was renamed. |
| `@task` / `@workflow` / `@dynamic` | `@env.task` | A single task decorator off a `flyte.TaskEnvironment`. Workflows and dynamic tasks are no longer distinct constructs: everything is a task, and orchestration is plain Python. |
| `map_task()` | `flyte.map()` | Plus `asyncio.gather()` for async fan-out. |
| `conditional()` | native `if` / `elif` / `else` | Branching is now ordinary Python control flow, not a DSL. |
| `ImageSpec` | `flyte.Image` | Container image definition. |
| `current_context()` | `flyte.ctx()` | Runtime context access. |
| `FlyteFile` / `FlyteDirectory` | `flyte.io.File` / `flyte.io.Dir` | Offloaded file and directory references. |
| `StructuredDataset` | `flyte.io.DataFrame` | Offloaded tabular data. |
| `LaunchPlan` | `flyte.Trigger` | Scheduling and parameterized entry points. |
| `CronSchedule` | `flyte.Cron` | Cron-based scheduling, used with a `flyte.Trigger`. |
| Decks (`enable_deck=True`) | Reports (`report=True`) | Custom HTML rendered in the UI during/after a run. |
## The two mechanical changes behind (almost) every migration
Most of a migration comes down to two moves.
### 1. Move task configuration into a `TaskEnvironment`
Instead of configuring the image, resources, and caching on each task decorator, configure them once on a `flyte.TaskEnvironment` and share it across tasks:
```python
env = flyte.TaskEnvironment(
name="training",
image=flyte.Image.from_debian_base().with_pip_packages("scikit-learn", "pandas"),
resources=flyte.Resources(cpu="2", memory="4Gi"),
cache="auto",
)
```
### 2. Replace `@task` / `@workflow` / `@dynamic` with `@env.task`
Every decorated function becomes an `@env.task`. There is no separate workflow or dynamic construct: a "workflow" is simply a task that calls other tasks, and orchestration is plain Python. The `env` in `@env.task` is just the variable you assigned your `TaskEnvironment` to — name it whatever you like.
## Package imports
The package is renamed from `flytekit` to `flyte`, and the workflow/dynamic/map_task imports disappear.
### Flyte 1
```python
import flytekit
from flytekit import task, workflow, dynamic, map_task
from flytekit import ImageSpec, Resources, Secret
from flytekit import current_context, LaunchPlan, CronSchedule
```
### Flyte 2
```python
import flyte
from flyte import TaskEnvironment, Resources, Secret
from flyte import Image, Trigger, Cron
```
## Before and after: pure Python execution
### Flyte 1
```python
import flytekit
image = flytekit.ImageSpec(
name="hello-world-image",
packages=["requests"],
)
@flytekit.task(container_image=image)
def mean(data: list[float]) -> float:
return sum(list) / len(list)
@flytekit.workflow
def main(data: list[float]) -> float:
output = mean(data)
# ❌ performing trivial operations in a workflow is not allowed
# output = output / 100
# ❌ if/else is not allowed
# if output < 0:
# raise ValueError("Output cannot be negative")
return output
```
### Flyte 2
```python
import flyte
env = flyte.TaskEnvironment(
"hello_world",
image=flyte.Image.from_debian_base().with_pip_packages("requests"),
)
@env.task
def mean(data: list[float]) -> float:
return sum(data) / len(data)
@env.task
def main(data: list[float]) -> float:
output = mean(data)
# ✅ performing trivial operations in a workflow is allowed
output = output / 100
# ✅ if/else is allowed
if output < 0:
raise ValueError("Output cannot be negative")
return output
```
## Quick reference: minimal Flyte 2 module
```python
import asyncio
import flyte
# 1. Define an image
image = (
flyte.Image.from_debian_base(python_version=(3, 11))
.with_pip_packages("pandas", "numpy")
)
# 2. Create a TaskEnvironment
env = flyte.TaskEnvironment(
name="my_env",
image=image,
resources=flyte.Resources(cpu="1", memory="2Gi"),
)
# 3. Define tasks
@env.task
async def process(x: int) -> int:
return x * 2
# 4. Define the entrypoint task
@env.task
async def main(items: list[int]) -> list[int]:
results = await asyncio.gather(*[process(x) for x in items])
return list(results)
# 5. Run it
if __name__ == "__main__":
flyte.init_from_config()
run = flyte.run(main, items=[1, 2, 3, 4, 5])
print(run.url)
run.wait()
```
```bash
# CLI
flyte run my_module.py main --items '[1,2,3,4,5]' # remote (default)
flyte run --local my_module.py main --items '[1,2,3,4,5]'
flyte deploy my_module.py my_env
```
## Recommended migration strategy
Migrations rarely happen all at once. Work incrementally and lean on hybrid pipelines while the transition is in progress.
1. **Assess the codebase.** Inventory every `@task`, `@workflow`, `@dynamic`, and `map_task`; the images (`ImageSpec`), resources, and secrets; the control-flow constructs (`conditional`, `on_failure`, `>>`); the data types (`FlyteFile`, `FlyteDirectory`, `StructuredDataset`); and any schedules (`LaunchPlan`, `CronSchedule`).
2. **Establish the `TaskEnvironment`(s).** Group tasks by their image/resource/cache needs and define a `flyte.TaskEnvironment` for each group. This is mechanical change #1 and unblocks everything else.
3. **Port leaf tasks first, then orchestration.** Convert atomic compute tasks (`@task` → `@env.task`), then rebuild the `@workflow`/`@dynamic` orchestration as plain-Python driver tasks that call them.
4. **Migrate control flow and I/O.** Replace `conditional()` with `if`/`elif`/`else`, `on_failure` with `try`/`except`, `map_task` with `flyte.map` / `asyncio.gather`, and the `FlyteFile`/`FlyteDirectory`/`StructuredDataset` types with their `flyte.io` equivalents.
5. **Update config, CLI, and schedules.** Swap `pyflyte` for `flyte`, migrate config files, and convert `LaunchPlan`/`CronSchedule` to `flyte.Trigger`/`flyte.Cron`.
6. **Run hybrid during the transition.** Keep unported v1 workflows callable via bridge tasks (see below) until every piece is on v2.
### Sibling skills to route to
Migrate by theme. Start with tasks and workflows, then jump to whatever the workload needs:
- **`flyte-migrate-tasks-workflows`** — the structural shift: `@task`/`@workflow` → `@env.task`, sequential ordering, nested "subworkflows", and the `@task` → `TaskEnvironment` parameter mapping.
- **`flyte-migrate-config`** — moving image/resources/cache to the `TaskEnvironment`, GPUs, secrets, caching, scheduling with triggers, and the `pyflyte` → `flyte` command/config-file changes.
- **`flyte-migrate-control-flow`** — `conditional()` and `@dynamic` become plain Python `if`/loops, `on_failure` becomes `try`/`except`, and `map_task` → `flyte.map` / `asyncio.gather`.
- **`flyte-migrate-data-io`** — `FlyteFile`/`FlyteDirectory` → `flyte.io.File`/`Dir`, `StructuredDataset` → `flyte.io.DataFrame`, dataclasses, and ETL patterns.
- **`flyte-migrate-ml`** — small-model training, hyperparameter optimization, deep learning, batch inference, and end-to-end pipelines.
## Hybrid v1 and v2 pipelines
For a while you'll have Flyte 1 and Flyte 2 workloads running side by side, and you'll want them to call each other: a Flyte 1 workflow that kicks off a newly ported Flyte 2 task, or a Flyte 2 task that triggers a workflow that hasn't been migrated yet.
You can bridge the two in both directions. The idea is the same each way: one task installs **both** SDKs, authenticates to the **other** control plane, fetches the entity it wants to run, and launches it. Keep the bridging task lightweight and focused on orchestration.
### Running a Flyte 2 task from a Flyte 1 workflow
The bridge is a single Flyte 1 task that runs the Flyte 2 client. Give it an image with **both** `flytekit` and `flyte` installed, provide a Flyte 2 API key as a secret, authenticate inside the task with `flyte.init_from_api_key()`, fetch the deployed task with `flyte.remote.Task.get(...)`, and run it.
```python
import flytekit
from flytekit import task, workflow, ImageSpec, Secret, current_context
# The bridge image needs BOTH the v1 (flytekit) and v2 (flyte) SDKs.
bridge_image = ImageSpec(
name="v1-to-v2-bridge",
packages=["flytekit", "flyte"],
)
@task(
container_image=bridge_image,
secret_requests=[Secret(group="flyte", key="flyte_api_key")],
)
def launch_v2_from_v1(x: int) -> str:
import flyte
import flyte.remote
# Authenticate to the Flyte 2 control plane with the API key.
# Option A: read the mounted secret and pass it explicitly.
api_key = current_context().secrets.get(group="flyte", key="flyte_api_key")
flyte.init_from_api_key(api_key=api_key)
# Option B: if FLYTE_API_KEY is set as an env var, no argument is needed:
# flyte.init_from_api_key()
# Fetch the deployed Flyte 2 task and run it.
remote_v2_task = flyte.remote.Task.get(
"my_v2_env.process",
auto_version="latest",
)
run = flyte.run(remote_v2_task, x=x)
run.wait() # optional: block until the v2 run finishes
return run.url
@workflow
def main(x: int) -> str:
return launch_v2_from_v1(x=x)
```
The referenced Flyte 2 task (`my_v2_env.process` above) must be **deployed** before the bridge runs. Use `flyte.init_from_api_key()` here — do **not** use `flyte.init_from_config()`, which reads a `config.yaml` that has no API-key field.
### Running a Flyte 1 workflow from a Flyte 2 task
The reverse works the same way: a Flyte 2 task installs the Flyte 1 client and uses `FlyteRemote` to launch a Flyte 1 workflow.
```python
import flyte
env = flyte.TaskEnvironment(
name="v2_to_v1_bridge",
# The image needs the Flyte 1 client installed.
image=flyte.Image.from_debian_base().with_pip_packages("flytekit"),
# Supply credentials for the Flyte 1 control plane (config or API key).
secrets=[flyte.Secret(key="v1_client_secret", as_env_var="V1_CLIENT_SECRET")],
)
@env.task
async def launch_v1_from_v2(x: int) -> str:
from flytekit.remote import FlyteRemote
from flytekit.configuration import Config
# Point the client at your Flyte 1 cluster.
remote = FlyteRemote(
config=Config.for_endpoint(endpoint="my-v1-cluster.example.com"),
default_project="flytesnacks",
default_domain="development",
)
# Fetch the deployed Flyte 1 workflow and execute it.
wf = remote.fetch_workflow(name="my_v1_module.main", version="v1.2.3")
execution = remote.execute(wf, inputs={"x": x}, wait=True)
return execution.id.name
```
### Hybrid considerations
- **Both SDKs in one image.** The bridging task installs `flytekit` and `flyte` together. Pin versions and watch for dependency conflicts; keep the bridge image minimal.
- **Deploy the callee first.** For v1→v2, the Flyte 2 task must be deployed (`flyte deploy`) before `flyte.remote.Task.get()` can resolve it. For v2→v1, the Flyte 1 workflow must be registered on its cluster.
- **Wait vs. fire-and-forget.** Both `run.wait()` (v2) and `execute(..., wait=True)` (v1) block until the launched run finishes. Omit them to launch and return immediately.
- **Credentials cross a boundary.** The bridge authenticates to a *different* control plane than the one it runs on. Store the API key or client credentials as a secret — never hard-code them.
- **Keep the bridge lightweight.** Like any orchestrating task, it should mostly launch and assemble results rather than do heavy compute.
## Gotchas
Flyte 2 lets each Python task act as its own engine, launching sub-tasks and assembling their outputs. That flexibility warrants some caveats.
### Common gotchas
- **`flyte.map` returns a generator.** Wrap it in `list()` to materialize results, unlike `map_task` which returned a list directly.
- **`memory`, not `mem`.** The `Resources` parameter was renamed, and there are no separate `requests`/`limits` — a single value serves as both.
- **GPUs use a `"T4:1"` string.** Type and count are combined; the separate `accelerator=` argument is gone.
- **Image, resources, and cache live on the `TaskEnvironment`.** Set them once at the env level instead of repeating them on every task decorator.
- **`current_context()` is gone.** Read secrets from environment variables and use `flyte.ctx()` for runtime context.
- **The `>>` ordering operator is gone.** Sequential (sync) calls and sequential `await`s are naturally ordered.
- **Retries no longer have a platform cap.** In Flyte 1 the control plane capped attempts at 3; in Flyte 2 total attempts equal `retries + 1`. Audit any large `retries` values before deploying.
- **You can only `await` async tasks.** Call a sync task from an async context with `.aio()`.
- **Pick an entrypoint task name.** There's no `@workflow`, so the top-level task is just a task (commonly `main`); run it with `flyte run module.py main`.
- **Type annotations are more lenient.** Flyte 2 will pickle untyped I/O rather than rejecting it at registration.
- **Keep orchestration lightweight.** A task that calls other tasks acts as a driver pod. Avoid heavy CPU work in it.
## Anti-Patterns
1. **Don't introduce non-determinism into orchestration.** When a task launches another task, a new Action ID is determined as a hash of the inputs and task definition — consistent hashing is what makes recovery and replay work. Branching on `datetime.now()` or other non-deterministic values breaks that guarantee: on retry, a *different* downstream task may get kicked off. If non-determinism is unavoidable, decorate sub-task functions with `@trace` for fine-grained checkpointing and observability.
2. **Don't do heavy compute in a driver task.** When a task runs other tasks and assembles their outputs, it becomes a driver pod (work that Flyte Propeller did in v1). A CPU-bound function between two `await`s makes the driver pod hang and slows downstream kickoff. Keep parent tasks focused on orchestration:
```python
@env.task
async def t_main():
await t1()
local_cpu_intensive_function() # ❌ blocks the driver pod between t1 and t2
await t2()
```
3. **Don't rely on global state across tasks.** Each task runs in its own isolated container; globals are not carried across task containers. Any state that must persist has to be reconstructable through repeated deterministic execution.
4. **Don't materialize huge in-memory I/O between tasks.** Outputs are materialized in the parent pod's memory, so passing a 1 GB `list[float]` requires the pod to hold all of it, risking OOM. Use `flyte.io.File`, `flyte.io.Dir`, and `flyte.io.DataFrame` — they're materialized only as pointers to offloaded data, so their memory footprint stays low.
5. **Don't skip type hints at the "workflow" level.** The top-level task now runs at runtime, so the system can't guarantee type safety across the DAG the way the v1 DSL did. Use Python type hints and a type checker like `mypy` at all levels, including the top-most task.
flyte-migrate-config15.6 KB
---
name: flyte-migrate-config
description: Migrates Flyte 1 task configuration, container images, resources, caching, secrets, scheduling, and CLI/config-file usage to Flyte 2 equivalents. Use when migrating Flyte 1 task configuration, images, resources, secrets, scheduling, or CLI/config-file usage to Flyte 2. Trigger words include resources, ImageSpec, cache_version, secrets, LaunchPlan, CronSchedule, pyflyte, config, register, and deploy.
---
# Flyte 1 to Flyte 2 Migration: Task Configuration and CLI/Config
In Flyte 1, image, resources, caching, secrets, and scheduling were configured per-task on the `@task` decorator or per-workflow on a `LaunchPlan`. In Flyte 2 most of this moves to the `flyte.TaskEnvironment`, so it is declared once and shared. The CLI is renamed from `pyflyte` to `flyte` and the config file is trimmed down. This skill covers migrating those settings and commands.
## Grounding References
| Resource | URL |
|---|---|
| Migration guide (Task configuration) | https://www.union.ai/docs/v2/flyte/user-guide/migration/flyte-2/configuration/ |
| Migration guide (CLI) | https://www.union.ai/docs/v2/flyte/user-guide/migration/flyte-2/cli-and-configuration/ |
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| CLI API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-cli/ |
| Example code | https://github.com/unionai/unionai-examples |
## Image, resources, and caching move to the TaskEnvironment
Image, resources, and caching move from the `@task` decorator to the `TaskEnvironment`. Per-task settings like `retries` and `timeout` stay on `@env.task`. Note that `mem` is renamed to `memory`, and there are no separate `requests`/`limits` — a single `Resources` value serves as both.
### Flyte 1
```python
from datetime import timedelta
import flytekit
from flytekit import Resources
image = flytekit.ImageSpec(
name="training-image",
packages=["scikit-learn", "pandas"],
)
@flytekit.task(
container_image=image,
requests=Resources(cpu="2", mem="4Gi"),
limits=Resources(cpu="4", mem="8Gi"),
cache=True,
cache_version="1.0",
retries=3,
timeout=timedelta(minutes=30),
)
def train_epoch(step: int) -> float:
# A stand-in for a training step that returns the current loss.
return 1.0 / (step + 1)
@flytekit.workflow
def main(step: int) -> float:
return train_epoch(step=step)
```
### Flyte 2
```python
from datetime import timedelta
import flyte
# Image, resources, and caching move to the TaskEnvironment, so they are declared
# once and shared by every task in the environment.
env = flyte.TaskEnvironment(
name="training",
image=flyte.Image.from_debian_base().with_pip_packages("scikit-learn", "pandas"),
resources=flyte.Resources(cpu="2", memory="4Gi"), # "memory", not "mem"
cache="auto",
)
# retries and timeout stay on the task decorator.
@env.task(retries=3, timeout=timedelta(minutes=30))
def train_epoch(step: int) -> float:
# A stand-in for a training step that returns the current loss.
return 1.0 / (step + 1)
@env.task
def main(step: int) -> float:
return train_epoch(step)
```
## Container images: ImageSpec to flyte.Image
Flyte 1's `ImageSpec` is replaced by Flyte 2's `flyte.Image` with a fluent builder API. Instead of one constructor with many arguments, you start from a base and chain builder methods.
```python
from flyte import Image
image = (
Image.from_debian_base(name="my-image", registry="ghcr.io/myorg", python_version=(3, 11))
.with_pip_packages("pandas", "numpy")
.with_apt_packages("curl", "git")
.with_env_vars({"MY_VAR": "value"})
)
```
| Constructor | Use case |
|---|---|
| `Image.from_debian_base()` | Most common; includes the Flyte SDK |
| `Image.from_base(image_uri)` | Start from any existing image |
| `Image.from_dockerfile(path)` | Complex custom builds |
| `Image.from_uv_script(path)` | UV-based projects |
Common chainable builder methods: `.with_pip_packages(...)`, `.with_requirements(path)`, `.with_uv_project(path)`, `.with_apt_packages(...)`, `.with_commands([...])`, `.with_source_file(path, dst=...)`, `.with_source_folder(path, dst=...)`, `.with_env_vars({...})`, and `.with_workdir(...)`.
| Flyte 1 `ImageSpec` | Flyte 2 `Image` | Notes |
|---|---|---|
| `name` | `name` (constructor) | Same |
| `registry` | `registry` (constructor) | Same |
| `python_version` | `python_version` (tuple) | `"3.11"` becomes `(3, 11)` |
| `packages` | `.with_pip_packages()` | Method instead of param |
| `apt_packages` | `.with_apt_packages()` | Method instead of param |
| `requirements` | `.with_requirements()` | Supports txt, poetry.lock, uv.lock |
| `env` | `.with_env_vars()` | Method instead of param |
| `commands` | `.with_commands()` | Method instead of param |
| `copy` / `source_root` | `.with_source_file()` / `.with_source_folder()` | More explicit methods |
| `base_image` | `Image.from_base()` | Different constructor |
| `builder` | Config file or `flyte.init()` | Global setting |
| `platform` | `platform` (constructor) | Tuple: `("linux/amd64", "linux/arm64")` |
For a private registry, create an image-pull secret and reference it:
```bash
flyte create secret --type image_pull my-registry-secret --from-file ~/.docker/config.json
```
```python
image = Image.from_debian_base(
registry="private.registry.com",
name="my-image",
registry_secret="my-registry-secret",
)
```
## Resources and GPUs
A single `flyte.Resources` value serves as both request and limit — there are no separate `requests`/`limits`. Several parameters were renamed.
| Flyte 1 | Flyte 2 | Notes |
|---|---|---|
| `cpu="1"` | `cpu="1"` | Same |
| `mem="2Gi"` | `memory="2Gi"` | Renamed |
| `gpu="1"` | `gpu="A100:1"` | `Type:count` format |
| `ephemeral_storage="10Gi"` | `disk="10Gi"` | Renamed |
| N/A | `shm="auto"` | New: shared memory |
GPU type and count are combined into one string, replacing the separate Flyte 1 `accelerator=` argument:
```python
env = flyte.TaskEnvironment(
name="gpu_env",
resources=flyte.Resources(
cpu="4",
memory="32Gi",
gpu="A100:2", # Type:count
# gpu="A100 80G:1" # 80GB variant
# gpu=flyte.GPU("A100", count=1, partition="1g.5gb") # MIG partition
),
)
```
Supported GPU types include A10, A10G, A100, A100 80G, B200, H100, H200, L4, L40s, T4, V100, RTX PRO 6000, and GB10.
## Caching: cache_version to cache="auto" / CachePolicy
Caching is enabled at the env level with `cache="auto"` (or per-task on `@env.task`). The explicit `cache_version` string moves into a `flyte.Cache` object.
| Behavior | Description |
|---|---|
| `"auto"` | Cache results and reuse if available |
| `"override"` | Always execute and overwrite the cache |
| `"disable"` | No caching (default for a `TaskEnvironment`) |
```python
# Flyte 1: @task(cache=True, cache_version="1.0")
# Flyte 2:
@env.task(cache="auto")
def cached_task(x: int) -> int:
return x * 2
# Advanced control (replaces cache_version, serialize, ignored_inputs, ...)
@env.task(cache=flyte.Cache(
behavior="auto",
version_override="v1.0",
serialize=True,
ignored_inputs=("debug",),
))
def advanced(x: int, debug: bool = False) -> int:
return x * 2
```
## Secrets: current_context().secrets to env vars
Secrets move from `secret_requests` on the task to `secrets` on the `TaskEnvironment`, and you read them from environment variables instead of `current_context().secrets` — for example, an API key for a model registry or hosted LLM.
### Flyte 1
```python
from flytekit import task, workflow, Secret, current_context
@task(secret_requests=[Secret(group="openai", key="api_key")])
def call_api() -> str:
token = current_context().secrets.get(group="openai", key="api_key")
return f"token has {len(token)} chars"
@workflow
def main() -> str:
return call_api()
```
### Flyte 2
```python
import os
import flyte
# Secrets are declared on the TaskEnvironment and injected as environment
# variables (instead of read through current_context().secrets).
env = flyte.TaskEnvironment(
name="secrets",
secrets=[flyte.Secret(key="openai_api_key", as_env_var="OPENAI_API_KEY")],
)
@env.task
def call_api() -> str:
token = os.getenv("OPENAI_API_KEY", "")
return f"token has {len(token)} chars"
@env.task
def main() -> str:
return call_api()
```
A `flyte.Secret` can be mounted as an environment variable or as a file, and the access convention changes:
```python
flyte.Secret(key="openai-key", as_env_var="OPENAI_API_KEY") # mount as env var
flyte.Secret(key="access-key", group="aws") # env var: AWS_ACCESS_KEY
flyte.Secret(key="ssl-cert", mount="/etc/flyte/secrets") # mount as a file
```
| Flyte 1 pattern | Flyte 2 pattern |
|---|---|
| `ctx.secrets.get(key="mykey", group="mygroup")` | `os.environ["MYGROUP_MYKEY"]` (auto-named) |
| `ctx.secrets.get(key="mykey")` | `os.environ["MY_SECRET"]` (with `as_env_var="MY_SECRET"`) |
Create and manage secrets from the CLI:
```bash
flyte create secret MY_SECRET_KEY --value my_secret_value
flyte create secret MY_SECRET_KEY --from-file /path/to/secret
flyte get secret
flyte delete secret MY_SECRET_KEY
```
## Scheduling: LaunchPlan + CronSchedule to flyte.Trigger + flyte.Cron
A `LaunchPlan` with a `CronSchedule` (say, a nightly retraining job) becomes a `flyte.Trigger` attached directly to the task. Use `flyte.TriggerTime` to bind the scheduled fire time to an input, and deploy the trigger with `flyte deploy`.
### Flyte 1
```python
from flytekit import task, workflow, LaunchPlan, CronSchedule
@task
def retrain(kickoff_time: str) -> str:
return f"retrained model at {kickoff_time}"
@workflow
def main(kickoff_time: str) -> str:
return retrain(kickoff_time=kickoff_time)
# A LaunchPlan attaches a schedule (and default inputs) to a workflow.
nightly_retrain = LaunchPlan.get_or_create(
workflow=main,
name="nightly_retrain",
schedule=CronSchedule(
schedule="0 2 * * *", # 2 AM daily
kickoff_time_input_arg="kickoff_time",
),
)
```
### Flyte 2
```python
from datetime import datetime
import flyte
env = flyte.TaskEnvironment(name="scheduling")
# A Trigger replaces LaunchPlan + CronSchedule. It is attached directly to the
# task and deployed with it (flyte deploy). flyte.TriggerTime binds the
# scheduled fire time to a task input.
nightly_retrain = flyte.Trigger(
name="nightly_retrain",
automation=flyte.Cron("0 2 * * *"), # 2 AM daily
inputs={"trigger_time": flyte.TriggerTime},
auto_activate=True,
)
@env.task(triggers=nightly_retrain)
def main(trigger_time: datetime = datetime(2024, 1, 1, 2, 0)) -> str:
return f"retrained model at {trigger_time.isoformat()}"
```
Triggers support `flyte.Cron("0 9 * * *", timezone="America/New_York")` and `flyte.FixedRate(timedelta(hours=1))` as automations, plus convenience constructors like `flyte.Trigger.hourly()` and `flyte.Trigger.daily()`.
## CLI command mapping: pyflyte to flyte
The command-line tool is renamed from `pyflyte` to `flyte`, and remote is now the default.
| Flyte 1 | Flyte 2 | Notes |
|---|---|---|
| `pyflyte run` | `flyte run` | Similar, different flags |
| `pyflyte run --remote` | `flyte run` | Remote is the default in Flyte 2 |
| `pyflyte run` (local) | `flyte run --local` | Local execution is now explicit |
| `pyflyte register` | `flyte deploy` | Different concept |
| `pyflyte package` | N/A | Not needed in Flyte 2 |
| `pyflyte serialize` | N/A | Not needed in Flyte 2 |
### Running tasks — Flyte 1
```bash
# Local
pyflyte run my_module.py my_workflow --arg1 value1
# Remote
pyflyte --config config.yaml run --remote my_module.py my_workflow --arg1 value1
```
### Running tasks — Flyte 2
```bash
# Remote (default)
flyte run my_module.py my_task --arg1 value1
# Local
flyte run --local my_module.py my_task --arg1 value1
# With an explicit config file
flyte --config config.yaml run my_module.py my_task --arg1 value1
```
### Deploying (register to deploy)
In Flyte 1 you registered a module; in Flyte 2 you deploy task environments.
#### Flyte 1
```bash
pyflyte register my_module.py -p my-project -d development
```
#### Flyte 2
```bash
# Deploy a task environment
flyte deploy my_module.py my_env --project my-project --domain development
# Deploy all environments in a file
flyte deploy --all my_module.py
# Deploy with an explicit version, or recursively
flyte deploy --version v1.0.0 my_module.py my_env
flyte deploy --recursive --all ./src
```
### Key flag differences
| Flyte 1 flag | Flyte 2 flag | Notes |
|---|---|---|
| `--remote` | (default) | Remote is the default |
| `--copy-all` | `--copy-style all` | File copying |
| N/A | `--copy-style loaded_modules` | Default: only imported modules |
| `-p, --project` | `--project` | Same |
| `-d, --domain` | `--domain` | Same |
| `-i, --image` | `--image` | Same format |
| N/A | `--follow, -f` | Follow execution logs |
## Configuration files
The config file lives in the same place (`~/.flyte/config.yaml`), but the environment variable changes from `FLYTECTL_CONFIG` to `FLYTE_CONFIG`, and the format is simpler.
### Flyte 1
```yaml
admin:
endpoint: dns:///your-cluster.hosted.unionai.cloud
insecure: false
authType: Pkce
```
### Flyte 2
```yaml
admin:
endpoint: dns:///your-cluster.hosted.unionai.cloud
image:
builder: remote # or "local"
task:
domain: development
org: your-org
project: your-project
```
| Setting | Flyte 1 | Flyte 2 |
|---|---|---|
| Endpoint | `admin.endpoint` | `admin.endpoint` |
| Auth type | `admin.authType` | Auto-detected (PKCE default) |
| Project | CLI flag `-p` | `task.project` (default) |
| Domain | CLI flag `-d` | `task.domain` (default) |
| Organization | CLI flag `--org` | `task.org` (default) |
| Image builder | N/A | `image.builder` (`local` or `remote`) |
### Configuring in code
```python
import flyte
# From a config file (auto-discovers, or pass a path)
flyte.init_from_config()
flyte.init_from_config("path/to/config.yaml")
# Programmatically
flyte.init(
endpoint="flyte.example.com",
project="my-project",
domain="development",
)
```
For API-key authentication in non-interactive environments, use `flyte.init_from_api_key()`.
## Anti-Patterns
1. **Don't keep `image`, `resources`, and `cache` on `@env.task`** — move them onto the shared `flyte.TaskEnvironment`; only per-task settings like `retries` and `timeout` stay on `@env.task`.
2. **Don't use `mem`, `ephemeral_storage`, or separate `requests`/`limits`** — use `memory`, `disk`, and a single `flyte.Resources` value that serves as both.
3. **Don't pass GPUs with `gpu="1"` plus `accelerator=`** — combine type and count into one `Type:count` string like `gpu="A100:2"`.
4. **Don't rebuild `ImageSpec`'s many constructor args** — start from a base (`Image.from_debian_base()`) and chain `.with_*` builder methods.
5. **Don't keep `cache_version="1.0"`** — use `cache="auto"` for the common case, or `flyte.Cache(version_override=...)` for advanced control.
6. **Don't read secrets via `current_context().secrets.get(...)`** — declare them on the `TaskEnvironment` and read the injected environment variable with `os.environ` / `os.getenv`.
7. **Don't recreate `LaunchPlan` + `CronSchedule`** — use `flyte.Trigger` with `flyte.Cron` attached to the task, and deploy it with `flyte deploy`.
8. **Don't run `pyflyte ... --remote`** — `flyte run` is remote by default; add `--local` explicitly for in-process runs.
9. **Don't use `pyflyte register`** — use `flyte deploy` to deploy task environments.
10. **Don't set `FLYTECTL_CONFIG` or rely on `admin.authType`** — use `FLYTE_CONFIG` and the simpler config format with auto-detected auth.
flyte-migrate-control-flow11.8 KB
---
name: flyte-migrate-control-flow
description: "Migrates Flyte 1 branching, dynamic workflows, failure handling, and fan-out to native Flyte 2 Python. Use when migrating Flyte 1 branching, dynamic workflows, failure handling, or map_task/fan-out to Flyte 2. Trigger words: conditional, @dynamic, map_task, on_failure, branching, parallelism, fan-out, flyte.map, asyncio.gather."
---
# Flyte 1 to 2 Migration: Control Flow and Parallelism
Flyte 1 expressed branching, dynamic fan-out, and failure handling through DSL constructs (`conditional()`, `@dynamic`, `@workflow(on_failure=...)`) and `map_task`. In Flyte 2 these are all ordinary Python, because orchestration runs as real Python at runtime. Native `if`/`elif`/`else` replaces the conditional DSL, plain task loops replace `@dynamic`, `try`/`except` replaces `on_failure`, and `flyte.map` / `asyncio.gather` replace `map_task`.
## Grounding References
| Resource | URL |
|---|---|
| Migration guide (Control flow) | https://www.union.ai/docs/v2/flyte/user-guide/migration/flyte-2/control-flow/ |
| Migration guide (Parallelism) | https://www.union.ai/docs/v2/flyte/user-guide/migration/flyte-2/parallelism/ |
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via `flyte-mcp` server |
## Conditional Execution
The `conditional()` DSL becomes ordinary Python `if` / `elif` / `else` — for example, choosing a model based on dataset size.
### Flyte 1
```python
from flytekit import task, workflow, conditional
@task
def train_gradient_boosting(n_rows: int) -> str:
return f"trained gradient boosting on {n_rows} rows"
@task
def train_logistic_regression(n_rows: int) -> str:
return f"trained logistic regression on {n_rows} rows"
@workflow
def main(n_rows: int) -> str:
# Pick the model based on dataset size.
return (
conditional("model_choice")
.if_(n_rows > 10_000)
.then(train_gradient_boosting(n_rows=n_rows))
.else_()
.then(train_logistic_regression(n_rows=n_rows))
)
```
### Flyte 2
```python
import flyte
env = flyte.TaskEnvironment(name="conditional")
@env.task
def train_gradient_boosting(n_rows: int) -> str:
return f"trained gradient boosting on {n_rows} rows"
@env.task
def train_logistic_regression(n_rows: int) -> str:
return f"trained logistic regression on {n_rows} rows"
# Branching is now ordinary Python control flow -- no conditional() DSL.
@env.task
def main(n_rows: int) -> str:
if n_rows > 10_000:
return train_gradient_boosting(n_rows)
return train_logistic_regression(n_rows)
```
## Dynamic Workflows
`@dynamic` existed so a task could generate a variable number of subtask calls at runtime (e.g. one per data partition discovered at runtime). In Flyte 2 every task can do this natively, so `@dynamic` simply disappears — loop over runtime data in an ordinary `@env.task`.
### Flyte 1
```python
from flytekit import task, workflow, dynamic
@task
def list_partitions(n: int) -> list[int]:
return list(range(n))
@task
def process_partition(partition_id: int) -> int:
# Aggregate one data partition.
return partition_id * 2
@dynamic
def process_all(partitions: list[int]) -> list[int]:
results = []
for partition_id in partitions:
results.append(process_partition(partition_id=partition_id))
return results
@workflow
def main(n: int) -> list[int]:
partitions = list_partitions(n=n)
return process_all(partitions=partitions)
```
### Flyte 2
```python
import flyte
env = flyte.TaskEnvironment(name="dynamic")
@env.task
def process_partition(partition_id: int) -> int:
# Aggregate one data partition.
return partition_id * 2
# No @dynamic decorator needed: a plain task can loop over runtime data (e.g. a
# variable number of partitions discovered at runtime) and call other tasks.
@env.task
def main(n: int) -> list[int]:
return [process_partition(partition_id) for partition_id in range(n)]
```
## Error Handling
Flyte 1's `@workflow(on_failure=...)` handler becomes ordinary Python `try` / `except` — catch a failed training run, run cleanup, and recover or re-raise.
### Flyte 1
```python
from flytekit import task, workflow
@task
def train_fold(max_depth: int) -> float:
if max_depth <= 0:
raise ValueError("max_depth must be positive")
# Return validation accuracy for this hyperparameter.
return 0.90 + 0.001 * max_depth
@task
def notify_failure() -> None:
print("training run failed -- sending alert")
# The on_failure handler runs if any node in the workflow fails. There is no
# try/except inside a Flyte 1 workflow.
@workflow(on_failure=notify_failure)
def main(max_depth: int) -> float:
return train_fold(max_depth=max_depth)
```
### Flyte 2
```python
import flyte
env = flyte.TaskEnvironment(name="error_handling")
@env.task
async def train_fold(max_depth: int) -> float:
if max_depth <= 0:
raise ValueError("max_depth must be positive")
return 0.90 + 0.001 * max_depth
# Failure handling is ordinary Python try/except -- no on_failure handler.
@env.task
async def main(max_depth: int) -> float:
try:
return await train_fold(max_depth)
except ValueError as e:
print(f"invalid hyperparameter ({e}); falling back to a safe default")
# Recover with a safe default instead of failing the whole run.
return await train_fold(max_depth=6)
```
Flyte 2 also exposes typed errors, so you can catch a specific failure and retry with more resources — a common need for memory-hungry training jobs:
```python
try:
return await train_fold(sample_size)
except flyte.errors.OOMError:
# Retry the same task with a larger memory request.
return await train_fold.override(
resources=flyte.Resources(memory="16Gi")
)(sample_size)
```
## Fan-out: map_task
`map_task()` becomes `flyte.map()`, a near drop-in replacement. The one catch: `flyte.map` returns a generator, so wrap it in `list()`. For new code, the idiomatic approach is Python `async`/`await` with `asyncio.gather()`, which gives finer control over concurrency and error handling.
### Flyte 1
```python
from functools import partial
from flytekit import task, workflow, map_task
@task
def get_shards(n: int) -> list[int]:
return list(range(n))
@task
def score_shard(shard_id: int, model_version: int) -> int:
# Score one shard of records with the given model version.
return shard_id * model_version
@workflow
def main(n: int, model_version: int) -> list[int]:
shards = get_shards(n=n)
return map_task(
partial(score_shard, model_version=model_version),
concurrency=10,
)(shard_id=shards)
```
### Flyte 2 (flyte.map)
```python
import flyte
from functools import partial
env = flyte.TaskEnvironment(name="map_task")
@env.task
def score_shard(shard_id: int, model_version: int) -> int:
# Score one shard of records with the given model version.
return shard_id * model_version
@env.task
def main(n: int, model_version: int) -> list[int]:
bound = partial(score_shard, model_version=model_version)
# flyte.map is a drop-in for map_task, but it returns a generator, so wrap
# it in list() to materialize the results.
return list(flyte.map(bound, range(n), concurrency=10))
```
### Flyte 2 (asyncio.gather)
```python
import asyncio
import flyte
env = flyte.TaskEnvironment(name="map_task")
@env.task
async def score_shard_async(shard_id: int, model_version: int) -> int:
return shard_id * model_version
@env.task
async def main_async(n: int, model_version: int) -> list[int]:
# asyncio.gather is the idiomatic Flyte 2 way to fan out.
coros = [score_shard_async(i, model_version) for i in range(n)]
return list(await asyncio.gather(*coros))
```
### Choosing flyte.map vs asyncio.gather
| Feature | `flyte.map` (sync) | `asyncio.gather` (async) |
|---|---|---|
| Syntax | `list(flyte.map(fn, items))` | `await asyncio.gather(*tasks)` |
| Concurrency limit | Built-in `concurrency=N` | Use `asyncio.Semaphore` |
| Streaming / as-completed | No | Yes, via `asyncio.as_completed()` |
| Error handling | `return_exceptions=True` | Check return type |
Use `flyte.map` for the smallest change from Flyte 1 `map_task`, or when stuck in synchronous code. Use `asyncio.gather` for new code where you want streaming results or fine-grained concurrency control.
### Concurrency Control and Error Handling
`map_task`'s `concurrency` and `min_success_ratio` become an `asyncio.Semaphore` and `return_exceptions=True`:
```python
import asyncio
@env.task
async def main(items: list[int], max_concurrent: int = 5) -> list[str]:
sem = asyncio.Semaphore(max_concurrent)
async def process_with_limit(item: int) -> str:
async with sem:
return await process_item(item)
tasks = [process_with_limit(i) for i in items]
results = await asyncio.gather(*tasks, return_exceptions=True)
return [r for r in results if not isinstance(r, Exception)]
```
## Data Backfills
Reprocessing a range of dates is a textbook `@dynamic` use case in Flyte 1, because the number of days is only known at runtime. In Flyte 2 it's a plain task that builds the date range and fans the days out with `asyncio.gather`.
### Flyte 1
```python
from datetime import date, timedelta
from flytekit import task, workflow, dynamic
@task
def process_day(day: str) -> int:
# Reprocess a single day's partition; return the row count.
return len(day)
# @dynamic is needed because the number of days is only known at runtime.
@dynamic
def backfill(start: str, days: int) -> list[int]:
base = date.fromisoformat(start)
results = []
for i in range(days):
day = (base + timedelta(days=i)).isoformat()
results.append(process_day(day=day))
return results
@workflow
def main(start: str, days: int) -> list[int]:
return backfill(start=start, days=days)
```
### Flyte 2
```python
import asyncio
from datetime import date, timedelta
import flyte
env = flyte.TaskEnvironment(name="data_backfill")
@env.task
async def process_day(day: str) -> int:
# Reprocess a single day's partition; return the row count.
return len(day)
# A plain task builds the date range at runtime and fans the days out in
# parallel with asyncio.gather -- no @dynamic and no map_task needed.
@env.task
async def main(start: str, days: int) -> list[int]:
base = date.fromisoformat(start)
coros = [
process_day((base + timedelta(days=i)).isoformat())
for i in range(days)
]
return list(await asyncio.gather(*coros))
```
## Anti-Patterns
1. **Don't import `conditional`, `dynamic`, or `map_task` from `flytekit`** — none exist in Flyte 2. Branching is native `if`/`elif`/`else`, dynamic fan-out is a plain task loop, and `map_task` becomes `flyte.map`.
2. **Don't keep the `conditional().if_().then().else_()` DSL** — rewrite it as ordinary Python control flow inside an `@env.task`.
3. **Don't reach for `@dynamic`** — every Flyte 2 task can loop over runtime data and call other tasks, so drop the decorator entirely.
4. **Don't pass `on_failure=...` to `@workflow`** — there is no workflow decorator in Flyte 2; handle failures with ordinary `try`/`except` inside a task.
5. **Don't forget to `list()` a `flyte.map` result** — it returns a generator, not a materialized list.
6. **Don't forget to `await` async fan-out** — `asyncio.gather(*coros)` returns a coroutine; without `await` you get a coroutine object instead of results.
7. **Don't drop concurrency limits** — port `concurrency=N` to `flyte.map(..., concurrency=N)` or an `asyncio.Semaphore`, and `min_success_ratio` to `return_exceptions=True` with filtering.
8. **Don't use Union-only features** — avoid `ReusePolicy` and other Union-specific APIs.
flyte-migrate-data-io10.3 KB
---
name: flyte-migrate-data-io
description: Migrates Flyte 1 data types and offloaded I/O to Flyte 2. Use when migrating Flyte 1 data types and I/O (files, directories, dataframes, dataclasses) to Flyte 2, converting FlyteFile, FlyteDirectory, or StructuredDataset to flyte.io.File, flyte.io.Dir, and flyte.io.DataFrame. Trigger words are FlyteFile, FlyteDirectory, StructuredDataset, DataFrame, dataclass, Pydantic, type, I/O, and serialization.
---
# Flyte 1 to 2 Migration: Data Types and I/O
Flyte 2 renames the offloaded-data types and makes their I/O `async`, but the mental model is the same: pass lightweight references to large data between tasks, not the materialized bytes. `FlyteFile`, `FlyteDirectory`, and `StructuredDataset` become `flyte.io.File`, `flyte.io.Dir`, and `flyte.io.DataFrame`. Plain dataclasses and Pydantic `BaseModel`s work directly as task I/O with no JSON mixin.
## Grounding References
| Resource | URL |
|---|---|
| Migration guide | https://www.union.ai/docs/v2/flyte/user-guide/migration/flyte-2/data-io/ |
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via `flyte-mcp` server |
## Type Mapping
| Flyte 1 | Flyte 2 | Notes |
|---|---|---|
| `flytekit.types.file.FlyteFile` | `flyte.io.File` | I/O is `async` |
| `flytekit.types.directory.FlyteDirectory` | `flyte.io.Dir` | I/O is `async` |
| `flytekit.types.structured.StructuredDataset` | `flyte.io.DataFrame` | build with `from_df`, read with `open(...).all()` |
| `@dataclass_json` + `@dataclass` | plain `@dataclass` | no mixin needed |
| Pydantic `BaseModel` (+ config) | plain Pydantic `BaseModel` | works directly as task I/O |
## Offloaded Data: The Mental Model
`File`, `Dir`, and `DataFrame` are lightweight references (pointers) to data offloaded in blob storage — not the materialized bytes. In Flyte 2 the read/write operations are `async`: upload with `await File.from_local(local_path)`, read with `async with f.open("rb") as fh: await fh.read()`, build a frame with `flyte.io.DataFrame.from_df(df)` (sync constructor), and read it with `await fdf.open(pandas.DataFrame).all()`.
## Files and Directories
`FlyteFile` and `FlyteDirectory` become `flyte.io.File` and `flyte.io.Dir` — the way you pass model artifacts and datasets between tasks. Use `await File.from_local(...)` to upload and `async with file.open(...)` to read.
### Flyte 1
```python
import os
from flytekit import task, workflow, current_context
from flytekit.types.file import FlyteFile
@task
def write_file(content: str) -> FlyteFile:
path = os.path.join(current_context().working_directory, "out.txt")
with open(path, "w") as f:
f.write(content)
return FlyteFile(path=path)
@task
def read_file(f: FlyteFile) -> str:
with open(f.download()) as fh:
return fh.read()
@workflow
def main(content: str) -> str:
f = write_file(content=content)
return read_file(f=f)
```
### Flyte 2
```python
import flyte
from flyte.io import File
env = flyte.TaskEnvironment(name="files")
@env.task
async def write_file(content: str) -> File:
with open("out.txt", "w") as f:
f.write(content)
# File.from_local uploads the file to blob storage and returns a reference
# (a lightweight pointer, not the materialized bytes).
return await File.from_local("out.txt")
@env.task
async def read_file(f: File) -> str:
async with f.open("rb") as fh:
return (await fh.read()).decode("utf-8")
@env.task
async def main(content: str) -> str:
f = await write_file(content)
return await read_file(f)
```
Directories follow the same pattern: import `Dir` from `flyte.io` and use its `async` upload/read methods in place of `FlyteDirectory`. See [Files and directories](https://www.union.ai/docs/v2/flyte/user-guide/task-programming/files-and-directories) for more.
## DataFrames
`StructuredDataset` becomes `flyte.io.DataFrame`. Construct one with `flyte.io.DataFrame.from_df(df)` and read it back with `await df.open(pandas.DataFrame).all()`.
### Flyte 1
```python
import pandas as pd
from flytekit import task, workflow
from flytekit.types.structured import StructuredDataset
@task
def make_df() -> StructuredDataset:
df = pd.DataFrame({"employee_id": [1, 2, 3], "salary": [50000, 60000, 70000]})
return StructuredDataset(dataframe=df)
@task
def total_payroll(sd: StructuredDataset) -> float:
df = sd.open(pd.DataFrame).all()
return float(df["salary"].sum())
@workflow
def main() -> float:
return total_payroll(sd=make_df())
```
### Flyte 2
```python
import pandas as pd
import flyte
import flyte.io
env = flyte.TaskEnvironment(
name="dataframe",
image=flyte.Image.from_debian_base().with_pip_packages("pandas", "pyarrow"),
)
@env.task
async def make_df() -> flyte.io.DataFrame:
df = pd.DataFrame({"employee_id": [1, 2, 3], "salary": [50000, 60000, 70000]})
# StructuredDataset becomes flyte.io.DataFrame.
return flyte.io.DataFrame.from_df(df)
@env.task
async def total_payroll(fdf: flyte.io.DataFrame) -> float:
df = await fdf.open(pd.DataFrame).all()
return float(df["salary"].sum())
@env.task
async def main() -> float:
return await total_payroll(await make_df())
```
Add the dataframe dependencies (for example `pandas` and `pyarrow`) to the `TaskEnvironment` image. See [DataFrames](https://www.union.ai/docs/v2/flyte/user-guide/task-programming/dataframes) for more.
## Dataclasses and Structured Types
Flyte 1 required a `@dataclass_json` mixin for dataclass I/O. In Flyte 2, plain dataclasses (and Pydantic `BaseModel`s) work directly as task inputs and outputs — handy for passing around a training config.
### Flyte 1
```python
from dataclasses import dataclass
from dataclasses_json import dataclass_json
from flytekit import task, workflow
@dataclass_json
@dataclass
class TrainingConfig:
learning_rate: float
n_estimators: int
max_depth: int = 6
@task
def make_config(learning_rate: float, n_estimators: int) -> TrainingConfig:
return TrainingConfig(learning_rate=learning_rate, n_estimators=n_estimators)
@task
def train(config: TrainingConfig) -> str:
return (
f"trained with lr={config.learning_rate}, "
f"n_estimators={config.n_estimators}, max_depth={config.max_depth}"
)
@workflow
def main(learning_rate: float, n_estimators: int) -> str:
config = make_config(learning_rate=learning_rate, n_estimators=n_estimators)
return train(config=config)
```
### Flyte 2
```python
from dataclasses import dataclass
import flyte
env = flyte.TaskEnvironment(name="dataclasses")
# Plain dataclasses work directly as task I/O -- no @dataclass_json mixin needed.
# Pydantic BaseModels work the same way.
@dataclass
class TrainingConfig:
learning_rate: float
n_estimators: int
max_depth: int = 6
@env.task
def make_config(learning_rate: float, n_estimators: int) -> TrainingConfig:
return TrainingConfig(learning_rate=learning_rate, n_estimators=n_estimators)
@env.task
def train(config: TrainingConfig) -> str:
return (
f"trained with lr={config.learning_rate}, "
f"n_estimators={config.n_estimators}, max_depth={config.max_depth}"
)
@env.task
def main(learning_rate: float, n_estimators: int) -> str:
config = make_config(learning_rate, n_estimators)
return train(config)
```
## Data ETL: Putting It Together
Extract, clean, aggregate, and write out a feature table. `StructuredDataset` becomes `flyte.io.DataFrame`, and the tasks become `async`.
### Flyte 1
```python
import pandas as pd
from flytekit import task, workflow
from flytekit.types.structured import StructuredDataset
@task
def extract() -> pd.DataFrame:
# Read raw transaction records (stand-in for a real source).
return pd.DataFrame(
{
"user_id": [1, 1, 2, 3, 3, 3],
"amount": [10.0, 5.0, 20.0, 7.5, 2.5, 1.0],
}
)
@task
def transform(df: pd.DataFrame) -> StructuredDataset:
# Clean and aggregate into a per-user feature table.
df = df[df["amount"] > 0]
agg = df.groupby("user_id", as_index=False)["amount"].sum()
return StructuredDataset(dataframe=agg)
@task
def load(sd: StructuredDataset) -> int:
df = sd.open(pd.DataFrame).all()
return len(df)
@workflow
def main() -> int:
raw = extract()
features = transform(df=raw)
return load(sd=features)
```
### Flyte 2
```python
import pandas as pd
import flyte
import flyte.io
env = flyte.TaskEnvironment(
name="data_etl",
image=flyte.Image.from_debian_base().with_pip_packages("pandas", "pyarrow"),
)
@env.task
async def extract() -> pd.DataFrame:
# Read raw transaction records (stand-in for a real source).
return pd.DataFrame(
{
"user_id": [1, 1, 2, 3, 3, 3],
"amount": [10.0, 5.0, 20.0, 7.5, 2.5, 1.0],
}
)
@env.task
async def transform(df: pd.DataFrame) -> flyte.io.DataFrame:
# Clean and aggregate into a per-user feature table.
df = df[df["amount"] > 0]
agg = df.groupby("user_id", as_index=False)["amount"].sum()
# StructuredDataset becomes flyte.io.DataFrame.
return flyte.io.DataFrame.from_df(agg)
@env.task
async def load(sd: flyte.io.DataFrame) -> int:
df = await sd.open(pd.DataFrame).all()
return len(df)
@env.task
async def main() -> int:
raw = await extract()
features = await transform(raw)
return await load(features)
```
## Anti-Patterns
1. **Don't call the offloaded-data I/O synchronously** — `File.from_local`, `file.open(...).read()`, and `DataFrame.open(...).all()` are `async` in Flyte 2; `await` them inside `async` tasks.
2. **Don't keep the `@dataclass_json` mixin** — plain `@dataclass` and Pydantic `BaseModel`s serialize as task I/O directly; drop `dataclasses_json`.
3. **Don't return `StructuredDataset(dataframe=df)`** — use `flyte.io.DataFrame.from_df(df)` instead.
4. **Don't materialize large data into task outputs** — return `File`, `Dir`, or `DataFrame` references, not the raw bytes or full frames.
5. **Don't forget the dataframe dependencies** — add `pandas` and `pyarrow` (or your engine) to the `TaskEnvironment` image so DataFrame I/O works remotely.
6. **Don't import from `flytekit.types.*`** — import `File` and `Dir` from `flyte.io`, and use `flyte.io.DataFrame`.
flyte-migrate-ml21.7 KB
---
name: flyte-migrate-ml
description: Migrates Flyte 1 machine learning code to Flyte 2 and unlocks net-new v2 patterns. Use when migrating Flyte 1 ML workloads (training, HPO, GPU/deep learning, batch inference) to Flyte 2, specifying GPU resources, or building the end-to-end pipeline pattern. Trigger words - migrate training, HPO, GPU, deep learning, batch inference, model serving, pytorch.
---
# Flyte 1 to Flyte 2 ML Migration Skill
Migrate existing Flyte 1 ML workloads — small-model training, hyperparameter optimization, deep learning on GPUs, and batch inference — to Flyte 2, then take advantage of patterns that were not possible in Flyte 1 (real-time serving, apps, sandboxed execution).
This skill is specifically about **migrating existing v1 ML code**. For greenfield authoring in Flyte 2, use the companion skills:
- `flyte-sdk-ml` — writing new ML training / inference tasks in Flyte 2.
- `flyte-sdk-app` — writing new apps and serving endpoints.
- `flyte-sdk-agent` — writing new agents and sandboxed / code-mode workloads.
## Grounding References
| Resource | URL |
|---|---|
| Migration guide (ML workloads) | https://www.union.ai/docs/v2/flyte/user-guide/migration/flyte-2/ml-workloads/ |
| Migration guide (New in Flyte 2) | https://www.union.ai/docs/v2/flyte/user-guide/migration/flyte-2/new-in-flyte-2/ |
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via `flyte-mcp` server |
## Migration Cheat Sheet
| Flyte 1 | Flyte 2 |
|---|---|
| `ImageSpec(name=..., packages=[...])` | `flyte.Image.from_debian_base().with_pip_packages(...)` |
| `@task(container_image=..., requests=..., cache=...)` | Set `image`, `resources`, `cache` once on `flyte.TaskEnvironment`, then `@env.task` |
| `Resources(cpu=..., mem=...)` | `flyte.Resources(cpu=..., memory=...)` (note `mem` becomes `memory`) |
| `Resources(gpu="1")` + `accelerator=T4` | `flyte.Resources(gpu="T4:1")` |
| `FlyteFile` / `FlyteFile(path=...)` | `flyte.io.File` / `await File.from_local(...)` |
| `model_file.download()` | `await model_file.download()` |
| `current_context().working_directory` | `os.getcwd()` |
| `@workflow` | An orchestrating `@env.task` (plain `async` Python) |
| `map_task(fn)(x=xs)` | `await asyncio.gather(*[fn(x) for x in xs])` |
| A "pick the best" task | Plain Python after `gather` |
## Small model training (scikit-learn / XGBoost)
Train a model, persist it as a `File`, and evaluate it. Image, resources, and caching move to the `TaskEnvironment`; `FlyteFile` becomes `flyte.io.File`.
### Flyte 1
```python
import os
import joblib
from flytekit import task, workflow, ImageSpec, Resources, current_context
from flytekit.types.file import FlyteFile
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier
image = ImageSpec(
name="xgb-image",
packages=["xgboost", "scikit-learn", "joblib"],
)
@task(container_image=image, requests=Resources(cpu="2", mem="4Gi"))
def train_model(n_estimators: int, max_depth: int) -> FlyteFile:
data = load_breast_cancer()
X_train, _, y_train, _ = train_test_split(data.data, data.target, random_state=42)
model = XGBClassifier(n_estimators=n_estimators, max_depth=max_depth)
model.fit(X_train, y_train)
model_path = os.path.join(current_context().working_directory, "model.json")
joblib.dump(model, model_path)
return FlyteFile(path=model_path)
@task(container_image=image)
def evaluate(model_file: FlyteFile) -> float:
model = joblib.load(model_file.download())
data = load_breast_cancer()
_, X_test, _, y_test = train_test_split(data.data, data.target, random_state=42)
return float(model.score(X_test, y_test))
@workflow
def main(n_estimators: int, max_depth: int) -> float:
model = train_model(n_estimators=n_estimators, max_depth=max_depth)
return evaluate(model_file=model)
```
### Flyte 2
```python
import os
import joblib
import flyte
from flyte.io import File
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier
env = flyte.TaskEnvironment(
name="train_xgboost",
image=flyte.Image.from_debian_base().with_pip_packages(
"xgboost", "scikit-learn", "joblib"
),
resources=flyte.Resources(cpu="2", memory="4Gi"),
)
@env.task
async def train_model(n_estimators: int, max_depth: int) -> File:
data = load_breast_cancer()
X_train, _, y_train, _ = train_test_split(data.data, data.target, random_state=42)
model = XGBClassifier(n_estimators=n_estimators, max_depth=max_depth)
model.fit(X_train, y_train)
model_path = os.path.join(os.getcwd(), "model.json")
joblib.dump(model, model_path)
return await File.from_local(model_path)
@env.task
async def evaluate(model_file: File) -> float:
local_path = await model_file.download()
model = joblib.load(local_path)
data = load_breast_cancer()
_, X_test, _, y_test = train_test_split(data.data, data.target, random_state=42)
return float(model.score(X_test, y_test))
@env.task
async def main(n_estimators: int, max_depth: int) -> float:
model = await train_model(n_estimators, max_depth)
return await evaluate(model)
```
## Hyperparameter optimization
Fan out one training run per hyperparameter, then pick the best. In Flyte 1 the grid search runs through `map_task` and the "pick the best" step must itself be a task. In Flyte 2 you `gather` the runs and select the winner in plain Python.
### Flyte 1
```python
from flytekit import task, workflow, map_task
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
@task
def get_grid() -> list[int]:
return [2, 4, 8, 16]
@task
def train_eval(max_depth: int) -> float:
data = load_iris()
model = RandomForestClassifier(max_depth=max_depth, random_state=42)
scores = cross_val_score(model, data.data, data.target, cv=3)
return float(scores.mean())
@task
def best_score(scores: list[float]) -> float:
return max(scores)
@workflow
def main() -> float:
grid = get_grid()
# Fan out one training run per hyperparameter value.
scores = map_task(train_eval)(max_depth=grid)
return best_score(scores=scores)
```
### Flyte 2
```python
import asyncio
import flyte
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
env = flyte.TaskEnvironment(
name="hpo",
image=flyte.Image.from_debian_base().with_pip_packages("scikit-learn"),
)
@env.task
async def train_eval(max_depth: int) -> float:
data = load_iris()
model = RandomForestClassifier(max_depth=max_depth, random_state=42)
scores = cross_val_score(model, data.data, data.target, cv=3)
return float(scores.mean())
@env.task
async def main() -> dict:
grid = [2, 4, 8, 16]
# Fan out one training run per hyperparameter value...
scores = await asyncio.gather(*[train_eval(d) for d in grid])
# ...then pick the best in plain Python (impossible in a Flyte 1 workflow).
best_idx = max(range(len(scores)), key=lambda i: scores[i])
return {"best_max_depth": grid[best_idx], "best_score": scores[best_idx]}
```
## Large model training (deep learning)
GPU configuration moves to the `TaskEnvironment`: the Flyte 1 `Resources(gpu="1")` plus a separate `accelerator=T4` become a single `gpu="T4:1"` string on `flyte.Resources`.
### Flyte 1
```python
from flytekit import task, workflow, ImageSpec, Resources
from flytekit.extras.accelerators import T4
import torch
import torch.nn as nn
image = ImageSpec(
name="dl-image",
packages=["torch"],
)
@task(
container_image=image,
requests=Resources(cpu="4", mem="16Gi", gpu="1"),
accelerator=T4,
)
def train(epochs: int) -> float:
device = "cuda" if torch.cuda.is_available() else "cpu"
model = nn.Linear(10, 1).to(device)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
loss_fn = nn.MSELoss()
X = torch.randn(128, 10, device=device)
y = torch.randn(128, 1, device=device)
loss = torch.tensor(0.0)
for _ in range(epochs):
optimizer.zero_grad()
loss = loss_fn(model(X), y)
loss.backward()
optimizer.step()
return float(loss.item())
@workflow
def main(epochs: int) -> float:
return train(epochs=epochs)
```
### Flyte 2
```python
import flyte
import torch
import torch.nn as nn
# GPU type and count go in a single "T4:1"-style string. For multi-node
# distributed training, wrap the training task with the torch elastic plugin.
env = flyte.TaskEnvironment(
name="train_deep_learning",
image=flyte.Image.from_debian_base().with_pip_packages("torch"),
resources=flyte.Resources(cpu="4", memory="16Gi", gpu="T4:1"),
)
@env.task
async def train(epochs: int) -> float:
device = "cuda" if torch.cuda.is_available() else "cpu"
model = nn.Linear(10, 1).to(device)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
loss_fn = nn.MSELoss()
X = torch.randn(128, 10, device=device)
y = torch.randn(128, 1, device=device)
loss = torch.tensor(0.0)
for _ in range(epochs):
optimizer.zero_grad()
loss = loss_fn(model(X), y)
loss.backward()
optimizer.step()
return float(loss.item())
@env.task
async def main(epochs: int) -> float:
return await train(epochs)
```
For multi-node distributed training (PyTorch elastic, etc.), wrap the training task with the torch elastic plugin. See the Resources docs and plugin integrations at https://www.union.ai/docs/v2/flyte/user-guide/task-configuration/resources.
## Batch inference
Load a trained model once and score many batches in parallel. `map_task` with a `partial`-bound model becomes `asyncio.gather` over the batches, reusing the same model reference.
### Flyte 1
```python
import os
from functools import partial
import joblib
from flytekit import task, workflow, map_task, ImageSpec, current_context
from flytekit.types.file import FlyteFile
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
image = ImageSpec(name="inference-image", packages=["scikit-learn", "joblib"])
@task(container_image=image)
def train_model() -> FlyteFile:
data = load_iris()
model = RandomForestClassifier().fit(data.data, data.target)
model_path = os.path.join(current_context().working_directory, "model.joblib")
joblib.dump(model, model_path)
return FlyteFile(path=model_path)
@task(container_image=image)
def get_batches() -> list[list[list[float]]]:
data = load_iris()
rows = data.data.tolist()
# Split the rows into batches of 30.
return [rows[i : i + 30] for i in range(0, len(rows), 30)]
@task(container_image=image)
def score_batch(model_file: FlyteFile, batch: list[list[float]]) -> list[int]:
model = joblib.load(model_file.download())
return [int(p) for p in model.predict(batch)]
@workflow
def main() -> list[list[int]]:
model = train_model()
batches = get_batches()
return map_task(partial(score_batch, model_file=model))(batch=batches)
```
### Flyte 2
```python
import asyncio
import os
import joblib
import flyte
from flyte.io import File
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
env = flyte.TaskEnvironment(
name="batch_inference",
image=flyte.Image.from_debian_base().with_pip_packages("scikit-learn", "joblib"),
)
@env.task
async def train_model() -> File:
data = load_iris()
model = RandomForestClassifier().fit(data.data, data.target)
model_path = os.path.join(os.getcwd(), "model.joblib")
joblib.dump(model, model_path)
return await File.from_local(model_path)
@env.task
async def score_batch(model_file: File, batch: list[list[float]]) -> list[int]:
local_path = await model_file.download()
model = joblib.load(local_path)
return [int(p) for p in model.predict(batch)]
@env.task
async def main() -> list[list[int]]:
model = await train_model()
rows = load_iris().data.tolist()
batches = [rows[i : i + 30] for i in range(0, len(rows), 30)]
# Score every batch in parallel, reusing the same model reference.
coros = [score_batch(model, batch) for batch in batches]
return list(await asyncio.gather(*coros))
```
## A complete example: end-to-end ML pipeline
Putting it together — a load / train / evaluate pipeline shows the image, resources, caching, file I/O, and orchestration changes in one place. Image, resources, and cache are set **once** on the `TaskEnvironment`, and the "workflow" is just an orchestrating task.
### Flyte 1
```python
import os
import joblib
import pandas as pd
from flytekit import task, workflow, ImageSpec, Resources, current_context
from flytekit.types.file import FlyteFile
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
image = ImageSpec(
name="ml-image",
packages=["pandas", "scikit-learn", "joblib"],
)
@task(
container_image=image,
requests=Resources(cpu="2", mem="4Gi"),
cache=True,
cache_version="1.0",
)
def load_data() -> pd.DataFrame:
data = load_iris(as_frame=True)
df = data.frame
df["species"] = data.target
return df
@task(container_image=image)
def train_model(data: pd.DataFrame) -> FlyteFile:
model = RandomForestClassifier()
X = data.drop("species", axis=1)
y = data["species"]
model.fit(X, y)
model_path = os.path.join(current_context().working_directory, "model.joblib")
joblib.dump(model, model_path)
return FlyteFile(path=model_path)
@task(container_image=image)
def evaluate(model_file: FlyteFile, data: pd.DataFrame) -> float:
model = joblib.load(model_file.download())
X = data.drop("species", axis=1)
y = data["species"]
return float(model.score(X, y))
@workflow
def main() -> float:
data = load_data()
model = train_model(data=data)
return evaluate(model_file=model, data=data)
```
### Flyte 2
```python
import os
import joblib
import pandas as pd
import flyte
from flyte.io import File
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
# Image, resources, and cache are set once on the TaskEnvironment.
env = flyte.TaskEnvironment(
name="ml_pipeline",
image=flyte.Image.from_debian_base().with_pip_packages(
"pandas", "scikit-learn", "joblib"
),
resources=flyte.Resources(cpu="2", memory="4Gi"),
cache="auto",
)
@env.task
async def load_data() -> pd.DataFrame:
data = load_iris(as_frame=True)
df = data.frame
df["species"] = data.target
return df
@env.task
async def train_model(data: pd.DataFrame) -> File:
model = RandomForestClassifier()
X = data.drop("species", axis=1)
y = data["species"]
model.fit(X, y)
model_path = os.path.join(os.getcwd(), "model.joblib")
joblib.dump(model, model_path)
return await File.from_local(model_path)
@env.task
async def evaluate(model_file: File, data: pd.DataFrame) -> float:
local_path = await model_file.download()
model = joblib.load(local_path)
X = data.drop("species", axis=1)
y = data["species"]
return float(model.score(X, y))
# The "workflow" is just an orchestrating task.
@env.task
async def main() -> float:
data = await load_data()
model = await train_model(data)
return await evaluate(model, data)
```
## New in Flyte 2
Flyte 1 was a batch orchestration system: everything ran as a finite DAG that started, did work, and finished. Flyte 2 keeps all of that and adds long-running services, high-throughput batch inference, and sandboxed code execution — so the same project that trains your model can also serve it, host a dashboard, saturate a GPU, or safely run LLM-generated code. There is no v1 counterpart to migrate here; these are net-new capabilities that your migrated training code unlocks. For greenfield authoring of these, see the `flyte-sdk-app` and `flyte-sdk-agent` skills.
### Real-time inference and model serving
Instead of scoring a batch and exiting, you can stand up an always-on REST endpoint from a `FastAPIAppEnvironment` and deploy it with `flyte.deploy`. The app can load a model artifact produced by one of your migrated training tasks.
```python
app = FastAPI(title="ML Model API")
# Define request/response models
class PredictionRequest(BaseModel):
feature1: float
feature2: float
feature3: float
class PredictionResponse(BaseModel):
prediction: float
probability: float
# Load model (you would typically load this from storage)
model = None
@asynccontextmanager
async def lifespan(app: FastAPI):
global model
model_path = os.getenv("MODEL_PATH", "/app/models/model.joblib")
# In production, load from your storage
if os.path.exists(model_path):
with open(model_path, "rb") as f:
model = joblib.load(f)
yield
@app.post("/predict", response_model=PredictionResponse)
async def predict(request: PredictionRequest):
# Make prediction
# prediction = model.predict([[request.feature1, request.feature2, request.feature3]])
# Dummy prediction for demo
prediction = 0.85
probability = 0.92
return PredictionResponse(
prediction=prediction,
probability=probability,
)
env = FastAPIAppEnvironment(
name="ml-model-api",
app=app,
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"fastapi",
"uvicorn",
"scikit-learn",
"pydantic",
"joblib",
),
parameters=[
flyte.app.Parameter(
name="model_file",
value=flyte.io.File.from_existing_remote("s3://bucket/models/model.joblib"),
mount="/app/models",
env_var="MODEL_PATH",
),
],
resources=flyte.Resources(cpu=2, memory="2Gi"),
requires_auth=False,
)
```
For serving large language models, the `flyteplugins-vllm` integration gives you a production-grade vLLM server (with autoscaling to zero) via `VLLMAppEnvironment`. Any web app — a Streamlit dashboard, a Gradio demo, a Flask backend — runs as a `flyte.app.AppEnvironment` that you configure with image, resources, port, autoscaling, and a custom subdomain, then `flyte.serve`.
### Dynamic batching for GPU inference
For in-process batch inference, `DynamicBatcher` from `flyte.extras` keeps an expensive GPU saturated: async producers load and preprocess data concurrently while a single consumer feeds the model in optimally-sized batches, with built-in backpressure. This replaces the Flyte 1 pattern of standing up a separate inference server just to get request batching.
```python
import asyncio
from flyte.extras import DynamicBatcher
async with DynamicBatcher(
process_fn=run_inference, # takes a batch, returns results in the same order
target_batch_cost=1000, # cost budget per batch
max_batch_size=64, # hard cap on records per batch
batch_timeout_s=0.05, # max wait before dispatching a partial batch
) as batcher:
futures = [await batcher.submit(record) for record in records]
results = await asyncio.gather(*futures)
```
`submit()` is non-blocking and returns a `Future`; when the queue is full it applies backpressure automatically. See the batch inference docs for `TokenBatcher` (token-aware LLM batching).
### Sandboxed code execution
`flyte.sandbox.create()` runs arbitrary Python code or shell commands inside an ephemeral, single-use Docker container — built on demand from declared dependencies, executed once, then discarded. Only declared inputs go in and only declared outputs come back, which makes it the safe way to run untrusted code, most importantly code generated by an LLM.
```python
# sandbox_environment provides the base runtime for code sandboxes.
# Include it in depends_on so the sandbox runtime is available when tasks execute.
env = flyte.TaskEnvironment(
name="sandbox-demo",
image=flyte.Image.from_debian_base(name="sandbox-demo"),
depends_on=[sandbox_environment],
)
# Auto-IO mode: pure computation. The code string runs in an isolated sandbox;
# only the declared inputs go in and only the declared outputs come back.
sum_sandbox = flyte.sandbox.create(
name="sum-to-n",
code="total = sum(range(n + 1)) if conditional else 0",
inputs={"n": int, "conditional": bool},
outputs={"total": int},
)
```
Call it from a task with `await sum_sandbox.run.aio(n=10, conditional=True)`. This also powers **code mode** (programmatic tool calling), where an agent writes a whole program instead of emitting one tool call at a time.
## Anti-Patterns
1. **Don't keep `@task` / `@workflow` per-task config** — move `image`, `resources`, and `cache` onto a single `flyte.TaskEnvironment` and decorate with `@env.task`.
2. **Don't leave a separate "pick the best" task** — after `asyncio.gather`, select the winner in plain Python inside the orchestrating task.
3. **Don't carry `map_task` + `partial` into v2** — fan out with `asyncio.gather` over coroutines, reusing the same model reference.
4. **Don't split GPU type and count** — replace `Resources(gpu="1")` + `accelerator=T4` with a single `gpu="T4:1"` string on `flyte.Resources`.
5. **Don't use `mem=` or `current_context().working_directory`** — use `memory=` on `flyte.Resources` and `os.getcwd()` for local paths.
6. **Don't forget `await`** — `File.from_local`, `download`, and task calls are all async in v2.
7. **Don't hand-roll a serving container or a request-batching server** — use a `FastAPIAppEnvironment` / `AppEnvironment` for serving and `DynamicBatcher` for GPU batching.
8. **Don't run untrusted or LLM-generated code inline** — use `flyte.sandbox.create()` with `sandbox_environment` in `depends_on`.
flyte-migrate-slurm25.6 KB
---
name: flyte-migrate-slurm
description: Migrates Slurm (sbatch/srun) HPC workloads to Flyte 2 (the flyte Python SDK) — job scripts become typed tasks, `#SBATCH` pragmas become TaskEnvironment config, job arrays become flyte.map, and multi-node training becomes a clustered task environment. Use when porting an HPC or supercomputer cluster workload off Slurm, translating `#SBATCH` pragmas, or replacing sbatch chains, job arrays, and module load with Flyte. Trigger words are sbatch, srun, SLURM, `#SBATCH`, HPC, job array, partition, squeue, sinfo, module load, scancel, supercomputer, cluster migration.
---
# Slurm to Flyte 2 Migration Skill
Slurm schedules jobs. Somewhere along the way ML work stopped being jobs and became pipelines — a data prep step, a training step, an eval step, a sweep over configs, each with different hardware and failure characteristics. A Slurm job is a bash script with `#SBATCH` pragmas at the top; in Flyte 2 that script becomes a typed Python function decorated with `@env.task`, a pipeline is just a task that calls other tasks, and control flow is plain Python. This skill maps the Slurm surface area onto the Flyte 2 SDK and is honest about the places where Slurm still wins.
## Grounding References
| Resource | URL |
| ----------------- | ---------------------------------------------------------------- |
| Migration guide | https://www.union.ai/docs/v2/flyte/user-guide/migration/flyte-2/ |
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via `flyte-mcp` server |
## Three mental shifts
Almost every point of friction for a Slurm migrant traces back to one of these three.
### 1. Modules become images
On Slurm you `module load cuda/12.1`, activate a venv on NFS, and hope the login node and the compute node agree. In Flyte 2 the environment is declared once, in Python, as a `flyte.Image`. There is no Dockerfile to write: images are built remotely and content-hashed, so an unchanged spec is a cache hit and a changed one rebuilds automatically.
```python
image = (
flyte.Image.from_debian_base(python_version=(3, 12))
.with_pip_packages("torch", "transformers", "datasets")
.with_env_vars({"HF_HUB_ENABLE_HF_TRANSFER": "1"})
)
```
### 2. The shared filesystem becomes explicit data
There is no shared `/home` or `/scratch` that every node sees. Instead you pass `flyte.io.File` and `flyte.io.Dir` — typed references to object storage that stream rather than copy. This is more typing than `/scratch/$USER/run17/ckpt.pt`, and in exchange you get lineage for free: every input and output of every task is recorded, so "which dataset produced this checkpoint" is a question the system answers instead of a question you grep for. If you genuinely need a parallel filesystem (Lustre, GPFS, FSx), mount it through a CSI driver and a `flyte.PodTemplate` — that path stays open.
### 3. Job scripts become functions
`sbatch` chains held together by `--dependency=afterok`, sentinel files on NFS, and a cron job that polls for them all collapse into ordinary Python: call a function, await it, branch on the result. Binaries you can't or won't rewrite in Python still run — as container tasks with typed inputs and outputs.
## Migration Cheat Sheet
| Slurm | Flyte 2 |
| ------------------------------------------------- | ----------------------------------------------------------- |
| `sbatch train.sh` | `flyte run train.py main` |
| `srun --pty python train.py` (interactive) | `flyte run --local train.py main`, or a devbox |
| `#SBATCH --gres=gpu:a100:8` | `flyte.Resources(gpu="A100:8")` |
| `#SBATCH --cpus-per-task=16 --mem=64G` | `flyte.Resources(cpu=16, memory="64Gi")` |
| `#SBATCH --tmp=100G` | `flyte.Resources(disk="100Gi")` |
| `#SBATCH --array=0-999` | `flyte.map(step, range(1000), concurrency=200)` |
| `#SBATCH --nodes=4 --ntasks-per-node=8` | `ClusteredTaskEnvironment(replicas=4, nproc_per_node=8)` |
| `#SBATCH --partition=gpu --qos=high` | `queue="gpu-high"` (queue must exist in cluster config) |
| `#SBATCH --requeue` | `retries=3` plus `interruptible=True` for spot |
| `#SBATCH --time=04:00:00` | `timeout=timedelta(hours=4)` |
| `#SBATCH --begin=...` / a crontab entry | `flyte.Trigger(...)` with `flyte.Cron(...)` |
| `#SBATCH --dependency=afterok:$JOBID` | Plain Python — call the next task after the first returns |
| `module load cuda && source venv/bin/activate` | `flyte.Image.from_debian_base().with_pip_packages(...)` |
| `$SLURM_PROCID`, `$SLURM_NNODES`, `$SLURM_NTASKS` | `flyte.ctx().rank`, `.nnodes`, `.world_size` |
| `$SLURM_ARRAY_TASK_ID` | The argument you mapped over |
| `/scratch/$USER/data.parquet` | `flyte.io.File` / `flyte.io.Dir` passed between tasks |
| `squeue`, `sacct` | `flyte get run`, `flyte get logs`, the UI |
| `scancel <jobid>` | `flyte stop`, or abort from the UI |
| `srun --pty bash` / `ssh node042` | `flyte run --debug ...`, or `flyte debug <run-name>` (beta) |
## The job script becomes a task
This is the canonical translation. Everything above the `module load` line is configuration and moves onto the `TaskEnvironment` or the task decorator; everything below it is the function body.
### Slurm
```bash
#!/bin/bash
#SBATCH --job-name=train
#SBATCH --partition=gpu
#SBATCH --gres=gpu:a100:8
#SBATCH --cpus-per-task=16
#SBATCH --mem=64G
#SBATCH --time=04:00:00
#SBATCH --requeue
module load cuda/12.1
source ~/venvs/train/bin/activate
srun python train.py --lr 3e-4
```
### Flyte 2
```python
from datetime import timedelta
import flyte
from flyte.io import File
env = flyte.TaskEnvironment(
name="training",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages("torch"),
resources=flyte.Resources(cpu=16, memory="64Gi", gpu="A100:8"),
)
@env.task(retries=3, timeout=timedelta(hours=4))
async def train(lr: float = 3e-4) -> File:
import torch
model = build_model().cuda()
optimizer = torch.optim.AdamW(model.parameters(), lr=lr)
... # the same training loop that was in train.py
torch.save(model.state_dict(), "model.pt")
return await File.from_local("model.pt")
```
```bash
flyte run train.py train --lr 3e-4 # remote (the sbatch equivalent)
flyte run --local train.py train # same code, on your laptop
flyte deploy train.py env # register the environment + triggers
```
The `#SBATCH --time` line has a richer counterpart than a single number. `timeout=` accepts a `timedelta`, an int number of seconds, or a `flyte.Timeout` object that separates the budgets Slurm collapses into one:
```python
@env.task(
retries=2,
timeout=flyte.Timeout(
max_runtime=timedelta(hours=4), # per-attempt wall clock
max_queued_time=timedelta(minutes=30), # fail fast if capacity never appears
deadline=timedelta(hours=10), # absolute budget across all attempts
),
)
async def train(...) -> File: ...
```
## Job arrays and sweeps
A Slurm array plus a results-collection script is two artifacts held together by a filename convention. In Flyte 2 the fan-out and the reduction live in the same function.
### Slurm
```bash
#!/bin/bash
#SBATCH --array=0-63
#SBATCH --gres=gpu:1
CONFIG=$(sed -n "$((SLURM_ARRAY_TASK_ID + 1))p" configs.txt)
python train.py --config "$CONFIG" --out "/scratch/$USER/sweep/$SLURM_ARRAY_TASK_ID.json"
# ...then a separate job, after the array drains, to read 64 JSON files and pick a winner.
```
### Flyte 2
```python
import asyncio
import flyte
env = flyte.TaskEnvironment(
name="sweep",
image=flyte.Image.from_debian_base().with_pip_packages("torch"),
resources=flyte.Resources(cpu=8, memory="32Gi", gpu="L4:1"),
)
@env.task
async def train_one(lr: float, batch_size: int) -> float:
... # returns a validation metric
return val_loss
@env.task
async def sweep() -> dict:
configs = [(lr, bs) for lr in (1e-4, 3e-4, 1e-3) for bs in (16, 32, 64)]
# Small fan-out: gather is the most direct translation of an array job.
losses = await asyncio.gather(*[train_one(lr, bs) for lr, bs in configs])
# Picking the winner is plain Python — no second job, no sentinel files.
best = min(range(len(losses)), key=lambda i: losses[i])
return {"lr": configs[best][0], "batch_size": configs[best][1], "loss": losses[best]}
```
For a 10,000-element array, `flyte.map` gives you a bounded fan-out with a concurrency cap — the `%` throttle in `--array=0-9999%200`:
```python
@env.task
async def big_sweep(n: int = 10_000) -> float:
# flyte.map returns a generator; wrap it in list() to materialize.
losses = list(flyte.map(train_one_indexed, range(n), concurrency=200))
return min(losses)
```
## Job dependencies become function calls
`--dependency=afterok:$JOBID` is the construct that most often turns into a shell script full of `sbatch --parsable` and `awk`. It has no counterpart in Flyte 2 because it doesn't need one: awaiting a task _is_ the dependency, and the value it returns _is_ the handoff.
### Slurm
```bash
PREP=$(sbatch --parsable prep.sh)
TRAIN=$(sbatch --parsable --dependency=afterok:$PREP train.sh)
sbatch --dependency=afterok:$TRAIN eval.sh
```
### Flyte 2
```python
from flyte.io import Dir, File
@env.task
async def pipeline(raw: Dir) -> float:
prepared = await prep(raw) # runs first
model = await train(prepared) # waits for prep, gets its output directly
return await evaluate(model, prepared)
```
Branching that Slurm can't express at all — `afterok` on one job but `afternotok` on another, or "promote only if the eval didn't regress" — is just `try`/`except` and `if`:
```python
@env.task
async def pipeline_with_gate(raw: Dir) -> str:
prepared = await prep(raw)
try:
model = await train(prepared)
except Exception:
model = await train_with_fallback_config(prepared)
if await evaluate(model, prepared) < 0.85:
return "held back"
await promote(model)
return "promoted"
```
## Data: from `/scratch` to typed references
The Slurm version writes to a path both jobs happen to agree on. The Flyte version passes a value.
### Slurm
```bash
# prep.sh
python prep.py --in /scratch/$USER/raw --out /scratch/$USER/prepared
# train.sh — coupled to prep.sh only by this string
python train.py --data /scratch/$USER/prepared --ckpt /scratch/$USER/ckpt.pt
```
### Flyte 2
```python
import flyte
from flyte.io import Dir, File
@env.task
async def prep(raw: Dir) -> Dir:
local = await raw.download()
... # write outputs into ./prepared
return await Dir.from_local("prepared")
@env.task
async def train(prepared: Dir) -> File:
local = await prepared.download() # streams from object storage to this pod
...
return await File.from_local("ckpt.pt")
```
Scratch space _inside_ a task is still just the local filesystem — request it with `flyte.Resources(disk="100Gi")` and use `os.getcwd()`. What changes is that anything another task needs must leave as a typed output. If a parallel filesystem is non-negotiable (a dataset too large or too latency-sensitive to stream), mount it with a CSI driver through `pod_template=flyte.PodTemplate.from_spec(pod_spec_with_lustre_pvc)` on the `TaskEnvironment`.
## Multi-node training
`--nodes=4 --ntasks-per-node=8` with `srun` as the launcher becomes a `ClusteredTaskEnvironment`. It launches its replicas as a single Kubernetes JobSet, runs `torchrun` rendezvous across them, and exposes the standard `RANK` / `WORLD_SIZE` / `MASTER_ADDR` environment variables — so training code written for `torchrun` needs no changes. The same values are on `flyte.ctx()`.
### Slurm
```bash
#!/bin/bash
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=8
#SBATCH --gres=gpu:h100:8
export MASTER_ADDR=$(scontrol show hostnames "$SLURM_JOB_NODELIST" | head -n1)
srun python -m torch.distributed.run --nnodes=4 --nproc_per_node=8 pretrain.py
```
### Flyte 2
```python
import flyte
from flyte.clustered import ClusteredTaskEnvironment, ClusterFailurePolicy, TorchRun
env = ClusteredTaskEnvironment(
name="pretrain",
image=image,
resources=flyte.Resources(cpu=16, memory="64Gi", gpu="H100:8", shm="auto"),
replicas=4, # pods == nodes
nproc_per_node=8, # processes per pod => world size 32
runtime=TorchRun(rdzv_backend="c10d"), # "static" relies on JobSet restarts instead
failure_policy=ClusterFailurePolicy(max_restarts=2, restart_on_host_maintenance=True),
)
@env.task
async def pretrain(steps: int = 10_000) -> File:
import torch
import torch.distributed as dist
ctx = flyte.ctx()
torch.cuda.set_device(ctx.local_rank or 0)
dist.init_process_group(backend="nccl") # torchrun already set RANK/WORLD_SIZE/MASTER_ADDR
print(f"rank {ctx.rank}/{ctx.world_size} on node {ctx.node_rank}/{ctx.nnodes}", flush=True)
... # the same DDP/FSDP loop you ran under srun
dist.barrier()
dist.destroy_process_group()
# Only rank 0 has anything to return.
return await File.from_local("ckpt.pt")
```
`restart_on_host_maintenance=True` is the piece with no Slurm analogue: a node reclaimed by the cloud provider (spot reclaim, host maintenance, drain) restarts the whole set _for free_, leaving the `max_restarts` budget untouched, so an unreliable cluster can't burn the budget you reserved for real bugs.
A clustered task is a worker, not a driver — it cannot launch subtasks. To compose distributed steps, orchestrate from a plain `TaskEnvironment` that declares `depends_on=[clustered_env]`:
```python
driver_env = flyte.TaskEnvironment(
name="driver",
image=image,
resources=flyte.Resources(cpu=1, memory="1Gi"),
depends_on=[env], # without this, awaiting pretrain() fails on image-cache lookup
)
@driver_env.task
async def main(steps: int = 10_000) -> float:
ckpt = await pretrain(steps) # JobSet #1
return await evaluate(ckpt) # JobSet #2
```
Ephemeral, per-task Ray / Spark / Dask clusters (via the `flyteplugins-ray`, `flyteplugins-spark`, and `flyteplugins-dask` integrations) replace the long-lived clusters an HPC site usually stands up by hand — they exist for the duration of the task and are torn down with it.
## Fault tolerance: `--requeue`, decomposed
`#SBATCH --requeue` restarts the job from the top and hopes you wrote your own checkpoint logic. Flyte 2 splits that into four independent mechanisms you compose.
**Retries, declaratively.** `retries=3` on the decorator, or a `RetryStrategy` when you want backoff:
```python
@env.task(
retries=flyte.RetryStrategy(
count=4,
backoff=flyte.Backoff(base=timedelta(seconds=10), factor=2.0, cap=timedelta(minutes=5)),
),
)
async def flaky_download(url: str) -> File: ...
```
**Spot capacity, safely.** `interruptible=True` says the task may run on preemptible instances. Preemptions are tracked as _system_ failures and do not consume your retry budget, and the final attempt falls back to on-demand — so `interruptible=True, retries=2` means two spot attempts and one guaranteed on-demand attempt. Set it on the environment, override it per task:
```python
env = flyte.TaskEnvironment(name="sweep", image=image, interruptible=True)
@env.task(interruptible=False) # the one step you don't want preempted
async def publish(model: File) -> str: ...
```
**Checkpoints that survive node changes.** Checkpoints go to object storage, not `/scratch`, so a retry resumes on whatever node it lands on. No shared filesystem required:
```python
@env.task(retries=3)
async def train(n_epochs: int = 100) -> int:
checkpoint = flyte.ctx().checkpoint
path = await checkpoint.load() # None on the first attempt
start = int(path.read_bytes()) if path else 0
for epoch in range(start, n_epochs):
...
await checkpoint.save(f"{epoch + 1}".encode())
return n_epochs
```
**Durable function calls.** `@flyte.trace` records the result of an individual function call inside a task. On a retry, recorded calls are skipped instead of re-executed — the granularity Slurm has no way to express:
```python
@flyte.trace
async def call_expensive_api(prompt: str) -> str:
... # on retry, an already-recorded call replays instead of re-running
```
**Caching.** Task-level caching is keyed on the code and the inputs, so rerunning a twelve-hour pipeline after fixing step nine starts at step nine:
```python
env = flyte.TaskEnvironment(name="etl", image=image, cache="auto")
```
## Warm pools
Slurm feels fast at submit time because the allocation is already running — `srun` inside an existing allocation starts in milliseconds. `flyte.ReusePolicy` is the equivalent: containers stay warm between tasks, keeping in-memory state, so a model loaded once serves thousands of task invocations.
```python
env = flyte.TaskEnvironment(
name="scorer",
image=image,
resources=flyte.Resources(cpu=4, memory="16Gi", gpu="L4:1"),
reusable=flyte.ReusePolicy(
replicas=(2, 10), # autoscaling range; a bare int pins the count
concurrency=4, # concurrent tasks per replica (async tasks only)
idle_ttl=300, # shut the environment down after 5 idle minutes
),
)
```
The caveat is the same one that bites long-lived Slurm allocations: the process outlives the task. Treat module-level globals and caches deliberately, and don't let one task's state leak into the next.
## Existing binaries
A Fortran solver, a C++ simulator, a genomics tool — anything you're not rewriting runs as a `ContainerTask` with typed inputs and outputs. Inputs are staged into `input_data_dir`, and whatever the command writes into `output_data_dir` is read back as the declared types.
```python
from flyte.extras import ContainerTask
from flyte.io import File
align = ContainerTask(
name="align_reads",
image="quay.io/biocontainers/bwa:0.7.17",
resources=flyte.Resources(cpu=16, memory="64Gi"),
inputs={"reference": File, "reads": File},
outputs={"alignment": File},
input_data_dir="/var/inputs",
output_data_dir="/var/outputs",
file_input_layout="NAMED_DIR", # preserves original filenames + extensions
command=[
"/bin/sh", "-c",
"bwa mem /var/inputs/reference/* /var/inputs/reads/* > /var/outputs/alignment",
],
)
@env.task
async def main(reference: File, reads: File) -> File:
return await align(reference=reference, reads=reads)
```
## Scheduling, monitoring, and interactive work
`#SBATCH --begin=` and the crontab that wraps most recurring HPC work become a `flyte.Trigger` attached to the task and deployed with it:
```python
from datetime import datetime
nightly = flyte.Trigger(
name="nightly_retrain",
automation=flyte.Cron("0 2 * * *", timezone="America/Los_Angeles"),
inputs={"start_time": flyte.TriggerTime, "lr": 3e-4},
auto_activate=True,
)
@env.task(triggers=nightly)
async def retrain(start_time: datetime, lr: float) -> File: ...
```
For monitoring, `squeue` and `sacct` become `flyte get run` and `flyte get logs` (or the UI, which shows the pipeline structure rather than a flat job list):
```bash
flyte get run # like squeue
flyte get run <run_name> # detail for one run
flyte get logs <run_name> --attempt 0 # like sacct + tailing a slurm-*.out
flyte stop <run_name> # like scancel
```
`srun --pty bash` and `ssh node042` have two counterparts. `flyte run --debug` opens a browser-based VS Code session in the task pod:
```bash
flyte run --debug train.py train --lr 3e-4
```
```python
run = flyte.with_runcontext(debug=True).run(train, lr=3e-4)
print(run.get_debug_url())
```
SSH into the running task is available in beta (requires `flyteplugins-union`) and is closer to the muscle memory of `ssh` onto a compute node:
```bash
flyte debug <run-name> --write-config
ssh flyte-debug
```
## Migration order that works
Do not start with the thing Slurm does best.
1. **Pipeline-shaped work first** — data processing, evals, sweeps, batch inference, RL rollouts. These are multi-step, embarrassingly parallel, and failure-prone in boring ways, so composition, caching, and retries pay off on day one. They also exercise images and data plumbing on workloads where a bad hour costs little.
2. **Single-node training next** — one `TaskEnvironment`, one GPU resource string, checkpoints to object storage. At this point you've validated that your image builds, your data streams, and your logs are where you expect.
3. **Multi-node training last** — once images, data, and observability are proven. `ClusteredTaskEnvironment` is the piece with the most moving parts and the least tolerance for a half-migrated environment.
Nothing forces a big bang: Slurm and Flyte can run side by side indefinitely, and a Flyte task can `subprocess` out to `sbatch` during the overlap if you need a bridge.
## Gotchas
- **`flyte.map` returns a generator.** Wrap it in `list()` to materialize results.
- **`memory`, not `mem`.** And GPUs use a combined `"A100:8"` string — type and count together, not `--gres` plus a separate accelerator argument.
- **`shm="auto"` matters for PyTorch DataLoader.** The container default `/dev/shm` is tiny; multi-worker data loading will fail cryptically without it.
- **A clustered task cannot launch subtasks.** Orchestrate from a plain `TaskEnvironment` with `depends_on=[clustered_env]`, or you'll hit an image-cache lookup failure at runtime.
- **`nproc_per_node` must not exceed the GPU count.** `ClusteredTaskEnvironment` validates this locally and raises `ValueError` before anything is submitted.
- **Only rank 0 should return outputs.** Every replica runs the task body; have non-zero ranks return early or return a trivial value.
- **Retries have no platform cap.** Total attempts equal `retries + 1`, so audit any large values ported from a `--requeue` habit.
- **`interruptible=True` with zero retries runs on-demand.** The final attempt always falls back off spot, and a single attempt _is_ the final attempt.
- **Clustered tasks are torchrun-focused.** Classic MPI HPC codes (`mpirun`, tightly-coupled CFD, molecular dynamics) are not the target. Keep those on Slurm.
- **Gang scheduling and topology-aware placement are still maturing on Kubernetes.** Slurm's scheduler has decades of work behind co-scheduling N nodes on the same rack or fabric; the Kubernetes ecosystem is closing the gap but is not there.
- **Queues order work; they don't preempt it.** `queue="gpu-high"` is the nearest analogue to `--partition` plus `--qos`, but a lower-priority task already running is not interrupted. Queue names must exist in your cluster configuration, and the feature is platform-dependent — check what your deployment supports before designing around it.
- **Frontier scale is still Slurm's.** Hundreds of GPUs per job with explicit InfiniBand topology control is where Slurm keeps winning. Migrate the pipeline layer; be deliberate about the rest.
## Anti-Patterns
1. **Don't recreate `/scratch` as a hardcoded bucket path.** Pass `flyte.io.File` / `flyte.io.Dir` between tasks. Two tasks agreeing on a string is exactly the coupling you're migrating away from — and it forfeits the lineage you get for free.
2. **Don't port `#SBATCH` pragmas onto every task decorator.** Image, resources, cache, and interruptibility belong on a shared `flyte.TaskEnvironment`; override per task only where a task genuinely differs.
3. **Don't translate `--dependency=afterok` into a sentinel-file poll.** Await the task and use its return value.
4. **Don't keep the array-plus-collector split.** After `asyncio.gather` or `flyte.map`, reduce in plain Python inside the same task.
5. **Don't `srun` inside a task.** The task body _is_ the rank's process. For multi-node, use `ClusteredTaskEnvironment` and let torchrun do the launching.
6. **Don't checkpoint to a local path and expect a retry to find it.** Use `flyte.ctx().checkpoint` or write a `File` to object storage — a retry may land on a different node.
7. **Don't do heavy compute in an orchestrating task.** A task that calls other tasks is a driver pod; CPU-bound work between awaits stalls everything downstream.
8. **Don't lean on global state in a reusable environment.** `ReusePolicy` keeps the process alive across tasks — cache the model deliberately, and don't let mutable state leak between invocations.
9. **Don't migrate tightly-coupled MPI simulation first (or at all).** Start with pipeline-shaped work; leave the workloads Slurm is genuinely better at on Slurm.
## Related skills
- **`flyte-sdk-ml`** — greenfield ML authoring in Flyte 2 (training, HPO, inference) once the migration shape is clear.
- **`flyte-migrate`** — the entry point if you _also_ have Flyte 1 (`flytekit`) code to port.
- **`flyte-sdk-ship`** — image specs, dependency management, and reproducible builds, i.e. everything that replaces `module load`.
- **`flyte-sdk-app`** — serving and endpoints, for the step after training that Slurm never covered.
flyte-migrate-tasks-workflows8.83 KB
---
name: flyte-migrate-tasks-workflows
description: >-
Migrates Flyte 1 tasks and workflows to Flyte 2, where the @task, @workflow,
and @dynamic decorators collapse into a single @env.task on a
flyte.TaskEnvironment and a workflow becomes a task that calls other tasks.
Use when the user is migrating Flyte 1 tasks/workflows to Flyte 2. Trigger
words: migrate task, migrate workflow, @task, @workflow, @dynamic,
TaskEnvironment, env.task.
---
# Flyte 1 to 2 Migration: Tasks and Workflows
The biggest structural change in Flyte 2 is that everything is a task. The Flyte 1 `@task`, `@workflow`, and `@dynamic` decorators all collapse into a single `@env.task` on a `flyte.TaskEnvironment`, and a "workflow" is now just a task that calls other tasks. This skill covers the structural shift, sequential ordering without the `>>` operator, nested subworkflows, TaskEnvironment configuration basics, and the full `@task` parameter mapping.
## Grounding References
| Resource | URL |
|---|---|
| Migration guide | https://www.union.ai/docs/v2/flyte/user-guide/migration/flyte-2/tasks-and-workflows/ |
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via `flyte-mcp` server |
## The Structural Shift
In Flyte 1 you decorated units of work with `@task`, composed them with `@workflow`, and used `@dynamic` for runtime-generated graphs. In Flyte 2 you create one `flyte.TaskEnvironment` that carries the configuration, then decorate every function — leaf tasks and orchestrating "workflows" alike — with `@env.task`. There is no separate `@workflow` decorator: the entrypoint is just a task that calls other tasks.
## Hello World: Tasks and Workflows
A `@task` plus `@workflow` becomes two `@env.task`s, where the entrypoint task calls the others. Sequential calls are naturally ordered — no `>>` operator required.
### Flyte 1
```python
import flytekit
@flytekit.task
def say_hello(name: str) -> str:
return f"Hello, {name}!"
@flytekit.task
def to_upper(greeting: str) -> str:
return greeting.upper()
@flytekit.workflow
def main(name: str) -> str:
greeting = say_hello(name=name)
return to_upper(greeting=greeting)
```
### Flyte 2
```python
import flyte
env = flyte.TaskEnvironment(name="hello_world")
@env.task
def say_hello(name: str) -> str:
return f"Hello, {name}!"
@env.task
def to_upper(greeting: str) -> str:
return greeting.upper()
# The "workflow" is now just a task that calls other tasks.
@env.task
def main(name: str) -> str:
greeting = say_hello(name)
return to_upper(greeting)
```
Note that Flyte 2 task calls pass arguments positionally (`say_hello(name)`) rather than requiring keyword arguments as in Flyte 1 (`say_hello(name=name)`).
## Chaining and Ordering
In Flyte 1 you sometimes used `>>` to force ordering between tasks with no data dependency. In Flyte 2, sequential (synchronous) calls run in the order they are written, and `await`ing async tasks in sequence does the same. The `>>` operator is gone.
### Flyte 1
```python
from flytekit import task, workflow
@task
def clear_staging_table() -> None:
# Side effect only: truncate the staging table.
print("cleared staging table")
@task
def load_into_staging() -> None:
# Side effect only: load fresh rows into staging.
print("loaded staging table")
@task
def publish_to_prod() -> None:
# Side effect only: swap staging into the production table.
print("published to prod")
@workflow
def main() -> None:
clear = clear_staging_table()
load = load_into_staging()
publish = publish_to_prod()
# These tasks pass no data between them, so use the >> operator to force
# ordering: clear must finish before load, which must finish before publish.
clear >> load >> publish
```
### Flyte 2
```python
import flyte
env = flyte.TaskEnvironment(name="staging_publish")
@env.task
def clear_staging_table() -> None:
print("cleared staging table")
@env.task
def load_into_staging() -> None:
print("loaded staging table")
@env.task
def publish_to_prod() -> None:
print("published to prod")
# Sequential (synchronous) calls run in the order they're written, even when no
# data flows between them. The Flyte 1 `>>` ordering operator is gone.
@env.task
def main() -> None:
clear_staging_table()
load_into_staging()
publish_to_prod()
```
## Subworkflows
A `@workflow` invoked by another `@workflow` (for example, a reusable preprocessing pipeline) becomes a task that calls other tasks — nest them as deeply as you like.
### Flyte 1
```python
from flytekit import task, workflow
@task
def impute(value: float) -> float:
# Replace missing/negative sentinel values with 0.
return value if value >= 0 else 0.0
@task
def scale(value: float) -> float:
return value / 100.0
@workflow
def preprocess(value: float) -> float:
imputed = impute(value=value)
return scale(value=imputed)
@workflow
def main(raw_value: float) -> float:
return preprocess(value=raw_value)
```
### Flyte 2
```python
import flyte
env = flyte.TaskEnvironment(name="subworkflow")
@env.task
def impute(value: float) -> float:
# Replace missing/negative sentinel values with 0.
return value if value >= 0 else 0.0
@env.task
def scale(value: float) -> float:
return value / 100.0
# A preprocessing "subworkflow" is just a task that calls other tasks.
@env.task
def preprocess(value: float) -> float:
imputed = impute(value)
return scale(imputed)
@env.task
def main(raw_value: float) -> float:
return preprocess(raw_value)
```
## TaskEnvironment Configuration
The `TaskEnvironment` holds the configuration that Flyte 1 spread across the `@task` decorator. The task decorator can still override a few settings per-task.
```python
import flyte
env = flyte.TaskEnvironment(
name="my_env", # Required: unique name
image=flyte.Image.from_debian_base(), # Or a string, or "auto"
resources=flyte.Resources(
cpu="2",
memory="4Gi",
gpu="A100:1",
disk="10Gi",
),
env_vars={"LOG_LEVEL": "INFO"},
secrets=[flyte.Secret(key="api-key", as_env_var="API_KEY")],
cache="auto", # "auto", "override", "disable", or a Cache object
reusable=flyte.ReusePolicy(replicas=5, idle_ttl=60),
interruptible=True,
)
# The task decorator can override some settings:
@env.task(
short_name="my_task", # Display name
cache="disable", # Override cache
retries=3, # Retry count
timeout=3600, # Seconds or a timedelta
report=True, # Generate an HTML report
)
def my_task(x: int) -> int:
return x
```
## Parameter Mapping: `@task` to `TaskEnvironment` + `@env.task`
| Flyte 1 `@task` parameter | Flyte 2 location | Notes |
|---|---|---|
| `container_image` | `TaskEnvironment(image=...)` | Env-level only |
| `requests` | `TaskEnvironment(resources=...)` | Env-level only |
| `limits` | `TaskEnvironment(resources=...)` | Combined with requests (single value) |
| `environment` | `TaskEnvironment(env_vars=...)` | Env-level only |
| `secret_requests` | `TaskEnvironment(secrets=...)` | Env-level only |
| `cache` | Both | Can override at task level |
| `cache_version` | `flyte.Cache(version_override=...)` | In a `Cache` object |
| `retries` | `@env.task(retries=...)` | Task-level only |
| `timeout` | `@env.task(timeout=...)` | Task-level only |
| `interruptible` | Both | Can override at task level |
| `pod_template` | Both | Can override at task level |
| `deprecated` | N/A | Not in Flyte 2 |
| `docs` | `@env.task(docs=...)` | Task-level only |
For image, resource, secret, and caching detail, see the Task configuration migration page.
## Anti-Patterns
1. **Don't reach for `>>`** — the ordering operator is gone. Sequential synchronous calls already run in written order; `await` async tasks in sequence for the same effect.
2. **Don't look for a `@workflow` decorator** — there isn't one. The orchestrating entrypoint is just another `@env.task` that calls other tasks.
3. **Don't look for a `@dynamic` decorator** — dynamic graphs also collapse into ordinary `@env.task` functions that call other tasks at runtime.
4. **Don't put heavy compute in the orchestrating task** — keep the entrypoint task focused on calling other tasks; push CPU/GPU/memory-intensive work into leaf tasks whose resources you can tune per environment.
5. **Don't set image, resources, or secrets on the `@env.task` decorator** — those are env-level and belong on `TaskEnvironment`. Only per-task overrides like `retries`, `timeout`, `cache`, and `short_name` go on `@env.task`.
6. **Don't keep passing every argument by keyword** — Flyte 2 task calls accept positional arguments (`say_hello(name)`).
flyte-sdk-agent14.1 KB
---
name: flyte-sdk-agent
description: 'Builds durable agents with Flyte 2 — ReAct patterns, Plan-and-Execute, LangGraph integration, PydanticAI integration, OpenAI Agents SDK integration, agent memory, MCP tool integration, skills, and agent chat UI. Use when the user wants to build AI agents, implement ReAct loops, integrate agent frameworks, add agent memory, or build agent-powered workflows. Trigger words: "agent", "ReAct", "LangGraph", "PydanticAI", "OpenAI agents", "MCP", "tool calling", "memory", "skills", "agentic".'
---
# Flyte 2 SDK Agent Skill
Build durable, observable AI agents with Flyte 2.
## Grounding References
| Resource | URL |
|---|---|
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| CLI API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-cli/ |
| flyte-sdk source | https://github.com/flyteorg/flyte-sdk |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via the `flyte-cluster` and `flyte-docs` MCP servers |
**Ground unfamiliar APIs in real examples.** When unsure of a current Flyte 2 API, or for a pattern not shown below, and the `flyte-docs` search tools are available, search them first — by exact symbol (`TaskEnvironment`, `flyte.io.File`, `map_task`), since matching is literal substring, not semantic — then adapt a real example rather than inventing one, and cite the file or section you pulled it from. (Flyte 2 is not `flytekit`; priors are often wrong.)
## Pure Python Agents (No Framework)
### ReAct Pattern — Reason, Act, Observe
```python
import flyte
env = flyte.TaskEnvironment(
name="react-agent",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"openai", "boto3",
),
)
@env.task
async def think(observation: str) -> str:
"""LLM generates next action."""
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": f"Observation: {observation}\nWhat do you do next?"},
],
)
return response.choices[0].message.content
@env.task
async def act(thought: str) -> dict:
"""Parse thought and execute tool call."""
# Parse the thought to extract tool name and arguments
# Then call the appropriate tool
return {"tool": "search", "result": "search results..."}
@env.task
async def observe(result: dict) -> str:
"""Format tool result for the next reasoning step."""
return f"Tool {result['tool']} returned: {result['result']}"
@env.task
async def react_loop(initial_query: str, max_steps: int = 5) -> str:
"""ReAct loop: think → act → observe → think → ..."""
observation = initial_query
for i in range(max_steps):
thought = await think(observation)
if "FINAL_ANSWER" in thought:
return thought.split("FINAL_ANSWER:")[-1].strip()
result = await act(thought)
observation = await observe(result)
return "Max steps reached"
if __name__ == "__main__":
import asyncio
result = asyncio.run(react_loop("What is the weather in Tokyo?"))
print(result)
```
### Plan-and-Execute with Parallel Fan-out
```python
@env.task
async def plan(query: str) -> list[str]:
"""Generate a plan of sub-tasks."""
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4",
messages=[
{"role": "system", "content": "Break this query into sub-tasks."},
{"role": "user", "content": query},
],
)
# Parse response into list of sub-tasks
return response.choices[0].message.content.split("\n")
@env.task
async def execute_subtask(task: str) -> str:
"""Execute a single sub-task."""
...
return result
@env.task
async def synthesize(results: list[str]) -> str:
"""Synthesize results into a final answer."""
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4",
messages=[
{"role": "system", "content": "Synthesize these results."},
{"role": "user", "content": "\n".join(results)},
],
)
return response.choices[0].message.content
@env.task
async def plan_and_execute(query: str) -> str:
"""Plan sub-tasks, execute in parallel, synthesize results."""
tasks = await plan(query)
# Parallel execution of sub-tasks
results = await flyte.map(execute_subtask, tasks)
return await synthesize(results)
```
## Agent Framework Integrations
### LangGraph Agents
```python
import flyte
from langgraph.prebuilt import create_react_agent
from langchain_openai import ChatOpenAI
env = flyte.TaskEnvironment(
name="langgraph-agent",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"langgraph", "langchain-openai", "langchain",
),
)
@env.task
async def langgraph_agent(query: str) -> str:
"""Run a LangGraph agent as a Flyte task."""
agent = create_react_agent(
model=ChatOpenAI(model="gpt-4"),
tools=[search_tool, calculate_tool],
)
result = agent.invoke({"messages": [("user", query)]})
return result["messages"][-1].content
@env.task
async def parallel_agents(queries: list[str]) -> list[str]:
"""Run multiple LangGraph agents in parallel."""
results = await flyte.map(langgraph_agent, queries)
return results
```
### PydanticAI Agents
```python
import flyte
from pydantic_ai import Agent, RunContext
from pydantic import BaseModel
class AnalysisResult(BaseModel):
summary: str
confidence: float
recommendations: list[str]
env = flyte.TaskEnvironment(
name="pydantic-agent",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"pydantic-ai", "openai",
),
)
@env.task
async def pydantic_agent(query: str) -> AnalysisResult:
"""Run a PydanticAI agent with structured output."""
agent = Agent(
"openai:gpt-4",
result_type=AnalysisResult,
)
result = await agent.run(query)
return result.data
@env.task
async def parallel_pydantic_agents(queries: list[str]) -> list:
"""Run multiple PydanticAI agents in parallel."""
results = await flyte.map(pydantic_agent, queries)
return results
```
### OpenAI Agents SDK
```python
import flyte
from openai.agents import Agent, Tool
env = flyte.TaskEnvironment(
name="openai-agent",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"openai",
),
)
@env.task
async def search_tool(query: str) -> str:
"""A tool that is also a durable Flyte task."""
# Tools become durable — results are cached and replayable
...
return results
@env.task
async def openai_agent(query: str) -> str:
"""Run an OpenAI Agents SDK agent."""
agent = Agent(
name="researcher",
instructions="Research the query thoroughly.",
tools=[search_tool],
)
result = agent.run(query)
return result.final_output
@env.task
async def main(query: str) -> str:
"""Wrap the agent in a Flyte workflow for durability."""
return await openai_agent(query)
```
## Building Agents with Flyte Primitives
### How Flyte maps to the agent stack
| Agent Concept | Flyte Primitive |
|---|---|
| Tool call | `@env.task` — each tool is a durable task |
| Reasoning step | `@env.task` with LLM call |
| Observation | Output of tool task → input to reasoning task |
| Loop | `flyte.new_condition` for external gates, or dynamic workflows for internal loops |
| Parallel execution | `flyte.map` for fan-out sub-tasks |
| Traces | `flyte.trace` for lightweight LLM calls within a step |
### Durable agent pattern
```python
@env.task
async def llm_call(prompt: str, model: str = "gpt-4") -> str:
"""Durable LLM call — cached by input, replayable."""
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(model=model, messages=[{"role": "user", "content": prompt}])
return response.choices[0].message.content
@env.task
async def tool_call(tool_name: str, args: dict) -> str:
"""Durable tool execution — each tool call is a Flyte action."""
if tool_name == "search":
return perform_search(args["query"])
elif tool_name == "calc":
return str(evaluate(args["expression"]))
raise ValueError(f"Unknown tool: {tool_name}")
@env.task
async def agent_step(
history: list[dict],
) -> dict:
"""Single agent step: decide next action."""
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4",
messages=[
{"role": "system", "content": "Decide next action based on history."},
*history,
],
response_format={"type": "json_object"},
)
return eval(response.choices[0].message.content) # parse JSON
@env.task
async def run_agent(
query: str,
max_steps: int = 10,
) -> str:
"""Run a durable agent loop."""
history = [{"role": "user", "content": query}]
for step in range(max_steps):
decision = await agent_step(history)
if decision["type"] == "final_answer":
return decision["answer"]
# Execute tool
result = await tool_call(decision["tool"], decision["args"])
history.append({"role": "assistant", "content": f"Tool {decision['tool']} → {result}"})
return "Max steps reached"
```
## Agent Memory
### Keyed MemoryStore
```python
import flyte
from flyte.extend import MemoryStore
env = flyte.TaskEnvironment(
name="agent-with-memory",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"openai",
),
)
@env.task
async def agent_with_memory(query: str, user_id: str) -> str:
"""Agent with persistent memory per user."""
memory = MemoryStore(keyed_by=user_id)
# Load previous context
context = await memory.get("conversation", default=[])
# Add new message
context.append({"role": "user", "content": query})
# Generate response
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4",
messages=context,
)
answer = response.choices[0].message.content
# Store updated context
context.append({"role": "assistant", "content": answer})
await memory.set("conversation", context)
return answer
```
### Run-level context
```python
@env.task
async def agent_step(query: str) -> str:
"""Access run-level context for memory."""
ctx = flyte.ctx()
# Use run ID as a key for temporary memory
run_memory_key = f"run:{ctx.run_id}:memory"
# Store intermediate results
...
```
## Agent Chat UI
### Built-in chat UI
```python
import flyte
from flyte.extend import Agent, tool
env = flyte.TaskEnvironment(
name="chat-agent",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"openai",
),
)
class ChatAgent(Agent):
"""Built-in agent with chat UI."""
@tool
async def search(self, query: str) -> str:
"""Search the web."""
...
@tool
async def calculate(self, expression: str) -> str:
"""Evaluate a math expression."""
...
async def run(self, message: str) -> str:
"""Main agent loop."""
# Use tools to respond
return "Response"
if __name__ == "__main__":
flyte.init_from_config()
agent = ChatAgent()
agent.run(message="Hello!")
```
### Custom FastAPI chat app
```python
from fastapi import FastAPI
import flyte
from flyte.app.extras import FastAPIAppEnvironment
app = FastAPI()
env = FastAPIAppEnvironment(
name="chat-app",
app=app,
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"fastapi", "uvicorn", "openai",
),
)
@app.post("/chat")
async def chat(message: str):
"""Chat endpoint — delegates to a Flyte agent task."""
result = await flyte.run(agent_task, inputs={"message": message})
return {"response": result.outputs}
if __name__ == "__main__":
flyte.init_from_config()
flyte.serve(env)
```
## Deploying Agents
### As a task
```bash
# Run agent as a one-shot task
flyte run agent.py run_agent --query "Research X"
```
### As a scheduled task (Trigger)
```bash
# Create a trigger for periodic agent execution
flyte create trigger agent_task daily-agent \
--schedule "0 9 * * 1" # every Monday at 9am
```
### Behind a webhook
```python
# Agent behind a webhook app
@app.post("/agent")
async def agent_webhook(payload: dict):
flyte.run(agent_task, inputs={"message": payload["text"]})
return {"status": "queued"}
```
### Chat app pattern
```bash
# Deploy as a persistent app
flyte deploy agent_app.py env
```
## MCP Integration
### Building an MCP server for agents
```python
import flyte
from flyte.extend import MCPServer, tool
env = flyte.TaskEnvironment(
name="mcp-server",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"fastmcp",
),
)
@env.task
async def build_mcp_server() -> MCPServer:
"""Build an MCP server with durable tools."""
@tool
async def search(query: str) -> str:
return perform_search(query)
@tool
async def fetch(url: str) -> str:
return fetch_url(url)
return MCPServer(tools=[search, fetch])
```
### Connecting an MCP client
```python
# Claude Code — local (stdio)
# Configure inClaude Code settings to connect to the Flyte MCP server
# OpenCode — local
# Configure in opencode.json to connect to the Flyte MCP server
```
## Agent Anti-Patterns
1. **Don't put LLM calls in a loop without durability** — each LLM call should be a `@env.task` for caching and replay.
2. **Don't use Union-only features** — avoid `ReusePolicy` and other Union-specific APIs.
3. **Don't pass large prompts inline** — use `flyte.io.File` for large context windows.
4. **Don't forget to set resources** — LLM agent tasks need CPU for the orchestration container.
5. **Don't hardcode API keys** — use Flyte secrets for LLM API keys.
flyte-sdk-app11.4 KB
---
name: flyte-sdk-app
description: 'Builds and serves Flyte 2 apps — FastAPI, Streamlit, vLLM, SGLang, WebSocket, and browser apps. Use when the user wants to serve a model, create a REST API, build a dashboard, deploy an LLM backend, or create a web app with Flyte. Trigger words: "app", "serve", "deploy app", "FastAPI", "Streamlit", "vLLM", "SGLang", "REST API", "dashboard", "serving", "endpoint", "webhook", "WebSocket".'
---
# Flyte 2 SDK App Skill
Build and serve applications with Flyte 2.
## Grounding References
| Resource | URL |
|---|---|
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| CLI API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-cli/ |
| flyte-sdk source | https://github.com/flyteorg/flyte-sdk |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via the `flyte-cluster` and `flyte-docs` MCP servers |
## App Types
| App Type | Use Case | Import |
|---|---|---|
| `FastAPIAppEnvironment` | REST APIs, model serving | `from flyte.app.extras import FastAPIAppEnvironment` |
| `StreamlitAppEnvironment` | Dashboards, data apps | `from flyte.app.extras import StreamlitAppEnvironment` |
| `vLLMAppEnvironment` | LLM serving | `from flyte.app.extras import vLLMAppEnvironment` |
| `SGLangAppEnvironment` | Structured generation | `from flyte.app.extras import SGLangAppEnvironment` |
| Custom (`AppEnvironment`) | Any HTTP server | `import flyte` |
## FastAPI App — Model Serving
### Basic FastAPI app
```python
from fastapi import FastAPI
import flyte
from flyte.app.extras import FastAPIAppEnvironment
app = FastAPI()
env = FastAPIAppEnvironment(
name="my-model",
app=app,
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"fastapi", "uvicorn", "torch",
),
)
@app.get("/predict")
async def predict(x: float) -> dict:
return {"result": x * 2 + 5}
if __name__ == "__main__":
flyte.init_from_config()
flyte.serve(env)
```
### Model serving with loading
```python
from fastapi import FastAPI
import flyte
from flyte.app.extras import FastAPIAppEnvironment
app = FastAPI()
env = FastAPIAppEnvironment(
name="text-classifier",
app=app,
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"fastapi", "uvicorn", "torch", "transformers",
),
)
model = None # Loaded once at startup
@app.on_event("startup")
async def load_model():
global model
model = transformers.AutoModelForSequenceClassification.from_pretrained("bert-base")
@app.get("/predict")
async def predict(text: str) -> dict:
assert model is not None
outputs = model(transformers.encode(text))
return {"prediction": outputs.argmax().item(), "confidence": outputs.softmax().max().item()}
if __name__ == "__main__":
flyte.init_from_config()
flyte.serve(env)
```
### Multi-file FastAPI app
```
app/
__init__.py
main.py # FastAPI app entry
routes/
__init__.py
predict.py
health.py
models/
__init__.py
classifier.py
```
```python
# app/main.py
from fastapi import FastAPI
from .routes import predict, health
app = FastAPI()
app.include_router(predict.router, prefix="/api")
app.include_router(health.router, prefix="/health")
```
## Streamlit App — Data Dashboards
### Basic Streamlit app
```python
import streamlit as st
import flyte
from flyte.app.extras import StreamlitAppEnvironment
st.title("Data Dashboard")
df = st.dataframe(load_data())
if st.button("Refresh"):
st.rerun()
env = StreamlitAppEnvironment(
name="dashboard",
script="app.py",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"streamlit", "pandas", "matplotlib",
),
)
if __name__ == "__main__":
flyte.init_from_config()
flyte.serve(env)
```
### Streamlit with upstream app dependency
```python
import streamlit as st
import requests
import flyte
from flyte.app.extras import StreamlitAppEnvironment
# Access upstream app endpoint
MODEL_ENDPOINT = flyte.app.AppEndpoint(app_name="model-serving")
st.title("Model Results")
text = st.text_input("Enter text:")
if text:
response = requests.post(
f"{MODEL_ENDPOINT.url}/predict",
json={"text": text},
)
st.json(response.json())
env = StreamlitAppEnvironment(
name="results-dashboard",
script="app.py",
depends_on=[MODEL_ENDPOINT],
)
```
## vLLM App — LLM Serving
### Basic vLLM app
```python
import flyte
from flyte.app.extras import vLLMAppEnvironment
env = vLLMAppEnvironment(
name="llm-serving",
model="meta-llama/Llama-3-8b-Instruct",
image=flyte.Image.from_base("vllm/vllm-openai:latest"),
resources=flyte.Resources(
cpu="8",
memory="32Gi",
gpu="1",
gpu_model="nvidia-a10g",
),
)
if __name__ == "__main__":
flyte.init_from_config()
flyte.serve(env)
```
### vLLM with model prefetch
```python
env = vLLMAppEnvironment(
name="llm-serving",
model="meta-llama/Llama-3-8b-Instruct",
prefetch=True, # prefetch model weights at deploy time
image=flyte.Image.from_base("vllm/vllm-openai:latest"),
resources=flyte.Resources(
cpu="8",
memory="32Gi",
gpu="1",
gpu_model="nvidia-a10g",
),
)
```
### vLLM multi-GPU
```python
env = vLLMAppEnvironment(
name="llm-serving",
model="meta-llama/Llama-3-70b-Instruct",
tensor_parallel_size=4, # shard across 4 GPUs
prefetch=True,
image=flyte.Image.from_base("vllm/vllm-openai:latest"),
resources=flyte.Resources(
cpu="16",
memory="128Gi",
gpu="4",
gpu_model="nvidia-a100",
),
)
```
## SGLang App — Structured Generation
### Basic SGLang app
```python
import flyte
from flyte.app.extras import SGLangAppEnvironment
env = SGLangEnvironment(
name="structured-gen",
model="meta-llama/Llama-3-8b-Instruct",
prefetch=True,
image=flyte.Image.from_base("sgl-project/sglang:latest"),
resources=flyte.Resources(
cpu="4",
memory="16Gi",
gpu="1",
gpu_model="nvidia-a10g",
),
)
if __name__ == "__main__":
flyte.init_from_config()
flyte.serve(env)
```
## WebSocket Apps
```python
import asyncio
import flyte
from flyte.app.extras import FastAPIAppEnvironment
from fastapi import FastAPI, WebSocket
app = FastAPI()
env = FastAPIAppEnvironment(
name="websocket-app",
app=app,
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"fastapi", "uvicorn", "websockets",
),
)
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
await websocket.accept()
try:
while True:
data = await websocket.receive_text()
result = process(data)
await websocket.send_text(result)
except WebSocketDisconnect:
pass
if __name__ == "__main__":
flyte.init_from_config()
flyte.serve(env)
```
## Serving vs Deploying
### Serve (ephemeral, for development)
```bash
# Serve an app locally
flyte serve app.py env
```
```python
# Serve programmatically
result = flyte.serve(env)
print(f"App URL: {result.url}")
```
### Deploy (persistent, for production)
```bash
# Deploy an app
flyte deploy app.py env
```
```python
# Deploy programmatically
result = flyte.deploy(env)
print(f"App URL: {result.url}")
```
### Activating and deactivating apps
```bash
# Activate a deployed app
flyte update app <app_name> --activate --project flytesnacks --domain development
# Deactivate
flyte update app <app_name> --deactivate --project flytesnacks --domain development
# Check status
flyte get app <app_name> --project flytesnacks --domain development
```
### Using Flyte MCP for app management
Getting an app's status, activating it, and deactivating it are all available as MCP
tools, each taking the app name.
## App Parameters
### Passing parameters into apps
```python
env = FastAPIAppEnvironment(
name="model-serving",
app=app,
parameters={
"model_name": flyte.app.Parameter(name="model_name", mount="/models/model.safetensors"),
"api_key": flyte.app.Parameter(name="api_key", env_var="API_KEY"),
},
)
```
### Overriding parameters at serve time
```bash
flyte serve app.py env --parameter model_name=/custom/path
```
## App Autoscaling
### Auto-scaling apps
```python
from datetime import timedelta
env = FastAPIAppEnvironment(
name="auto-scaling-app",
app=app,
scaling=flyte.app.Scaling(
replicas=(1, 10), # autoscale between (min, max) replicas
scaledown_after=timedelta(minutes=10),
),
)
```
## App Dependencies (Serving Graphs)
### Deploying multiple apps together
```python
model_env = FastAPIAppEnvironment(
name="model-serving",
app=model_app,
image=model_image,
)
dashboard_env = StreamlitAppEnvironment(
name="results-dashboard",
script="dashboard.py",
depends_on=[model_env], # upstream dependency
image=dashboard_image,
)
# Deploy both together
flyte.deploy(model_env)
flyte.deploy(dashboard_env)
# Access upstream endpoint
model_url = model_env.endpoint.url
```
### GPU/CPU split serving graph
```python
# GPU app: model inference
gpu_env = FastAPIAppEnvironment(
name="model-gpu",
app=gpu_app,
image=flyte.Image.from_base("nvidia/cuda:12.1-py3").with_pip_packages(
"torch", "fastapi", "uvicorn",
),
resources=flyte.Resources(
cpu="4", memory="16Gi", gpu="1", gpu_model="nvidia-a10g",
),
)
# CPU app: pre/post processing
cpu_env = FastAPIAppEnvironment(
name="preprocess-cpu",
app=cpu_app,
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"fastapi", "uvicorn", "pillow", "numpy",
),
depends_on=[gpu_env],
resources=flyte.Resources(cpu="2", memory="4Gi"),
)
```
## Webhook Apps
### Basic webhook
```python
import flyte
from flyte.app.extras import FastAPIAppEnvironment
from fastapi import FastAPI, Request
app = FastAPI()
env = FastAPIAppEnvironment(
name="webhook-receiver",
app=app,
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"fastapi", "uvicorn",
),
)
@app.post("/webhook")
async def webhook(request: Request):
payload = await request.json()
# Trigger a Flyte workflow
flyte.run(process_webhook, inputs={"payload": payload})
return {"status": "received"}
if __name__ == "__main__":
flyte.init_from_config()
flyte.serve(env)
```
## App Secrets
### Secret-based authentication
```python
# Create a secret (via CLI or SDK)
# flyte create secret my-api-key --value "sk-xxx"
env = FastAPIAppEnvironment(
name="authenticated-app",
app=app,
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"fastapi", "uvicorn",
),
secrets={"api_key": flyte.Secret(key="my-api-key", group="default")},
)
# Access secret inside the app
api_key = os.environ["FLYTE_SECRET_MY_API_KEY"]
```
## Anti-Patterns
1. **Don't use `flyte.run()` inside apps** — use `flyte.serve()` for apps, `flyte.run()` for workflows.
2. **Don't forget `flyte.init_from_config()`** — required before `flyte.serve()`.
3. **Don't hardcode model paths** — use `flyte.app.AppEndpoint` for upstream app URLs.
4. **Don't use Union-only features** — avoid `ReusePolicy` and other Union-specific APIs.
5. **Don't serve GPU apps without GPU resources** — always specify `gpu` and `gpu_model` in resources.
flyte-sdk-data15.8 KB
---
name: flyte-sdk-data
description: 'Handles data engineering patterns: ETL pipelines, data processing, data quality checks, fanout/map tasks, conditions, dynamic workflows, and batch data transformations. Use when the user wants to build ETL pipelines, process large datasets, run data quality checks, fan out data processing tasks, or handle batch data transformations. Trigger words: "ETL", "data pipeline", "data processing", "fanout", "map", "transform", "data quality", "parquet", "CSV", "batch", "extract", "load", "validate", "schema".'
---
# Flyte 2 SDK Data Engineering Skill
Build ETL pipelines, data processing workflows, and data quality checks with Flyte 2.
## Grounding References
| Resource | URL |
|---|---|
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| CLI API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-cli/ |
| flyte-sdk source | https://github.com/flyteorg/flyte-sdk |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via the `flyte-cluster` and `flyte-docs` MCP servers |
**Ground unfamiliar APIs in real examples.** When unsure of a current Flyte 2 API, or for a pattern not shown below, and the `flyte-docs` search tools are available, search them first — by exact symbol (`TaskEnvironment`, `flyte.io.File`, `map_task`), since matching is literal substring, not semantic — then adapt a real example rather than inventing one, and cite the file or section you pulled it from. (Flyte 2 is not `flytekit`; priors are often wrong.)
## ETL Pipeline Patterns
### Basic Extract-Transform-Load
```python
import flyte
import flyte.io
env = flyte.TaskEnvironment(
name="etl-pipeline",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"pandas", "polars", "pyarrow", "boto3", "sqlalchemy",
),
)
@env.task(retries=3, cache="auto")
async def extract(source_uri: str) -> flyte.io.DataFrame:
"""Extract data from various sources."""
import polars as pl
if source_uri.endswith(".csv"):
df = pl.read_csv(source_uri)
elif source_uri.endswith(".parquet"):
df = pl.read_parquet(source_uri)
else:
raise ValueError(f"Unsupported format: {source_uri}")
return flyte.io.DataFrame(df)
@env.task(retries=2, cache="auto")
async def transform(df: flyte.io.DataFrame) -> flyte.io.DataFrame:
"""Clean and transform data."""
inner = df.to_polars()
cleaned = (
inner
.drop_nulls()
.unique()
.with_columns([
pl.col("date").str.strptime(pl.Date, "%Y-%m-%d").alias("date_parsed"),
])
)
return flyte.io.DataFrame(cleaned)
@env.task(retries=1, cache="auto")
async def load(df: flyte.io.DataFrame, destination: str) -> str:
"""Load transformed data to destination."""
inner = df.to_polars()
if destination.endswith(".parquet"):
inner.write_parquet(destination)
elif destination.endswith(".csv"):
inner.write_csv(destination)
return destination
@env.task
async def etl_pipeline(source_uri: str, destination: str) -> dict:
"""Orchestrate the ETL pipeline."""
raw = await extract(source_uri)
cleaned = await transform(raw)
loaded_path = await load(cleaned, destination)
return {"source": source_uri, "destination": loaded_path}
```
### Multi-step ETL with intermediate storage
```python
@env.task(cache="auto")
async def extract_raw(source_uri: str) -> flyte.io.File:
"""Extract and save raw data to remote storage."""
import polars as pl
df = pl.read_csv(source_uri)
path = "/tmp/raw.parquet"
df.write_parquet(path)
return flyte.io.File(path=path)
@env.task(cache="auto")
async def validate_raw(raw: flyte.io.File) -> dict:
"""Validate raw data quality."""
df = pl.read_parquet(raw.path)
return {
"row_count": len(df),
"column_count": len(df.columns),
"null_counts": df.null_count().to_dict(),
}
@env.task(cache="auto")
async def clean(raw: flyte.io.File) -> flyte.io.File:
"""Clean and normalize data."""
df = pl.read_parquet(raw.path)
cleaned = df.drop_nulls().unique()
path = "/tmp/cleaned.parquet"
cleaned.write_parquet(path)
return flyte.io.File(path=path)
@env.task(cache="auto")
async def enrich(cleaned: flyte.io.File) -> flyte.io.File:
"""Enrich data with external features."""
df = pl.read_parquet(cleaned.path)
# Join with external feature store
...
path = "/tmp/enriched.parquet"
df.write_parquet(path)
return flyte.io.File(path=path)
@env.task
async def load_enriched(enriched: flyte.io.File, destination: str) -> str:
"""Load enriched data to final destination."""
...
return destination
@env.task
async def etl_with_validation(source_uri: str, destination: str) -> dict:
"""ETL pipeline with validation gates."""
raw = await extract_raw(source_uri)
quality = await validate_raw(raw)
# Quality gate: fail if too many nulls
if quality["null_counts"].get("critical_field", 0) / quality["row_count"] > 0.5:
raise ValueError("Too many nulls in critical field")
cleaned = await clean(raw)
enriched = await enrich(cleaned)
loaded = await load_enriched(enriched, destination)
return {"quality": quality, "destination": loaded}
```
## Fan-out Data Processing
### Map over large datasets
```python
@env.task(cache="auto")
async def process_file(file_uri: str) -> flyte.io.DataFrame:
"""Process a single data file."""
import polars as pl
df = pl.read_parquet(file_uri)
cleaned = df.drop_nulls().unique()
return flyte.io.DataFrame(cleaned)
@env.task
async def process_dataset(file_uris: list[str]) -> list:
"""Fan out processing across all files in parallel."""
results = await flyte.map(process_file, file_uris)
return results
@env.task
async def merge_results(results: list) -> flyte.io.DataFrame:
"""Merge processed results into a single DataFrame."""
import polars as pl
combined = pl.concat([r.to_polars() for r in results])
return flyte.io.DataFrame(combined)
@env.task
async def main(file_uris: list[str]) -> flyte.io.DataFrame:
processed = await process_dataset(file_uris)
return await merge_results(processed)
```
### Fan-out with error handling
```python
@env.task
async def process_file_safe(file_uri: str) -> dict:
"""Process a file with error handling."""
try:
df = await process_file(file_uri)
return {"status": "success", "file": file_uri, "rows": len(df.to_polars())}
except Exception as e:
return {"status": "error", "file": file_uri, "error": str(e)}
@env.task
async def process_with_errors(file_uris: list[str]) -> dict:
"""Process files, collecting both successes and errors."""
results = await flyte.map(process_file_safe, file_uris)
successes = [r for r in results if r["status"] == "success"]
errors = [r for r in results if r["status"] == "error"]
return {"successes": successes, "errors": errors, "total": len(results)}
```
### Limited concurrency fan-out
```python
@env.task
async def main(file_uris: list[str]) -> list:
"""Fan out with limited concurrency."""
import asyncio
sem = asyncio.Semaphore(20) # max 20 concurrent
async def bounded(uri):
async with sem:
return await process_file(uri)
return await asyncio.gather(*(bounded(u) for u in file_uris))
```
## Data Quality Checks
### Comprehensive data quality
```python
@env.task(cache="auto")
async def validate_schema(df: flyte.io.DataFrame, expected_schema: dict) -> dict:
"""Validate DataFrame schema matches expected schema."""
inner = df.to_polars()
checks = {}
# Column names
expected_cols = set(expected_schema.keys())
actual_cols = set(inner.columns)
checks["columns_match"] = expected_cols == actual_cols
checks["missing_columns"] = list(expected_cols - actual_cols)
checks["extra_columns"] = list(actual_cols - expected_cols)
# Column types
for col, expected_type in expected_schema.items():
if col in inner.columns:
actual_type = str(inner[col].dtype)
checks[f"type_{col}"] = {
"expected": expected_type,
"actual": actual_type,
"match": expected_type in actual_type,
}
return checks
@env.task(cache="auto")
async def validate_nulls(df: flyte.io.DataFrame, max_null_pct: float = 0.1) -> dict:
"""Validate null percentages per column."""
inner = df.to_polars()
row_count = len(inner)
checks = {}
for col in inner.columns:
null_count = inner[col].null_count()
null_pct = null_count / row_count if row_count > 0 else 0
checks[col] = {
"null_count": null_count,
"null_pct": null_pct,
"passed": null_pct <= max_null_pct,
}
return checks
@env.task(cache="auto")
async def validate_values(df: flyte.io.DataFrame, constraints: dict) -> dict:
"""Validate value constraints (ranges, enums, patterns)."""
inner = df.to_polars()
checks = {}
for col, constraint in constraints.items():
if col not in inner.columns:
continue
if "min" in constraint:
checks[f"{col}_min"] = inner[col].min() >= constraint["min"]
if "max" in constraint:
checks[f"{col}_max"] = inner[col].max() <= constraint["max"]
if "allowed_values" in constraint:
unique = set(inner[col].unique())
checks[f"{col}_values"] = unique.issubset(set(constraint["allowed_values"]))
return checks
@env.task
async def data_quality_gate(
df: flyte.io.DataFrame,
schema: dict,
max_null_pct: float = 0.1,
constraints: dict = None,
) -> dict:
"""Run all data quality checks and pass/fail."""
schema_check = await validate_schema(df, schema)
null_check = await validate_nulls(df, max_null_pct)
value_checks = await validate_values(df, constraints or {})
all_passed = (
schema_check["columns_match"]
and all(c["passed"] for c in null_check.values())
and all(value_checks.values())
)
return {
"passed": all_passed,
"schema": schema_check,
"nulls": null_check,
"values": value_checks,
}
```
### Data quality with custom checks
```python
@env.task(cache="auto")
async def check_distribution(df: flyte.io.DataFrame, column: str, expected_stats: dict) -> dict:
"""Check if data distribution matches expected statistics."""
inner = df.to_polars()
col_data = inner[column].drop_nulls()
actual_mean = col_data.mean()
actual_std = col_data.std()
actual_min = col_data.min()
actual_max = col_data.max()
return {
"column": column,
"mean": {"actual": actual_mean, "expected": expected_stats.get("mean"),
"within_tolerance": abs(actual_mean - expected_stats.get("mean", 0)) < expected_stats.get("tolerance", 0.1)},
"std": {"actual": actual_std, "expected": expected_stats.get("std"),
"within_tolerance": abs(actual_std - expected_stats.get("std", 0)) < expected_stats.get("tolerance", 0.1)},
}
```
## Dynamic Workflows for Data
### Dynamic file processing
```python
@env.task
async def discover_files(prefix: str) -> list[str]:
"""Discover data files in a storage prefix."""
import boto3
s3 = boto3.client("s3")
files = []
paginator = s3.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket="my-data-bucket", Prefix=prefix):
for obj in page.get("Contents", []):
if obj["Key"].endswith((".parquet", ".csv")):
files.append(f"s3://{obj['Bucket']}/{obj['Key']}")
return files
@env.task
async def process_file(file_uri: str) -> flyte.io.File:
"""Process a single file."""
import polars as pl
df = pl.read_parquet(file_uri)
cleaned = df.drop_nulls()
path = f"/tmp/cleaned_{file_uri.split('/')[-1]}"
cleaned.write_parquet(path)
return flyte.io.File(path=path)
@env.task
async def merge_files(files: list[flyte.io.File]) -> flyte.io.File:
"""Merge processed files."""
import polars as pl
dfs = [pl.read_parquet(f.path) for f in files]
combined = pl.concat(dfs)
path = "/tmp/merged.parquet"
combined.write_parquet(path)
return flyte.io.File(path=path)
@env.task
async def dynamic_etl(prefix: str, destination: str) -> dict:
"""Dynamic ETL: discover files, process, merge."""
files = await discover_files(prefix)
processed = await flyte.map(process_file, files)
merged = await merge_files(processed)
# Copy to destination
return {"source_prefix": prefix, "destination": destination, "file_count": len(files)}
```
### Conditional data routing
```python
@env.task
async def route_data(df: flyte.io.DataFrame, threshold: float) -> dict:
"""Route data based on quality score."""
score = compute_quality_score(df)
if score >= threshold:
return {"route": "production", "score": score}
else:
return {"route": "review", "score": score}
@env.task
async def process_production(df: flyte.io.DataFrame) -> flyte.io.File:
"""Process data for production."""
...
@env.task
async def process_review(df: flyte.io.DataFrame) -> flyte.io.File:
"""Flag data for manual review."""
...
@env.task
async def conditional_pipeline(df: flyte.io.DataFrame, threshold: float) -> dict:
"""Route data based on quality."""
routed = await route_data(df, threshold)
if routed["route"] == "production":
result = await process_production(df)
else:
result = await process_review(df)
return {**routed, "result": result}
```
## JsonlFile and JsonlDir for Large Datasets
### JsonlFile — streaming JSONL
```python
@env.task
async def process_jsonl(path: str) -> int:
"""Process a JSONL file with streaming."""
from flyte.extend import JsonlFile
jf = JsonlFile(path)
count = 0
async for record in jf.stream():
process(record)
count += 1
return count
```
### JsonlDir — batched JSONL directories
```python
@env.task
async def process_jsonl_dir(dir_path: str) -> dict:
"""Process JSONL directory with batched streaming."""
from flyte.extend import JsonlDir
jd = JsonlDir(dir_path)
total = 0
async for batch in jd.stream_batches():
total += len(batch)
return {"records": total}
```
## Data Format Reference
| Format | Flyte Type | Best For |
|---|---|---|
| Parquet | `flyte.io.DataFrame` | Tabular data, ETL |
| CSV | `flyte.io.File` | Small datasets, interchange |
| JSONL | `JsonlFile` / `JsonlDir` | Streaming records |
| JSON | inline (dict) | Small structured data |
| Pickle | `flyte.io.File` | Python objects |
| NumPy (.npy/.npz) | `flyte.io.File` | Arrays, embeddings |
| PNG/JPEG | `flyte.io.File` | Images |
| Model (.pt/.safetensors) | `flyte.io.File` | Model checkpoints |
## Performance Tips for Data Pipelines
1. **Use Parquet over CSV** — columnar format, compressed, faster I/O
2. **Cache idempotent transforms** — `cache="auto"` on ETL steps
3. **Fan out with `flyte.map`** — parallel processing for independent files
4. **Use `flyte.trace` for lightweight ops** — no container spin-up cost
5. **Set `raw_data_path`** — control where intermediate data is stored
6. **Use `inline_output_limit`** — control when data goes by reference vs inline
7. **Use `interruptible=True`** — spot instances for fault-tolerant data processing
## Anti-Patterns
1. **Don't load entire datasets into memory** — use streaming (`JsonlFile`, Polars lazy) for large data.
2. **Don't pass DataFrames inline** — they go by reference automatically, but small dicts do go inline.
3. **Don't skip data quality gates** — always validate before and after transforms.
4. **Don't use Union-only features** — avoid `ReusePolicy` and other Union-specific APIs.
flyte-sdk-eval11.8 KB
---
name: flyte-sdk-eval
description: 'Builds minimal evaluation harnesses (unit tests + small-run workflows) and suggests ways to validate correctness and performance early. Use when the user wants to test Flyte tasks, validate pipeline outputs, set up evaluation pipelines, or write unit tests for ML/data workflows. Trigger words: "test", "evaluate", "validation", "unit test", "verify", "assert", "data quality", "metrics", "benchmark".'
---
# Flyte 2 SDK Eval Skill
Build evaluation harnesses, unit tests, and validation pipelines for Flyte 2 workflows.
## Grounding References
| Resource | URL |
|---|---|
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| CLI API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-cli/ |
| flyte-sdk source | https://github.com/flyteorg/flyte-sdk |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via the `flyte-cluster` and `flyte-docs` MCP servers |
## Testing Patterns
### Direct Task Invocation (unit testing)
Test task logic directly without remote execution:
```python
import pytest
from pipeline import preprocess, train, evaluate
def test_preprocess():
"""Test preprocessing logic in isolation."""
result = preprocess(["a", "b", "c"])
assert result is not None
assert len(result) == 3
def test_train():
"""Test training with a small dataset."""
import flyte
import flyte.io
data = flyte.io.DataFrame(polars.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]}))
model = train(data)
assert model is not None
def test_evaluate():
"""Test evaluation metrics."""
import flyte
model = flyte.io.File(path="/tmp/mock_model.pt")
metrics = evaluate(model)
assert "accuracy" in metrics
assert 0 <= metrics["accuracy"] <= 1
```
### Using flyte.run() for Integration Testing
Test the full workflow execution locally:
```python
import pytest
import flyte
from pipeline import main
def test_full_pipeline():
"""Run the full pipeline locally with test data."""
result = flyte.run(main, inputs={"data": ["test1", "test2"]})
assert result is not None
assert "accuracy" in result.outputs
def test_full_pipeline_with_inputs():
"""Test with specific inputs via flyte.run()."""
result = flyte.run(
main,
inputs={"data": ["a", "b", "c"]},
)
assert result.status == "SUCCEEDED"
```
### Testing Async Tasks
```python
import asyncio
import pytest
def test_async_task():
"""Test async task by running in event loop."""
result = asyncio.run(preprocess(["a", "b"]))
assert result is not None
@pytest.mark.asyncio
async def test_async_task_mark():
"""Test async task with pytest-asyncio."""
result = await preprocess(["a", "b"])
assert len(result) == 2
```
## Evaluation Pipeline Patterns
### ML Model Evaluation
```python
import flyte
import flyte.io
env = flyte.TaskEnvironment(
name="eval",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"scikit-learn", "scipy", "pandas", "matplotlib",
),
)
@env.task
async def load_test_data() -> flyte.io.DataFrame:
"""Load ground truth test data."""
...
return flyte.io.DataFrame(df)
@env.task
async def load_model(model_uri: str) -> object:
"""Load a trained model."""
...
@env.task
async def predict(model: object, data: flyte.io.DataFrame) -> flyte.io.DataFrame:
"""Run model predictions on test data."""
...
@env.task
async def compute_metrics(
predictions: flyte.io.DataFrame,
ground_truth: flyte.io.DataFrame,
) -> dict:
"""Compute evaluation metrics."""
from sklearn.metrics import (
accuracy_score, f1_score, precision_score, recall_score,
roc_auc_score, mean_squared_error,
)
y_true = ground_truth.to_polars()["label"].to_list()
y_pred = predictions.to_polars()["prediction"].to_list()
y_prob = predictions.to_polars()["probability"].to_list()
return {
"accuracy": accuracy_score(y_true, y_pred),
"f1": f1_score(y_true, y_pred),
"precision": precision_score(y_true, y_pred),
"recall": recall_score(y_true, y_pred),
"auc": roc_auc_score(y_true, y_prob),
}
@env.task
async def generate_report(metrics: dict) -> flyte.io.File:
"""Generate an evaluation report."""
import matplotlib.pyplot as plt
fig, axes = plt.subplots(1, 2, figsize=(12, 4))
# Confusion matrix, ROC curve, etc.
path = "/tmp/eval_report.png"
fig.savefig(path)
return flyte.io.File(path=path)
@env.task
async def evaluate_pipeline(
model_uri: str,
test_data_uri: str,
) -> dict:
"""Full evaluation pipeline."""
data = await load_test_data()
model = await load_model(model_uri)
preds = await predict(model, data)
metrics = await compute_metrics(preds, data)
report = await generate_report(metrics)
return {"metrics": metrics, "report": report}
```
### A/B Model Comparison
```python
@env.task
async def compare_models(
model_a_uri: str,
model_b_uri: str,
test_data: flyte.io.DataFrame,
) -> dict:
"""Compare two models on the same test data."""
model_a = await load_model(model_a_uri)
model_b = await load_model(model_b_uri)
preds_a = await predict(model_a, test_data)
preds_b = await predict(model_b, test_data)
metrics_a = await compute_metrics(preds_a, test_data)
metrics_b = await compute_metrics(preds_b, test_data)
winner = "A" if metrics_a["accuracy"] > metrics_b["accuracy"] else "B"
return {
"model_a_metrics": metrics_a,
"model_b_metrics": metrics_b,
"winner": winner,
"improvement": metrics_a["accuracy"] - metrics_b["accuracy"],
}
```
### Data Quality Validation
```python
@env.task
async def validate_data(df: flyte.io.DataFrame) -> dict:
"""Run data quality checks."""
inner = df.to_polars()
checks = {}
# Row count check
row_count = len(inner)
checks["row_count"] = row_count
if row_count == 0:
raise ValueError("Dataset is empty")
# Null check
null_counts = inner.null_count().to_dict()
checks["null_counts"] = null_counts
for col, count in null_counts.items():
if count > 0 and count / row_count > 0.5:
raise ValueError(f"Column {col} has >50% nulls")
# Type check
checks["dtypes"] = {str(k): str(v) for k, v in inner.schema.items()}
# Value range check
for col in inner.columns:
if inner[col].dtype.is_float64():
min_val = inner[col].min()
max_val = inner[col].max()
if min_val < 0 or max_val > 1:
checks[f"range_{col}"] = {"min": min_val, "max": max_val}
return {"passed": True, "checks": checks}
@env.task
async def data_quality_gate(
data: flyte.io.DataFrame,
threshold: float = 0.9,
) -> bool:
"""Pass/fail gate based on data quality score."""
result = await validate_data(data)
score = result["checks"].get("quality_score", 1.0)
if score < threshold:
raise ValueError(f"Data quality gate failed: {score} < {threshold}")
return True
```
### Pipeline Output Validation
```python
@env.task
async def validate_output(
model_path: flyte.io.File,
metrics: dict,
min_accuracy: float = 0.8,
) -> dict:
"""Validate that pipeline outputs meet quality thresholds."""
validation = {
"model_exists": model_path is not None,
"metrics_valid": all(v >= 0 and v <= 1 for v in metrics.values()),
"accuracy_threshold": metrics.get("accuracy", 0) >= min_accuracy,
}
if not validation["accuracy_threshold"]:
raise ValueError(
f"Model accuracy {metrics['accuracy']} below threshold {min_accuracy}"
)
return validation
```
## Experiment Tracking
### Manual Experiment Tracking
```python
import json
import datetime
import flyte
import flyte.io
@env.task
async def track_experiment(
experiment_name: str,
hyperparams: dict,
metrics: dict,
) -> flyte.io.File:
"""Track experiment results as a JSON file."""
record = {
"experiment": experiment_name,
"timestamp": datetime.datetime.now().isoformat(),
"hyperparameters": hyperparams,
"metrics": metrics,
}
path = f"/tmp/experiments/{experiment_name}.json"
with open(path, "w") as f:
json.dump(record, f, indent=2)
return flyte.io.File(path=path)
@env.task
async def run_experiment(
config: dict,
data: flyte.io.DataFrame,
) -> dict:
"""Run a single experiment and track results."""
model = await train(data, config)
metrics = await evaluate(model, data)
await track_experiment(config["name"], config, metrics)
return metrics
```
### Hyperparameter Search with Tracking
```python
@env.task
async def hpo_search(
param_grid: list[dict],
data: flyte.io.DataFrame,
) -> dict:
"""Run hyperparameter search with experiment tracking."""
results = await flyte.map(
lambda cfg: run_experiment(cfg, data),
param_grid,
)
best = max(results, key=lambda r: r["accuracy"])
return best
```
## Performance Benchmarking
### Task-level Benchmarking
```python
import time
import flyte
@env.task
async def benchmark_task(
task_fn,
inputs: dict,
num_runs: int = 5,
) -> dict:
"""Benchmark a task's performance."""
durations = []
for _ in range(num_runs):
start = time.time()
await task_fn(**inputs)
durations.append(time.time() - start)
return {
"mean_ms": sum(durations) / len(durations) * 1000,
"min_ms": min(durations) * 1000,
"max_ms": max(durations) * 1000,
"p95_ms": sorted(durations)[int(len(durations) * 0.95)] * 1000,
}
```
### Throughput Testing
```python
@env.task
async def throughput_test(
batch_sizes: list[int],
) -> dict:
"""Test throughput at different batch sizes."""
results = {}
for size in batch_sizes:
data = create_batch(size)
start = time.time()
await process_batch(data)
elapsed = time.time() - start
results[size] = {
"throughput": size / elapsed if elapsed > 0 else 0,
"latency_ms": elapsed * 1000 / size,
}
return results
```
## Testing with Flyte MCP
### Inspecting runs with MCP
If Flyte MCP tools are available, use them to read a past run's inputs and outputs for
validation, list recent runs for a task to compare against, and block until a run finishes
before checking its status.
## pytest Configuration
```ini
# pytest.ini
[pytest]
testpaths = tests
asyncio_mode = auto
markers =
slow: marks tests as slow (deselect with '-m "not slow"')
integration: marks tests as integration tests
```
```toml
# pyproject.toml
[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto"
markers = [
"slow: marks tests as slow",
"integration: marks tests as integration tests",
]
```
## Test Structure
```
tests/
__init__.py
test_preprocess.py # unit tests for preprocessing
test_train.py # unit tests for training
test_evaluate.py # unit tests for evaluation
test_integration.py # integration tests (flyte.run)
test_data_quality.py # data quality validation
conftest.py # shared fixtures
```
## Anti-Patterns
1. **Don't test against remote runs in unit tests** — use direct function invocation for unit tests. Reserve `flyte.run()` for integration tests.
2. **Don't hardcode test data paths** — use `flyte.io.File(path="/tmp/test_data")` with temp directories.
3. **Don't skip data quality gates** — always validate data before and after transformations.
4. **Don't use Union-only features** — avoid `ReusePolicy` and other Union-specific APIs.
5. **Don't test ML models with random data** — use representative test datasets that match production distribution.
flyte-sdk-ml20.4 KB
---
name: flyte-sdk-ml
description: 'Handles ML workload patterns: model training, hyperparameter optimization, experiment tracking, model evaluation and selection, batch inference, real-time serving, and model monitoring. Use when the user wants to train models, run hyperparameter search, track experiments, evaluate models, do batch or real-time inference, or set up model monitoring. Trigger words: "train", "training", "hyperparameter", "HPO", "experiment", "tracking", "evaluation", "inference", "batch inference", "model serving", "monitoring", "GPU", "PyTorch", "TensorFlow", "scikit-learn", "HuggingFace", "model".'
---
# Flyte 2 SDK ML Skill
Build ML training, HPO, evaluation, and inference pipelines with Flyte 2.
## Grounding References
| Resource | URL |
|---|---|
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| CLI API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-cli/ |
| flyte-sdk source | https://github.com/flyteorg/flyte-sdk |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via the `flyte-cluster` and `flyte-docs` MCP servers |
**Ground unfamiliar APIs in real examples.** When unsure of a current Flyte 2 API, or for a pattern not shown below, and the `flyte-docs` search tools are available, search them first — by exact symbol (`TaskEnvironment`, `flyte.io.File`, `map_task`), since matching is literal substring, not semantic — then adapt a real example rather than inventing one, and cite the file or section you pulled it from. (Flyte 2 is not `flytekit`; priors are often wrong.)
## Model Training
### PyTorch Training
```python
import flyte
import flyte.io
env = flyte.TaskEnvironment(
name="training",
image=flyte.Image.from_base("pytorch/pytorch:2.1-cuda12.1-cudnn8-devel").with_pip_packages(
"transformers", "datasets", "accelerate",
),
)
@env.task(
requests=flyte.Resources(
cpu="4", memory="16Gi", gpu="1", gpu_model="nvidia-a10g",
),
)
async def train(
train_data: flyte.io.File,
val_data: flyte.io.File,
hyperparams: dict,
) -> flyte.io.File:
"""Train a model and save checkpoint."""
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
# Load data
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModelForSequenceClassification.from_pretrained(
"bert-base-uncased", num_labels=2
)
# Train
for epoch in range(hyperparams["epochs"]):
# ... training loop ...
pass
# Save checkpoint
output_path = "/tmp/model_checkpoint"
model.save_pretrained(output_path)
tokenizer.save_pretrained(output_path)
return flyte.io.File(path=output_path)
@env.task
async def main(
train_uri: str,
val_uri: str,
lr: float = 0.001,
batch_size: int = 32,
epochs: int = 3,
) -> dict:
hyperparams = {"lr": lr, "batch_size": batch_size, "epochs": epochs}
checkpoint = await train(
train_data=flyte.io.File(path=train_uri),
val_data=flyte.io.File(path=val_uri),
hyperparams=hyperparams,
)
return {"checkpoint": checkpoint, "hyperparams": hyperparams}
```
### scikit-learn Training
```python
import flyte
import flyte.io
env = flyte.TaskEnvironment(
name="sklearn-training",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"scikit-learn", "pandas", "polars", "joblib",
),
)
@env.task
async def train_sklearn(
train_data: flyte.io.DataFrame,
val_data: flyte.io.DataFrame,
model_type: str = "random_forest",
) -> flyte.io.File:
"""Train a scikit-learn model."""
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
import joblib
X_train = train_data.to_polars().drop("label").to_numpy()
y_train = train_data.to_polars()["label"].to_numpy()
X_val = val_data.to_polars().drop("label").to_numpy()
y_val = val_data.to_polars()["label"].to_numpy()
if model_type == "random_forest":
model = RandomForestClassifier(n_estimators=100)
elif model_type == "gbm":
model = GradientBoostingClassifier(n_estimators=100)
else:
model = LogisticRegression()
model.fit(X_train, y_train)
accuracy = model.score(X_val, y_val)
path = f"/tmp/{model_type}_model.joblib"
joblib.dump(model, path)
return flyte.io.File(path=path)
```
### HuggingFace Trainer
```python
import flyte
import flyte.io
env = flyte.TaskEnvironment(
name="hf-training",
image=flyte.Image.from_base("pytorch/pytorch:2.1-cuda12.1-cudnn8-devel").with_pip_packages(
"transformers", "datasets", "accelerate", "evaluate",
),
)
@env.task(
requests=flyte.Resources(
cpu="4", memory="16Gi", gpu="1", gpu_model="nvidia-a10g",
),
)
async def train_hf(
dataset_name: str,
model_name: str,
hyperparams: dict,
) -> flyte.io.File:
"""Train with HuggingFace Trainer."""
from datasets import load_dataset
from transformers import (
AutoModelForSequenceClassification,
AutoTokenizer,
Trainer,
TrainingArguments,
)
train_dataset = load_dataset(dataset_name, split="train")
val_dataset = load_dataset(dataset_name, split="validation")
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(
model_name, num_labels=2
)
def tokenize(examples):
return tokenizer(examples["text"], truncation=True, padding="max_length", max_length=512)
train_dataset = train_dataset.map(tokenize)
val_dataset = val_dataset.map(tokenize)
training_args = TrainingArguments(
output_dir="/tmp/training_output",
learning_rate=hyperparams.get("lr", 2e-5),
per_device_train_batch_size=hyperparams.get("batch_size", 16),
num_train_epochs=hyperparams.get("epochs", 3),
evaluation_strategy="epoch",
save_strategy="epoch",
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_dataset,
eval_dataset=val_dataset,
)
trainer.train()
trainer.save_model("/tmp/final_model")
tokenizer.save_pretrained("/tmp/final_model")
return flyte.io.File(path="/tmp/final_model")
```
## Hyperparameter Optimization
### Manual HPO with fan-out
```python
import flyte
env = flyte.TaskEnvironment(
name="hpo",
image=flyte.Image.from_base("pytorch/pytorch:2.1-cuda12.1-cudnn8-devel").with_pip_packages(
"transformers", "datasets",
),
)
@env.task(
requests=flyte.Resources(
cpu="4", memory="16Gi", gpu="1", gpu_model="nvidia-a10g",
),
)
async def train_trial(hyperparams: dict) -> dict:
"""Run a single hyperparameter trial."""
# hyperparams = {"model": "bert-base", "lr": 2e-5, "batch_size": 16, "epochs": 3}
checkpoint = await train_hf(
dataset_name="glue/mnli",
model_name=hyperparams["model"],
hyperparams=hyperparams,
)
# Evaluate
metrics = await evaluate(checkpoint, "glue/mnli", split="validation")
return {
"hyperparams": hyperparams,
"accuracy": metrics["accuracy"],
"checkpoint": checkpoint,
}
@env.task
async def hpo_search(
param_grid: list[dict],
) -> dict:
"""Run hyperparameter search with parallel trials."""
# Fan out all trials in parallel
results = await flyte.map(train_trial, param_grid)
best = max(results, key=lambda r: r["accuracy"])
return best
@env.task
async def main() -> dict:
param_grid = [
{"model": "bert-base", "lr": 1e-5, "batch_size": 16, "epochs": 3},
{"model": "bert-base", "lr": 2e-5, "batch_size": 16, "epochs": 3},
{"model": "bert-base", "lr": 5e-5, "batch_size": 16, "epochs": 3},
{"model": "bert-base", "lr": 2e-5, "batch_size": 32, "epochs": 3},
]
return await hpo_search(param_grid)
```
### Grid search pattern
```python
from itertools import product
@env.task
async def grid_search() -> dict:
"""Grid search over hyperparameter combinations."""
lr_values = [1e-5, 2e-5, 5e-5]
batch_sizes = [16, 32]
epochs = [2, 3]
param_grid = [
{"model": "bert-base", "lr": lr, "batch_size": bs, "epochs": ep}
for lr, bs, ep in product(lr_values, batch_sizes, epochs)
]
results = await flyte.map(train_trial, param_grid)
best = max(results, key=lambda r: r["accuracy"])
return best
```
## Experiment Tracking
### Manual experiment tracking
```python
import json
import datetime
import flyte
import flyte.io
@env.task
async def track_experiment(
experiment_name: str,
hyperparams: dict,
metrics: dict,
checkpoint: flyte.io.File,
) -> flyte.io.File:
"""Track experiment results as a JSON file in remote storage."""
record = {
"experiment": experiment_name,
"timestamp": datetime.datetime.now().isoformat(),
"hyperparameters": hyperparams,
"metrics": metrics,
"checkpoint_uri": checkpoint.path,
}
path = f"/tmp/experiments/{experiment_name}_{datetime.datetime.now().strftime('%Y%m%d_%H%M%S')}.json"
with open(path, "w") as f:
json.dump(record, f, indent=2)
return flyte.io.File(path=path)
@env.task
async def compare_experiments(
experiment_names: list[str],
) -> dict:
"""Compare multiple experiments."""
reports = []
for name in experiment_names:
report = await load_experiment(name)
reports.append(report)
# Find best by metric
best = max(reports, key=lambda r: r["metrics"].get("accuracy", 0))
return {"best_experiment": best, "all": reports}
```
### Inference result tracking
```python
@env.task
async def track_inference(
model_uri: str,
test_data: flyte.io.File,
metrics: dict,
) -> flyte.io.File:
"""Track inference results."""
record = {
"model_uri": model_uri,
"test_data": test_data.path,
"metrics": metrics,
"timestamp": datetime.datetime.now().isoformat(),
}
path = f"/tmp/inference/{model_uri.split('/')[-1]}_{datetime.datetime.now().strftime('%Y%m%d')}.json"
with open(path, "w") as f:
json.dump(record, f, indent=2)
return flyte.io.File(path=path)
```
## Model Evaluation and Selection
### Evaluation pipeline
```python
import flyte
import flyte.io
env = flyte.TaskEnvironment(
name="evaluation",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"scikit-learn", "scipy", "pandas", "matplotlib", "seaborn",
),
)
@env.task
async def evaluate_model(
model_path: flyte.io.File,
test_data: flyte.io.DataFrame,
) -> dict:
"""Evaluate a model and return metrics."""
import joblib
from sklearn.metrics import (
accuracy_score, f1_score, precision_score, recall_score,
roc_auc_score, confusion_matrix, classification_report,
)
model = joblib.load(model_path.path)
X_test = test_data.to_polars().drop("label").to_numpy()
y_test = test_data.to_polars()["label"].to_numpy()
y_pred = model.predict(X_test)
y_prob = model.predict_proba(X_test)[:, 1] if hasattr(model, "predict_proba") else y_pred
return {
"accuracy": accuracy_score(y_test, y_pred),
"f1": f1_score(y_test, y_pred),
"precision": precision_score(y_test, y_pred),
"recall": recall_score(y_test, y_pred),
"auc": roc_auc_score(y_test, y_prob),
"confusion_matrix": confusion_matrix(y_test, y_pred).tolist(),
"report": classification_report(y_test, y_pred, output_dict=True),
}
@env.task
async def select_best_model(
candidate_models: list[flyte.io.File],
test_data: flyte.io.DataFrame,
) -> dict:
"""Evaluate all candidates and select the best."""
evaluations = await flyte.map(
lambda m: evaluate_model(m, test_data),
candidate_models,
)
best = max(evaluations, key=lambda e: e["accuracy"])
return {"best_metrics": best, "all_evaluations": evaluations}
```
### Model comparison report
```python
@env.task
async def generate_comparison_report(
evaluations: list[dict],
model_names: list[str],
) -> flyte.io.File:
"""Generate a model comparison report."""
import matplotlib.pyplot as plt
import pandas as pd
df = pd.DataFrame({
"model": model_names,
"accuracy": [e["accuracy"] for e in evaluations],
"f1": [e["f1"] for e in evaluations],
"precision": [e["precision"] for e in evaluations],
"recall": [e["recall"] for e in evaluations],
"auc": [e["auc"] for e in evaluations],
})
fig, axes = plt.subplots(1, 3, figsize=(18, 5))
metrics = ["accuracy", "f1", "precision", "recall", "auc"]
for i, metric in enumerate(metrics[:3]):
axes[i].bar(df["model"], df[metric])
axes[i].set_title(metric)
axes[i].tick_params(axis="x", rotation=45)
path = "/tmp/model_comparison.png"
fig.savefig(path, bbox_inches="tight")
return flyte.io.File(path=path)
```
## Batch Inference
### Large-scale batch inference
```python
import flyte
import flyte.io
env = flyte.TaskEnvironment(
name="batch-inference",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"torch", "transformers", "pandas", "polars", "boto3",
),
)
@env.task(
requests=flyte.Resources(
cpu="4", memory="16Gi", gpu="1", gpu_model="nvidia-a10g",
),
)
async def load_model(model_uri: str) -> object:
"""Load model into memory."""
from transformers import AutoModelForSequenceClassification, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(model_uri)
model = AutoModelForSequenceClassification.from_pretrained(model_uri)
model.eval()
return {"model": model, "tokenizer": tokenizer}
@env.task(
requests=flyte.Resources(
cpu="2", memory="8Gi", gpu="1", gpu_model="nvidia-a10g",
),
)
async def batch_predict(
model_ctx: object,
data_file: flyte.io.File,
batch_size: int = 32,
) -> flyte.io.File:
"""Run inference on a batch of data."""
import torch
import polars as pl
model = model_ctx["model"]
tokenizer = model_ctx["tokenizer"]
df = pl.read_parquet(data_file.path)
texts = df["text"].to_list()
all_preds = []
all_probs = []
for i in range(0, len(texts), batch_size):
batch = texts[i:i + batch_size]
inputs = tokenizer(batch, padding=True, truncation=True, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
probs = torch.softmax(outputs.logits, dim=1)
preds = torch.argmax(probs, dim=1)
all_preds.extend(preds.tolist())
all_probs.extend(probs.tolist())
results = pl.DataFrame({"prediction": all_preds, "probability": all_probs})
path = f"/tmp/predictions_{data_file.path.split('/')[-1]}"
results.write_parquet(path)
return flyte.io.File(path=path)
@env.task
async def batch_inference(
model_uri: str,
data_files: list[str],
) -> list:
"""Run batch inference on multiple data files."""
model_ctx = await load_model(model_uri)
# Fan out inference across files
results = await flyte.map(
lambda f: batch_predict(model_ctx, flyte.io.File(path=f)),
data_files,
)
return results
```
### GPU batch inference optimization
```python
@env.task
async def optimized_batch_inference(
model_uri: str,
data_files: list[str],
) -> list:
"""Optimized batch inference with dynamic batching."""
# Use dynamic batcher for better GPU utilization
# Combine small batches and shard large ones
...
```
## Real-time Model Serving
### FastAPI model serving (covered in flyte-sdk-app)
```python
from fastapi import FastAPI
import flyte
from flyte.app.extras import FastAPIAppEnvironment
app = FastAPI()
model = None
@app.on_event("startup")
async def load_model():
global model
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model = AutoModelForSequenceClassification.from_pretrained("model-checkpoint")
model.tokenizer = AutoTokenizer.from_pretrained("model-checkpoint")
@app.get("/predict")
async def predict(text: str) -> dict:
inputs = model.tokenizer(text, return_tensors="pt", padding=True, truncation=True)
with torch.no_grad():
outputs = model(**inputs)
probs = torch.softmax(outputs.logits, dim=1)
return {
"prediction": int(torch.argmax(probs, dim=1)[0]),
"confidence": float(probs.max().item()),
}
env = FastAPIAppEnvironment(
name="model-serving",
app=app,
image=flyte.Image.from_base("pytorch/pytorch:2.1-cuda12.1-cudnn8-devel").with_pip_packages(
"fastapi", "uvicorn", "torch", "transformers",
),
resources=flyte.Resources(cpu="4", memory="16Gi", gpu="1", gpu_model="nvidia-a10g"),
)
```
## Model Monitoring
### Drift detection
```python
@env.task(cache="auto")
async def detect_drift(
baseline_data: flyte.io.DataFrame,
current_data: flyte.io.DataFrame,
) -> dict:
"""Detect data drift between baseline and current distributions."""
import scipy.stats as stats
drift_results = {}
baseline_df = baseline_data.to_polars()
current_df = current_data.to_polars()
for col in baseline_df.columns:
if baseline_df[col].dtype.is_float64():
# Kolmogorov-Smirnov test
stat, p_value = stats.ks_2samp(
baseline_df[col].to_list(),
current_df[col].to_list(),
)
drift_results[col] = {
"statistic": stat,
"p_value": p_value,
"drift_detected": p_value < 0.05,
}
return drift_results
@env.task
async def monitor_model(
model_uri: str,
baseline_data: flyte.io.DataFrame,
current_data: flyte.io.DataFrame,
predictions: flyte.io.DataFrame,
) -> dict:
"""Monitor model health: drift, performance, prediction distribution."""
drift = await detect_drift(baseline_data, current_data)
# Prediction distribution analysis
pred_dist = predictions.to_polars()["prediction"].value_counts().to_dict()
# Confidence distribution
conf_stats = {
"mean": float(predictions.to_polars()["probability"].mean()),
"std": float(predictions.to_polars()["probability"].std()),
"min": float(predictions.to_polars()["probability"].min()),
"max": float(predictions.to_polars()["probability"].max()),
}
return {
"drift": drift,
"prediction_distribution": pred_dist,
"confidence_stats": conf_stats,
"alert": any(d["drift_detected"] for d in drift.values()),
}
```
### Prediction quality monitoring
```python
@env.task
async def monitor_prediction_quality(
predictions: flyte.io.DataFrame,
ground_truth: flyte.io.DataFrame,
) -> dict:
"""Monitor prediction quality over time."""
merged = predictions.to_polars().join(ground_truth.to_polars(), on="id")
accuracy = (merged["prediction"] == merged["label"]).mean()
# Per-class performance
per_class = {}
for label in merged["label"].unique():
mask = merged["label"] == label
per_class[int(label)] = {
"count": int(mask.sum()),
"accuracy": int(merged[mask]["prediction"] == merged[mask]["label"]).mean(),
}
return {"accuracy": float(accuracy), "per_class": per_class}
```
## ML Resource Recommendations
| ML Workload | CPU | Memory | GPU |
|---|---|---|---|
| scikit-learn (small data) | 2-4 | 4-8 Gi | none |
| scikit-learn (large data) | 4-8 | 16-32 Gi | none |
| PyTorch training (small model) | 4 | 16 Gi | 1x A10G |
| PyTorch training (large model) | 8 | 32+ Gi | 4-8x A100 |
| HuggingFace fine-tuning | 4-8 | 16-32 Gi | 1-4x A10G/A100 |
| Batch inference (CPU) | 4-8 | 16-32 Gi | none |
| Batch inference (GPU) | 4 | 16 Gi | 1-4x A10G/A100 |
| LLM serving | 8-16 | 32-64 Gi | 1-8x A100/H100 |
## ML Anti-Patterns
1. **Don't train without experiment tracking** — always log hyperparams, metrics, and model artifacts.
2. **Don't skip evaluation** — always evaluate on held-out test data with multiple metrics.
3. **Don't over-provision GPUs** — start with 1x A10G for most fine-tuning, scale only when needed.
4. **Don't do batch inference one-by-one** — use `flyte.map` for parallel file-level inference.
5. **Don't use Union-only features** — avoid `ReusePolicy` and other Union-specific APIs.
6. **Don't forget to set `cache="auto"`** on evaluation tasks — same model + same data = same result.
flyte-sdk-optimize8.8 KB
---
name: flyte-sdk-optimize
description: 'Suggests performance improvements (task granularity, caching, resource requests, data format changes) using observed run metadata when available. Use when the user wants to optimize workflow performance, debug slow tasks, configure caching, tune resources, or improve throughput. Trigger words: "optimize", "performance", "slow", "cache", "caching", "resource", "throughput", "latency", "speed up", "bottleneck", "profiling", "metadata".'
---
# Flyte 2 SDK Optimize Skill
Optimize Flyte 2 workflows for performance, cost, and reliability.
## Grounding References
| Resource | URL |
|---|---|
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| CLI API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-cli/ |
| flyte-sdk source | https://github.com/flyteorg/flyte-sdk |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via the `flyte-cluster` and `flyte-docs` MCP servers |
## Optimization Strategy Overview
Performance optimization in Flyte follows a hierarchy:
1. **Reduce container overhead** — use traces for lightweight ops
2. **Parallelize work** — use `flyte.map` for fan-out
3. **Cache results** — use `cache="auto"` for idempotent tasks
4. **Tune resources** — set appropriate CPU/memory/GPU
5. **Optimize data transfer** — choose efficient formats, reduce inline I/O
6. **Use reusable containers** — shared environments reduce image pull time
## Caching
### Enable automatic caching
```python
@env.task(cache="auto") # versioned by function body + inputs
async def preprocess(data: list[str]) -> flyte.io.File:
...
```
### Cache key strategies
```python
@env.task(cache="auto") # default: function body + inputs
async def task_a(data: str) -> flyte.io.File:
...
@env.task(cache="override", salt="v2") # add salt for cache key variation
async def task_b(data: str) -> flyte.io.File:
...
@env.task(cache="disable") # always re-run
async def task_c(data: str) -> flyte.io.File:
...
```
### Content-based caching for DataFrames
```python
@env.task(cache="auto")
async def transform(df: flyte.io.DataFrame) -> flyte.io.DataFrame:
"""Cache key includes DataFrame content hash."""
...
```
### Ignoring specific inputs in cache key
```python
@env.task(cache="auto", cache_ignore_inputs=["api_key"])
async def fetch_data(api_key: str, url: str) -> flyte.io.File:
"""Don't include api_key in cache key."""
...
```
### Cache policies
```python
@env.task(cache="auto", cache_policy=flyte.CachePolicy(min_cached_age="1h"))
async def cached_task(data: str) -> flyte.io.File:
"""Only use cache if result is at least 1 hour old."""
...
```
## Resource Tuning
### Setting task resources
```python
@env.task(
requests=flyte.Resources(cpu="500m", memory="1Gi"),
limits=flyte.Resources(cpu="2", memory="4Gi"),
)
async def light_task(data: str) -> str:
"""Lightweight task — small resources."""
...
@env.task(
requests=flyte.Resources(cpu="4", memory="16Gi"),
limits=flyte.Resources(cpu="8", memory="32Gi"),
)
async def heavy_task(data: flyte.io.DataFrame) -> flyte.io.DataFrame:
"""Heavy data processing — large resources."""
...
@env.task(
requests=flyte.Resources(cpu="1", memory="4Gi", gpu="1", gpu_model="nvidia-a10g"),
limits=flyte.Resources(cpu="2", memory="8Gi", gpu="1", gpu_model="nvidia-a10g"),
)
async def train_model(data: flyte.io.File) -> flyte.io.File:
"""GPU training task."""
...
```
### GPU resource configuration
```python
@env.task(
requests=flyte.Resources(
cpu="2",
memory="8Gi",
gpu="1",
gpu_model="nvidia-a10g", # or "nvidia-a100", "nvidia-h100"
),
)
async def inference(batch: flyte.io.DataFrame) -> flyte.io.DataFrame:
...
```
### Resource recommendations by workload
| Workload | CPU | Memory | GPU |
|---|---|---|---|
| Light ETL | 500m-1 | 1-2 Gi | none |
| Data processing | 2-4 | 8-16 Gi | none |
| Embedding | 2-4 | 8-16 Gi | none |
| Model training | 4-8 | 16-32 Gi | 1-8 |
| Batch inference | 2-4 | 8-16 Gi | 1-4 |
| LLM serving | 8-16 | 32-64 Gi | 1-8 |
| Data quality | 1-2 | 4-8 Gi | none |
## Parallelization Patterns
### flyte.map for parallel execution
```python
@env.task
async def process_item(item: dict) -> dict:
"""Process a single item."""
...
@env.task
async def main(items: list[dict]) -> list:
"""Fan out processing in parallel."""
results = await flyte.map(process_item, items)
return results
```
### flyte.trace for lightweight parallelism
```python
@env.task
async def fetch_url(url: str) -> str:
"""Lightweight HTTP fetch — use trace (no container overhead)."""
...
@env.task
async def main(urls: list[str]) -> list:
"""Use trace for light ops (no container spin-up cost)."""
results = await flyte.trace(fetch_url, urls)
return results
```
### asyncio.gather for sequential fan-out
```python
@env.task
async def main(data: list[str]) -> dict:
"""Chain tasks with parallel fan-out at each step."""
# Step 1: parallel preprocessing
preprocessed = await asyncio.gather(*(preprocess(d) for d in data))
# Step 2: sequential aggregation
aggregated = aggregate(preprocessed)
# Step 3: parallel evaluation
metrics = await asyncio.gather(*(evaluate(p) for p in preprocessed))
return {"aggregated": aggregated, "metrics": metrics}
```
### Controlling concurrency
```python
@env.task
async def main(urls: list[str]) -> list:
"""Limit concurrency with asyncio.Semaphore."""
import asyncio
sem = asyncio.Semaphore(10) # max 10 concurrent
async def bounded(item):
async with sem:
return await process_item(item)
return await asyncio.gather(*(bounded(u) for u in urls))
```
## Data Format Optimization
### Choosing efficient formats
| Use case | Recommended format | Why |
|---|---|---|
| Tabular data | Parquet | Columnar, compressed, fast |
| JSON data | JSONL | Line-delimited, streaming |
| Images | PNG/WebP | Lossless/lossy compression |
| Audio | WAV/FLAC | Lossless |
| Model checkpoints | .pt/.safetensors | Native framework format |
| Embeddings | .npy/.npz | NumPy binary format |
### Reducing inline I/O
```python
# Bad: large dict passed inline (JSON serialization overhead)
@env.task
async def process(large_data: dict) -> dict:
...
# Good: pass by reference
@env.task
async def process(data_file: flyte.io.File) -> flyte.io.File:
...
# Good: set inline output limit
@env.task(inline_output_limit="5MB")
async def process(data: dict) -> dict:
...
```
## Run Metadata Inspection
### Using MCP to inspect runs
If Flyte MCP tools are available, use them to list a task's recent runs for performance
analysis, fetch a run's metadata (status, duration), and read its inputs and outputs.
### Performance analysis checklist
1. **Check run duration** — fetch the run's metadata and read `durationMs`
2. **Check cache status** — `CACHE_HIT` vs `CACHE_MISS` in run metadata
3. **Check resource utilization** — compare requested vs actual usage
4. **Check data transfer** — large inline I/O indicates format issues
5. **Check retry count** — frequent retries indicate instability
## Retry and Timeout Configuration
### Retries for resilience
```python
@env.task(retries=3) # retry up to 3 times on failure
async def flaky_task(data: str) -> str:
"""Task that may fail transiently."""
...
@env.task(retries=flyte.RetryStrategy(count=3))
async def critical_task(data: str) -> str:
"""Always retry, never fail fast."""
...
```
### Timeouts for bounding execution
```python
@env.task(max_runtime="1h") # bound single attempt
async def long_task(data: str) -> str:
...
@env.task(max_queued_time="30m") # fail fast if no capacity
async def urgent_task(data: str) -> str:
...
@env.task(deadline="2h") # bound total wall-clock (all attempts)
async def deadline_task(data: str) -> str:
...
```
## Interruptible Tasks (Spot Instances)
```python
@env.task(interruptible=True) # can be preempted, falls back to on-demand
async def spot_task(data: str) -> str:
"""Cost-effective for fault-tolerant workloads."""
...
```
## Optimization Anti-Patterns
1. **Don't over-cache** — avoid `cache="auto"` on tasks with side effects or non-deterministic outputs
2. **Don't set resources too high** — over-provisioning wastes money; too low causes OOM
3. **Don't use `asyncio.gather` for heavy workloads** — use `flyte.map` for parallel container execution
4. **Don't skip caching on ETL** — idempotent data transforms should always cache
5. **Don't use Union-only features** — avoid `ReusePolicy` and other Union-specific APIs
6. **Don't ignore run metadata** — always check `durationMs` and cache status when debugging performance
flyte-sdk-run10.1 KB
---
name: flyte-sdk-run
description: 'Runs Flyte 2 workflows, interacts with runs and actions, retrieves logs and data, and manages run lifecycle. Use when the user wants to run a workflow, check run status, view logs, get run outputs, re-run a workflow, or manage runs programmatically. Trigger words: "run", "execute", "logs", "status", "output", "input", "watch", "rerun", "cancel", "abort", "run metadata", "action".'
---
# Flyte 2 SDK Run Skill
Run workflows, interact with runs, and manage the execution lifecycle.
## Grounding References
| Resource | URL |
|---|---|
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| CLI API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-cli/ |
| flyte-sdk source | https://github.com/flyteorg/flyte-sdk |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via the `flyte-cluster` and `flyte-docs` MCP servers |
## Tool Priority
1. **Flyte MCP** — if the harness has Flyte MCP tools, prefer them over shelling out to
the CLI. They cover listing runs, fetching run details and inputs/outputs, polling to
completion, executing a task, and aborting a run, and they return structured data
instead of text you have to parse.
2. **`flyte` CLI** — for local run commands, and anything MCP does not expose
3. **Python SDK** — for programmatic run control
## Running Workflows
### Via Python SDK
```python
import flyte
if __name__ == "__main__":
# Run with defaults from config
result = flyte.run(main, inputs={"data": ["a", "b", "c"]})
print(f"Run name: {result.name}")
print(f"Status: {result.status}")
```
### Via CLI
```bash
# Run with local config
flyte run pipeline.py main --data '[1,2,3]'
# Run with specific project/domain
flyte run pipeline.py main --data '[1,2,3]' --project flytesnacks --domain development
# Run with custom run name
flyte run pipeline.py main --data '[1,2,3]' --name my-custom-run
# Run with specific image
flyte run pipeline.py main --data '[1,2,3]' --image ghcr.io/myorg/task:v1.0
# Run with local mode (in-process, no remote)
flyte run --local pipeline.py main --data '[1,2,3]'
# Run with TUI
flyte run --tui --local pipeline.py main --data '[1,2,3]'
# Pass arguments by type
flyte run pipeline.py main \
--data '[1,2,3]' \
--learning-rate 0.001 \
--batch-size 32 \
--train-data s3://bucket/train.parquet \
--flag true
```
### Run command options
| Flag | Description |
|---|---|
| `--project` / `--domain` | Target project and domain |
| `--run-project` / `--run-domain` | Override run project/domain |
| `--local` | Run locally (in-process) |
| `--tui` | Terminal UI for local runs |
| `--name` | Custom run name |
| `--image` | Image mapping (named or default) |
| `--copy-style` | `loaded_modules` (default), `all`, `none` |
| `--root-dir` | Set root directory for code bundling |
| `--raw-data-path` | Override raw data path |
| `--service-account` | K8s service account |
| `--follow` | Follow run progress |
| `--no-sync-local-sys-paths` | Skip local sys path sync |
### Passing inputs by type
```bash
# List
flyte run pipeline.py main --data '[1,2,3]'
# Dict
flyte run pipeline.py main --config '{"lr": 0.001, "epochs": 10}'
# Boolean
flyte run pipeline.py main --flag true
# Datetime
flyte run pipeline.py main --date '2025-01-01T00:00:00'
# Duration
flyte run pipeline.py main --timeout '1h'
# File
flyte run pipeline.py main --input-file s3://bucket/data.parquet
# DataFrame (via file path)
flyte run pipeline.py main --data-file /path/to/data.parquet
```
## Interacting with Runs
### Using Flyte MCP
If Flyte MCP tools are available, prefer them for all of the above — listing runs,
fetching a run's details, polling until it completes, and reading its inputs and outputs.
### Using CLI
```bash
# List runs
flyte get run --project flytesnacks --domain development
# Get run info
flyte get run <run_name> --project flytesnacks --domain development
# Watch run progress
flyte get run <run_name> --project flytesnacks --domain development
# Get run outputs
flyte get io <run_name> --project flytesnacks --domain development
# Download run artifacts
flyte get io <run_name> --outputs-only --project flytesnacks --domain development
```
### Using Python SDK
```python
import flyte
# Run and get handle
result = flyte.run(main, inputs={"data": ["a", "b"]})
# Check status
print(result.status) # RUNNING, SUCCEEDED, FAILED, CANCELED
# Wait for completion
result.wait()
# Get outputs
print(result.outputs)
# Get URL in console
print(result.url)
```
## Viewing Logs
### Using CLI
```bash
# Stream logs
flyte get logs <run_name> --project flytesnacks --domain development
# View logs for a specific attempt
flyte get logs <run_name> --attempt 0
# Filter system logs
flyte get logs <run_name> --filter-system
# Scope to project/domain
flyte get logs <run_name> --project flytesnacks --domain development
```
### CLI log options
| Flag | Description |
|---|---|
| `--attempt` / `-a` | View specific attempt logs |
| `--filter-system` | Filter out system logs |
| `--pretty` | Auto-scrolling box (limited to `--lines`) |
| `--project` / `--domain` | Scope logs |
### Using Python SDK
```python
import flyte
result = flyte.run(main, inputs={"data": ["a"]})
# Logs are retrieved via the CLI: `flyte get logs <run_name>`
print(result.url) # open the run in the UI to view logs
```
## Re-running Runs
### CLI
```bash
# Re-run with original code and inputs
flyte rerun <run_name> --project flytesnacks --domain development
# Re-run with new local code
flyte run --rerun-from <run_name> pipeline.py main --data '[4,5,6]'
```
### Python SDK
```python
import flyte
# Re-run with new inputs
result = flyte.run(
main,
inputs={"data": [4, 5, 6]},
run_context=flyte.with_runcontext(run_name="rerun-of-abc123"),
)
```
## Running Tasks (vs Workflows)
### Run a single task
```bash
# Ephemeral run (deploy + run in one command)
flyte run pipeline.py preprocess --data '[1,2,3]'
# Run a deployed task
flyte run --task-name preprocess --project flytesnacks --domain development \
--inputs '{"data": "[1,2,3]"}'
```
### Using Flyte MCP
Executing a registered task is available as an MCP tool, taking project, domain, task name,
version, and inputs.
## Run Context Configuration
### Programmatic run context
```python
import flyte
# Configure a run programmatically
result = flyte.run(
main,
inputs={"data": ["a", "b"]},
run_context=flyte.with_runcontext(
project="flytesnacks",
domain="development",
raw_data_path="s3://my-bucket/{run_id}/",
service_account="my-sa",
),
)
```
### Reading run context inside a task
```python
@env.task
async def my_task(data: str) -> str:
# Access run metadata inside the task
ctx = flyte.ctx()
print(f"Run: {ctx.run_id}")
print(f"Project: {ctx.project}")
print(f"Domain: {ctx.domain}")
print(f"Version: {ctx.version}")
return data
```
## Abort and Cancel Runs
### CLI
```bash
# Abort a run
flyte abort run <run_name> --project flytesnacks --domain development
```
### Python SDK
```python
import flyte
result = flyte.run(main, inputs={"data": ["a"]})
result.abort()
```
### Using Flyte MCP
Aborting a run is available as an MCP tool, taking the run name.
## Programmatic Abort from Within a Task
```python
@env.task
async def long_task(data: str) -> str:
import asyncio
import signal
async def check_abort():
while True:
if asyncio.current_task().cancelled():
raise asyncio.CancelledError("Run was aborted")
await asyncio.sleep(1)
# Start abort watcher
watcher = asyncio.create_task(check_abort())
try:
# Long-running work
await asyncio.sleep(3600)
finally:
watcher.cancel()
await watcher
return data
```
## Run Data Access
### Accessing large data from cloud storage
```python
import flyte
import flyte.io
@env.task
async def get_run_data(run_name: str) -> flyte.io.File:
"""Download artifacts from a past run."""
# Flyte stores outputs in the metadata bucket
# Access via the SDK's data retrieval methods
...
@env.task
async def upload_local_data(file_path: str) -> flyte.io.File:
"""Upload local file to remote storage for a run."""
return flyte.io.File(path=file_path)
```
### S3 / GCS / Azure access
```python
# S3
import boto3
s3 = boto3.client("s3")
obj = s3.get_object(Bucket="my-bucket", Key="run-artifacts/output.parquet")
# GCS
from google.cloud import storage
client = storage.Client()
bucket = client.bucket("my-bucket")
blob = bucket.blob("run-artifacts/output.parquet")
# Azure
from azure.storage.blob import BlobServiceClient
client = BlobServiceClient(account_url="https://myacct.blob.core.windows.net/")
blob = client.get_blob_client(container="my-container", blob="output.parquet")
```
## Run Modes
### Local execution
```bash
# In-process (no remote backend needed)
flyte run --local pipeline.py main --data '[1,2,3]'
# With TUI
flyte run --tui --local pipeline.py main --data '[1,2,3]'
```
### Devbox
```bash
# Start local dev environment
flyte start devbox
# Create config for devbox
flyte create config \
--endpoint localhost:30080 \
--project flytesnacks \
--domain development \
--builder local \
--insecure
# Run on devbox
flyte run pipeline.py main --data '[1,2,3]'
```
### Remote execution
```bash
# Create config for remote backend
flyte create config \
--endpoint <host> \
--project flytesnacks \
--domain development \
--builder local \
--insecure
# Run on remote backend
flyte run pipeline.py main --data '[1,2,3]'
```
## Anti-Patterns
1. **Don't confuse `flyte run` (workflow) with `flyte run --task-name` (single task)** — use the right command for your intent.
2. **Don't skip `--follow`** when running long workflows — you won't see progress.
3. **Don't hardcode run names** — let Flyte generate them, or use meaningful prefixes.
4. **Don't access run data directly from S3/GCS** — use Flyte's data retrieval methods when possible.
5. **Don't use Union-only features** — avoid `ReusePolicy` and other Union-specific APIs.
flyte-sdk-ship9.12 KB
---
name: flyte-sdk-ship
description: 'Generates flyte.Image specs, Dockerfiles, dependency management, image tagging strategy, and reproducible build instructions for Flyte tasks. Use when the user needs to build container images for Flyte tasks, configure custom images, manage dependencies, or set up reproducible builds. Trigger words: "image", "Docker", "build", "dependency", "pip package", "debian base", "image builder", "push image", "container", "Dockerfile", "uv", "requirements".'
---
# Flyte 2 SDK Ship Skill
Generate images, Dockerfiles, and dependency configurations for Flyte 2 tasks.
## Grounding References
| Resource | URL |
|---|---|
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| CLI API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-cli/ |
| flyte-sdk source | https://github.com/flyteorg/flyte-sdk |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via the `flyte-cluster` and `flyte-docs` MCP servers |
**Ground unfamiliar APIs in real examples.** When unsure of a current Flyte 2 API, or for a pattern not shown below, and the `flyte-docs` search tools are available, search them first — by exact symbol (`TaskEnvironment`, `flyte.io.File`, `map_task`), since matching is literal substring, not semantic — then adapt a real example rather than inventing one, and cite the file or section you pulled it from. (Flyte 2 is not `flytekit`; priors are often wrong.)
## flyte.Image — Programmatic Image Definition
Use `flyte.Image` to define task container images in Python. Flyte builds and pushes them automatically.
### From Debian Base
```python
import flyte
env = flyte.TaskEnvironment(
name="etl-pipeline",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"pandas", "polars", "pyarrow", "boto3",
).with_system_packages(
"git", "curl", "wget",
),
)
```
### From Existing Image
```python
env = flyte.TaskEnvironment(
name="ml-training",
image=flyte.Image.from_base(
"ghcr.io/flyteorg/flyte:py3.12-v2", # base image
).with_pip_packages(
"torch", "transformers", "datasets",
),
)
```
### Image from uv Script Metadata
When using a `# /// script` header, Flyte can derive the image from the script's dependencies:
```python
# /// script
# requires-python = ">=3.12"
# dependencies = [
# "pandas",
# "polars",
# ]
# ///
# Flyte reads the script metadata and builds the image automatically
```
## Image Configuration Methods
### with_pip_packages
```python
image = flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"pandas", # latest version
"torch>=2.0", # version constraint
"transformers==4.40", # pinned version
)
```
### with_system_packages
```python
image = image.with_system_packages(
"git", "curl", "wget", "jq", "ffmpeg",
)
```
### with_commands (apt-get run)
```python
image = image.with_commands(
"apt-get update && apt-get install -y libgl1-mesa-glx", # for OpenCV
"pip install --upgrade pip",
)
```
### with_env_vars
```python
image = image.with_env_vars({
"HF_HUB_DISABLE_TELEMETRY": "1",
"PYTHONDONTWRITEBYTECODE": "1",
})
```
### with_local_rs_controller
```python
# For development: bake the Rust controller wheel into the image
image = image.with_local_rs_controller()
```
## Custom Dockerfile
For complex builds, use a custom `Dockerfile`:
```dockerfile
# Dockerfile
FROM python:3.12-slim
RUN apt-get update && apt-get install -y \
git \
libgl1-mesa-glx \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir \
torch \
transformers \
datasets
WORKDIR /app
COPY . /app
ENV HF_HUB_DISABLE_TELEMETRY=1
```
```python
env = flyte.TaskEnvironment(
name="ml-training",
image=flyte.Image.from_dockerfile("Dockerfile"),
)
```
## Image Builder Configuration
### Local builder (default for development)
```yaml
# .flyte/config.yaml
image:
builder: local
```
Builds images locally using Docker. Fast iteration, requires Docker installed.
### Remote builder (CI/production)
```yaml
# .flyte/config.yaml
image:
builder: remote
registry: "ghcr.io/myorg"
repository: "flyte-tasks"
```
Builds images in a remote Docker build service. No local Docker needed.
### Push to registry
```python
# Programmatically configure the builder
env = flyte.TaskEnvironment(
name="training",
image=flyte.Image.from_debian_base(python_version=(3, 12)),
)
# Set registry via config or CLI
```
## Image Tagging Strategy
### Version tags (recommended for production)
```bash
# Tag with git sha for reproducibility
VERSION=$(git rev-parse --short HEAD)
flyte deploy --version $VERSION
```
### Semantic versioning
```bash
# Tag with semver
flyte deploy --version 1.2.3
```
### Auto versioning (development)
```bash
# Flyte auto-generates a version based on code hash
flyte deploy --version auto
```
## Dependency Management Patterns
### Using pyproject.toml
```toml
[project]
name = "my-flyte-pipeline"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"pandas",
"polars",
"flyte",
]
[project.optional-dependencies]
ml = ["torch", "transformers", "datasets"]
dev = ["pytest", "ruff"]
```
### Using requirements.txt
```
pandas>=2.0
polars>=0.20
pyarrow>=14.0
boto3>=1.34
flyte
```
### uv pyproject.toml (monorepo)
```toml
[project]
name = "flyte-monorepo"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["flyte"]
[tool.uv.sources]
# Pin flyte to local path during development
flyte = { workspace = true }
```
## BYOI (Bring Your Own Image) Pattern
For multi-team setups where each team manages their own images:
```python
# team-a/pipeline.py
import flyte
# Reference an externally-built image
env = flyte.TaskEnvironment(
name="team-a-task",
image=flyte.Image.from_base("ghcr.io/team-a/base:v1.2.3"),
)
@env.task
async def process(data: str) -> str:
...
```
```python
# team-b/pipeline.py
import flyte
import flyte.io
env = flyte.TaskEnvironment(
name="team-b-task",
image=flyte.Image.from_base("ghcr.io/team-b/base:v2.0.0"),
)
@env.task
async def train(model_path: flyte.io.File) -> dict:
...
```
## Common Image Recipes
### Data Engineering
```python
env = flyte.TaskEnvironment(
name="etl",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"pandas", "polars", "pyarrow", "boto3", "sqlalchemy",
"db-dtypes", # for BigQuery
"google-cloud-bigquery",
).with_system_packages("git", "curl"),
)
```
### ML Training (GPU)
```python
env = flyte.TaskEnvironment(
name="training",
image=flyte.Image.from_base("nvidia/cuda:12.1-py3").with_pip_packages(
"torch", "torchvision", "transformers", "datasets",
"accelerate", "peft",
),
)
```
### LLM Inference
```python
env = flyte.TaskEnvironment(
name="inference",
image=flyte.Image.from_base("python:3.12-slim").with_pip_packages(
"fastapi", "uvicorn", "torch", "transformers",
"bitsandbytes", "vllm",
),
)
```
### Data Quality
```python
env = flyte.TaskEnvironment(
name="data-quality",
image=flyte.Image.from_debian_base(python_version=(3, 12)).with_pip_packages(
"pandas", "polars", "pyarrow",
"great-expectations", # or "pandera"
"soda-core",
"boto3",
),
)
```
## Build and Deploy Workflow
### Build locally
```bash
# Deploy will auto-build the image
flyte deploy pipeline.py
# Dry-run to see what would be built
flyte deploy --dry-run pipeline.py
```
### Build with remote builder
```bash
# Use remote builder (no Docker needed locally)
flyte deploy --image-builder remote pipeline.py
```
### Push image manually
```bash
# If using a custom registry
docker build -t ghcr.io/myorg/my-task:v1 .
docker push ghcr.io/myorg/my-task:v1
```
### Image caching
Flyte caches built images by content hash. If the image source hasn't changed, it reuses the cached image.
## Troubleshooting
| Issue | Fix |
|---|---|
| `Docker not found` | Install Docker, or use `builder: remote` in config |
| `Permission denied` on Docker socket | Add user to `docker` group: `sudo usermod -aG docker $USER` |
| `Image build failed` | Check Dockerfile syntax, apt package names, pip requirements |
| `Registry push failed` | Verify registry credentials, network connectivity |
| `CUDA not found in container` | Use `nvidia/cuda` base image or install CUDA toolkit in Dockerfile |
| `pip install fails for torch` | Use the correct CUDA index: `--extra-index-url https://download.pytorch.org/whl/cu121` |
| `Image too large` | Use slim base images, multi-stage builds, `.dockerignore` |
## Anti-Patterns
1. **Don't bake secrets into images** — use Flyte secrets instead (`flyte.Secret`).
2. **Don't use `latest` tags in production** — pin to specific versions or git SHAs.
3. **Don't install unnecessary system packages** — they increase image size and build time.
4. **Don't forget `.dockerignore`** — exclude `.git`, `__pycache__`, `.venv`, etc.
5. **Don't use Union-only features** — avoid `ReusePolicy` and other Union-specific APIs.
flyte-sdk-types13.1 KB
---
name: flyte-sdk-types
description: 'Guides correct types, I/O, and serialization for common data (Pandas, Arrow, Parquet, images, audio, HF datasets), including data locality and storage best practices. Use when the user needs help with type annotations, data serialization, file I/O between tasks, custom type transformers, DataFrame handling, or choosing the right Flyte type for their data. Trigger words: "type", "serialize", "deserialize", "DataFrame", "File", "Directory", "custom type", "data format", "Parquet", "Arrow", "Pandas", "type transformer".'
---
# Flyte 2 SDK Types Skill
Guide correct type annotations, I/O patterns, and serialization for Flyte 2 workflows.
## Grounding References
| Resource | URL |
|---|---|
| Official docs | https://www.union.ai/docs/v2/flyte |
| Docs index (LLMs) | https://www.union.ai/docs/v2/flyte/llms.txt |
| SDK API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-sdk/ |
| CLI API reference | https://www.union.ai/docs/v2/union/api-reference/flyte-cli/ |
| flyte-sdk source | https://github.com/flyteorg/flyte-sdk |
| Example code | https://github.com/unionai/unionai-examples |
| Flyte MCP tools | Available via the `flyte-cluster` and `flyte-docs` MCP servers |
**Ground unfamiliar APIs in real examples.** When unsure of a current Flyte 2 API, or for a pattern not shown below, and the `flyte-docs` search tools are available, search them first — by exact symbol (`TaskEnvironment`, `flyte.io.File`, `map_task`), since matching is literal substring, not semantic — then adapt a real example rather than inventing one, and cite the file or section you pulled it from. (Flyte 2 is not `flytekit`; priors are often wrong.)
## Type System Overview
Flyte 2 uses **Python type hints** for serialization. Every task input/output must have a type annotation. Flyte's type transformer system handles conversion between Python types and remote storage (S3, GCS, etc.).
### Supported Native Types
| Python Type | Flyte Type | Remote Transport |
|---|---|---|
| `int`, `float`, `bool`, `str` | Literal | Inline (JSON) |
| `list[T]`, `dict[K, V]` | Collection / Map | Inline (JSON) for small, blob for large |
| `flyte.io.File` | Blob | Uploaded to metadata bucket |
| `flyte.io.Dir` | Blob (directory) | Uploaded to metadata bucket |
| `flyte.io.DataFrame` | DataFrame | Parquet in metadata bucket |
| `dataclass` | Struct | JSON in metadata bucket |
| `pydantic.BaseModel` | Struct | JSON in metadata bucket |
| `datetime`, `timedelta` | DateTime / Duration | Inline |
## flyte.io.File — Single Files
Use `flyte.io.File` for any single file that flows between tasks. Files are **automatically uploaded** to the metadata bucket at runtime.
```python
import flyte
import flyte.io
@env.task
async def download_url(url: str) -> flyte.io.File:
"""Download a file and return as flyte.io.File."""
import urllib.request
local_path = f"/tmp/{url.split('/')[-1]}"
urllib.request.urlretrieve(url, local_path)
return flyte.io.File(path=local_path)
@env.task
async def process(file: flyte.io.File) -> dict:
"""Read a file — flyte.io.File downloads it automatically."""
# file.path gives the local path (already downloaded)
with open(file.path) as f:
content = f.read()
return {"lines": len(content.splitlines())}
@env.task
async def main(url: str) -> dict:
downloaded = await download_url(url)
return await process(downloaded) # type: flyte.io.File flows as remote reference
```
### flyte.io.File best practices
- **Always pass `flyte.io.File` between tasks** — never pass file paths as strings. Flyte serializes the reference to the remote blob.
- **`file.path`** gives the local download path inside the task container.
- **Don't hardcode paths** — let Flyte manage the download/upload lifecycle.
- **Compression**: Flyte infers format from extension (`.parquet`, `.csv`, `.json`, `.pt`, `.png`, etc.).
## flyte.io.Dir — Directories
Use `flyte.io.Dir` for a collection of files (e.g., model checkpoints, output artifacts).
```python
import flyte
import flyte.io
@env.task
async def train(checkpoint_dir: flyte.io.Dir) -> flyte.io.Dir:
"""Train and save checkpoints to a directory."""
# Write checkpoints
for i in range(10):
path = f"{checkpoint_dir.path}/checkpoint_{i}.pt"
save_model(path)
return checkpoint_dir
@env.task
async def evaluate(checkpoints: flyte.io.Dir) -> dict:
"""Load checkpoints from a directory."""
# List all files in the directory
files = list(checkpoints.path.glob("*.pt"))
...
```
## flyte.io.DataFrame — Polars DataFrames
Flyte 2 has built-in support for Polars DataFrames. They are **passed by reference** (Parquet in the metadata bucket), not inline.
```python
import flyte
import flyte.io
@env.task
async def load_csv(url: str) -> flyte.io.DataFrame:
"""Load a CSV and return as Polars DataFrame."""
import polars as pl
df = pl.read_csv(url)
return flyte.io.DataFrame(df)
@env.task
async def clean(df: flyte.io.DataFrame) -> flyte.io.DataFrame:
"""Clean the DataFrame."""
inner = df.to_polars() # Get the underlying Polars DataFrame
cleaned = inner.drop_nulls()
return flyte.io.DataFrame(cleaned)
@env.task
async def save_parquet(df: flyte.io.DataFrame, path: str) -> flyte.io.File:
"""Save DataFrame to Parquet."""
inner = df.to_polars()
inner.write_parquet(path)
return flyte.io.File(path=path)
@env.task
async def main(url: str) -> flyte.io.DataFrame:
raw = await load_csv(url)
cleaned = await clean(raw)
return cleaned # Flows as Parquet reference
```
### Polars DataFrame patterns
```python
# Convert to Polars for manipulation
inner_df = df.to_polars()
# Convert from Polars
df = flyte.io.DataFrame(inner_df)
# Common operations
df = flyte.io.DataFrame(inner_df.filter(pl.col("age") > 18))
df = flyte.io.DataFrame(inner_df.group_by("category").agg(pl.col("value").mean()))
# Check shape
print(df.shape) # (rows, cols)
print(df.schema) # column names and types
```
### Eager vs Lazy DataFrames
```python
# Eager (loaded into memory)
df = flyte.io.DataFrame(inner_df)
# Lazy (streaming, for large datasets)
df = flyte.io.DataFrame(inner_df.lazy())
# Materialize lazy to eager
eager = df.to_polars() # materializes
```
## Dataclass and Pydantic Models
For structured data, use Python dataclasses or Pydantic models. Flyte serializes them to JSON.
```python
from dataclasses import dataclass
from pydantic import BaseModel
import flyte
import flyte.io
@dataclass
class TrainingConfig:
learning_rate: float
batch_size: int
epochs: int
class PredictionOutput(BaseModel):
predictions: list[float]
confidence: list[float]
model_version: str
@env.task
async def train(config: TrainingConfig) -> flyte.io.File:
# config.learning_rate, config.batch_size, etc.
...
@env.task
async def predict(model: flyte.io.File, data: flyte.io.DataFrame) -> PredictionOutput:
return PredictionOutput(
predictions=[0.5, 0.8, 0.3],
confidence=[0.9, 0.7, 0.95],
model_version="v1.0",
)
```
## Custom Type Transformers
Extend Flyte's type system to support custom types (e.g., PIL Images, HuggingFace datasets).
### PIL Image transformer
```python
from PIL import Image
import flyte
import flyte.io
from flyte.types import TypeTransformer
class PILImageTransformer(TypeTransformer[Image.Image]):
_type = Image.Image
def get_type(self, input: Image.Image) -> type:
return Image.Image
def save(self, img: Image.Image, path: str) -> None:
img.save(path)
def load(self, path: str) -> Image.Image:
return Image.open(path)
# Register the transformer
flyte.types.TypeEngine.register(PILImageTransformer())
# Now use it in tasks
@env.task
async def process_image(img: Image.Image) -> flyte.io.File:
# img is a PIL Image, already downloaded
...
```
### HuggingFace Dataset transformer
```python
from datasets import Dataset
import flyte
class HFDatasetTransformer(TypeTransformer[Dataset]):
_type = Dataset
def get_type(self, input: Dataset) -> type:
return Dataset
def save(self, ds: Dataset, path: str) -> None:
ds.save_to_disk(path)
def load(self, path: str) -> Dataset:
return Dataset.load_from_disk(path)
flyte.types.TypeEngine.register(HFDatasetTransformer())
```
## Data I/O Patterns by Domain
### ETL / Data Engineering
```python
# CSV → Parquet conversion
@env.task(cache="auto")
async def csv_to_parquet(csv_file: flyte.io.File, output_path: str) -> flyte.io.File:
import polars as pl
df = pl.read_csv(csv_file.path)
df.write_parquet(output_path)
return flyte.io.File(path=output_path)
# JsonlFile — batched JSONL reading
@env.task
async def process_jsonl(path: str) -> int:
from flyte.extend import JsonlFile
jf = JsonlFile(path)
count = 0
async for record in jf.stream():
process(record)
count += 1
return count
# JsonlDir — batched JSONL directory
@env.task
async def process_jsonl_dir(dir_path: str) -> dict:
from flyte.extend import JsonlDir
jd = JsonlDir(dir_path)
total = 0
async for batch in jd.stream_batches():
total += len(batch)
return {"records": total}
```
### Image Processing
```python
from PIL import Image
import flyte
import flyte.io
@env.task
async def resize_image(input_file: flyte.io.File, size: tuple[int, int]) -> flyte.io.File:
img = Image.open(input_file.path)
resized = img.resize(size)
output_path = f"/tmp/resized_{size[0]}x{size[1]}.png"
resized.save(output_path)
return flyte.io.File(path=output_path)
@env.task
async def batch_resize(files: list[flyte.io.File], size: tuple[int, int]) -> list:
import asyncio
return await asyncio.gather(*(resize_image(f, size) for f in files))
```
### Audio Processing
```python
import librosa
import numpy as np
import flyte
import flyte.io
@env.task
async def extract_features(audio_file: flyte.io.File) -> flyte.io.File:
"""Extract MFCC features from audio file."""
y, sr = librosa.load(audio_file.path, sr=None)
mfccs = librosa.feature.mfcc(y=y, sr=sr)
# Save as numpy array
output_path = "/tmp/mfccs.npz"
np.savez(output_path, mfccs=mfccs, sr=sr)
return flyte.io.File(path=output_path)
```
### HuggingFace Datasets
```python
from datasets import load_dataset, Dataset
import flyte
import flyte.io
@env_task
async def load_dataset_from_hub(dataset_name: str, split: str = "train") -> flyte.io.File:
"""Load a HF dataset and save locally."""
ds = load_dataset(dataset_name, split=split)
path = f"/tmp/{dataset_name.replace('/', '_')}_{split}"
ds.save_to_disk(path)
return flyte.io.File(path=path)
@env.task
async def process_dataset(file: flyte.io.File) -> flyte.io.DataFrame:
"""Convert HF dataset to Flyte DataFrame."""
ds = Dataset.load_from_disk(file.path)
df = ds.to_pandas()
return flyte.io.DataFrame(df)
```
## Data Locality and Storage Best Practices
### How data flows between tasks
1. **By reference (default)** — large data (DataFrames, Files, Directories) is uploaded to the metadata bucket. Tasks receive a remote reference and download on demand.
2. **Inline (small data)** — primitives (int, float, str, bool) and small collections are passed inline as JSON.
### Choosing the right transport
| Data type | Transport | Max size |
|---|---|---|
| `int`, `float`, `bool`, `str` | Inline | None |
| `list`, `dict` (small) | Inline | ~10 MB |
| `flyte.io.File` | Reference (S3/GCS) | Unlimited |
| `flyte.io.Dir` | Reference (S3/GCS) | Unlimited |
| `flyte.io.DataFrame` (Polars) | Reference (Parquet) | Unlimited |
| `dataclass` / `BaseModel` | Inline (JSON) | ~10 MB |
### Storage best practices
1. **Use `flyte.io.File` for files** — don't pass strings. Flyte manages the upload/download.
2. **Use `flyte.io.DataFrame` for tabular data** — stored as Parquet, efficient for ETL pipelines.
3. **Use `flyte.io.Dir` for collections** — model checkpoints, output artifacts.
4. **Set `cache="auto"`** on idempotent tasks (ETL, transforms) to avoid re-processing.
5. **Use `raw_data_path`** for per-run customization: `flyte.with_runcontext(raw_data_path="s3://my-bucket/{run_id}/")`
6. **Large data should never be inline** — if your dataclass exceeds ~10 MB, switch to `flyte.io.File` or `flyte.io.Dir`.
## Inline I/O Threshold
Control when data is passed inline vs by reference:
```python
@env.task(inline_output_limit="10MB") # data > 10MB goes by reference
async def process(data: dict) -> dict:
...
```
Default threshold is generous. For ML training outputs or large DataFrames, set it lower to avoid overhead.
## Common Type Mistakes
1. **Missing type hints** — Flyte 2 requires type annotations on all task inputs/outputs. No type hint = serialization error.
2. **Passing `flyte.io.File` as a string** — always use `flyte.io.File(path=...)` objects. The path string alone won't serialize.
3. **Using Pandas instead of Polars** — Flyte 2's native DataFrame is Polars. Use `df.to_polars()` to get the underlying DataFrame.
4. **Not registering custom transformers** — if you register a custom type transformer, it must be registered before task execution.
5. **Forgetting `.path` on flyte.io.File** — inside a task, `file` is a FlyteFile object, not a string. Use `file.path` for the local path.
start-dex-local13.8 KB
---
name: start-dex-local
description: Deploy Dex as a local in-cluster OIDC provider (IdP stand-in) for a kind-based Flyte deployment, so oauth2-proxy can be tested with no cloud account or real users. Use when the user wants to stand up Dex locally for testing Flyte authentication. For local testing only — in-memory storage, static test passwords.
---
# Start a local IdP with Dex (for testing)
Replace the **external** OIDC provider (Okta, Google, …) that oauth2-proxy
expects with [Dex](https://dexidp.io/) running **inside the same kind cluster**,
so you can test the whole Flyte auth flow with no cloud account and no real
users.
This assumes a kind cluster with Flyte and Traefik already up (steps 1–6 of the
kind deployment / the `deploy-flyte-kind` skill's OIDC section), about to wire
oauth2-proxy. Deploy Dex first, then point oauth2-proxy at it.
> **For local testing only.** Dex here uses in-memory storage and a static test
> password baked into its config. Never use this configuration anywhere real.
## The issuer-URL constraint (why the setup looks the way it does)
OIDC requires the **issuer URL to be identical everywhere it's seen**:
- oauth2-proxy (in-cluster) reaches Dex over a Kubernetes service name.
- The browser reaches Dex to log in, and must land on the *same* issuer the
token was minted for, or validation fails.
A service name (`dex.flyte.svc.cluster.local`) isn't resolvable from the
browser; a `localhost` URL isn't resolvable from inside the cluster. The fix:
serve Dex under the **same host as Flyte** (`flyte.local`) at a sub-path
(`/dex`), routed through Traefik. One URL — `http://flyte.local/dex` — works
from both sides.
## Step 0: Prerequisites
```bash
for t in kubectl helm; do command -v $t >/dev/null || echo "MISSING: $t"; done
kubectl --context kind-flyte -n traefik get deploy traefik >/dev/null 2>&1 || echo "MISSING: traefik"
kubectl --context kind-flyte -n flyte get svc flyte-http >/dev/null 2>&1 || echo "MISSING: flyte"
docker ps --filter name=flyte-control-plane --format '{{.Ports}}' | grep -q '80->30080' \
|| echo "MISSING: hostPort 80->30080 mapping (recreate the cluster — see deploy-flyte-kind Step 1)"
```
If anything is `MISSING`, stop. Flyte + Traefik must already be deployed, and the
cluster must have been created with the `hostPort: 80 → 30080` mapping — Dex's
issuer (`http://flyte.local/dex`) is unreachable from the browser without it, and
the mapping can't be added to an existing cluster. Also confirm `127.0.0.1
flyte.local` is in `/etc/hosts`.
## Step 1: Write the Dex config
Write `dex-config.yaml` in the working directory. The issuer is the
through-Traefik URL; the two static clients are oauth2-proxy (confidential, with
a secret) and the Flyte CLI (public, for SDK login). `staticPasswords` gives a
login with no external user store:
```yaml
# dex-config.yaml
issuer: http://flyte.local/dex
storage:
type: memory
web:
http: 0.0.0.0:5556
oauth2:
skipApprovalScreen: true # auto-approve, no consent screen in dev
staticClients:
# oauth2-proxy — confidential client (matches the secret passed to oauth2-proxy)
- id: oauth2-proxy
name: oauth2-proxy
secret: oauth2-proxy-secret
redirectURIs:
- 'http://flyte.local/oauth2/callback'
- 'https://flyte.local/oauth2/callback' # console opened over TLS (websecure)
# Flyte CLI — public client for SDK/CLI PKCE login
- id: flytectl
name: 'Flyte CLI'
public: true
redirectURIs:
- 'http://localhost:53593/callback'
enablePasswordDB: true
staticPasswords:
# login: admin@example.com / password
- email: "admin@example.com"
username: "admin"
userID: "08a8684b-db88-4b73-90a9-3cd1661f5466"
# bcrypt hash of the literal string "password" — see the warning below
hash: "$2a$10$wi77Jcsjw08l416Q4./OCu6qNvYMaNSvA3Jbo30QeyZAvq9b4BSRK"
```
**`hash` must be a complete 60-character bcrypt string.** Dex crashes
(`CrashLoopBackOff`) with `malformed bcrypt hash: hashedSecret too short` if it's
even one char short. The hash above is for `password` and is known-good — but
**verify length 60 before pasting** (`echo -n "$HASH" | wc -c`), a char lost in
transit looks fine and crashes Dex. To use a different password:
```bash
htpasswd -bnBC 10 "" 'your-password' | tr -d ':\n' | sed 's/^\$2y/\$2a/'
```
## Step 2: Deploy Dex
The Dex chart renders whatever you pass under its `config` value into a Secret and
mounts it as `config.yaml`. So nest the Step 1 config under a top-level `config:`
key in a values file and hand that to the chart — don't mount your own ConfigMap
volume (the chart already defines a `config` volume; adding another collides with
`Duplicate value: "config"`).
Write `dex-values.yaml` (the Step 1 YAML indented one level under `config:`):
```yaml
# dex-values.yaml
config:
issuer: http://flyte.local/dex
storage:
type: memory
web:
http: 0.0.0.0:5556
oauth2:
skipApprovalScreen: true
staticClients:
- id: oauth2-proxy
name: oauth2-proxy
secret: oauth2-proxy-secret
redirectURIs:
- 'http://flyte.local/oauth2/callback'
- 'https://flyte.local/oauth2/callback' # console opened over TLS (websecure)
- id: flytectl
name: 'Flyte CLI'
public: true
redirectURIs:
- 'http://localhost:53593/callback'
enablePasswordDB: true
staticPasswords:
- email: "admin@example.com"
username: "admin"
userID: "08a8684b-db88-4b73-90a9-3cd1661f5466"
hash: "$2a$10$wi77Jcsjw08l416Q4./OCu6qNvYMaNSvA3Jbo30QeyZAvq9b4BSRK"
```
```bash
helm repo add dex https://charts.dexidp.io
helm repo update
helm install dex dex/dex -n flyte --kube-context kind-flyte -f dex-values.yaml
```
Confirm Dex came up (if it `CrashLoopBackOff`s, check the logs — a bad `hash` is
the usual cause, see Step 1):
```bash
kubectl --context kind-flyte -n flyte rollout status deploy/dex
kubectl --context kind-flyte -n flyte get svc dex # note the port (5556 by default)
```
## Step 3: Route the issuer path through Traefik
Apply an ingress so `http://flyte.local/dex` reaches the Dex service — this is
what makes the single issuer URL resolve from the browser:
```bash
kubectl --context kind-flyte apply -f - <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dex
namespace: flyte
spec:
ingressClassName: traefik
rules:
- host: flyte.local
http:
paths:
- path: /dex
pathType: Prefix
backend:
service:
name: dex
port:
number: 5556
EOF
```
Check discovery works through the host path (the URL oauth2-proxy will fetch):
```bash
curl -s http://flyte.local/dex/.well-known/openid-configuration | head
```
A JSON doc with `"issuer":"http://flyte.local/dex"` confirms Dex is reachable at
the issuer it advertises.
## Step 4: Point oauth2-proxy at Dex
Use these values for the oauth2-proxy install (instead of external-IdP
placeholders). If oauth2-proxy is already installed against a placeholder IdP,
`helm upgrade` it with these flags. The `hostAliases` setting is the one
addition Dex needs that an external IdP doesn't — see the warning below:
```bash
# Dex's issuer is flyte.local, which the pod can't otherwise resolve — point it at Traefik.
TRAEFIK_IP=$(kubectl -n traefik --context kind-flyte get svc traefik -o jsonpath='{.spec.clusterIP}')
helm install oauth2-proxy oauth2-proxy/oauth2-proxy -n flyte --kube-context kind-flyte \
--set config.clientID='oauth2-proxy' \
--set config.clientSecret='oauth2-proxy-secret' \
--set config.cookieSecret="$(openssl rand -base64 32)" \
--set extraArgs.provider=oidc \
--set extraArgs.oidc-issuer-url='http://flyte.local/dex' \
--set extraArgs.upstream='static://202' \
--set extraArgs.reverse-proxy='true' \
--set extraArgs.set-xauthrequest='true' \
--set extraArgs.email-domain='*' \
--set extraArgs.cookie-secure='false' \
--set extraArgs.ssl-insecure-skip-verify='true' \
--set "hostAliases[0].ip=$TRAEFIK_IP" \
--set "hostAliases[0].hostnames[0]=flyte.local" # resolve the issuer in-cluster
```
> **Why `hostAliases` is required for Dex.** Dex's issuer is `flyte.local`, a
> name that resolves on your host (via `/etc/hosts`) but **not inside the
> cluster** — CoreDNS doesn't know it, and it isn't a Kubernetes service name. So
> oauth2-proxy hangs on `Performing OIDC Discovery...` at startup and
> `CrashLoopBackOff`s. The `hostAliases` flag adds `flyte.local → Traefik's
> ClusterIP` to the pod's `/etc/hosts`, so `flyte.local/dex` resolves to the same
> issuer from both the pod and the browser, as OIDC requires. Quote the
> `hostAliases[0]...` args — in zsh the unquoted `[0]` is a glob and errors with
> `no matches found`. This pins the current ClusterIP; if Traefik's service is
> recreated with a new IP, `helm upgrade` with the new value.
Then continue with the rest of the oauth2-proxy wiring (the ForwardAuth
middlewares and the Flyte ingress) from the kind deployment guide.
## Step 5: Advertise Dex to the SDK/CLI
oauth2-proxy gates the **browser** path, but the SDK/CLI discover where to log in
from Flyte's auth metadata. Point it at Dex using the public `flytectl` client
from Step 1, then `helm upgrade flyte … -f values-local.yaml`. V2 has no auth
server of its own — it just advertises Dex:
```yaml
# add to values-local.yaml
flyte-core-components:
runs:
authMetadata:
externalAuthServerBaseUrl: http://flyte.local/dex
flyteClient:
clientId: flytectl
redirectUri: http://localhost:53593/callback
scopes:
- openid
- profile
- offline_access
```
This is the same `authMetadata` block as a real IdP — only the issuer URL points
at the in-cluster Dex.
## Step 6: Verify the flow
With Dex, oauth2-proxy, and the Flyte ingress all in place, check the flow from
the command line first. These use `curl --resolve` to point `flyte.local` at the
local Traefik node port, so they work **without** editing `/etc/hosts` (the
browser still needs the hosts entry):
```bash
# 1. The console is gated — an unauthenticated request is rejected by the auth middleware:
curl -s -o /dev/null -w "%{http_code}\n" --resolve flyte.local:80:127.0.0.1 \
http://flyte.local/v2
# → 401 (oauth2-auth ForwardAuth rejects it; a browser is then redirected by
# the oauth2-signin error middleware)
# 2. The sign-in page is served:
curl -s -o /dev/null -w "%{http_code}\n" --resolve flyte.local:80:127.0.0.1 \
"http://flyte.local/oauth2/sign_in?rd=http://flyte.local/v2"
# → 200
# 3. Starting login redirects all the way to Dex's login page:
curl -s -o /dev/null -w "%{url_effective}\n" -L --max-redirs 5 \
--resolve flyte.local:80:127.0.0.1 "http://flyte.local/oauth2/start?rd=http://flyte.local/v2"
# → http://flyte.local/dex/auth/local/login?... (oauth2-proxy → Dex)
```
A raw `curl` to `/v2` returns `401`, not a `302` — Traefik's `oauth2-signin`
middleware turns the 401 into a sign-in redirect via its `errors` handler, which
a browser follows but `curl` shows raw. The 401 still confirms the request is
gated; checks 2 and 3 confirm the redirect itself.
### Add the hosts entry for browser access
The `curl --resolve` checks above bypass DNS, but a browser can't — it needs
`flyte.local` to resolve to the local Traefik node port. Editing `/etc/hosts`
needs sudo, so **have the user run it themselves** rather than running it for
them:
```bash
echo "127.0.0.1 flyte.local" | sudo tee -a /etc/hosts
```
First check whether it's already there (idempotent — don't add a duplicate):
```bash
grep -q "flyte.local" /etc/hosts && echo "present" || echo "absent"
```
- **`present`** → nothing to do, continue.
- **`absent`** → tell the user to run the `tee` line above (suggest they type it
as `! echo "127.0.0.1 flyte.local" | sudo tee -a /etc/hosts` so it runs in this
session), then **ask whether they've added it or want to skip for now.**
- **Added** → re-run the `grep` to confirm it's present, then continue to the
browser step.
- **Skip** → that's fine; the deployment is complete and the `curl --resolve`
checks already proved the flow. Note that browser login won't work until the
entry is added, and stop here.
Don't reach for `127.0.0.1` as a workaround: Traefik has no route for that host
(404), and the OIDC issuer is minted as `flyte.local`, so login fails on an
issuer mismatch. The hostname must be `flyte.local` end to end.
Once the entry is present, open `http://flyte.local/v2` in a browser and log in
as **`admin@example.com` / `password`**. You should land in the console. The
`X-Auth-Request-Email` header Dex supplies flows through oauth2-proxy to Flyte
and populates `executed_by` on runs.
## Troubleshooting
| Symptom | Cause and fix |
|---|---|
| oauth2-proxy `CrashLoopBackOff`, logs stuck on `Performing OIDC Discovery...` | The pod can't resolve `flyte.local` in-cluster. Confirm the `hostAliases` from Step 4 are set (`kubectl -n flyte get deploy oauth2-proxy -o jsonpath='{.spec.template.spec.hostAliases}'`) and point at Traefik's current ClusterIP. |
| oauth2-proxy `CrashLoopBackOff`, logs show `could not fetch .well-known` | oauth2-proxy can't reach the issuer. Confirm Step 3's curl returns the discovery doc and that `oidc-issuer-url` matches `issuer` in `dex-config.yaml` **exactly**. |
| Browser: `Unregistered redirect_uri` / `redirect_uri did not match` | The `oauth2-proxy` static client's `redirectURIs` must list the callback for the scheme you open the console with — `http://flyte.local/oauth2/callback` **and** `https://flyte.local/oauth2/callback` (opening `/v2` over TLS uses the `https` one). List both. |
| Login succeeds but loops back to sign-in | Issuer mismatch between what the browser saw and what oauth2-proxy validated. Both must be `http://flyte.local/dex` — not a service name, not `localhost`. |
## Tear down
```bash
helm uninstall dex -n flyte --kube-context kind-flyte
kubectl --context kind-flyte -n flyte delete ingress dex
```
Package details
Publisher declarations from the archived package. These are separate from our research and the live service's terms.
- Package author
- SAMHITA ALLA
Package observed Oct 3, 2026.
Technical details
- First seen
- Sep 30, 2026 · 22:02 UTC
- Last seen
- Oct 3, 2026 · 18:00 UTC
- Collection status
- Collected
plugin_asdk_app_6a79985b0ef881918e389c82400ea95c
Download plugin data (JSON)Before you connect Flyte
How do I connect it?
Open the publisher's marketplace listing to check current availability and follow its connection instructions. This directory does not install plugins. Check the requested access and any account requirements before connecting.
Check marketplace availability ↗
Does it require paid access?
We have not established the pricing or subscription requirements for this plugin. An absent price does not mean free access.
Compare researched pricing and access models →
How can I evaluate it?
Check the declared skills and available files, then try a small task whose result you can verify. Our archived descriptions and instructions establish publisher claims, not tested runtime quality. Review sources and coverage limits.