Update to astronomer-data

{
  "name": "airflow",
  "description": "Queries, manages, and troubleshoots Apache Airflow using the `af` CLI. Use when working with anything related to Airflow - a DAG, a DAG run, a task log, an import or parse error, a broken DAG, or any Airflow operation. Covers listing and triggering DAGs, retrying runs, reading task logs, diagnosing failures, debugging import and parse errors, checking connections, variables and pools, exploring the REST API, and monitoring health (for example \"trigger a pipeline\", \"retry a run\", \"list connections\", \"check Airflow health\", \"why did my DAG fail\"). This is the entrypoint that routes to sibling skills for authoring, testing, deploying, and migrating Airflow 2 to 3. Not for warehouse/SQL analytics on Airflow metadata tables (use analyzing-data); for deep root-cause reports use debugging-dags or airflow-investigation.",
  "included_files": [
    {
      "relative_path": "api-reference.md",
      "size_in_bytes": 2806
    },
    {
      "relative_path": "hooks/warm-uvx-cache.sh",
      "size_in_bytes": 408
    }
  ],
  "skill_md_contents": "---\nname: airflow\ndescription: Queries, manages, and troubleshoots Apache Airflow using the `af` CLI. Use when working with anything related to Airflow - a DAG, a DAG run, a task log, an import or parse error, a broken DAG, or any Airflow operation. Covers listing and triggering DAGs, retrying runs, reading task logs, diagnosing failures, debugging import and parse errors, checking connections, variables and pools, exploring the REST API, and monitoring health (for example \"trigger a pipeline\", \"retry a run\", \"list connections\", \"check Airflow health\", \"why did my DAG fail\"). This is the entrypoint that routes to sibling skills for authoring, testing, deploying, and migrating Airflow 2 to 3. Not for warehouse/SQL analytics on Airflow metadata tables (use analyzing-data); for deep root-cause reports use debugging-dags or airflow-investigation.\n---\n\n# Airflow Operations\n\nUse `af` commands to query, manage, and troubleshoot Airflow workflows.\n\n## Astro CLI\n\nThe [Astro CLI](https://www.astronomer.io/docs/astro/cli/overview) is the recommended way to run Airflow locally and deploy to production. It provides a containerized Airflow environment that works out of the box:\n\n```bash\n# Initialize a new project\nastro dev init\n\n# Start local Airflow (webserver at http://localhost:8080)\nastro dev start\n\n# Parse DAGs to catch errors quickly (no need to start Airflow)\nastro dev parse\n\n# Run pytest against your DAGs\nastro dev pytest\n\n# Deploy to production\nastro deploy            # Full deploy (image + DAGs)\nastro deploy --dags     # DAG-only deploy (fast, no image build)\n```\n\nFor more details:\n- **New project?** See the **setting-up-astro-project** skill\n- **Local environment?** See the **managing-astro-local-env** skill\n- **Deploying?** See the **deploying-airflow** skill\n\n---\n\n## Running the CLI\n\nThese commands assume `af` is on PATH. Run via `astro otto` to get it automatically, or install standalone with `uv tool install astro-airflow-mcp`.\n\n## Instance Configuration\n\nManage multiple Airflow instances with persistent configuration:\n\n```bash\n# Add a new instance\naf instance add prod --url https://airflow.example.com --token \"$API_TOKEN\"\naf instance add staging --url https://staging.example.com --username admin --password admin\n\n# List and switch instances\naf instance list      # Shows all instances in a table\naf instance use prod  # Switch to prod instance\naf instance current   # Show current instance\naf instance delete old-instance\n\n# Auto-discover instances (use --dry-run to preview first)\naf instance discover --dry-run        # Preview all discoverable instances\naf instance discover                  # Discover from all backends (astro, local)\naf instance discover astro            # Discover Astro deployments only\naf instance discover astro --all-workspaces  # Include all accessible workspaces\naf instance discover local            # Scan common local Airflow ports\naf instance discover local --scan     # Deep scan all ports 1024-65535\n\n# IMPORTANT: Always run with --dry-run first and ask for user consent before\n# running discover without it. The non-dry-run mode creates API tokens in\n# Astro Cloud, which is a sensitive action that requires explicit approval.\n\n# Show where an instance came from (file path + scope)\naf instance show prod\n\n# Override instance for a single command via env vars\nAIRFLOW_API_URL=https://staging.example.com AIRFLOW_AUTH_TOKEN=$STG af dags list\n\n# Or switch persistently\naf instance use staging\n```\n\nConfig layout (mirrors `git config` system/global/local):\n\n| Scope | File | Committed? |\n|---|---|---|\n| Global | `~/.astro/config.yaml` | n/a (per-user) |\n| Project shared | `<root>/.astro/config.yaml` | yes |\n| Project local | `<root>/.astro/config.local.yaml` | no (gitignored) |\n\n`<root>` is found by walking up from cwd looking for `.astro/`. Default write routing inside a project: `add`/`discover` → project-shared, `use` → project-local. Override with `--global` / `--project` / `--local`. Set `AF_CONFIG=<path>` to bypass layering and use a single file.\n\nMigrate from the legacy `~/.af/config.yaml` with `af migrate` (idempotent; renames the old file to `.bak`).\n\nTokens in config can reference environment variables using `${VAR}` syntax:\n```yaml\ninstances:\n- name: prod\n  url: https://airflow.example.com\n  auth:\n    token: ${AIRFLOW_API_TOKEN}\n```\n\nOr use environment variables directly (no config file needed):\n\n```bash\nexport AIRFLOW_API_URL=http://localhost:8080\nexport AIRFLOW_AUTH_TOKEN=your-token-here\n# Or username/password:\nexport AIRFLOW_USERNAME=admin\nexport AIRFLOW_PASSWORD=admin\n```\n\nOr CLI flags: `af --airflow-url http://localhost:8080 --token \"$TOKEN\" <command>`\n\n## Quick Reference\n\n| Command | Description |\n|---------|-------------|\n| `af health` | System health check |\n| `af dags list` | List all DAGs |\n| `af dags get <dag_id>` | Get DAG details |\n| `af dags explore <dag_id>` | Full DAG investigation |\n| `af dags source <dag_id>` | Get DAG source code |\n| `af dags pause <dag_id>` | Pause DAG scheduling |\n| `af dags unpause <dag_id>` | Resume DAG scheduling |\n| `af dags errors` | List import errors |\n| `af dags warnings` | List DAG warnings |\n| `af dags stats` | DAG run statistics |\n| `af runs list` | List DAG runs |\n| `af runs get <dag_id> <run_id>` | Get run details |\n| `af runs trigger <dag_id>` | Trigger a DAG run |\n| `af runs trigger-wait <dag_id>` | Trigger and wait for completion |\n| `af runs delete <dag_id> <run_id>` | Permanently delete a DAG run |\n| `af runs clear <dag_id> <run_id>` | Clear a run for re-execution |\n| `af runs diagnose <dag_id> <run_id>` | Diagnose failed run |\n| `af tasks list <dag_id>` | List tasks in DAG |\n| `af tasks get <dag_id> <task_id>` | Get task definition |\n| `af tasks instance <dag_id> <run_id> <task_id>` | Get task instance |\n| `af tasks logs <dag_id> <run_id> <task_id>` | Get task logs |\n| `af config version` | Airflow version |\n| `af config show` | Full configuration |\n| `af config connections` | List connections |\n| `af config variables` | List variables |\n| `af config variable <key>` | Get specific variable |\n| `af config pools` | List pools |\n| `af config pool <name>` | Get pool details |\n| `af config plugins` | List plugins |\n| `af config providers` | List providers |\n| `af config assets` | List assets/datasets |\n| `af api <endpoint>` | Direct REST API access |\n| `af api ls` | List available API endpoints |\n| `af api ls --filter X` | List endpoints matching pattern |\n| `af registry providers` | List providers in the Airflow Registry |\n| `af registry modules <provider>` | List operators/hooks/sensors/transfers in a provider |\n| `af registry parameters <provider>` | Constructor signatures (name, type, default, required) for a provider's classes |\n| `af registry connections <provider>` | Connection types a provider exposes |\n\n## User Intent Patterns\n\n### Getting Started\n- \"How do I run Airflow locally?\" / \"Set up Airflow\" -> use the **managing-astro-local-env** skill (uses Astro CLI)\n- \"Create a new Airflow project\" / \"Initialize project\" -> use the **setting-up-astro-project** skill (uses Astro CLI)\n- \"How do I install Airflow?\" / \"Get started with Airflow\" -> use the **setting-up-astro-project** skill\n\n### DAG Operations\n- \"What DAGs exist?\" / \"List all DAGs\" -> `af dags list`\n- \"Tell me about DAG X\" / \"What is DAG Y?\" -> `af dags explore <dag_id>`\n- \"What's the schedule for DAG X?\" -> `af dags get <dag_id>`\n- \"Show me the code for DAG X\" -> `af dags source <dag_id>`\n- \"Stop DAG X\" / \"Pause this workflow\" -> `af dags pause <dag_id>`\n- \"Resume DAG X\" -> `af dags unpause <dag_id>`\n- \"Are there any DAG errors?\" -> `af dags errors`\n- \"Create a new DAG\" / \"Write a pipeline\" -> use the **authoring-dags** skill\n\n### Run Operations\n- \"What runs have executed?\" -> `af runs list`\n- \"Run DAG X\" / \"Trigger the pipeline\" -> `af runs trigger <dag_id>`\n- \"Run DAG X and wait\" -> `af runs trigger-wait <dag_id>`\n- \"Why did this run fail?\" -> `af runs diagnose <dag_id> <run_id>`\n- \"Delete this run\" / \"Remove stuck run\" -> `af runs delete <dag_id> <run_id>`\n- \"Clear this run\" / \"Retry this run\" / \"Re-run this\" -> `af runs clear <dag_id> <run_id>`\n- \"Test this DAG and fix if it fails\" -> use the **testing-dags** skill\n\n### Task Operations\n- \"What tasks are in DAG X?\" -> `af tasks list <dag_id>`\n- \"Get task logs\" / \"Why did task fail?\" -> `af tasks logs <dag_id> <run_id> <task_id>`\n- \"Full root cause analysis\" / \"Diagnose and fix\" -> use the **debugging-dags** skill\n\n### Data Operations\n- \"Is the data fresh?\" / \"When was this table last updated?\" -> use the **checking-freshness** skill\n- \"Where does this data come from?\" -> use the **tracing-upstream-lineage** skill\n- \"What depends on this table?\" / \"What breaks if I change this?\" -> use the **tracing-downstream-lineage** skill\n\n### Deployment Operations\n- \"Deploy my DAGs\" / \"Push to production\" -> use the **deploying-airflow** skill\n- \"Set up CI/CD\" / \"Automate deploys\" -> use the **deploying-airflow** skill\n- \"Deploy to Kubernetes\" / \"Set up Helm\" -> use the **deploying-airflow** skill\n- \"astro deploy\" / \"DAG-only deploy\" -> use the **deploying-airflow** skill\n\n### System Operations\n- \"What version of Airflow?\" -> `af config version`\n- \"What connections exist?\" -> `af config connections`\n- \"Are pools full?\" -> `af config pools`\n- \"Is Airflow healthy?\" -> `af health`\n\n### API Exploration\n- \"What API endpoints are available?\" -> `af api ls`\n- \"Find variable endpoints\" -> `af api ls --filter variable`\n- \"Access XCom values\" / \"Get XCom\" -> `af api xcom-entries -F dag_id=X -F task_id=Y`\n- \"Get event logs\" / \"Audit trail\" -> `af api event-logs -F dag_id=X`\n- \"Create connection via API\" -> `af api connections -X POST --body '{...}'`\n- \"Create variable via API\" -> `af api variables -X POST -F key=name -f value=val`\n\n### Registry Discovery\n- \"What operators does provider X have?\" -> `af registry modules <provider>`\n- \"What are the constructor params for operator Y?\" -> `af registry parameters <provider>`\n- \"What providers exist?\" / \"Is there a provider for Z?\" -> `af registry providers`\n- \"What connection types does provider X expose?\" -> `af registry connections <provider>`\n- \"Writing a DAG with a specific operator\" -> use registry to verify current signature before copying examples\n\n## Common Workflows\n\n### Validate DAGs Before Deploying\n\nIf you're using the Astro CLI, you can validate DAGs without a running Airflow instance:\n\n```bash\n# Parse DAGs to catch import errors and syntax issues\nastro dev parse\n\n# Run unit tests\nastro dev pytest\n```\n\nOtherwise, validate against a running instance:\n\n```bash\naf dags errors     # Check for parse/import errors\naf dags warnings   # Check for deprecation warnings\n```\n\n### Discover Operator Signatures Before Writing Code\n\nThe Airflow Registry at `airflow.apache.org/registry` is the authoritative source for provider classes and their current constructor signatures. Prefer it over memory or stale documentation when authoring DAGs — the registry reflects the live provider release.\n\n```bash\n# List all providers and pick the one you need\naf registry providers | jq '.providers[] | {id, name, version}'\n\n# List every operator / hook / sensor in a provider (e.g. standard, amazon, google)\naf registry modules standard \\\n  | jq '.modules[] | {name, type, import_path, docs_url}'\n\n# Get the current constructor signature for a specific class\naf registry parameters standard \\\n  | jq '.classes[\"airflow.providers.standard.operators.hitl.ApprovalOperator\"].parameters'\n\n# Filter modules by substring (useful when you know the concept but not the class)\naf registry modules standard \\\n  | jq '.modules[] | select(.import_path | test(\"hitl\"))'\n```\n\nResults are cached locally: 1 hour for the latest version, 30 days for pinned versions (which are immutable). Add `--version X.Y.Z` to any `modules` / `parameters` / `connections` call to target a specific release.\n\n### Investigate a Failed Run\n\n```bash\n# 1. List recent runs to find failure\naf runs list --dag-id my_dag\n\n# 2. Diagnose the specific run\naf runs diagnose my_dag manual__2024-01-15T10:00:00+00:00\n\n# 3. Get logs for failed task (from diagnose output)\naf tasks logs my_dag manual__2024-01-15T10:00:00+00:00 extract_data\n\n# 4. After fixing, clear the run to retry all tasks\naf runs clear my_dag manual__2024-01-15T10:00:00+00:00\n```\n\n### Morning Health Check\n\n```bash\n# 1. Overall system health\naf health\n\n# 2. Check for broken DAGs\naf dags errors\n\n# 3. Check pool utilization\naf config pools\n```\n\n### Understand a DAG\n\n```bash\n# Get comprehensive overview (metadata + tasks + source)\naf dags explore my_dag\n```\n\n### Check Why DAG Isn't Running\n\n```bash\n# Check if paused\naf dags get my_dag\n\n# Check for import errors\naf dags errors\n\n# Check recent runs\naf runs list --dag-id my_dag\n```\n\n### Trigger and Monitor\n\n```bash\n# Option 1: Trigger and wait (blocking)\naf runs trigger-wait my_dag --timeout 1800\n\n# Option 2: Trigger and check later\naf runs trigger my_dag\naf runs get my_dag <run_id>\n```\n\n## Output Format\n\nAll commands output JSON (except `instance` commands which use human-readable tables):\n\n```bash\naf dags list\n# {\n#   \"total_dags\": 5,\n#   \"returned_count\": 5,\n#   \"dags\": [...]\n# }\n```\n\nUse `jq` for filtering:\n\n```bash\n# Find failed runs\naf runs list | jq '.dag_runs[] | select(.state == \"failed\")'\n\n# Get DAG IDs only\naf dags list | jq '.dags[].dag_id'\n\n# Find paused DAGs\naf dags list | jq '[.dags[] | select(.is_paused == true)]'\n```\n\n## Task Logs Options\n\n```bash\n# Get logs for specific retry attempt\naf tasks logs my_dag run_id task_id --try 2\n\n# Get logs for mapped task index\naf tasks logs my_dag run_id task_id --map-index 5\n```\n\n## Direct API Access with `af api`\n\nUse `af api` for endpoints not covered by high-level commands (XCom, event-logs, backfills, etc).\n\n```bash\n# Discover available endpoints\naf api ls\naf api ls --filter variable\n\n# Basic usage\naf api dags\naf api dags -F limit=10 -F only_active=true\naf api variables -X POST -F key=my_var -f value=\"my value\"\naf api variables/old_var -X DELETE\n```\n\n**Field syntax**: `-F key=value` auto-converts types, `-f key=value` keeps as string.\n\n**Full reference**: See [api-reference.md](api-reference.md) for all options, common endpoints (XCom, event-logs, backfills), and examples.\n\n## Related Skills\n\n| Skill | Use when... |\n|-------|-------------|\n| **authoring-dags** | Creating or editing DAG files with best practices |\n| **testing-dags** | Iterative test -> debug -> fix -> retest cycles |\n| **debugging-dags** | Deep root cause analysis and failure diagnosis |\n| **checking-freshness** | Checking if data is up to date or stale |\n| **tracing-upstream-lineage** | Finding where data comes from |\n| **tracing-downstream-lineage** | Impact analysis -- what breaks if something changes |\n| **deploying-airflow** | Deploying DAGs to production (Astro, Docker Compose, Kubernetes) |\n| **migrating-airflow-2-to-3** | Upgrading DAGs from Airflow 2.x to 3.x |\n| **managing-astro-local-env** | Starting, stopping, or troubleshooting local Airflow |\n| **setting-up-astro-project** | Initializing a new Astro/Airflow project |\n| **airflow-state-store** | Per-task checkpointing, watermarks, crash-safe operators (Airflow 3.3+) |\n| **airflow-hitl** | Pausing a DAG for human approval or input (Airflow 3.1+) |\n"
}