Update to astronomer-data

{
  "name": "authoring-dags",
  "description": "Workflow and best practices for writing Apache Airflow DAGs. Use when creating a new DAG, write pipeline code, handling questions about DAG patterns and conventions or extending an existing DAG with a follow-up/downstream task. ANY request shaped like 'add a DAG named X', 'write a pipeline', 'add a task that runs after Y', or 'extend the DAG'. For testing and debugging DAGs, see the testing-dags skill.",
  "included_files": [
    {
      "relative_path": "reference/best-practices.md",
      "size_in_bytes": 10834
    }
  ],
  "skill_md_contents": "---\nname: authoring-dags\ndescription: Workflow and best practices for writing Apache Airflow DAGs. Use when creating a new DAG, write pipeline code, handling questions about DAG patterns and conventions or extending an existing DAG with a follow-up/downstream task. ANY request shaped like 'add a DAG named X', 'write a pipeline', 'add a task that runs after Y', or 'extend the DAG'. For testing and debugging DAGs, see the testing-dags skill.\nhooks:\n  Stop:\n    - hooks:\n        - type: command\n          command: \"echo 'Remember to test your DAG with the testing-dags skill'\"\n---\n\n# DAG Authoring Skill\n\nThis skill guides you through creating and validating Airflow DAGs using best practices and `af` CLI commands.\n\n> **For testing and debugging DAGs**, see the **testing-dags** skill which covers the full test -> debug -> fix -> retest workflow.\n\n---\n\n## Running the CLI\n\nThese commands assume `af` is on PATH. Run via `astro otto` to get it automatically, or install standalone with `uv tool install astro-airflow-mcp`.\n\n---\n\n## Workflow Overview\n\n```\n+-----------------------------------------+\n| 1. DISCOVER                             |\n|    Understand codebase & environment    |\n+-----------------------------------------+\n                 |\n+-----------------------------------------+\n| 2. PLAN                                 |\n|    Propose structure, get approval      |\n+-----------------------------------------+\n                 |\n+-----------------------------------------+\n| 3. IMPLEMENT                            |\n|    Write DAG following patterns         |\n+-----------------------------------------+\n                 |\n+-----------------------------------------+\n| 4. VALIDATE                             |\n|    Check import errors, warnings        |\n+-----------------------------------------+\n                 |\n+-----------------------------------------+\n| 5. TEST (with user consent)             |\n|    Trigger, monitor, check logs         |\n+-----------------------------------------+\n                 |\n+-----------------------------------------+\n| 6. ITERATE                              |\n|    Fix issues, re-validate              |\n+-----------------------------------------+\n```\n\n---\n\n## Phase 1: Discover\n\nBefore writing code, understand the context.\n\n### Explore the Codebase\n\nUse file tools to find existing patterns:\n- `Glob` for `**/dags/**/*.py` to find existing DAGs\n- `Read` similar DAGs to understand conventions\n- Check `requirements.txt` for available packages\n\n### Query the Airflow Environment\n\nUse `af` CLI commands to understand what's available:\n\n| Command | Purpose |\n|---------|---------|\n| `af config connections` | What external systems are configured |\n| `af config variables` | What configuration values exist |\n| `af config providers` | What operator packages are installed |\n| `af config version` | Version constraints and features |\n| `af dags list` | Existing DAGs and naming conventions |\n| `af config pools` | Resource pools for concurrency |\n\n**Example discovery questions:**\n- \"Is there a Snowflake connection?\" -> `af config connections`\n- \"What Airflow version?\" -> `af config version`\n- \"Are S3 operators available?\" -> `af config providers`\n\n---\n\n## Phase 2: Plan\n\nBased on discovery, propose:\n\n1. **DAG structure** - Tasks, dependencies, schedule\n2. **Operators to use** - Based on available providers\n3. **Connections needed** - Existing or to be created\n4. **Variables needed** - Existing or to be created\n5. **Packages needed** - Additions to requirements.txt\n\n**Get user approval before implementing.**\n\n---\n\n## Phase 3: Implement\n\nWrite the DAG following best practices (see below). Key steps:\n\n1. Create DAG file in appropriate location\n2. Update `requirements.txt` if needed\n3. Save the file\n\n---\n\n## Phase 4: Validate\n\n**Use `af` CLI as a feedback loop to validate your DAG.**\n\n### Step 1: Check Import Errors\n\nAfter saving, check for parse errors (Airflow will have already parsed the file):\n\n```bash\naf dags errors\n```\n\n- If your file appears -> **fix and retry**\n- If no errors -> **continue**\n\nCommon causes: missing imports, syntax errors, missing packages.\n\n### Step 2: Verify DAG Exists\n\n```bash\naf dags get <dag_id>\n```\n\nCheck: DAG exists, schedule correct, tags set, paused status.\n\n### Step 3: Check Warnings\n\n```bash\naf dags warnings\n```\n\nLook for deprecation warnings or configuration issues.\n\n### Step 4: Explore DAG Structure\n\n```bash\naf dags explore <dag_id>\n```\n\nReturns in one call: metadata, tasks, dependencies, source code.\n\n### On Astro\n\nIf you're running on Astro, you can also validate locally before deploying:\n\n- **Parse check**: Run `astro dev parse` to catch import errors and DAG-level issues without starting a full Airflow environment\n- **DAG-only deploy**: Once validated, use `astro deploy --dags` for fast DAG-only deploys that skip the Docker image build — ideal for iterating on DAG code\n\n---\n\n## Phase 5: Test\n\n> See the **testing-dags** skill for comprehensive testing guidance.\n\nOnce validation passes, test the DAG using the workflow in the **testing-dags** skill:\n\n1. **Get user consent** -- Always ask before triggering\n2. **Trigger and wait** -- `af runs trigger-wait <dag_id> --timeout 300`\n3. **Analyze results** -- Check success/failure status\n4. **Debug if needed** -- `af runs diagnose <dag_id> <run_id>` and `af tasks logs <dag_id> <run_id> <task_id>`\n\n### Quick Test (Minimal)\n\n```bash\n# Ask user first, then:\naf runs trigger-wait <dag_id> --timeout 300\n```\n\nFor the full test -> debug -> fix -> retest loop, see **testing-dags**.\n\n---\n\n## Phase 6: Iterate\n\nIf issues found:\n1. Fix the code\n2. Check for import errors: `af dags errors`\n3. Re-validate (Phase 4)\n4. Re-test using the **testing-dags** skill workflow (Phase 5)\n\n---\n\n## CLI Quick Reference\n\n| Phase | Command | Purpose |\n|-------|---------|---------|\n| Discover | `af config connections` | Available connections |\n| Discover | `af config variables` | Configuration values |\n| Discover | `af config providers` | Installed operators |\n| Discover | `af config version` | Version info |\n| Validate | `af dags errors` | Parse errors (check first!) |\n| Validate | `af dags get <dag_id>` | Verify DAG config |\n| Validate | `af dags warnings` | Configuration warnings |\n| Validate | `af dags explore <dag_id>` | Full DAG inspection |\n\n> **Testing commands** -- See the **testing-dags** skill for `af runs trigger-wait`, `af runs diagnose`, `af tasks logs`, etc.\n\n---\n\n## Best Practices & Anti-Patterns\n\nFor code patterns and anti-patterns, see **[reference/best-practices.md](reference/best-practices.md)**.\n\n**Read this reference when writing new DAGs or reviewing existing ones.** It covers what patterns are correct (including Airflow 3-specific behavior) and what to avoid.\n\n---\n\n## Related Skills\n\n- **testing-dags**: For testing DAGs, debugging failures, and the test -> fix -> retest loop\n- **debugging-dags**: For troubleshooting failed DAGs\n- **deploying-airflow**: For deploying DAGs to production (Astro or open-source)\n- **migrating-airflow-2-to-3**: For migrating DAGs to Airflow 3\n"
}