← astronomer-dataCONTENT HISTORY

Update to astronomer-data

Snapshot Sep 30, 2026 · 23:17 UTC · version 0.1.0

Collection source: not recorded for this historical snapshot.

WHAT CHANGED · RULE-BASED ANALYSIS

First saved snapshot

No earlier snapshot is available to establish a change.

Compare saved observations

Download comparison JSON
Full technical diff · 0 changed fields
Full snapshot data
{
  "name": "cosmos-dbt-fusion",
  "description": "Run a dbt Fusion project with Astronomer Cosmos. Use when running a dbt Fusion project with Astronomer Cosmos (Cosmos 1.11+, ExecutionMode.LOCAL on Snowflake/Databricks). Before implementing, verify dbt engine is Fusion (not Core), the warehouse is supported, and local execution is acceptable. Does not cover dbt Core.",
  "included_files": [
    {
      "relative_path": "reference/cosmos-config.md",
      "size_in_bytes": 3912
    }
  ],
  "skill_md_contents": "---\nname: cosmos-dbt-fusion\ndescription: Run a dbt Fusion project with Astronomer Cosmos. Use when running a dbt Fusion project with Astronomer Cosmos (Cosmos 1.11+, ExecutionMode.LOCAL on Snowflake/Databricks). Before implementing, verify dbt engine is Fusion (not Core), the warehouse is supported, and local execution is acceptable. Does not cover dbt Core.\n---\n\n# Cosmos + dbt Fusion: Implementation Checklist\n\nExecute steps in order. This skill covers Fusion-specific constraints only.\n\n> **Version note**: dbt Fusion support was introduced in Cosmos 1.11.0. Requires Cosmos ≥1.11.\n>\n> **Reference**: See **[reference/cosmos-config.md](reference/cosmos-config.md)** for ProfileConfig, operator_args, and Airflow 3 compatibility details.\n\n> **Before starting**, confirm: (1) dbt engine = Fusion (not Core → use **cosmos-dbt-core**), (2) warehouse = Snowflake, Databricks, Bigquery and Redshift only.\n\n### Fusion-Specific Constraints\n\n| Constraint | Details |\n|------------|---------|\n| No async | `AIRFLOW_ASYNC` not supported |\n| No virtualenv | Fusion is a binary, not a Python package |\n| Warehouse support | Snowflake, Databricks, Bigquery and Redshift support [while in preview](https://github.com/dbt-labs/dbt-fusion) |\n\n---\n\n## 1. Confirm Cosmos Version\n\n> **CRITICAL**: Cosmos 1.11.0 introduced dbt Fusion compatibility.\n\n```bash\n# Check installed version\npip show astronomer-cosmos\n\n# Install/upgrade if needed\npip install \"astronomer-cosmos>=1.11.0\"\n```\n\n**Validate**: `pip show astronomer-cosmos` reports version ≥ 1.11.0\n\n---\n\n## 2. Install the dbt Fusion Binary (REQUIRED)\n\ndbt Fusion is NOT bundled with Cosmos or dbt Core. Install it into the Airflow runtime/image.\n\nDetermine where to install the Fusion binary (Dockerfile / base image / runtime).\n\n### Example Dockerfile Install\n\n```dockerfile\nUSER root\nRUN apt-get update && apt-get install -y curl\nENV SHELL=/bin/bash\nRUN curl -fsSL https://public.cdn.getdbt.com/fs/install/install.sh | sh -s -- --update\nUSER astro\n```\n\n### Common Install Paths\n\n| Environment | Typical path |\n|-------------|--------------|\n| Astro Runtime | `/home/astro/.local/bin/dbt` |\n| System-wide | `/usr/local/bin/dbt` |\n\n**Validate**: The `dbt` binary exists at the chosen path and `dbt --version` succeeds.\n\n---\n\n## 3. Choose Parsing Strategy (RenderConfig)\n\nParsing strategy is the same as dbt Core. Pick ONE:\n\n| Load mode | When to use | Required inputs |\n|-----------|-------------|-----------------|\n| `dbt_manifest` | Large projects; fastest parsing | `ProjectConfig.manifest_path` |\n| `dbt_ls` | Complex selectors; need dbt-native selection | Fusion binary accessible to scheduler |\n| `automatic` | Simple setups; let Cosmos pick | (none) |\n\n```python\nfrom cosmos import RenderConfig, LoadMode\n\n_render_config = RenderConfig(\n    load_method=LoadMode.AUTOMATIC,  # or DBT_MANIFEST, DBT_LS\n)\n```\n\n---\n\n## 4. Configure Warehouse Connection (ProfileConfig)\n\n> **Reference**: See **[reference/cosmos-config.md](reference/cosmos-config.md#profileconfig-warehouse-connection)** for full ProfileConfig options and examples.\n\n\n```python\nfrom cosmos import ProfileConfig\nfrom cosmos.profiles import SnowflakeUserPasswordProfileMapping\n\n_profile_config = ProfileConfig(\n    profile_name=\"default\",\n    target_name=\"dev\",\n    profile_mapping=SnowflakeUserPasswordProfileMapping(\n        conn_id=\"snowflake_default\",\n    ),\n)\n```\n\n---\n\n## 5. Configure ExecutionConfig (LOCAL Only)\n\n> **CRITICAL**: dbt Fusion with Cosmos requires `ExecutionMode.LOCAL` with `dbt_executable_path` pointing to the Fusion binary.\n\n```python\nfrom cosmos import ExecutionConfig\nfrom cosmos.constants import InvocationMode\n\n_execution_config = ExecutionConfig(\n    invocation_mode=InvocationMode.SUBPROCESS,\n    dbt_executable_path=\"/home/astro/.local/bin/dbt\",  # REQUIRED: path to Fusion binary\n    # execution_mode is LOCAL by default - do not change\n)\n```\n\n---\n\n## 6. Configure Project (ProjectConfig)\n\n```python\nfrom cosmos import ProjectConfig\n\n_project_config = ProjectConfig(\n    dbt_project_path=\"/path/to/dbt/project\",\n    # manifest_path=\"/path/to/manifest.json\",  # for dbt_manifest load mode\n    # install_dbt_deps=False,  # if deps precomputed in CI\n)\n```\n\n---\n\n## 7. Assemble DAG / TaskGroup\n\n### Option A: DbtDag (Standalone)\n\n```python\nfrom cosmos import DbtDag, ProjectConfig, ProfileConfig, ExecutionConfig, RenderConfig\nfrom cosmos.profiles import SnowflakeUserPasswordProfileMapping\nfrom pendulum import datetime\n\n_project_config = ProjectConfig(\n    dbt_project_path=\"/usr/local/airflow/dbt/my_project\",\n)\n\n_profile_config = ProfileConfig(\n    profile_name=\"default\",\n    target_name=\"dev\",\n    profile_mapping=SnowflakeUserPasswordProfileMapping(\n        conn_id=\"snowflake_default\",\n    ),\n)\n\n_execution_config = ExecutionConfig(\n    dbt_executable_path=\"/home/astro/.local/bin/dbt\",  # Fusion binary\n)\n\n_render_config = RenderConfig()\n\nmy_fusion_dag = DbtDag(\n    dag_id=\"my_fusion_cosmos_dag\",\n    project_config=_project_config,\n    profile_config=_profile_config,\n    execution_config=_execution_config,\n    render_config=_render_config,\n    start_date=datetime(2025, 1, 1),\n    schedule=\"@daily\",\n)\n```\n\n### Option B: DbtTaskGroup (Inside Existing DAG)\n\n```python\nfrom airflow.sdk import dag, task  # Airflow 3.x\n# from airflow.decorators import dag, task  # Airflow 2.x\nfrom airflow.models.baseoperator import chain\nfrom cosmos import DbtTaskGroup, ProjectConfig, ProfileConfig, ExecutionConfig\nfrom pendulum import datetime\n\n_project_config = ProjectConfig(dbt_project_path=\"/usr/local/airflow/dbt/my_project\")\n_profile_config = ProfileConfig(profile_name=\"default\", target_name=\"dev\")\n_execution_config = ExecutionConfig(dbt_executable_path=\"/home/astro/.local/bin/dbt\")\n\n@dag(start_date=datetime(2025, 1, 1), schedule=\"@daily\")\ndef my_dag():\n    @task\n    def pre_dbt():\n        return \"some_value\"\n\n    dbt = DbtTaskGroup(\n        group_id=\"dbt_fusion_project\",\n        project_config=_project_config,\n        profile_config=_profile_config,\n        execution_config=_execution_config,\n    )\n\n    @task\n    def post_dbt():\n        pass\n\n    chain(pre_dbt(), dbt, post_dbt())\n\nmy_dag()\n```\n\n---\n\n## 8. Final Validation\n\nBefore finalizing, verify:\n\n- [ ] **Cosmos version**: ≥1.11.0\n- [ ] **Fusion binary installed**: Path exists and is executable\n- [ ] **Warehouse supported**: Snowflake, Databricks, Bigquery or Redshift only\n- [ ] **Secrets handling**: Airflow connections or env vars, NOT plaintext\n\n### Troubleshooting\n\nIf user reports dbt Core regressions after enabling Fusion:\n\n```bash\nAIRFLOW__COSMOS__PRE_DBT_FUSION=1\n```\n\n### User Must Test\n\n- [ ] The DAG parses in the Airflow UI (no import/parse-time errors)\n- [ ] A manual run succeeds against the target warehouse (at least one model)\n\n---\n\n## Reference\n\n- Cosmos dbt Fusion docs: https://astronomer.github.io/astronomer-cosmos/configuration/dbt-fusion.html\n- dbt Fusion install: https://docs.getdbt.com/docs/core/pip-install#dbt-fusion\n\n---\n\n## Related Skills\n\n- **cosmos-dbt-core**: For dbt Core projects (not Fusion)\n- **authoring-dags**: General DAG authoring patterns\n- **testing-dags**: Testing DAGs after creation\n"
}

SHA-256: 7ef60ec25b2874e04fa44e2db0fffd13c088a83e80e1b522044e6db7bf921589