Update to Domino Data Lab

{
  "description": "Orchestrate multi-step ML workflows using Domino Flows (built on Flyte). Define DAGs with typed inputs/outputs, heterogeneous environments, automatic lineage, and reproducibility. Use when building data pipelines, multi-stage training workflows, or processes requiring orchestration and monitoring.",
  "included_files": [
    {
      "relative_path": "EXAMPLES.md",
      "size_in_bytes": 9942
    },
    {
      "relative_path": "FLOW-BASICS.md",
      "size_in_bytes": 8143
    }
  ],
  "name": "domino-flows",
  "skill_md_contents": "---\nname: domino-flows\ndescription: Orchestrate multi-step ML workflows using Domino Flows (built on Flyte). Define DAGs with typed inputs/outputs, heterogeneous environments, automatic lineage, and reproducibility. Use when building data pipelines, multi-stage training workflows, or processes requiring orchestration and monitoring.\n---\n\n# Domino Flows Skill\n\nThis skill provides comprehensive knowledge for orchestrating ML workflows using Domino Flows, built on the Flyte platform.\n\n## Key Concepts\n\n### What are Domino Flows?\n\nDomino Flows enable:\n- **DAG-based orchestration**: Define workflows as directed acyclic graphs\n- **Typed interfaces**: Strong typing for inputs and outputs\n- **Heterogeneous environments**: Different environments per task\n- **Automatic lineage**: Track data and model provenance\n- **Reproducibility**: Version-controlled workflows\n- **Scalability**: Distributed execution across compute resources\n\n### Core Components\n\n| Component | Description |\n|-----------|-------------|\n| **Task** | Single unit of work (runs as a Domino Job) |\n| **Workflow** | DAG connecting tasks |\n| **Artifact** | Typed input/output passed between tasks |\n| **Launch Plan** | Configured workflow execution |\n\n## Related Documentation\n\n- [FLOW-BASICS.md](./FLOW-BASICS.md) - DAG concepts, task definitions\n- [EXAMPLES.md](./EXAMPLES.md) - Common flow patterns\n\n## Quick Start\n\n> ⚠️ **Critical**: Domino Flows does NOT support native Flyte `@task` decorators.\n> Tasks must use `DominoJobTask` + `DominoJobConfig`. Only `@workflow` is unchanged.\n\n### Basic Flow\n\nEach task runs as a Domino Job. Stage scripts read from `/workflow/inputs/<name>`\nand write to `/workflow/outputs/o0`. Pass `PYTHONPATH=/mnt/code` in the command.\n\n```python\nfrom flytekit import workflow\nfrom flytekitplugins.domino.task import DominoJobConfig, DominoJobTask\n\npreprocess_task = DominoJobTask(\n    name=\"Preprocess Data\",\n    domino_job_config=DominoJobConfig(\n        Command=\"bash -c 'PYTHONPATH=/mnt/code python /mnt/code/stages/preprocess.py'\",\n    ),\n    inputs={\"input_path\": str},\n    outputs={\"o0\": str},\n    use_latest=True,\n)\n\ntrain_task = DominoJobTask(\n    name=\"Train Model\",\n    domino_job_config=DominoJobConfig(\n        Command=\"bash -c 'PYTHONPATH=/mnt/code python /mnt/code/stages/train.py'\",\n    ),\n    inputs={\"preprocess_output\": str},\n    outputs={\"o0\": str},\n    use_latest=True,\n)\n\n@workflow\ndef training_pipeline(input_path: str = \"/mnt/data/raw.csv\") -> str:\n    preprocess_output = preprocess_task(input_path=input_path)\n    result = train_task(preprocess_output=preprocess_output)\n    return result\n```\n\n### Stage Script Pattern\n\n```python\n# stages/preprocess.py\nimport json, os\n\nINPUTS, OUTPUTS = \"/workflow/inputs\", \"/workflow/outputs\"\n\ndef main():\n    input_path = open(f\"{INPUTS}/input_path\").read().strip()\n    # ... do work ...\n    os.makedirs(OUTPUTS, exist_ok=True)\n    with open(f\"{OUTPUTS}/o0\", \"w\") as f:\n        f.write(json.dumps({\"output_path\": \"/mnt/artifacts/processed.parquet\"}))\n\nif __name__ == \"__main__\":\n    main()\n```\n\n### Running the Flow\n\n```bash\n# Always commit and push first — jobs run against remote repo state\ngit add -A && git commit -m \"...\" && git push\n\n# Trigger remotely\nPYTHONPATH=/mnt/code pyflyte run --remote \\\n    my_flow.py training_pipeline \\\n    --input_path \"/mnt/data/raw.csv\"\n```\n\n## When to Use Flows\n\n### Good Use Cases\n\n- Data processing → Model training pipelines\n- ETL with ML steps\n- Multi-stage training with different environments\n- Processes requiring reproducibility and lineage\n- Scheduled/triggered workflows\n\n### Not Ideal For\n\n- Single dataset with many small computations\n- Tasks that write to mutable shared state\n- Simple single-step processes\n- Real-time inference (use Model APIs instead)\n\n## Documentation Links\n\n- Domino Flows: https://docs.dominodatalab.com/en/latest/user_guide/78acf5/orchestrate-with-flows/\n- Flyte Documentation: https://docs.flyte.org/\n"
}