Update to Data Engineering Copilot

{
  "description": "Implement or improve a SQL, Python, PySpark or orchestration pipeline with version-aware safe writes and test evidence.",
  "included_files": [],
  "name": "build-data-pipeline",
  "skill_md_contents": "---\nname: build-data-pipeline\ndescription: Implement or improve a SQL, Python, PySpark or orchestration pipeline with version-aware safe writes and test evidence.\n---\n\n\n# Build a data pipeline\n**Trigger:** User requests runnable code, modifications to a repository or a working ETL/ELT workflow.\n\n1. Inspect actual files, runtime, SDKs, integration permissions, source/target contracts and existing tests before editing. Preserve existing structure; no unrequested rewrite.\n2. Freeze one vertical slice: read bounded sample → validate schema and grain → transform → write to isolated target → reconcile and test.\n3. Make retries safe: deterministic input scope, idempotent side effects, stable keys, source ordering, explicit handling of deletes and rejects.\n4. Provide configuration through environment/secret stores, never hardcoded passwords. Bound retries/timeouts/concurrency; record provenance, failed rows and run metadata.\n5. Include typed/contract tests, sample fixtures, integration smoke tests and rollback/reprocessing instructions. Verify runtime-specific APIs via official docs.\n6. Run code/tests only with available authorized tools. Never state a build passed based on generated code alone. Ask for approval before production write or deploy.\n\nDeliver actual code/diff, commands, measured tests vs untested items, safety/rollback notes. Read `../../references/testing-standards.md` and `../../references/schema-contracts.md`.\n"
}