Update to ElevenLabs

{
  "name": "dubbing",
  "description": "Dub audio and video into other languages using the ElevenLabs Dubbing API (dubbing_v2), preserving the original speakers' voices. Use when translating videos, podcasts, or recordings into other languages, localizing media content, reviewing or correcting dubbing transcripts and translations, or regenerating a dub after edits.",
  "included_files": [
    {
      "relative_path": "references/api-reference.md",
      "size_in_bytes": 13788
    },
    {
      "relative_path": "references/installation.md",
      "size_in_bytes": 2667
    }
  ],
  "skill_md_contents": "---\nname: dubbing\ndescription: Dub audio and video into other languages using the ElevenLabs Dubbing API (dubbing_v2), preserving the original speakers' voices. Use when translating videos, podcasts, or recordings into other languages, localizing media content, reviewing or correcting dubbing transcripts and translations, or regenerating a dub after edits.\nlicense: MIT\ncompatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY).\nmetadata: {\"openclaw\": {\"requires\": {\"env\": [\"ELEVENLABS_API_KEY\"]}, \"primaryEnv\": \"ELEVENLABS_API_KEY\"}}\n---\n\n# ElevenLabs Dubbing\n\nDub audio or video into other languages while preserving the original speakers' voices. Create a project from a file or URL, review and edit the source transcript, add one or more target languages, refine translations per segment, and regenerate outputs.\n\n> **Important:** Use the Dubbing Projects API — `elevenlabs.dubbing.project.*` in the SDKs, or the `/v1/dubbing/project` REST endpoints. Do **not** use the legacy v1 dubbing surface (`client.dubbing.create()`, `client.dubbing.get()`, `client.dubbing.audio.get()`, or bare `/v1/dubbing` routes) — that is the older dubbing API, now under Legacy in the API reference.\n\n> **Setup:** See [Installation Guide](references/installation.md). The `elevenlabs` CLI and the SDKs read `ELEVENLABS_API_KEY` automatically; REST base URL is `https://api.elevenlabs.io` with your API key in the `xi-api-key` header.\n\n## Concepts\n\n| Concept | Meaning |\n|---------|---------|\n| **Project** | One source of media (file or URL) plus its source transcript. Prepared (transcribed) once, then rests in `ready` while you add languages. |\n| **Source transcript** | Editable segments (text, speaker, timing) transcribed from the source. The single source of truth every language is translated from. |\n| **Language (target)** | One dubbed output language. Each has its own transcript (source segments + a translation per segment) and its own dubbed audio output. |\n| **Revisions** | Independent monotonic counters. The project's `revision` bumps on source-transcript edits; a language's `revision` bumps on translation edits or source edits that affect it. A language's `output_revision` is the revision its current audio was generated from — when it's behind `revision`, the output is out of date. |\n\n**Recommended order of operations:** finalize the source transcript **before** adding any languages. Translations are produced from the source, so correcting the source first means every language starts from the right text — editing the source after a language completes marks it `stale` and requires a (charged) regeneration.\n\n> **Enterprise:** Transcript editing and regeneration are available to enterprise workspaces only. Creating projects, adding languages, and downloading dubs work on all plans.\n\n## Workflow\n\n1. **Create** the project from a file or URL → `queued`\n2. **Poll** the project until `ready`\n3. **Review and finalize the source transcript** (edit/add/delete segments)\n4. **Add** one language per target → `queued` → `processing` → `completed`\n5. **Download** each language's `outputs.lossless_audio` when `completed`\n6. **Refine** translations per segment if needed → the language goes `stale`\n7. **Regenerate** the language → `completed` again with fresh output\n\n## Quick Start (Python)\n\n```python\nimport os\nimport time\nimport requests\nfrom elevenlabs.client import ElevenLabs\n\nelevenlabs = ElevenLabs(api_key=os.getenv(\"ELEVENLABS_API_KEY\"))\n\n# 1. Create a project from a local file (or pass source_url=... instead of file)\nwith open(\"promo.mp4\", \"rb\") as f:\n    project = elevenlabs.dubbing.project.create(\n        file=f,\n        source_language=\"en\",\n        reference=\"Q3 marketing video\",\n    )\n\n# 2. Wait for the source media to be transcribed\nwhile True:\n    project = elevenlabs.dubbing.project.get(project.project_id)\n    if project.status == \"ready\":\n        break\n    if project.status == \"failed\":\n        raise RuntimeError(\"Project preparation failed\")\n    time.sleep(5)\n\n# 3. Add a Spanish language target\nlanguage = elevenlabs.dubbing.project.language.create(\n    project.project_id,\n    target_language=\"es\",\n)\n\n# 4. Wait for the dub to finish generating\nwhile True:\n    language = elevenlabs.dubbing.project.language.get(\n        project.project_id, language.language_id\n    )\n    if language.status == \"completed\":\n        break\n    if language.status == \"failed\":\n        raise RuntimeError(\"Dub generation failed\")\n    time.sleep(5)\n\n# 5. Download the dubbed audio (signed URL, valid ~1 hour — re-fetch the language for a fresh one)\naudio = requests.get(language.outputs.lossless_audio)\nwith open(\"promo_es.wav\", \"wb\") as f:\n    f.write(audio.content)\n```\n\n## Quick Start (JavaScript)\n\n```typescript\nimport { ElevenLabsClient } from \"@elevenlabs/elevenlabs-js\";\nimport { writeFile } from \"fs/promises\";\n\nconst elevenlabs = new ElevenLabsClient();\n\n// 1. Create a project (sourceUrl shown; file upload is also supported)\nlet project = await elevenlabs.dubbing.project.create({\n  sourceUrl: \"https://example.com/promo.mp4\",\n  sourceLanguage: \"en\",\n  reference: \"Q3 marketing video\",\n});\n\n// 2. Wait for the source media to be transcribed\nwhile (true) {\n  project = await elevenlabs.dubbing.project.get(project.projectId);\n  if (project.status === \"ready\") break;\n  if (project.status === \"failed\") throw new Error(\"Project preparation failed\");\n  await new Promise((resolve) => setTimeout(resolve, 5000));\n}\n\n// 3. Add a Spanish language target\nlet language = await elevenlabs.dubbing.project.language.create(project.projectId, {\n  targetLanguage: \"es\",\n});\n\n// 4. Wait for the dub to finish generating\nwhile (true) {\n  language = await elevenlabs.dubbing.project.language.get(project.projectId, language.languageId);\n  if (language.status === \"completed\") break;\n  if (language.status === \"failed\") throw new Error(\"Dub generation failed\");\n  await new Promise((resolve) => setTimeout(resolve, 5000));\n}\n\n// 5. Download the dubbed audio from the signed URL\nconst response = await fetch(language.outputs!.losslessAudio!);\nawait writeFile(\"promo_es.wav\", Buffer.from(await response.arrayBuffer()));\n```\n\n## Quick Start (CLI)\n\nThe `elevenlabs` CLI reads `ELEVENLABS_API_KEY` from the environment automatically.\n\n```bash\n# 1. Create a project (use --source-url \"https://...\" instead of --file to dub from a URL)\nelevenlabs dubbing project create --file promo.mp4 --source-language en\n# → {\"project_id\": \"proj_...\", \"status\": \"queued\", ...}\n\n# 2. Poll until status is \"ready\"\nelevenlabs dubbing project get --project-id proj_...\n\n# 3. Add a target language\nelevenlabs dubbing project language create --project-id proj_... --target-language es\n\n# 4. Poll the language until \"completed\", then download outputs.lossless_audio\nelevenlabs dubbing project language get --project-id proj_... --language-id lang_...\n```\n\n## Create Options\n\n`elevenlabs dubbing project create` (REST: `POST /v1/dubbing/project`, `multipart/form-data`) takes **either** `file` **or** `source_url` (not both):\n\n| Field | Required | Notes |\n|-------|----------|-------|\n| `file` | one of file/source_url | Source media to dub (audio or video), up to 3 GiB |\n| `source_url` | one of file/source_url | Public URL to fetch the source media from |\n| `source_language` | no | ISO 639 code (e.g. `en`). Omit to auto-detect — the detected language is reported on the source transcript's `language` field |\n| `reference` | no | Free-form label to identify the project on your end (max 500 chars) |\n| `model_id` | no | `dubbing_v2` (default) |\n| `target_language` | no | Optionally queue the first language target at creation; add more with `language.create` |\n| `keyterms` | no | Terms to bias transcription/translation toward (product/brand names). Up to 1000 terms; each at most 50 chars and 5 words; `<>{}[]\\` not allowed. Repeat the field once per term in multipart |\n\n## Editing the Source Transcript\n\nOnce the project is `ready`, read the transcript, then correct it before adding languages. Every edit bumps the project's `revision`. Each segment has a stable `id` used to edit or delete it. (Enterprise workspaces only.)\n\n```python\n# Read the source transcript\ntranscript = elevenlabs.dubbing.project.transcript.get(project_id)\n\n# Correct a segment's text — send only the fields to change (text, speaker_id, start_s, end_s)\nelevenlabs.dubbing.project.transcript.update_segment(\n    project_id,\n    segment_id=transcript.segments[0].id,\n    text=\"Welcome to our latest product demo.\",\n)\n\n# Add a segment (reuse an existing speaker_id so it's dubbed with that speaker's voice)\nadded = elevenlabs.dubbing.project.transcript.create_segment(\n    project_id,\n    text=\"Thanks for watching.\",\n    speaker_id=transcript.segments[0].speaker_id,\n    start_s=40.0,\n    end_s=42.0,\n)\n\n# Delete a segment\nelevenlabs.dubbing.project.transcript.delete_segment(project_id, segment_id=added.segment.id)\n```\n\nVia the CLI: `elevenlabs dubbing project transcript get --project-id proj_...`, then update a segment with only the changed fields (`--text`, `--speaker-id`, `--start-s`, `--end-s`):\n\n```bash\nelevenlabs dubbing project transcript update_segment \\\n  --project-id proj_... --segment-id seg_... \\\n  --text \"Welcome to our latest product demo.\"\n```\n\n## Refining Translations and Regenerating\n\nA language's transcript pairs each source segment with its `translation` (`null` = not yet translated; segment ids match the source). Edit a single translation, then regenerate. (Enterprise workspaces only.)\n\n```python\n# Read the language's translations\ntarget = elevenlabs.dubbing.project.language.transcript.get(project_id, language_id)\n\n# Refine a single translation (pass translation=None to clear it and mark for re-translation)\nelevenlabs.dubbing.project.language.transcript.update_segment(\n    project_id,\n    language_id,\n    segment_id=target.segments[0].id,\n    translation=\"Bienvenido a nuestra última demostración de producto.\",\n)\n\n# Regenerate the dub from the current transcript (charged like a generation)\nelevenlabs.dubbing.project.language.transcript.regenerate(project_id, language_id)\n```\n\nVia the CLI: `elevenlabs dubbing project language transcript update_segment --project-id proj_... --language-id lang_... --segment-id seg_... --translation \"...\"`, then `elevenlabs dubbing project language transcript regenerate --project-id proj_... --language-id lang_...` (returns `202 Accepted`).\n\nA translation edit affects only that language. After the edit, a `completed` language becomes `stale` — it keeps serving its previous output until you regenerate. Poll until `completed`; `output_revision` then equals `revision` and `outputs.lossless_audio` reflects the current transcript.\n\n## Dubbing into Multiple Languages\n\nAdd one language target per language — each generates independently. Track them all with `language.list` instead of polling one by one:\n\n```python\nfor lang in [\"es\", \"fr\", \"de\", \"ja\"]:\n    elevenlabs.dubbing.project.language.create(project_id, target_language=lang)\n\nwhile True:\n    result = elevenlabs.dubbing.project.language.list(project_id)\n    if not any(l.status in (\"queued\", \"processing\") for l in result.languages):\n        break\n    time.sleep(5)\n```\n\n## States\n\n**Project:**\n\n| Status | Meaning |\n|--------|---------|\n| `queued` | Created; source fetch + preparation enqueued |\n| `preparing` | Preparation (transcription) running |\n| `ready` | Source transcript available; add/generate languages. Projects **stay** `ready` — per-language progress lives on the languages |\n| `failed` | Preparation failed (e.g. source couldn't be fetched or decoded) |\n\n**Language:**\n\n| Status | Meaning |\n|--------|---------|\n| `queued` | Waiting on the project becoming `ready`, or on a generation worker |\n| `processing` | The dub is being generated |\n| `completed` | Finished; `outputs` populated with a signed download URL (valid ~1 hour — re-fetch for a fresh one) |\n| `stale` | Previously completed, but the transcript changed; keeps the last output until regenerated |\n| `failed` | Generation failed |\n\nYou can add a language before the project is `ready` — it stays `queued` and starts automatically once the project becomes `ready`. Adding a language accepts optional `model_id` (defaults to the project's) and `voice_settings` (e.g. `{\"cloning_strength\": 7}`, range 0–10, default 7 — controls how strongly dubbed speakers clone the source voices).\n\n## Error Handling\n\n- **401**: Invalid API key\n- **409 Conflict** on regenerate: The project isn't `ready` or the language isn't settled (e.g. already generating) — wait and retry\n- **Expired download URL**: `outputs.lossless_audio` is signed and valid ~1 hour; re-fetch the language for a fresh URL\n- **Transcript editing / regeneration unavailable**: These endpoints are enterprise-only — on other plans, create the project with a finalized source and add languages directly\n\n## References\n\n- [Installation Guide](references/installation.md)\n- [API Reference](references/api-reference.md) — every endpoint with full request/response schemas and SDK method names\n"
}