sync. labs

View saved version →

---
name: edit-dialogue
description: Change or remove a few spoken words in a single-speaker video with sync. labs, keeping the speaker's own voice and re-syncing the lips. Use for fixing a misspoken word, a wrong name, number, price or date, a mispronunciation, or cutting a filler word or phrase without reshooting. Not for replacing a whole script (lipsync-video with a script), translation (translate-and-dub) or switching to a different voice.
---

# Fix spoken words

Delivers one MP4 where only the agreed words change. Everything else keeps the original picture and voice.

## Inputs

- **Video**: a single-speaker video, as a file in this conversation, a public URL or a Sync video asset ID.
- **The fix**: which words to change or remove, and what to say instead. Never guess the replacement words.

## Tools

| Step | Tool |
| --- | --- |
| Check support | `get-dialogue-options` |
| Create the project | `projects_create` |
| Save the video | `upload-media`, `assets_create`, `upload-dialogue-source` |
| Transcribe | `create-dialogue-transcription`, `get-dialogue-transcription` |
| Preview the new audio | `create-dialogue-preview`, `get-dialogue-preview` |
| Quote | `estimate-dialogue-video` |
| Create | `create-dialogue-video` |
| Wait for the result | `generate_get-generation` |

## Steps

1. **Support.** Call `get-dialogue-options`. If Edit Dialogue isn't available, say so and stop. Note whether it advertises `keyedPreview`.
2. **Project.** Call `projects_create` with a new random UUID as `id`, a short `name` and `defaultWorkflow: "edit_dialogue"`. Pass the returned `projectId` with `projectIntent: "explicit"` in every later call.
3. **Save the video.** For a file, use `upload-media` with `mediaType: "video"`. For a URL, use `assets_create` with `type: "VIDEO"` and the project.
4. **Transcribe.** Call `create-dialogue-transcription`, then `get-dialogue-transcription` with the returned `id` until it completes. If the API rejects the video's storage, call `upload-dialogue-source` and use its new asset ID for every later step.
5. **Agree the edits.** Show the transcript lines around the fix. Turn the request into edits that use the transcript's word IDs:
   - `{ "kind": "change", "wordId": "w35", "replacement": "Mia" }`, adding `"pronunciation": "Meeah"` only for a name the voice might say wrong;
   - `{ "kind": "remove", "wordIds": ["w12", "w13"] }`.

   Confirm the edits, and confirm the user has permission to use this speaker's voice.
6. **Preview.** Call `create-dialogue-preview` with the edits and `transcriptionId`, plus a new `idempotencyKey` if `keyedPreview` is advertised. Poll `get-dialogue-preview` until it completes. If some edits failed, say which ones; don't present it as fully successful.
7. **Quote.** Call `estimate-dialogue-video` with the completed preview's ID as `dialogueEditId` and `model: "sync-3"`. Show the estimate and wait for approval of the preview and the cost.
8. **Create.** Call `create-dialogue-video` once with the same `dialogueEditId`, `model`, project and a new `idempotencyKey`. On a lost response, retry once with the same key and inputs.
9. **Wait and deliver.** Poll `generate_get-generation` with the returned `id` and `wait: true`. Give the exact `outputUrl`, unchanged.

## Limits

- One speaker only. If several people talk, say so before transcribing.
- Targeted words only. If the user wants a whole new script, explain that this keeps the original recording, and offer lipsync-video with a script and a chosen voice instead.
- Never split a new script into many word edits to get around this limit.

Referenced files: 1 — file references are metadata, not package contents.

View saved version →

---
name: lipsync-video
description: Lip-sync an existing video to new speech with sync. labs, from a voiceover, a re-recorded take, a song, or a typed script that sync. labs voices, including AI-generated clips and YouTube, TikTok, Instagram or Facebook links. Use for "lipsync this video to this audio", "make him say…", "match the lips to my voiceover" or "sync this clip to the song". Not for still pictures (talking-photo), translating speech into another language (translate-and-dub) or changing a few words in the original voice (edit-dialogue).
---

# Lipsync a video

Delivers one MP4 of the video with lips matched to the new speech.

## Inputs

- **Video**: a video file in this conversation, a public video URL, a Sync video asset ID, or a YouTube, TikTok, Instagram or Facebook link.
- **Speech**: an audio file, audio URL or Sync audio asset, or a script plus a voice.

Ask only for what is missing. Never invent the words.

## Tools

| Step | Tool |
| --- | --- |
| Save files from the chat | `upload-media` |
| Import a social link | `projects_create`, `assets_create` |
| Pick a voice (script only) | `voices_get-voices` |
| Quote | `generate_estimate-cost` |
| Shorten long audio | `trim-audio`, `get-audio-trim` |
| Create | `create-lipsync` |
| Wait for the result | `generate_get-generation` |

## Steps

1. **Save files.** For each file in this conversation (the video, or the user's audio), call `upload-media` once with `mediaType` `video` or `audio`. Use the returned asset IDs from here on, including for every retry; a file re-sent later becomes a different asset.
2. **Social links.** Create a project first with `projects_create`, using a new random UUID as `id`, a short `name` and `defaultWorkflow: "lipsync"`. Then call `assets_create` with the link, `type: "VIDEO"`, that `projectId` and `projectIntent: "explicit"`.
   - A 409 `import_in_progress` means the import is still running. Wait `retryAfterMs` (15 seconds if absent) and repeat the identical request until it returns an asset.
   - On `import_failed`, `import_not_enabled`, `import_unavailable` or a limit error, explain the reason, offer a file upload instead, and stop.
3. **Voice.** For a script, call `voices_get-voices`, suggest two or three fitting voices unless one was named, and use the returned `id` as `voiceId`.
4. **Length.** If the audio is noticeably longer or shorter than the video, ask how to handle it, and pass the answer as `options.sync_mode`:
   - `cut_off`: trim the longer one to match
   - `loop`: repeat the video from the start
   - `bounce`: play the video forward, then in reverse
   - `silence`: keep the video silent where the audio runs out
   - `remap`: speed up or slow down the video to match the audio

   If the audio is over a length limit, offer `trim-audio` to keep its first part, then quote again.
5. **Speakers.** If more than one face is visible, ask which person should speak. Leave `options.active_speaker_detection` unset unless the user picks a speaker.
6. **Quote.** Call `generate_estimate-cost` with the model (default `sync-3`) and the measured `audioDuration`, or the `script` for script input. For `cut_off` or `silence`, also pass `videoDuration` and `syncMode`. Label script quotes approximate and say speech synthesis is excluded.
7. **Confirm.** Show the video, speech, voice, any sync mode and the estimate, then wait for approval, unless this exact request and cost were already approved.
8. **Create.** Call `create-lipsync` once with:
   - the video as `videoAssetId` (saved or imported above) or `videoUrl`;
   - the speech as `audioAssetId`, `audioUrl`, or `script` with `voiceId`;
   - the model and any `options`;
   - a new `idempotencyKey`.

   For a project you just created in step 2, pass its `projectId` with `projectIntent: "explicit"`. Otherwise pass a new random UUID as `projectCreationKey` and a short `projectName`. On a lost response, retry once with the same keys and inputs.
9. **Wait and deliver.** Poll `generate_get-generation` with the same `id` and `wait: true` until `COMPLETED` or `FAILED`. Give the exact `outputUrl`, unchanged, and offer one next step.

## Rules

- Use `sync-3` unless the user picks another model; take model names only from `models_get`.
- Never reuse an older project unless the user explicitly asks for it.
- Never start a second paid generation to check progress or to refresh a result.

Referenced files: 1 — file references are metadata, not package contents.

View saved version →

---
name: setup
description: First-run setup for sync. labs after the plugin is installed. Confirms the connected account and its credits, shows what sync. labs can make, and walks the user to a first video. Use when the user runs setup for sync. labs or asks how to get started with it.
---

# Set up sync. labs

Gets a new user from install to a first result in about a minute. Setup itself never starts paid work.

## Tools

| Step | Tool |
| --- | --- |
| Account and credits | `get-account-context`, `get-account-usage` |

## Steps

1. **Account.** Call `get-account-context` and `get-account-usage`. Greet the user by their organization name and say what is available: credits, or the free allowance. Don't promote plans or pricing.
2. **Choices.** Offer the four things sync. labs makes, with one example prompt each:
   - **Make a photo talk:** "Make this photo say: welcome to my channel"
   - **Lipsync a video:** "Lipsync my video to this audio"
   - **Translate and dub:** "Dub this YouTube video into Spanish"
   - **Fix spoken words:** "Change the price I said in this video to $49"
3. **Fastest first win.** If the user isn't sure, suggest a talking photo, because it needs no upload. Create a character with image generation, write a one-line greeting together, and pick a voice. Then follow the talking-photo workflow: quote, get approval, create.
4. **Hand off.** Once the user picks, continue with the matching workflow (talking-photo, lipsync-video, translate-and-dub or edit-dialogue) in this conversation.

## Rules

- Quote before every paid video and wait for approval.
- If the account has no credits left, say so plainly and point to their sync. labs account; don't start a job that will fail.
- Keep it short: one message for the account and the choices, then follow the user's lead.

Referenced files: 1 — file references are metadata, not package contents.

View saved version →

---
name: talking-photo
description: Make a still photo, portrait, mascot, illustration or AI character talk with sync. labs, as a lip-synced video from a typed script (sync. labs voices it) or from the user's own audio. Use for "make this photo talk", "make this picture say…", a talking avatar or character, or animating a portrait with speech, including when the user has no picture yet and wants one created first. Not for moving video footage (lipsync-video), translation (translate-and-dub) or changing a few words in an existing video (edit-dialogue).
---

# Make a photo talk

Delivers one MP4 of the picture speaking the user's words.

## Inputs

- **Picture**: an image file in this conversation (uploaded, or generated here), a public image URL, or a Sync image asset ID. If the user has no picture, offer to create one with image generation first: one clear face, looking at the camera, mouth closed, head and shoulders, plain background.
- **Speech**: either a script, which needs a voice, or an audio file, audio URL or Sync audio asset.

Ask only for what is missing. Never invent the words the character says. If the user asks you to write them, draft a short script and get approval before quoting.

## Tools

| Step | Tool |
| --- | --- |
| Save files from the chat | `upload-media` |
| Pick a voice (script only) | `voices_get-voices` |
| Quote | `generate_estimate-cost` |
| Create | `create-lipsync` |
| Wait for the result | `generate_get-generation` |

## Steps

1. **Save files.** For each file in this conversation (the picture, or the user's audio), call `upload-media` once with `mediaType` `image` or `audio`. Use the returned asset IDs from here on, including for every retry; a file re-sent later becomes a different asset. Public URLs and Sync asset IDs need no upload.
2. **Voice.** For a script, call `voices_get-voices`. Unless the user named a voice, suggest two or three that fit the character, each with a few words on its style. Use the chosen voice's returned `id` as `voiceId`; never invent an ID.
3. **Quote.** Call `generate_estimate-cost` with `model: "sync-3"`, `visualType: "image"`, and either `script` or the measured `audioDuration`. Say that script quotes are approximate and exclude speech synthesis.
4. **Confirm.** Show the picture, the script or audio, the voice and the estimate, then wait for approval. Skip this only when the user already approved this exact request and cost.
5. **Create.** Call `create-lipsync` once with:
   - the picture as `imageAssetId` (saved in step 1) or `imageUrl`;
   - `script` with `voiceId`, or the audio as `audioAssetId` or `audioUrl`;
   - `model: "sync-3"`;
   - a new `idempotencyKey` and a new random UUID as `projectCreationKey`;
   - a short `projectName` such as "Talking fox".

   Do not pass `projectId` for new work. If the response is lost, retry once with the same keys and inputs; never generate new keys for a retry.
6. **Wait.** Call `generate_get-generation` with the returned `id` and `wait: true`. If it is still pending, call it again with the same `id`. Never create a second generation to check progress.
7. **Deliver.** When the status is `COMPLETED`, give the exact `outputUrl`, unchanged. Offer one next step, such as another line, another voice, or dubbing it into another language.

## When something fails

- **The picture is rejected** (no clear face, several faces, too small): explain the reason and suggest a clearer head-and-shoulders picture.
- **Out of credits or over a plan limit**: explain the returned message plainly. Don't promote plans or pricing.
- **Any other error**: quote the returned message, say no video was created, and stop. Don't retry with different inputs unless the user asks.

Referenced files: 1 — file references are metadata, not package contents.

View saved version →

---
name: translate-and-dub
description: Translate a video's speech into another language with sync. labs and re-sync the lips to the dubbed audio, from an uploaded video or a YouTube, TikTok, Instagram or Facebook link, into one or several languages. Use for "dub this video into Spanish", "translate my YouTube video to Japanese", AI dubbing, or localizing a video for other markets. Not for subtitles only, translating a text document, or lip-syncing audio the user already has (lipsync-video).
---

# Translate and dub a video

Delivers one dubbed MP4 per target language, with lips matched to the new audio.

## Inputs

- **Video**: a video file in this conversation, a Sync video asset ID, or a YouTube, TikTok, Instagram or Facebook link.
- **Target language(s)**: ask if none was given. Use `sourceLang: "auto"` unless the user names the spoken language.

## Tools

| Step | Tool |
| --- | --- |
| Check languages | `get-translation-options` |
| Create the project | `projects_create` |
| Save the video | `upload-media` (file) or `assets_create` (link) |
| Quote | `generate_estimate-cost` |
| Create | `create-translate-and-dub` |
| Wait for the result | `generate_get-generation` |

## Steps

1. **Languages.** Call `get-translation-options` and check that every requested language is supported. Name the closest supported option for any that is not.
2. **Project.** Call `projects_create` with a new random UUID as `id`, a short `name` and `defaultWorkflow: "translate_and_dub"`. Use the returned `projectId` with `projectIntent: "explicit"` in every later call.
3. **Save the video.**
   - **File:** call `upload-media` with `mediaType: "video"`.
   - **Link:** call `assets_create` with the URL, `type: "VIDEO"`, the `projectId` and `projectIntent: "explicit"`. A 409 `import_in_progress` means the import is running: wait `retryAfterMs` (15 seconds if absent) and repeat the identical request until it returns an asset. On `import_failed`, `import_not_enabled`, `import_unavailable` or a limit error, explain the reason, offer a file upload, and stop.

   Never pass an `importJobId` as an asset ID.
4. **Quote.** Call `generate_estimate-cost` with `workflow: "translate-and-dub"`, `model: "sync-3"` and the video's duration in seconds. Show the returned breakdown and say that any external voice-provider charges it lists are extra.
5. **Confirm.** Show the video, the language(s) and the estimate per language, then wait for approval, unless this exact request and cost were already approved.
6. **Create.** For each target language, call `create-translate-and-dub` once with `videoAssetId`, `targetLang`, `sourceLang`, `model: "sync-3"`, the `projectId` with `projectIntent: "explicit"`, and its own new `idempotencyKey`. On a lost response, retry that language once with the same key and inputs.
7. **Wait and deliver.** Poll `generate_get-generation` with each returned `id` and `wait: true` until it is `COMPLETED` or `FAILED`. Dubbing takes longer than plain lipsync, so tell the user it may take a few minutes. Give each exact `outputUrl`, unchanged, labelled with its language.

## Rules

- One paid job per language. Never resubmit a language to check progress.
- If a language fails, report it and keep the others.
- Don't offer subtitles, captions or a translated transcript as a substitute; this workflow produces dubbed video only.

Referenced files: 1 — file references are metadata, not package contents.