Seedream 5.0 Pro เปิดให้ใช้งานแล้ว | ลองใช้ในเครื่องสร้างรูปภาพ →
เข้าสู่ระบบ

Sync Lipsync 2

sync /

Sync Lipsync-2 synchronizes lip movements in any video to supplied audio, enabling realistic mouth alignment for films, podcasts, games, or animations. Ready-to-use REST inference API, best performance, no coldstarts, affordable pricing.

digital-human
อินพุต

ว่าง

$0.05ต่อครั้ง·~20 / $1

ตัวอย่างดูทั้งหมด

โมเดลที่เกี่ยวข้อง

README

Lipsync 2.0 — Video Re-Dub (Video + Audio to Lipsynced Video)

Lipsync 2.0 is a zero-shot lipsync model that takes an existing video and a separate audio track, then re-animates the mouth so lip movements match the speech. No training or fine-tuning is required, and it preserves the speaker’s style across languages, dubbing scenarios, and character types.

🔍 What it does

  • Zero-shot lipsync – Drop in any talking-face clip plus new audio; the model directly outputs a re-synced video.
  • Style preservation – Keeps the original speaker’s facial mannerisms and timing as much as possible, even for translated or re-recorded lines.
  • Cross-domain support – Works with live-action footage, stylised 3D characters, and AI-generated avatars.
  • Flexible editing workflows – Use it for dubbing, dialogue fixes in post, or re-animating entire performances while keeping the original shot.

🧩 Parameters

  • video* Source video to be re-dubbed (URL or upload). Use clips where the face is clearly visible and not heavily occluded.

  • audio* Target speech audio (URL or upload). The lips will be synced to this track.

  • sync_mode Strategy for matching video and audio durations when they differ:

  • bounce – Ping-pong the video to cover a longer audio span.

  • loop – Loop the video until the audio finishes.

  • cut_off – Truncate to the shorter of video/audio.

  • silence – Pad with silence or frozen frames where needed.

  • remap – Time-remap to better align audio and video over the full clip.

Output: a re-synced MP4 video with lips matching the provided audio.

💰 Pricing

Pricing is linear in video length:

  • Effective rate: $0.05 per second of input video

Examples:

Video lengthPrice
5 s$0.25
10 s$0.50
30 s$1.50
60 s$3.00

🚀 How to use

  1. Upload video under video (clear face, stable framing works best).
  2. Upload audio under audio (clean speech, minimal background noise).
  3. Choose a sync_mode depending on how you want to handle duration mismatches.
  4. Click Run and download the re-dubbed clip once processing completes.

🎛 Practical tips

  • Aim for good lighting and frontal or three-quarter views for more accurate mouth motion.
  • If you just need simple dubbing, start with cut_off; for longer audio over short clips, try loop or remap.
  • Keep audio free of strong music or over-compressed noise for cleaner alignment.
  • For multi-shot edits, process each shot separately, then assemble in your NLE (Premiere, DaVinci, etc.) for full control.

More Models to Try

  • WaveSpeedAI / InfiniteTalk WaveSpeedAI’s single-avatar talking-head model that turns one photo plus audio into smooth, lip-synced digital presenter videos for tutorials, marketing, and social content.

  • WaveSpeedAI / InfiniteTalk Multi Multi-avatar version of InfiniteTalk that drives several characters in one scene from separate audio tracks, ideal for dialog-style explainers, interviews, and role-play videos.

  • Kwaivgi / Kling V2 AI Avatar Standard Cost-effective Kling-based AI avatar model that generates natural talking-face videos from a single reference image and voice track, suitable for everyday content and customer support.

  • Kwaivgi / Kling V2 AI Avatar Pro Higher-fidelity Kling V2 avatar model for premium digital humans, offering smoother motion, better lip-sync, and more stable faces for commercials, brand spokespeople, and product demos.

หมายเหตุ:เว็บไซต์นี้ใช้โมเดล AI ที่จัดหาโดยบุคคลที่สาม

Lipsync 2 API — Quick start

Grab a WaveSpeedAI API key, then call POST https://api.wavespeed.ai/api/v3/sync/lipsync-2 with your input as JSON. The endpoint returns a prediction id. Start polling the result endpoint around every 2 seconds, increase the interval for long-running tasks, and stop on any terminal status. On completed, read output values from data.outputs. Examples for Lipsync 2 below.

HTTP example
set -euo pipefail

: "${WAVESPEED_API_KEY:?Set WAVESPEED_API_KEY}"

REQUEST_BODY=$(cat <<'JSON'
{
    "video": "https://interactive-examples.mdn.mozilla.net/media/cc0-videos/flower.mp4",
    "audio": "https://interactive-examples.mdn.mozilla.net/media/cc0-audio/t-rex-roar.mp3",
    "sync_mode": "cut_off"
}
JSON
)

# 1. Submit the prediction.
SUBMIT_RESPONSE=$(curl --silent --show-error --fail-with-body \
  -X POST "https://api.wavespeed.ai/api/v3/sync/lipsync-2" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $WAVESPEED_API_KEY" \
  -d "$REQUEST_BODY")

TASK=$(printf '%s' "$SUBMIT_RESPONSE" | jq 'if has("data") then .data else . end')
PREDICTION_ID=$(printf '%s' "$TASK" | jq -r '.id')
if [ -z "$PREDICTION_ID" ] || [ "$PREDICTION_ID" = "null" ]; then
  printf 'Submission response did not contain a prediction id
' >&2
  exit 1
fi
RESULT_URL=$(printf '%s' "$TASK" | jq -r '.urls.get // empty')
if [ -z "$RESULT_URL" ]; then
  RESULT_URL="https://api.wavespeed.ai/api/v3/predictions/$PREDICTION_ID/result"
fi

# 2. Poll until the prediction finishes.
while true; do
  RESPONSE=$(curl --silent --show-error --fail-with-body "$RESULT_URL" \
    -H "Authorization: Bearer $WAVESPEED_API_KEY")
  RESULT=$(printf '%s' "$RESPONSE" | jq 'if has("data") then .data else . end')
  STATUS=$(printf '%s' "$RESULT" | jq -r '.status')
  case "$STATUS" in
    completed) printf '%s\n' "$RESULT" | jq '.outputs'; break ;;
    failed|cancelled|timeout) printf '%s\n' "$RESULT" | jq . >&2; exit 1 ;;
    created|processing) sleep 2 ;;
    *) printf 'Unexpected status: %s
' "$STATUS" >&2; exit 1 ;;
  esac
done
Node.js example
const submitUrl = "https://api.wavespeed.ai/api/v3/sync/lipsync-2";
const apiKey = process.env.WAVESPEED_API_KEY;
if (!apiKey) throw new Error('Set WAVESPEED_API_KEY');

async function requestJson(url, options = {}) {
  const response = await fetch(url, options);
  if (!response.ok) throw new Error(await response.text());
  return response.json();
}

// 1. Submit the prediction.
const body = await requestJson(submitUrl, {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${apiKey}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
        "video": "https://interactive-examples.mdn.mozilla.net/media/cc0-videos/flower.mp4",
        "audio": "https://interactive-examples.mdn.mozilla.net/media/cc0-audio/t-rex-roar.mp3",
        "sync_mode": "cut_off"
}),
});
const task = body.data ?? body;
if (!task.id) throw new Error("Submission response did not contain a prediction id");
const resultUrl = task.urls?.get ||
  `https://api.wavespeed.ai/api/v3/predictions/${task.id}/result`;

// 2. Poll until the prediction finishes.
while (true) {
  const resultBody = await requestJson(resultUrl, {
    headers: { "Authorization": `Bearer ${apiKey}` },
  });
  const result = resultBody.data ?? resultBody;
  if (result.status === "completed") {
    console.log(result.outputs);
    break;
  }
  if (["failed", "cancelled", "timeout"].includes(result.status)) throw new Error(JSON.stringify(result));
  if (!["created", "processing"].includes(result.status)) throw new Error("Unexpected status: " + result.status);
  await new Promise(resolve => setTimeout(resolve, 2000));
}
Python example
import json
import os
import time
from urllib.request import Request, urlopen

api_key = os.environ["WAVESPEED_API_KEY"]
headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
payload = {
    "video": "https://interactive-examples.mdn.mozilla.net/media/cc0-videos/flower.mp4",
    "audio": "https://interactive-examples.mdn.mozilla.net/media/cc0-audio/t-rex-roar.mp3",
    "sync_mode": "cut_off"
}

def request_json(url, data=None):
    request = Request(url, data=data, headers=headers, method="POST" if data else "GET")
    with urlopen(request) as response:
        return json.load(response)

# 1. Submit the prediction.
body = request_json("https://api.wavespeed.ai/api/v3/sync/lipsync-2", json.dumps(payload).encode())
task = body.get("data", body)
if not task.get("id"):
    raise RuntimeError("Submission response did not contain a prediction id")
result_url = task.get("urls", {}).get("get") or f"https://api.wavespeed.ai/api/v3/predictions/{task['id']}/result"

# 2. Poll until the prediction finishes.
while True:
    result_body = request_json(result_url)
    result = result_body.get("data", result_body)
    status = result.get("status")
    if status == "completed":
        print(result.get("outputs", []))
        break
    if status in {"failed", "cancelled", "timeout"}:
        raise RuntimeError(result)
    if status not in {"created", "processing"}:
        raise RuntimeError(f"Unexpected status: {status}")
    time.sleep(2)

Lipsync 2 API — Frequently asked questions

What is the Lipsync 2 API?

Lipsync 2 is a Sync model for talking-avatar generation, exposed as a REST API on WaveSpeedAI. Sync Lipsync-2 synchronizes lip movements in any video to supplied audio, enabling realistic mouth alignment for films, podcasts, games, or animations. Ready-to-use REST inference API, best performance, no coldstarts, affordable pricing. You can call it programmatically or try it from the playground above.

How do I call the Lipsync 2 API?

POST your input parameters to the model's REST endpoint (shown in the API tab of this playground) with your WaveSpeedAI API key in the Authorization header. Submission returns a prediction ID. Poll the result endpoint starting around every 2 seconds, increase the interval for long-running tasks, and stop on any terminal status. The playground generates production-oriented Python, JavaScript, and cURL examples with timeouts, transient-error handling, and safe GET retries. Full request/response shape is documented at https://wavespeed.ai/docs/docs-api/sync/sync-lipsync-2.

How much does Lipsync 2 cost per run?

Lipsync 2 starts at $0.050 per run. That figure is the base price — the final charge scales with the parameters you set in the form (output size, length, count, references, or whatever knobs this model exposes), so a higher-quality or larger output costs more than a minimal one. The exact cost for your current input is shown live next to the Generate button before you submit, and the actual per-call charge is recorded on the prediction afterwards.

What inputs does Lipsync 2 accept?

Key inputs: `video`, `audio`, `sync_mode`. The full JSON schema (types, defaults, allowed values) is rendered above the Generate button and mirrored in the API reference at https://wavespeed.ai/docs/docs-api/sync/sync-lipsync-2.

How long does Lipsync 2 take to generate?

Median end-to-end generation time on WaveSpeedAI is around 143 seconds per request, based on recent successful runs. Queue time varies with global demand; live status is visible in the prediction record.

Can I use Lipsync 2 outputs commercially?

Commercial usage rights depend on the model's license, set by its provider (Sync). The license summary appears on the model card above; see WaveSpeedAI's Terms of Service for platform-level conditions.

Sync Lipsync 2 | AI Digital Human API | WaveSpeedAI