Sync Lipsync 2 | AI Digital Human API

Lipsync 2.0 — Video Re-Dub (Video + Audio to Lipsynced Video)

Lipsync 2.0 is a zero-shot lipsync model that takes an existing video and a separate audio track, then re-animates the mouth so lip movements match the speech. No training or fine-tuning is required, and it preserves the speaker’s style across languages, dubbing scenarios, and character types.

🔍 What it does

Zero-shot lipsync – Drop in any talking-face clip plus new audio; the model directly outputs a re-synced video.
Style preservation – Keeps the original speaker’s facial mannerisms and timing as much as possible, even for translated or re-recorded lines.
Cross-domain support – Works with live-action footage, stylised 3D characters, and AI-generated avatars.
Flexible editing workflows – Use it for dubbing, dialogue fixes in post, or re-animating entire performances while keeping the original shot.

🧩 Parameters

video* Source video to be re-dubbed (URL or upload). Use clips where the face is clearly visible and not heavily occluded.
audio* Target speech audio (URL or upload). The lips will be synced to this track.
sync_mode Strategy for matching video and audio durations when they differ:
bounce – Ping-pong the video to cover a longer audio span.
loop – Loop the video until the audio finishes.
cut_off – Truncate to the shorter of video/audio.
silence – Pad with silence or frozen frames where needed.
remap – Time-remap to better align audio and video over the full clip.

Output: a re-synced MP4 video with lips matching the provided audio.

💰 Pricing

Pricing is linear in video length:

Effective rate: $0.05 per second of input video

Examples:

Video length	Price
5 s	$0.25
10 s	$0.50
30 s	$1.50
60 s	$3.00

🚀 How to use

Upload video under video (clear face, stable framing works best).
Upload audio under audio (clean speech, minimal background noise).
Choose a sync_mode depending on how you want to handle duration mismatches.
Click Run and download the re-dubbed clip once processing completes.

🎛 Practical tips

Aim for good lighting and frontal or three-quarter views for more accurate mouth motion.
If you just need simple dubbing, start with cut_off; for longer audio over short clips, try loop or remap.
Keep audio free of strong music or over-compressed noise for cleaner alignment.
For multi-shot edits, process each shot separately, then assemble in your NLE (Premiere, DaVinci, etc.) for full control.

More Models to Try

WaveSpeedAI / InfiniteTalk WaveSpeedAI’s single-avatar talking-head model that turns one photo plus audio into smooth, lip-synced digital presenter videos for tutorials, marketing, and social content.
WaveSpeedAI / InfiniteTalk Multi Multi-avatar version of InfiniteTalk that drives several characters in one scene from separate audio tracks, ideal for dialog-style explainers, interviews, and role-play videos.
Kwaivgi / Kling V2 AI Avatar Standard Cost-effective Kling-based AI avatar model that generates natural talking-face videos from a single reference image and voice track, suitable for everyday content and customer support.
Kwaivgi / Kling V2 AI Avatar Pro Higher-fidelity Kling V2 avatar model for premium digital humans, offering smoother motion, better lip-sync, and more stable faces for commercials, brand spokespeople, and product demos.

Lipsync 2 API — Quick start

Grab a WaveSpeedAI API key, then call POST https://api.wavespeed.ai/api/v3/sync/lipsync-2 with your input as JSON. The endpoint returns a prediction id. Start polling the result endpoint around every 2 seconds, increase the interval for long-running tasks, and stop on any terminal status. On completed, read URLs from data.outputs. Examples for Lipsync 2 below.

HTTP example

# Submit the prediction
curl --fail-with-body --connect-timeout 10 --max-time 60 \
  -X POST "https://api.wavespeed.ai/api/v3/sync/lipsync-2" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $WAVESPEED_API_KEY" \
  -d '{
    "video": "https://example.com/your-input.mp4",
    "audio": "https://example.com/your-audio.mp3",
    "sync_mode": "cut_off"
}'

# Wait at least 2 seconds, then poll. Safe GET requests may be retried.
curl --fail-with-body --connect-timeout 10 --max-time 30 \
  --retry 4 --retry-all-errors --retry-delay 1 \
  -X GET "https://api.wavespeed.ai/api/v3/predictions/{request_id}/result" \
  -H "Authorization: Bearer $WAVESPEED_API_KEY"

# Start at 2 seconds and increase the interval for long-running tasks.
# Stop on completed, failed, cancelled, or timeout.

Node.js example

// npm install wavespeed
const { Client } = require('wavespeed');

const apiKey = process.env.WAVESPEED_API_KEY;
if (!apiKey) throw new Error('Set WAVESPEED_API_KEY');
const client = new Client(apiKey, {
  maxConnectionRetries: 5,
  retryInterval: 1.0,
});

try {
  const result = await client.run("sync/lipsync-2", {
        "video": "https://example.com/your-input.mp4",
        "audio": "https://example.com/your-audio.mp3",
        "sync_mode": "cut_off"
}, {
    timeout: 3600,
    pollInterval: 2.0,
  });
  console.log(result.outputs);
} catch (error) {
  console.error('Generation failed:', error);
  process.exitCode = 1;
}

Python example

# pip install wavespeed
import os
from wavespeed import Client

client = Client(
    api_key=os.environ["WAVESPEED_API_KEY"],
    max_connection_retries=5,
    retry_interval=1.0,
)

try:
    output = client.run(
        "sync/lipsync-2",
        {
    "video": "https://example.com/your-input.mp4",
    "audio": "https://example.com/your-audio.mp3",
    "sync_mode": "cut_off"
},
        timeout=3600.0,
        poll_interval=2.0,
    )
    print(output["outputs"])
except Exception as error:
    raise SystemExit(f"Generation failed: {error}") from error

Lipsync 2 API — Frequently asked questions

What is the Lipsync 2 API?

Lipsync 2 is a Sync model for talking-avatar generation, exposed as a REST API on WaveSpeedAI. Sync Lipsync-2 synchronizes lip movements in any video to supplied audio, enabling realistic mouth alignment for films, podcasts, games, or animations. Ready-to-use REST inference API, best performance, no coldstarts, affordable pricing. You can call it programmatically or try it from the playground above.

How do I call the Lipsync 2 API?

POST your input parameters to the model's REST endpoint (shown in the API tab of this playground) with your WaveSpeedAI API key in the Authorization header. Submission returns a prediction ID. Poll the result endpoint starting around every 2 seconds, increase the interval for long-running tasks, and stop on any terminal status. The playground generates production-oriented Python, JavaScript, and cURL examples with timeouts, transient-error handling, and safe GET retries. Full request/response shape is documented at https://wavespeed.ai/docs/docs-api/sync/sync-lipsync-2.

How much does Lipsync 2 cost per run?

Lipsync 2 starts at $0.050 per run. That figure is the base price — the final charge scales with the parameters you set in the form (output size, length, count, references, or whatever knobs this model exposes), so a higher-quality or larger output costs more than a minimal one. The exact cost for your current input is shown live next to the Generate button before you submit, and the actual per-call charge is recorded on the prediction afterwards.

What inputs does Lipsync 2 accept?

Key inputs: `video`, `audio`, `sync_mode`. The full JSON schema (types, defaults, allowed values) is rendered above the Generate button and mirrored in the API reference at https://wavespeed.ai/docs/docs-api/sync/sync-lipsync-2.

How long does Lipsync 2 take to generate?

Average end-to-end generation time on WaveSpeedAI is around 110 seconds per request — measured across recent runs. Queue time scales with global demand; live status is visible in the prediction record.

Can I use Lipsync 2 outputs commercially?

Commercial usage rights depend on the model's license, set by its provider (Sync). The license summary appears on the model card above; see WaveSpeedAI's Terms of Service for platform-level conditions.

ExamplesView all

Related Models

README