Sync Lipsync 2 Pro | AI Digital Human API

sync/lipsync-2-pro — Pro Audio-to-Lipsync

Lipsync-2-pro is a zero-shot model for generating realistic lip movements that match spoken audio. It works out of the box—no training or fine-tuning—and preserves the speaker’s natural style across languages, cameras, and video types. From live-action footage to animated or AI-generated faces, it brings broadcast-grade dubbing and dialogue editing into a simple API call.

What it does

Zero-shot lipsync – Just provide a video and an audio track; the model automatically re-animates the mouth to match the speech.
Style preservation – Learns each speaker’s characteristic timing and articulation and keeps that “signature delivery” even when the language changes.
Cross-domain support – Works on real humans, 2D/3D animation, and synthetic/AI avatars.
Flexible workflows – Use it for dubbing, rewriting individual lines in post, or re-animating entire performances.

Key capabilities

Expressiveness control – Internally balances how “subtle vs. animated” the lipsync is, so results can match calm talking-head news, energetic vlogs, or stylised characters.
Active speaker detection – In multi-person scenes, the system can focus lipsync on the currently speaking face instead of blindly animating everyone.
High-fidelity animation – Preserves identity, lighting, background and facial structure; only the mouth and local expressions are changed.
Record once, edit forever – Change lines after shooting without new takes; keep the original performance and camera work.
AI dubbing for any video – Pair with TTS or translation models to localise existing content into new languages.

Parameters

video* Input video to be re-synced (URL or upload). Works best with relatively stable talking-head or upper-body shots.
audio* Target speech (URL or upload). The new lip motion will follow this track.
sync_mode Controls how audio and video lengths are aligned:
cut_off – Trim to the shorter of audio or video (default, safest).
loop – Loop the shorter track until the longer one finishes.
bounce – Ping-pong the video when looping (forward–back–forward…).
silence – Pad missing audio with silence.
remap – Time-warp to better match durations.

Output: a new MP4 video with updated lipsync.

How to use

Upload or paste URLs for video and audio.
Choose an appropriate sync_mode based on whether you want trimming, looping, or padding.
Submit the job and wait for processing to complete.
Download the result and review; if needed, adjust audio, sync_mode or source video and re-run.

Pricing

Billing is purely based on audio length.

Base rate: $0.08 per second of audio

Audio length (seconds)	Price (USD)
5	$0.40
10	$0.80
15	$1.20
30	$2.40
60	$4.80

You can estimate other costs by multiplying the audio duration (in seconds) by $0.08/s; charges scale linearly with length.

More Models to Try

WaveSpeedAI / InfiniteTalk WaveSpeedAI’s single-avatar talking-head model that turns one photo plus audio into smooth, lip-synced digital presenter videos for tutorials, marketing, and social content.
WaveSpeedAI / InfiniteTalk Multi Multi-avatar version of InfiniteTalk that drives several characters in one scene from separate audio tracks, ideal for dialog-style explainers, interviews, and role-play videos.
Kwaivgi / Kling V2 AI Avatar Standard Cost-effective Kling-based AI avatar model that generates natural talking-face videos from a single reference image and voice track, suitable for everyday content and customer support.
Kwaivgi / Kling V2 AI Avatar Pro Higher-fidelity Kling V2 avatar model for premium digital humans, offering smoother motion, better lip-sync, and more stable faces for commercials, brand spokespeople, and product demos.

Lipsync 2 Pro API — Quick start

Grab a WaveSpeedAI API key, then call POST https://api.wavespeed.ai/api/v3/sync/lipsync-2-pro with your input as JSON. The endpoint returns a prediction id; poll the prediction endpoint until status flips to completed, then read the output URL from data.outputs[0]. Examples for Lipsync 2 Pro below.

HTTP example

# Submit the prediction
curl -X POST "https://api.wavespeed.ai/api/v3/sync/lipsync-2-pro" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $WAVESPEED_API_KEY" \
  -d '{
    "video": "https://example.com/your-input.mp4",
    "audio": "https://example.com/your-audio.mp3",
    "sync_mode": "cut_off"
}'

# Response includes a prediction id. Poll for the result:
curl -X GET "https://api.wavespeed.ai/api/v3/predictions/{request_id}/result" \
  -H "Authorization: Bearer $WAVESPEED_API_KEY"

# When status is "completed", read the output from data.outputs[0].

Node.js example

// npm install wavespeed
const WaveSpeed = require('wavespeed');

const client = new WaveSpeed(); // reads WAVESPEED_API_KEY from env

const result = await client.run("sync/lipsync-2-pro", {
        "video": "https://example.com/your-input.mp4",
        "audio": "https://example.com/your-audio.mp3",
        "sync_mode": "cut_off"
});

console.log(result.outputs[0]); // → URL of the generated output

Python example

# pip install wavespeed
import wavespeed

output = wavespeed.run(
    "sync/lipsync-2-pro",
    {
    "video": "https://example.com/your-input.mp4",
    "audio": "https://example.com/your-audio.mp3",
    "sync_mode": "cut_off"
}
)

print(output["outputs"][0])  # → URL of the generated output

Lipsync 2 Pro API — Frequently asked questions

What is the Lipsync 2 Pro API?

Lipsync 2 Pro is a Sync model for talking-avatar generation, exposed as a REST API on WaveSpeedAI. Lipsync-2-pro creates studio-grade lip synchronization for video-to-video editing in minutes, not weeks. Ready-to-use REST inference API, best performance, no coldstarts, affordable pricing. You can call it programmatically or try it from the playground above.

How do I call the Lipsync 2 Pro API?

POST your input parameters to the model's REST endpoint (shown in the API tab of this playground) with your WaveSpeedAI API key in the Authorization header. Submission returns a prediction ID; poll the prediction endpoint until status flips to "completed", then read the output URL from the result. The playground generates a ready-to-paste code sample in Python, JavaScript, or cURL for whatever inputs you've set. Full request/response shape is documented at https://wavespeed.ai/docs/docs-api/sync/sync-lipsync-2-pro.

How much does Lipsync 2 Pro cost per run?

Lipsync 2 Pro starts at $0.080 per run. That figure is the base price — the final charge scales with the parameters you set in the form (output size, length, count, references, or whatever knobs this model exposes), so a higher-quality or larger output costs more than a minimal one. The exact cost for your current input is shown live next to the Generate button before you submit, and the actual per-call charge is recorded on the prediction afterwards.

What inputs does Lipsync 2 Pro accept?

Key inputs: `video`, `audio`, `sync_mode`. The full JSON schema (types, defaults, allowed values) is rendered above the Generate button and mirrored in the API reference at https://wavespeed.ai/docs/docs-api/sync/sync-lipsync-2-pro.

How long does Lipsync 2 Pro take to generate?

Average end-to-end generation time on WaveSpeedAI is around 230 seconds per request — measured across recent runs. Queue time scales with global demand; live status is visible in the prediction record.

Can I use Lipsync 2 Pro outputs commercially?

Commercial usage rights depend on the model's license, set by its provider (Sync). The license summary appears on the model card above; see WaveSpeedAI's Terms of Service for platform-level conditions.

EsempiVedi tutto

Modelli correlati

README