Lipsync Audio to Video | AI Digital Human API

Lipsync — Audio-to-Video

Lipsync Audio-to-Video is an advanced AI-powered lip synchronization model that matches video lip movements to any audio track. Upload a video and audio file, and the model automatically adjusts the speaker's lips to perfectly sync with the new audio — ideal for dubbing, translation, and content localization.

Why It Stands Out

Precise lip synchronization: AI analyzes audio and adjusts lip movements frame by frame for natural results.
Audio replacement: Sync any audio track to existing video footage seamlessly.
High-quality output: Produces realistic lip movements that blend naturally with the original video.
Simple workflow: Just upload audio and video — no manual editing required.
Flexible input: Supports various audio and video formats via upload or public URL.

Parameters

Parameter	Required	Description
audio	Yes	Audio file to sync (upload or public URL).
video	Yes	Video file with the face to be synced (upload or public URL).

How to Use

Upload your audio — drag and drop a file or paste a public URL.
Upload your video — provide the video containing the face to be lip-synced.
Click Run and wait for processing.
Preview and download the lip-synced video.

Best Use Cases

Video Dubbing — Replace original audio with dubbed versions in different languages.
Content Localization — Adapt videos for international audiences with synced translations.
Film & TV Production — Fix audio sync issues or replace dialogue in post-production.
Marketing & Advertising — Create localized ad content without reshooting.
E-learning — Produce multilingual training videos from a single source.
Social Media — Create fun lip-sync content for entertainment.

Pricing

Metric	Price
Per 5 seconds of audio	$0.15
Minimum charge	$0.15 (5 seconds)
Maximum billable duration	600 seconds (10 minutes)

Total cost = (audio duration in seconds / 5) × $0.15

Examples

5s audio → $0.15 (minimum)
10s audio → $0.30
30s audio → $0.90
60s audio → $1.80
5 min (300s) audio → $9.00
10 min (600s) audio → $18.00 (maximum)

Pro Tips for Best Quality

Use clear, high-quality audio for best lip-sync accuracy.
Ensure the face in the video is clearly visible and well-lit.
Front-facing videos with minimal head movement produce the best results.
Keep audio and video durations matched for optimal synchronization.
For best results, use videos where the speaker's mouth is clearly visible throughout.

Notes

Ensure uploaded audio and video URLs are publicly accessible.
Maximum billable duration is 600 seconds (10 minutes) per job.
Processing time varies based on video length and current queue load.
Please ensure your content complies with usage guidelines.

Lipsync Audio To Video API — Quick start

Grab a WaveSpeedAI API key, then call POST https://api.wavespeed.ai/api/v3/bytedance/lipsync/audio-to-video with your input as JSON. The endpoint returns a prediction id. Start polling the result endpoint around every 2 seconds, increase the interval for long-running tasks, and stop on any terminal status. On completed, read URLs from data.outputs. Examples for Lipsync Audio To Video below.

HTTP example

# Submit the prediction
curl --fail-with-body --connect-timeout 10 --max-time 60 \
  -X POST "https://api.wavespeed.ai/api/v3/bytedance/lipsync/audio-to-video" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $WAVESPEED_API_KEY" \
  -d '{
    "audio": "https://example.com/your-audio.mp3",
    "video": "https://example.com/your-input.mp4"
}'

# Wait at least 2 seconds, then poll. Safe GET requests may be retried.
curl --fail-with-body --connect-timeout 10 --max-time 30 \
  --retry 4 --retry-all-errors --retry-delay 1 \
  -X GET "https://api.wavespeed.ai/api/v3/predictions/{request_id}/result" \
  -H "Authorization: Bearer $WAVESPEED_API_KEY"

# Start at 2 seconds and increase the interval for long-running tasks.
# Stop on completed, failed, cancelled, or timeout.

Node.js example

// npm install wavespeed
const { Client } = require('wavespeed');

const apiKey = process.env.WAVESPEED_API_KEY;
if (!apiKey) throw new Error('Set WAVESPEED_API_KEY');
const client = new Client(apiKey);

try {
  const result = await client.run("bytedance/lipsync/audio-to-video", {
        "audio": "https://example.com/your-audio.mp3",
        "video": "https://example.com/your-input.mp4"
}, {
    timeout: 3600,
    pollInterval: 2.0,
  });
  console.log(result.outputs);
} catch (error) {
  console.error('Generation failed:', error);
  process.exitCode = 1;
}

Python example

# pip install wavespeed
import os
from wavespeed import Client

client = Client(api_key=os.environ["WAVESPEED_API_KEY"])

try:
    output = client.run(
        "bytedance/lipsync/audio-to-video",
        {
    "audio": "https://example.com/your-audio.mp3",
    "video": "https://example.com/your-input.mp4"
},
        timeout=3600.0,
        poll_interval=2.0,
    )
    print(output["outputs"])
except Exception as error:
    raise SystemExit(f"Generation failed: {error}") from error

Lipsync Audio To Video API — Frequently asked questions

What is the Lipsync Audio To Video API?

Lipsync Audio To Video is a ByteDance model for talking-avatar generation, exposed as a REST API on WaveSpeedAI. LipSync turns audio into lifelike talking videos by generating precise lip movements fully synced to input audio. Ready-to-use REST inference API, best performance, no coldstarts, affordable pricing. You can call it programmatically or try it from the playground above.

How do I call the Lipsync Audio To Video API?

POST your input parameters to the model's REST endpoint (shown in the API tab of this playground) with your WaveSpeedAI API key in the Authorization header. Submission returns a prediction ID. Poll the result endpoint starting around every 2 seconds, increase the interval for long-running tasks, and stop on any terminal status. The playground generates production-oriented Python, JavaScript, and cURL examples with timeouts, transient-error handling, and safe GET retries. Full request/response shape is documented at https://wavespeed.ai/docs/docs-api/bytedance/bytedance-lipsync-audio-to-video.

How much does Lipsync Audio To Video cost per run?

Lipsync Audio To Video starts at $0.15 per run. That figure is the base price — the final charge scales with the parameters you set in the form (output size, length, count, references, or whatever knobs this model exposes), so a higher-quality or larger output costs more than a minimal one. The exact cost for your current input is shown live next to the Generate button before you submit, and the actual per-call charge is recorded on the prediction afterwards.

What inputs does Lipsync Audio To Video accept?

Key inputs: `video`, `audio`. The full JSON schema (types, defaults, allowed values) is rendered above the Generate button and mirrored in the API reference at https://wavespeed.ai/docs/docs-api/bytedance/bytedance-lipsync-audio-to-video.

How long does Lipsync Audio To Video take to generate?

Average end-to-end generation time on WaveSpeedAI is around 193 seconds per request — measured across recent runs. Queue time scales with global demand; live status is visible in the prediction record.

Can I use Lipsync Audio To Video outputs commercially?

Commercial usage rights depend on the model's license, set by its provider (ByteDance). The license summary appears on the model card above; see WaveSpeedAI's Terms of Service for platform-level conditions.

示例查看全部

相關模型

README