Bytedance Latentsync

Bytedance Latentsync

Playground

LatentSync combines Stable Diffusion and TREPA for high-res end-to-end lip-sync, delivering precise, realistic mouth motions in generated videos. Ready-to-use REST inference API, best performance, no coldstarts, affordable pricing.

Features

LatentSync is an advanced AI-powered lip synchronization model that matches video lip movements to any audio track. Upload a video and audio file, and the model automatically adjusts the speaker’s lips to perfectly sync with the new audio — ideal for dubbing, translation, and content localization.

Why It Stands Out

• Precise lip synchronization: AI analyzes audio and adjusts lip movements frame by frame for natural results.
• Latent-based approach: Advanced latent space manipulation for high-quality, realistic output.
• Audio replacement: Sync any audio track to existing video footage seamlessly.
• High-quality output: Produces realistic lip movements that blend naturally with the original video.
• Simple workflow: Just upload audio and video — no manual editing required.

Parameters

How to Use

1. Upload your audio — drag and drop a file or paste a public URL.
2. Upload your video — provide the video containing the face to be lip-synced.
3. Click Run and wait for processing.
4. Preview and download the lip-synced video.

Best Use Cases

• Video Dubbing — Replace original audio with dubbed versions in different languages.
• Content Localization — Adapt videos for international audiences with synced translations.
• Film & TV Production — Fix audio sync issues or replace dialogue in post-production.
• Marketing & Advertising — Create localized ad content without reshooting.
• E-learning — Produce multilingual training videos from a single source.
• Social Media — Create fun lip-sync content for entertainment.

Pricing

Billing Rules

• Billed per 5 seconds of audio at $0.15
• Minimum charge: 5 seconds
• Maximum billable duration: 600 seconds (10 minutes)

Pro Tips for Best Quality

• Use clear, high-quality audio for best lip-sync accuracy.
• Ensure the face in the video is clearly visible and well-lit.
• Front-facing videos with minimal head movement produce the best results.
• Keep audio and video durations matched for optimal synchronization.
• For best results, use videos where the speaker’s mouth is clearly visible throughout.

Notes

• Ensure uploaded audio and video URLs are publicly accessible.
• Maximum billable duration is 600 seconds (10 minutes) per job.
• Processing time varies based on video length and current queue load.
• Please ensure your content complies with usage guidelines.

Authentication

For authentication details, please refer to the Authentication Guide.

API Endpoints

Submit Task & Query Result

set -euo pipefail

export WAVESPEED_API_KEY="your-api-key"

REQUEST_BODY=$(cat <<'JSON'
{
  "audio": "https://interactive-examples.mdn.mozilla.net/media/cc0-audio/t-rex-roar.mp3",
  "video": "https://interactive-examples.mdn.mozilla.net/media/cc0-videos/flower.mp4"
}
JSON
)

# 1. Submit the prediction.
SUBMIT_RESPONSE=$(curl --silent --show-error --fail-with-body \
  -X POST "https://api.wavespeed.ai/api/v3/bytedance/latentsync" \
  -H "Authorization: Bearer ${WAVESPEED_API_KEY}" \
  -H "Content-Type: application/json" \
  -d "${REQUEST_BODY}")

TASK=$(printf '%s' "${SUBMIT_RESPONSE}" | jq 'if type == "object" and has("data") then .data else . end')
PREDICTION_ID=$(printf '%s' "${TASK}" | jq -r '.id // empty')
if [ -z "${PREDICTION_ID}" ]; then
  printf 'Submission response did not contain a prediction id
' >&2
  exit 1
fi
RESULT_URL=$(printf '%s' "${TASK}" | jq -r '.urls.get // empty')
if [ -z "${RESULT_URL}" ]; then RESULT_URL="https://api.wavespeed.ai/api/v3/predictions/${PREDICTION_ID}/result"; fi

# 2. Poll until the prediction finishes.
while true; do
  RESPONSE=$(curl --silent --show-error --fail-with-body \
    "${RESULT_URL}" \
    -H "Authorization: Bearer ${WAVESPEED_API_KEY}")
  RESULT=$(printf '%s' "${RESPONSE}" | jq 'if type == "object" and has("data") then .data else . end')
  STATUS=$(printf '%s' "${RESULT}" | jq -r '.status // empty')

  case "${STATUS}" in
    completed) printf '%s\n' "${RESULT}" | jq '.outputs'; break ;;
    failed|cancelled|timeout) printf '%s\n' "${RESULT}" | jq . >&2; exit 1 ;;
    created|processing) sleep 2 ;;
    *) printf 'Unexpected status: %s
' "${STATUS}" >&2; exit 1 ;;
  esac
done

Parameters

Task Submission Parameters

Request Parameters

Response Parameters

Result Request Parameters

Result Response Parameters