Multitalk

Playground

MultiTalk is an audio-driven conversational AI video generation model. Create talking or singing videos from a single image and audio input. Our endpoint starts with $0.15 per 5 seconds video generation and supports a maximum generation length of 60 seconds.

Features

MultiTalk

What is MultiTalk?

MultiTalk is a revolutionary framework for audio-driven multi-person conversational video generation developed by MeiGen-AI. Unlike traditional talking head generation methods that only animate facial movements, MultiTalk technology can generate realistic videos of people speaking, singing, and interacting while maintaining perfect lip synchronization with audio input. MultiTalk transforms static photos into dynamic speaking videos by making the person speak or sing exactly what you want them to say.

Pricing

Our endpoint starts with $0.15 per 5 seconds video generation and supports a maximum generation length of 60 seconds.

How MultiTalk Works

MultiTalk leverages advanced AI technology to understand both audio signals and visual information. This MultiTalk implementation combines MultiTalk + Wan2.1 + Uni3C for optimal results.

Audio Analysis: MultiTalk uses a powerful audio encoder (Wav2Vec) to understand the nuances of speech, including rhythm, tone, and pronunciation patterns.

Visual Understanding: Built on the robust Wan2.1 video diffusion model (you can visit our Wan2.1 workflow for t2v/i2v eneration), MultiTalk understands human anatomy, facial expressions, and body movements.

Camera Control: MultiTalk with Uni3C controlnet enables subtle camera movements and scene control, making the video more dynamic and professional-looking. Check out our Uni3C workflow for creating beautiful camera motion transfer.

Perfect Synchronization: Through sophisticated attention mechanisms, MultiTalk learns to perfectly align lip movements with audio while maintaining natural facial expressions and body language.

Instruction Following: Unlike simpler methods, MultiTalk can follow text prompts to control the scene, pose, and overall behavior while maintaining audio synchronization.

Authentication

For authentication details, please refer to the Authentication Guide.

API Endpoints

Submit Task & Query Result


# Submit the task
curl --location --request POST "https://api.wavespeed.ai/api/v3/wavespeed-ai/multitalk" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer ${WAVESPEED_API_KEY}" \
--data-raw '{
    "prompt": "",
    "seed": -1
}'

# Get the result
curl --location --request GET "https://api.wavespeed.ai/api/v3/predictions/${requestId}/result" \
--header "Authorization: Bearer ${WAVESPEED_API_KEY}"

Parameters

Task Submission Parameters

Request Parameters

Parameter	Type	Required	Default	Range	Description
image	string	Yes		-	The image for generating the output.
audio	string	Yes	-	-	The audio for generating the output.
prompt	string	No		-	The prompt for generating the output.
seed	integer	No	-1	-1 ~ 2147483647	The random seed to use for the generation. -1 means a random seed will be used.

Response Parameters

Parameter	Type	Description
code	integer	HTTP status code (e.g., 200 for success)
message	string	Status message (e.g., “success”)
data.id	string	Unique identifier for the prediction, Task Id
data.model	string	Model ID used for the prediction
data.outputs	array	Array of URLs to the generated content (empty when status is not `completed`)
data.urls	object	Object containing related API endpoints
data.urls.get	string	URL to retrieve the prediction result
data.has_nsfw_contents	array	Array of boolean values indicating NSFW detection for each output
data.status	string	Status of the task: `created`, `processing`, `completed`, or `failed`
data.created_at	string	ISO timestamp of when the request was created (e.g., “2023-04-01T12:34:56.789Z”)
data.error	string	Error message (empty if no error occurred)
data.timings	object	Object containing timing details
data.timings.inference	integer	Inference time in milliseconds

Result Query Parameters

Result Request Parameters

Parameter	Type	Required	Default	Description
id	string	Yes	-	Task ID

Result Response Parameters

Parameter	Type	Description
code	integer	HTTP status code (e.g., 200 for success)
message	string	Status message (e.g., “success”)
data	object	The prediction data object containing all details
data.id	string	Unique identifier for the prediction, the ID of the prediction to get
data.model	string	Model ID used for the prediction
data.outputs	array	Array of URLs to the generated content (empty when status is not `completed`)
data.urls	object	Object containing related API endpoints
data.urls.get	string	URL to retrieve the prediction result
data.has_nsfw_contents	array	Array of boolean values indicating NSFW detection for each output
data.status	string	Status of the task: `created`, `processing`, `completed`, or `failed`
data.created_at	string	ISO timestamp of when the request was created (e.g., “2023-04-01T12:34:56.789Z”)
data.error	string	Error message (empty if no error occurred)
data.timings	object	Object containing timing details
data.timings.inference	integer	Inference time in milliseconds

Mmaudio V2 Pika V2.0 Turbo I2v