Dia TTS
Dia directly generates realistic dialogue from transcripts. Audio conditioning enables emotion control. Produces natural nonverbals like laughter and throat clearing. will cost $0.04 per 1000 character.
Features
DIA-TTS is a 1.6 billion parameter TTS model that excels in producing lifelike dialogue from textual input. Unlike traditional TTS systems that often sound monotonous or robotic, DIA-TTS captures the nuances of human speech, including emotional tone and non-verbal cues. This capability is achieved through advanced audio conditioning, allowing users to guide the model's output by providing reference audio clips to influence the speaker’s emotional delivery and intonation.、
Key Features
- Emotion and Tone Control: DIA-TTS allows for audio conditioning, enabling precise control over the emotion and tone of the generated speech. Users can guide the model’s output by providing reference audio clips to influence the speaker’s emotional delivery and intonation .
- Nonverbal Communication Generation: The model can seamlessly interpret tags like (laughs), (coughs), and reproduce them as natural sounds, enhancing the realism of the synthesized speech .
- Multi-Speaker Dialogue Synthesis: DIA-TTS supports speaker tagging using [S1], [S2], etc., enabling the generation of multi-character conversations from text. This feature is particularly useful for applications like audiobooks and interactive storytelling .
- Open-Source and Accessible: DIA-TTS is available under an open-source Apache 2.0 license, with pretrained model checkpoints and inference code accessible via Hugging Face and GitHub, facilitating community-driven innovation and transparency.
ComfyUI
DIA-TTS image to video is available on ComfyUI, providing local inference capabilities through a node-based workflow, ensuring flexible and efficient image generation on your system.
Use Cases
- Virtual Assistants and Chatbots: Enhance user interactions by providing emotionally expressive and natural-sounding responses.
- Audiobook and Podcast Production: Generate engaging and dynamic narration with appropriate emotional tone and non-verbal cues.
- Interactive Storytelling and Gaming: Create immersive experiences by synthesizing multi-character dialogues with distinct voices and emotions.
- Accessibility Tools: Develop assistive technologies that offer more natural and expressive speech output for users with visual or reading impairments.
Accelerated Inference
Our accelerated inference approach leverages advanced optimization technology from WavespeedAI. This innovative fusion technique significantly reduces computational overhead and latency, enabling rapid image generation without compromising quality. The entire system is designed to efficiently handle large-scale inference tasks while ensuring that real-time applications achieve an optimal balance between speed and accuracy. For further details, please refer to the blog post.
Authentication
For authentication details, please refer to the Authentication Guide.
API Endpoints
Submit Task & Query Result
# Submit the task
curl --location --request POST "https://api.wavespeed.ai/api/v2/wavespeed-ai/dia-tts" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer ${WAVESPEED_API_KEY}" \
--data-raw '{
"prompt": "[S1] Dia is an open weights text to dialogue model. [S2] You get full control over scripts and voices. [S1] Wow. Amazing. (laughs) [S2] Try it now on WavesSpeed-AI."
}'
# Get the result
curl --location --request GET "https://api.wavespeed.ai/api/v2/requests/${requestId}/status" \
--header "Authorization: Bearer ${WAVESPEED_API_KEY}"
Parameters
Task Submission Parameters
Request Parameters
Parameter | Type | Required | Default | Range | Description |
---|---|---|---|---|---|
prompt | string | Yes | [S1] Dia is an open weights text to dialogue model. [S2] You get full control over scripts and voices. [S1] Wow. Amazing. (laughs) [S2] Try it now on WavesSpeed-AI. | - | The text to be converted to speech. |
Response Parameters
Parameter | Type | Description |
---|---|---|
code | integer | HTTP status code (e.g., 200 for success) |
message | string | Status message (e.g., “success”) |
data.id | string | Unique identifier for the prediction, Task Id |
data.model | string | Model ID used for the prediction |
data.outputs | array | Array of URLs to the generated content (empty when status is not completed ) |
data.urls | object | Object containing related API endpoints |
data.urls.get | string | URL to retrieve the prediction result |
data.has_nsfw_contents | array | Array of boolean values indicating NSFW detection for each output |
data.status | string | Status of the task: created , processing , completed , or failed |
data.created_at | string | ISO timestamp of when the request was created (e.g., “2023-04-01T12:34:56.789Z”) |
data.error | string | Error message (empty if no error occurred) |
data.timings | object | Object containing timing details |
data.timings.inference | integer | Inference time in milliseconds |
Result Query Parameters
Result Request Parameters
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
id | string | Yes | - | Task ID |
Result Response Parameters
Parameter | Type | Description |
---|---|---|
code | integer | HTTP status code (e.g., 200 for success) |
message | string | Status message (e.g., “success”) |
data | object | The prediction data object containing all details |
data.id | string | Unique identifier for the prediction |
data.model | string | Model ID used for the prediction |
data.outputs | array | Array of URLs to the generated content (empty when status is not completed ) |
data.urls | object | Object containing related API endpoints |
data.urls.get | string | URL to retrieve the prediction result |
data.has_nsfw_contents | array | Array of boolean values indicating NSFW detection for each output |
data.status | string | Status of the task: created , processing , completed , or failed |
data.created_at | string | ISO timestamp of when the request was created (e.g., “2023-04-01T12:34:56.789Z”) |
data.error | string | Error message (empty if no error occurred) |
data.timings | object | Object containing timing details |
data.timings.inference | integer | Inference time in milliseconds |