StepAudio 3 Explained: Which Audio Model Fits Your App?
StepAudio 3 explained for builders choosing among real-time voice, recognition, speech generation, and music without assuming every model has an API.
The brief says “audio model.” By sprint planning, that means a live agent, transcription, narration, and music. Routing all four to “StepAudio 3” hides different latency, rights, and recovery risks behind one name. I would stop integration there. This guide maps each task to a verified model line and a usable access path.
What the StepAudio 3 Name Covers
Separate the Realtime, Recognition, and Generation Lines
StepAudio 3 is a product family, not one interchangeable checkpoint.
| Task | Current model ID | Public integration surface |
|---|---|---|
| Full-duplex voice agent | stepaudio-3-realtime-preview | Persistent WebSocket |
| Turn-based audio understanding | stepaudio-3-chat-preview | Chat Completions; audio or text in, text out |
| Transcription | stepaudio-3-asr-max | HTTP request with SSE text output |
| Text-to-speech | stepaudio-3-tts | HTTP or streaming WebSocket |
| Mixed speech, effects, ambience, and music | stepaudio-3-gen-preview | Audio-generation request |
| Song-focused generation | stepaudio-3-music-preview | Asynchronous music task |
The Realtime model guide separates full-duplex Realtime from Chat. Text after an uploaded turn is not simultaneous listening, speaking, interruption handling, and tool use.
Distinguish a Model, Demo, Paper, and Public API
A model page names a capability; an API reference defines a contract. A studio or Hugging Face Space is not evidence that your account receives the same endpoint, quota, or weights. The StepAudio 3 Gen technical report supports research claims, not production latency or availability.
Do not switch models yet. Look at the workflow first.
Match Each Model Line to One Audio Task
Real-Time Conversation and Voice Agents
Choose Realtime for continuous listening, spoken responses, interruption, and in-call tools. Its preview route uses WebSocket with built-in transcription, VAD, and conversation state. Choose Chat for discrete audio turns and text replies. Test barge-in, first-audio latency, reconnection, and tool recovery on your network.
Speech Recognition and Transcription
Use ASR Max for incremental and final text. The ASR documentation lists POST /v1/audio/asr/sse and Chinese, English, mixed-language, dialect, noise, and singing scenarios. Treat these as a test matrix. Score your accents, domain terms, silence errors, partial-result churn, and final accuracy.
Speech, Sound, and Music Generation
Use StepAudio 3 TTS for narration. Use StepAudio 3 Gen to combine dialogue, designed voices, effects, ambience, singing, and music. Use StepAudio 3 Music when lyrics, reference audio, vocals, or song structure lead. Voice design is not persistent cloning; reference audio is not proof of rights.
Compare the Access Paths Builders Can Actually Use
Public API and Account Access
The StepAudio 3 API is several contracts, not one call. Confirm the model ID in your account: preview IDs may be retired when paid versions arrive. Global documentation uses .ai API hosts; do not blindly mix them with .com examples. In the public pages I checked, I did not find a complete country eligibility matrix, so verify dashboard access.
The pricing and rate-limit page lists ASR Max at $0.24 per audio hour and TTS at $0.36 per 10,000 characters. Realtime, Chat, Gen, and Music previews are free for a limited time, not permanently. Limits depend on account tier.
Hosted Demos, Research Releases, and Downloadable Assets
Voice Studio, showcases, and the official Music Space are evaluation surfaces, not proof of an endpoint, SLA, or self-hosting license. I found demos and research material, but no public checkpoint download supporting a local-deployment promise. Record the model ID, portal, date, and job metadata.
Choose a Starting Point for Your App
Start With the Required Input and Output
Write the contract in one line: microphone to spoken response, recording to transcript, script to narration, or lyrics and reference to song. Choose Realtime, ASR Max, TTS, or Music accordingly; use Gen for mixed scenes. Compose services when needed instead of assuming the family provides every control.
Check Latency, Control, Cost, and Failure Recovery
Run a production-shaped sample. Capture first-byte latency, completion time, accuracy, retries, policy failures, and review minutes. Force a streaming disconnect; verify restart behavior. For generation, measure cost per approved asset. Keep source, request, model ID, response IDs, and decision together.
Limits and Open Questions
Do Not Treat Reported Benchmarks as API Guarantees
Benchmarks may use internal settings, selected data, or unavailable controls. Re-run owned samples with one rubric, account tier, and network. A leaderboard cannot replace an app-level failure budget.
Keep StepAudio 2.5 and StepAudio 3 Names Separate
Some FAQ and cloning examples still reference 2.5 while pricing lists cloning for the newer TTS model. Never substitute IDs or assume identical schemas. Pin what your account accepts and regression-test migration.
FAQ
Does StepAudio 3 support speaker diarization for multi-speaker audio?
Not in the public contract I verified. ASR Max returns text and Realtime manages turns; neither documents speaker labels. Do not infer diarization from VAD.
Can StepAudio 3 outputs be used in commercial products?
There is no blanket yes from a model page. Review the StepFun Open Platform User Agreement, service rules, and preview terms; secure voice consent and content rights. This is general information, not legal advice. Send client delivery through legal or compliance review.
Which StepAudio 3 models support custom or cloned voices?
Pricing lists voice cloning for stepaudio-3-tts; Gen documents voice design. They are different. Realtime also discusses cloned IDs through an older TTS flow, so confirm the accepted creation model and schema before depending on identity continuity.
Does StepAudio 3 return word-level timestamps for recognized speech?
Not in the public ASR Max SSE contract I verified. It documents text, not word timing. For subtitle alignment, inspect a live response or obtain written support confirmation.
Does StepAudio 3 add watermarks or provenance metadata to audio?
No public page I checked specifies an audio watermark or machine-readable provenance field. That is not proof of absence. Store job IDs, inputs, consent, model version, and an export manifest; confirm local disclosure duties.
Conclusion
Choose by contract: Realtime for agents, ASR Max for transcription, TTS for narration, Gen for mixed scenes, and Music for songs. Verify account access, then test the failure that would break the app. A platform does not need to be everything. It needs not to break the critical path.
Previous posts:
