Realtime Voice (WebSocket) - API易文档中心

Overview

Realtime models run over a long-lived WebSocket connection: audio streams in, audio streams out, and the model can be interrupted mid-sentence — no “record, upload, wait, play back” cycle. The difference from stitching ASR + a text model + TTS is that this is end-to-end: the model hears tone, pauses and emotion directly, and speaks directly. Latency lands in the sub-second range. APIYI currently offers 4 models across 2 protocols, sharing one endpoint and one key:

gpt-realtime-2.1 / gpt-realtime-2.1-mini — OpenAI Realtime GA protocol
qwen3.5-omni-plus-realtime / qwen3.5-omni-flash-realtime — Alibaba Cloud Model Studio protocol

Status: private beta / integration in progress. Realtime voice is in limited supply and not yet open for self-service — you need to contact us to be enabled. Upstream protocol and behavior may still change during the beta; everything in the “Known Limitations” section below is measured, and will be updated as upstream changes. Please do not ship to production without a fallback path. If you plan to integrate, or need higher concurrency, reach us via WeCom support or email [email protected] / [email protected].

🎤 Highlights: bidirectional streaming audio over one connection, barge-in at any time, server_vad and semantic_vad turn detection, full function-calling round trip (including result injection), image input, and usage broken out per modality. All of the above verified on all four models (2026-08-24, UTC+8).

One thing to remember first: the 4 models use two different request protocols, with different field names and different event names. Changing only the model parameter without changing the request body will not work — this is by far the most common integration failure. The differences amount to 6 fields and 3 event names, all listed under “Protocol Comparison” below.

Request Beta Access

Contact WeCom support with your account and expected concurrency, and we will enable the beta group on your key.

API Manual

Key creation, base URL, billing modes and other general conventions.

Keys and Groups

Create keys, select groups and set quotas.

Call Logs

Inspect token usage and actual charges per call in the console.

This page is long. Three sections are required reading: Protocol Comparison (read before switching models), Start With Text (verify the whole chain without a microphone), and Known Limitations (four measured differences that affect client code).

Let an AI Agent Do the Integration

If you develop with Codex / Claude Code / Cursor, copy the prompt below to it. It will first fetch the plain-text version of this page (append .md to any docs URL), then write code for your stack — the two field families, the sample-rate red line, cancel semantics and idle disconnects are all baked into the requirements.

Have a coding agent integrate or troubleshoot Realtime voice. Copy and paste into Codex, Claude Code, Cursor and similar tools.

What this prompt saves you from

Requirement	Pitfall it avoids
Identify the protocol family first	Changing only `model` lets the handshake pass but `session.update` is then rejected
Sample rate pinned per family	Sending 16 kHz to Realtime GA fails with `integer_below_min_value`
Output-modality field was renamed	Writing `modalities` on the GA protocol is simply an unknown field
Event names changed too	Listening for `response.text.delta` on the GA protocol never fires
Close out on `output_item.done`	Model Studio omits `response.done` after cancel; waiting for it hangs the turn
Keepalive needed, ping does not count	Assuming a heartbeat prevents the 300-second idle disconnect
Voice pinned in the first frame	Changing it after audio has been produced fails with `cannot_update_voice`
Backend relay, no direct browser connection	Connecting from the browser hands the key to every visitor

Why APIYI for Realtime Voice

One key, four models

Same wss endpoint, same auth. Switching models means changing the model parameter and the matching field template — no second vendor account to maintain.

Direct access, no overseas setup

Reach api.apiyi.com from mainland data centers, home broadband or overseas nodes. No upstream vendor account, identity verification or prepayment required.

Protocol differences already mapped

Field comparison, event-name comparison, sample-rate limits and four measured limitations are all documented here so you do not have to rediscover them.

Zero-cost self-test over text

Verify handshake, auth, fields, tool wiring and concurrency without a microphone — audio tiers cost an order of magnitude more than text, so this is real money saved during integration.

Measured latency and concurrency

Handshake p50 0.65–1.08 s and first text delta p50 0.54–0.95 s at 20 concurrent sessions, with the test conditions and date stated under Technical Specs.

Direct support during the beta

Beta users get a direct WeCom channel for integration questions, concurrency increases and upstream behavior changes.

Core Capabilities

Bidirectional streaming, interruptible

Audio streams out as it is generated; the client can send response.cancel at any time. The session survives and context is preserved. Verified on all four models.

Two turn-detection modes

server_vad splits on silence duration, semantic_vad splits on intent (better at ignoring filler words like “uh-huh”). Both verified on all four models.

Full function-calling loop

The model triggers a tool, the client executes it, function_call_output injects the result, and the model continues speaking. Verified end to end on all four models.

Image input, per-modality usage

Send images mid-session for the model to read; usage returns text / audio / image tokens separately so cost can be attributed. Verified on all four models.

Supported Models

Model	Protocol family	Default voice	Input sample rate	Prompt caching	Positioning
`gpt-realtime-2.1`	Realtime GA	`marin`	≥ 24 kHz	✅ Supported	Flagship; strongest multilingual and reasoning
`gpt-realtime-2.1-mini`	Realtime GA	`marin`	≥ 24 kHz	✅ Supported	Cost-effective; ample for everyday dialogue
`qwen3.5-omni-plus-realtime`	Model Studio	`Tina`	16 kHz	⏸ Not observed	Flagship for Chinese-language scenarios
`qwen3.5-omni-flash-realtime`	Model Studio	`Tina`	16 kHz	⏸ Not observed	Cost-effective for Chinese-language scenarios

Output audio is PCM signed 16-bit / mono / 24 kHz on all four models.

The two protocol families share only the endpoint and the auth scheme. Request fields and server event names both differ. When you switch models you must switch field templates too — see “Protocol Comparison” below.

Pricing

Pricing in one sentence: billed per token, and audio costs an order of magnitude more than text (for gpt-realtime-2.1, audio input $32 vs text input $4; audio output $64 vs text output $24). Run text-only during integration and switch to audio once the chain is verified — see “Start With Text” below.

The tables below are the vendors’ official list prices, in USD per 1M tokens. Actual charges on APIYI are whatever the call logs show; the recharge bonus lowers the effective cost further.

Realtime GA protocol

Model	Text input	Text output	Cached read	Image input	Audio input	Audio output
`gpt-realtime-2.1`	$4	$24	$0.4	$5	$32	$64
`gpt-realtime-2.1-mini`	$0.6	$2.4	$0.06	$0.8	$10	$30

Model Studio protocol

The billing dimensions differ: image input is folded into the text tier, and output is split into “text only” and “text + audio” (only the audio portion is charged at the latter rate).

Model	Text / image input	Audio input	Text output	Text + audio output
`qwen3.5-omni-plus-realtime`	$1.38	$11	$8.25	$41.26
`qwen3.5-omni-flash-realtime`	$0.45	$3.71	$2.75	$14.71

Beta note: Realtime voice is in limited supply and billing is still being aligned with upstream. If your actual charges deviate noticeably from the tables above, please contact support so we can reconcile. Pricing may change with vendor policy and supply. This capability is offered to secure supply and serve customers, not as a profit-driven listing.

Access Group

How to get enabled during the beta: self-service group selection is not available yet; access is granted on request. Contact WeCom support with your account, use case and expected concurrency, and we will enable the beta group on your key and share the current caveats. General availability will be announced in the changelog; no key or code changes will be needed then.

Technical Specs

Dimension	Value
Transport	WebSocket (`wss://`), bidirectional streaming over one connection
Auth	`Authorization: Bearer <key>` header
Event format	Compatible with the OpenAI Realtime event model (client events / server events)
Input audio	PCM signed 16-bit, mono, Base64. Model Studio 16 kHz; Realtime GA ≥ 24 kHz
Output audio	PCM signed 16-bit, mono, 24 kHz
Output modalities	Text / audio (text-only is allowed)
Turn detection	`server_vad`, `semantic_vad`, or disabled for manual `commit`
Function calling	Supported, including `function_call_output` result injection
Image input	Supported (syntax differs per family, see below)
Session lifetime	Realtime GA: session carries `expires_at`, measured at roughly 30 minutes from connect. Model Studio: measured idle disconnect at 300 seconds

Measured latency and concurrency

Measured 2026-08-24 (UTC+8) over the public api.apiyi.com path, 20 concurrent sessions × 2 models, single-turn text-only exchange:

Metric	Measured
WebSocket handshake	p50 0.65–1.08 s
First text delta	p50 0.54–0.95 s
Full single turn	p50 < 1 s
Session success rate	99.6% (one handshake failure, recovered on reconnect)

These are point-in-time measurements at a specific concurrency level and are not a performance commitment. No availability SLA is offered during the beta — implement reconnection and graceful degradation on the client.

Endpoint

Endpoint	Purpose	Auth
`wss://api.apiyi.com/v1/realtime?model=<model-name>`	Open a realtime voice session	`Authorization: Bearer <key>`

All four models share this endpoint; the model query parameter selects which one you reach.

On connecting from the browser: this endpoint also accepts auth via the Sec-WebSocket-Protocol subprotocol (realtime, openai-insecure-api-key.<key>, openai-beta.realtime-v1), so a browser WebSocket can connect directly — but that hands your key to the browser, where any visitor can read it from the network panel. Use it for local verification only. In production, write a backend relay: the backend holds the key and opens the connection to APIYI, while the frontend talks only to your own service.

⚠️ Protocol Comparison (read before switching models)

The two families share the endpoint, the auth scheme and the overall event flow. The differences are concentrated in the session.update field structure and a few server event names.

Request field comparison

Purpose	Model Studio protocol	Realtime GA protocol
Output modality	`modalities: ["text","audio"]`	`output_modalities: ["audio"]`
Voice	`voice` (top level)	`audio.output.voice`
Speed	Not supported	`audio.output.speed` (0.7 / 1.0 / 1.5 measured as linear)
Input audio format	`input_audio_format: "pcm"`, 16 kHz	`audio.input.format: {"type":"audio/pcm","rate":24000}`
Output audio format	`output_audio_format: "pcm"`	`audio.output.format: {"type":"audio/pcm","rate":24000}`
Turn detection	`turn_detection` (top level)	`audio.input.turn_detection`
Input transcription	`input_audio_transcription`	`audio.input.transcription`

Server event comparison

Content	Model Studio protocol	Realtime GA protocol
Text delta	`response.text.delta`	`response.output_text.delta`
Audio delta	`response.audio.delta`	`response.output_audio.delta`
Audio transcript delta	`response.audio_transcript.delta`	`response.output_audio_transcript.delta`

All other events — session.created, session.updated, conversation.item.create, input_audio_buffer.append, input_audio_buffer.commit, response.create, response.cancel, response.done — are named identically in both.

Two minimal session.update payloads

The same thing written twice; copy directly. Model Studio protocol:

{
  "type": "session.update",
  "session": {
    "modalities": ["text", "audio"],
    "voice": "Ethan",
    "input_audio_format": "pcm",
    "output_audio_format": "pcm",
    "input_audio_transcription": { "model": "qwen3-asr-flash-realtime" },
    "turn_detection": { "type": "semantic_vad" }
  }
}

Realtime GA protocol:

{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "output_modalities": ["audio"],
    "audio": {
      "input": {
        "format": { "type": "audio/pcm", "rate": 24000 },
        "transcription": { "model": "whisper-1" },
        "turn_detection": { "type": "semantic_vad" }
      },
      "output": {
        "format": { "type": "audio/pcm", "rate": 24000 },
        "voice": "alloy",
        "speed": 1.0
      }
    }
  }
}

Sample rate is a hard constraint: audio.input.format.rate on the Realtime GA protocol must be ≥ 24000; sending 16000 fails immediately with integer_below_min_value: Expected a value >= 24000. The Model Studio protocol requires 16 kHz input. Resample on the client.

Start With Text: What the Text Channel Is For, and a Three-Step Self-Test

An audio pipeline involves microphone capture, resampling, chunking and turn detection. Any broken link shows up as “nothing happens”, which is hard to diagnose. So do not start with a microphone.

Text is the control plane, not a fallback input

In a realtime voice model, text is not “another way to send input” — it is the entire control channel other than the audio stream:

Combination	Typical use
Text → control plane	`instructions` system prompts, `function_call_output` tool results, retrieved context — all text, none reachable via microphone
Text → audio	Effectively a TTS with full conversational context: let a regular model think it through, then have the realtime model say it. Good for announcements and prompts
Text → text	The cheapest debugging channel. Plain text chat is better served by a regular chat model; its value here is verifying the chain at no cost

Three-step self-test

Step 1: text only, no microphone

Set output_modalities to text only, disable turn detection, and send one input_text. That alone verifies the handshake, the key and group, whether you picked the right field template, whether session.update took effect, whether tools inject correctly, whether multi-turn context holds, and how concurrency behaves. No audio tokens are produced at all.

Step 2: replay a local wav file

Use a fixed local audio file instead of a microphone, feeding it into input_audio_buffer.append in 100 ms chunks. This decouples the audio pipeline (format, sample rate, chunking, commit, VAD triggering) from your business logic and makes it repeatable — the same file should produce the same result twice.

Step 3: connect the live microphone

Once the first two steps pass, only capture and playback remain. If something breaks now, the search space is already small.

No test audio handy? On macOS, built-in tools generate a compliant file in one line:

say -v Samantha -o /tmp/ask.aiff "What is the weather in Beijing today? Answer in one sentence."

# Realtime GA protocol uses 24000
afconvert -f WAVE -d LEI16@24000 -c 1 /tmp/ask.aiff ask_24k.wav
# Model Studio protocol uses 16000
afconvert -f WAVE -d LEI16@16000 -c 1 /tmp/ask.aiff ask_16k.wav

Picking the wrong sample rate is the most common failure in step 2 — the two protocols differ, do not mix them up.

A runnable text smoke test

Depends only on websockets (pip install websockets). Flip one variable to switch protocols:

import asyncio, json, os, websockets

FAMILY = "ga"           # ga = gpt-realtime-2.1 series; omni = qwen3.5-omni series
MODEL = "gpt-realtime-2.1" if FAMILY == "ga" else "qwen3.5-omni-plus-realtime"
URL = f"wss://api.apiyi.com/v1/realtime?model={MODEL}"
HEADERS = {"Authorization": "Bearer " + os.environ["APIYI_API_KEY"]}

# The two protocols diverge only here: session structure and the text-delta event name.
SESSION = ({"type": "realtime", "output_modalities": ["text"],
            "audio": {"input": {"turn_detection": None}}}
           if FAMILY == "ga" else
           {"modalities": ["text"], "turn_detection": None})
TEXT_DELTA = "response.output_text.delta" if FAMILY == "ga" else "response.text.delta"

async def main():
    async with websockets.connect(URL, additional_headers=HEADERS) as ws:
        while json.loads(await ws.recv())["type"] != "session.created":
            pass
        await ws.send(json.dumps({"type": "session.update", "session": SESSION}))
        await ws.send(json.dumps({"type": "conversation.item.create", "item": {
            "type": "message", "role": "user",
            "content": [{"type": "input_text", "text": "Explain WebSocket in one sentence."}]}}))
        await ws.send(json.dumps({"type": "response.create"}))
        while True:
            e = json.loads(await ws.recv())
            if e["type"] == TEXT_DELTA:
                print(e["delta"], end="", flush=True)
            elif e["type"] == "response.done":
                print("\n\nusage =", json.dumps(e["response"]["usage"]))
                return
            elif e["type"] == "error":
                print("\nERROR:", json.dumps(e))
                return

asyncio.run(main())

If this runs, the endpoint, key, group and field template are all correct — move on to step 2.

Session Features: Voice, Turn Detection, Tools, Images

Voice

Item	Model Studio protocol	Realtime GA protocol
Default voice	`Tina`	`marin`
Verified working	`Tina`, `Ethan` and others	`alloy`, `marin`, `cedar`, `shimmer`, `verse`
Speed control	Not supported	`audio.output.speed`; duration scales linearly at 0.7 / 1.0 / 1.5
Invalid voice	`Voice 'xxx' is not supported.`	`invalid_value`

Pin the voice in the first session.update. Once a session has produced audio output, changing the voice fails with cannot_update_voice — this applies to both protocols. Open a new session to switch voices. Also, on the Model Studio protocol do not send an empty string as the voice; it falls back to an unsupported voice and returns 400. Simply omit the field if you do not need to set it.

Turn detection: server_vad and semantic_vad

server_vad — splits on silence duration, with straightforward parameters (threshold, silence_duration_ms, prefix_padding_ms).
semantic_vad — splits on conversational intent, ignoring filler words and meaningless background noise. More robust in multi-speaker environments.
You can also disable turn detection (null or none) and run in manual mode: send input_audio_buffer.commit yourself, then response.create. This suits push-to-talk interfaces where the UI controls turns.

In VAD mode you must keep streaming. After the speech ends, keep pushing a short stretch of silence (2 seconds is enough in testing) so the server can detect end-of-speech. If you push only the voiced part and then stop, speech_stopped never fires and no response is generated.

Function calling

Event order: the model emits a response.output_item.done of type function_call (carrying call_id and arguments) → the client executes it → the result is injected → another response.create lets the model continue.

{
  "type": "conversation.item.create",
  "item": {
    "type": "function_call_output",
    "call_id": "call_xxx",
    "output": "{\"city\":\"Beijing\",\"weather\":\"light rain\",\"temp_c\":21}"
  }
}

The full loop is verified on all four models — after injection the model correctly restates what the tool returned.

Image input

Realtime GA protocol: put input_image directly into the message; the value can be a data URI.

{
  "type": "conversation.item.create",
  "item": {
    "type": "message",
    "role": "user",
    "content": [
      { "type": "input_image", "image_url": "data:image/jpeg;base64,..." },
      { "type": "input_text", "text": "What does the image say?" }
    ]
  }
}

Model Studio protocol: images are treated as video frames, so audio must be appended first, otherwise you get Error append image before append audio.. In testing, the working approach is to interleave input_image_buffer.append into the input_audio_buffer.append stream at roughly one frame per second.

Known Limitations (beta)

All four items below are measured, and all of them affect client code. Read this before integrating.

Behavior	Affected family	Client workaround
`response.done` is not delivered after `response.cancel`, so the turn never closes out	Model Studio (reproduced in all 6 test runs)	Use `response.output_item.done` as the end-of-turn signal, plus a 5-second timeout. The session itself is unaffected and conversation continues normally after the interruption
Idle connections dropped at 300 seconds; WebSocket ping/pong does not count as activity	Model Studio	Send an application-level event periodically while idle, or accept the disconnect and reconnect automatically; replay `session.update` after reconnecting
Voice cannot be changed once the session has produced audio; fails with `cannot_update_voice`	Both families	Pin `voice` in the first `session.update`; open a new session to switch
Input-transcription completion event is not delivered in manual `commit` mode	The `flash` model on Model Studio	Switch to `server_vad` / `semantic_vad`, where transcription works normally. The conversation itself is unaffected — the model understands and answers the audio correctly

These behaviors may change as upstream evolves during the beta; this page will be kept current. If you hit something not listed here, please report it via WeCom support or [email protected], including the timestamp and session.id so we can trace it.

Best Practices

Pick the field template by protocol family first

Write the two session.update payloads as two config constants selected by model name, rather than scattering if-branches. This is the part most likely to break during maintenance six months later.

Pin session parameters in the first frame

Set output_modalities, voice, speed, turn_detection and transcription in the very first session.update. Voice especially — once audio has been produced it is too late.

Pass a text smoke test before adding audio

Run the text smoke test on this page to confirm endpoint, key, group and field template are all correct, then move to audio. Audio tiers cost an order of magnitude more than text, so this saves most of your integration budget.

Convert sample rate and channels on the client

PCM signed 16-bit, mono; 16 kHz for Model Studio, ≥ 24 kHz for Realtime GA. Do not expect server-side correction — a wrong format usually shows up as silence rather than an explicit error.

Close out on output_item.done with a timeout

Do not wait on response.done alone. This approach is correct on both families and avoids hanging a turn when the user interrupts.

Add keepalive and reconnect for long sessions

Watch the 300-second idle limit on Model Studio and expires_at on Realtime GA. After reconnecting, replay session.update and any required context, otherwise the new session runs with defaults.

Use a backend relay in production

Keep the key on the backend and have the frontend talk only to your own service. Direct browser connections work technically but expose the key.

Errors and Retries

Symptom	Meaning	What to do
Handshake returns 401	Invalid key, or the `Authorization` header was not sent	Check the key itself, whether you used `https://` by mistake, and whether the header is present
Handshake returns 503 with no available channel	The key is not enabled for the beta group, or the model name is wrong	Contact support to confirm the group is enabled; verify the `model` parameter
Handshake returns 400	The `model` query parameter is missing	The endpoint must include `?model=<model-name>`
`invalid_request_error` on an unknown field	Wrong protocol family	Switch to the field template for that model using the comparison tables above
`integer_below_min_value`	Input sample rate below 24000 on Realtime GA	Resample to 24 kHz or higher on the client
`cannot_update_voice`	Voice changed after the session produced audio	Pin the voice in the first frame; open a new session to switch
`Error append image before append audio.`	An image was appended before any audio on Model Studio	Append audio via `input_audio_buffer.append` first, then image frames
WebSocket 1006 / 1011	Network instability or an upstream disconnect	Reconnect with exponential backoff (1 s / 4 s / 16 s) and replay `session.update`
Connection drops after roughly 5 minutes of silence	The Model Studio idle limit	See the keepalive approach under Known Limitations

Troubleshooting tip: record the event_id of each event and the session.id of the session, and include them when reporting an issue — it shortens diagnosis considerably. Note also that Realtime GA error objects carry code and param (naming the exact field and its accepted values), while Model Studio error messages are coarser. Validate field syntax on the former first when debugging.

FAQ

Why is there no interactive playground on this page?

Interactive playgrounds are driven by OpenAPI specs, which describe a single request and a single response over HTTP. Realtime is dozens of event types flowing in both directions over one long-lived connection, which does not map onto that model. The alternative is the text smoke test in the “Start With Text” section — a few dozen lines, no microphone, and it confirms the chain works.

Can I swap between the four models by changing only the model name?

No. The endpoint and auth are the same, but request fields and event names belong to two protocols. At minimum you must change: modalities ↔ output_modalities, voice ↔ audio.output.voice, input_audio_format ↔ audio.input.format, turn_detection ↔ audio.input.turn_detection, input_audio_transcription ↔ audio.input.transcription, plus the event names response.text.delta ↔ response.output_text.delta and response.audio.delta ↔ response.output_audio.delta. See the Protocol Comparison section for the full mapping.

The handshake fails outright. How do I debug it?

Check five things in order: 1. the scheme is wss://, not https://; 2. the endpoint includes ?model=<model-name>; 3. the Authorization: Bearer <key> header is present; 4. the key is enabled for the beta group (503 with “no available channel” if not); 5. no reverse proxy in between is stripping the Upgrade header — this is a common issue when relaying through your own gateway.

Can I connect from the browser? Will my key leak?

Technically yes — the endpoint accepts auth via the Sec-WebSocket-Protocol subprotocol, so a browser WebSocket can connect directly. But that hands your key to the browser, where any visitor can read it from the network panel, so it is only suitable for local verification. In production write a backend relay: the backend holds the key and opens the connection to APIYI, and the frontend talks only to your own service.

Sending 16 kHz audio to gpt-realtime-2.1 fails. Why?

The Realtime GA protocol requires an input sample rate of at least 24000; 16000 returns integer_below_min_value. The correct form is "audio": {"input": {"format": {"type": "audio/pcm", "rate": 24000}}}. The two Model Studio models require 16 kHz instead — the two are not interchangeable.

I have no microphone / audio is hard to test. What now?

Follow the three-step self-test in the “Start With Text” section: verify the chain over text first (no audio tokens produced), then replay a local wav file to verify the audio pipeline, and only then connect a live microphone. Test audio can be generated in one line with the macOS built-ins say and afconvert — the commands are in that section.

After sending response.cancel I never receive response.done.

This is a known behavior of the two Model Studio models (reproduced in all 6 test runs): after an interruption you receive response.text.done, response.content_part.done and response.output_item.done, but response.done is not delivered. Use response.output_item.done as the end-of-turn signal and add a timeout as a backstop. The session itself is unaffected and the conversation continues normally. The two Realtime GA models behave correctly here.

My connection drops after about 5 minutes.

The Model Studio protocol drops connections after 300 seconds of inactivity, and WebSocket-level ping/pong does not count as activity — a heartbeat will not extend that timer. Either send an application-level event periodically while idle (a session.update, for instance), or accept the disconnect and reconnect automatically. Remember to replay session.update and any required context after reconnecting.

How long can a single session stay open?

On the Realtime GA protocol the session.created event carries expires_at, measured at roughly 30 minutes from connect, after which you need to reconnect. On the Model Studio protocol the constraint we mainly observed is the 300-second idle disconnect. Design long conversations on the assumption that sessions expire, and plan how context carries across sessions.

How do I set the voice, and why does changing it return cannot_update_voice?

Set the voice in session.update: top-level voice on Model Studio, audio.output.voice on Realtime GA. Once the session has produced audio output the voice can no longer be changed — this applies to both protocols and returns cannot_update_voice. Pin it in the first frame and open a new session to switch. Also, do not send an empty string as the voice on Model Studio; it returns 400.

I get no input transcription in manual commit mode.

The flash model on Model Studio does not deliver the transcription completion event in manual commit mode (reproduced consistently across runs); the plus model does, and both work in VAD mode. Switch to server_vad or semantic_vad. Testing shows the transcript text lands in an undocumented field on the delta events in this case, but that field may change at any time and should not be relied upon. Note this only affects displaying what the user said in your UI — the conversation is unaffected, and the model understands and answers the audio correctly.

Is image input supported? Why do I get Error append image before append audio.?

All four models support image input, but the syntax differs. On Realtime GA you place input_image directly in the message. On Model Studio images are treated as video frames, so audio must be appended before any image, which is what triggers that error. In testing the working approach is to interleave image frames into the audio stream at roughly one frame per second.

Is there prompt caching? How do I confirm a hit?

The two Realtime GA models support it and it applies automatically — in testing the second turn within a session already hit, with a value in usage.input_token_details.cached_tokens. No cache hits were observed on the two Model Studio models.

How do I estimate cost? Are text and audio billed separately?

The usage object on response.done reports tokens per modality (text / audio / image, separately for input and output), so cost can be attributed. Audio tiers are substantially higher than text, which is why text-only is recommended during integration. For actual charges, refer to the call logs.

API Manual

Key creation, base URL, billing modes and other general conventions.

Keys and Groups

Create keys, select groups and set quotas.

Text Generation

Regular chat models — a better fit for text-only conversation.

Model Pricing

Live pricing, endpoints and groups for every model on the platform.

Recharge Bonus

Lowers your effective cost further.

Request Beta Access

Contact WeCom support with your account and expected concurrency.

Realtime voice is currently in private beta. Every measured result on this page is dated 2026-08-24 (UTC+8) and will be updated as upstream changes. If you plan to integrate, hit something this page does not cover, or need higher concurrency, reach us at [email protected] / [email protected].

​Overview

Request Beta Access

API Manual

Keys and Groups

Call Logs

​Let an AI Agent Do the Integration

​Why APIYI for Realtime Voice

One key, four models

Direct access, no overseas setup

Protocol differences already mapped

Zero-cost self-test over text

Measured latency and concurrency

Direct support during the beta

​Core Capabilities

Bidirectional streaming, interruptible

Two turn-detection modes

Full function-calling loop

Image input, per-modality usage

​Supported Models

​Pricing

​Realtime GA protocol

​Model Studio protocol

​Access Group

​Technical Specs

​Measured latency and concurrency

​Endpoint

​⚠️ Protocol Comparison (read before switching models)

​Request field comparison

​Server event comparison

​Two minimal session.update payloads

​Start With Text: What the Text Channel Is For, and a Three-Step Self-Test

​Text is the control plane, not a fallback input

​Three-step self-test

​A runnable text smoke test

​Session Features: Voice, Turn Detection, Tools, Images

​Voice

​Turn detection: server_vad and semantic_vad

​Function calling

​Image input

​Known Limitations (beta)

​Best Practices

​Errors and Retries

​FAQ

​Related Documentation

API Manual

Keys and Groups

Text Generation

Model Pricing

Recharge Bonus

Request Beta Access

Overview

Let an AI Agent Do the Integration

Why APIYI for Realtime Voice

Core Capabilities

Supported Models

Pricing

Realtime GA protocol

Model Studio protocol

Access Group

Technical Specs

Measured latency and concurrency

Endpoint

⚠️ Protocol Comparison (read before switching models)

Request field comparison

Server event comparison

Two minimal session.update payloads

Start With Text: What the Text Channel Is For, and a Three-Step Self-Test

Text is the control plane, not a fallback input

Three-step self-test

A runnable text smoke test

Session Features: Voice, Turn Detection, Tools, Images

Voice

Turn detection: server_vad and semantic_vad

Function calling

Image input

Known Limitations (beta)

Best Practices

Errors and Retries

FAQ

Related Documentation