Sophon Realtime API

Sophon 是 AI降临派的全双工实时语音模型：它一直在听，自己判断你什么时候说完，边想边说，可以随时被打断。接口兼容 OpenAI Realtime / Chat Completions 协议，大多数情况下把 OpenAI 的地址和 key 换掉就能用。

Sophon is Adventists AI's full-duplex realtime voice model: it listens continuously, decides on its own when you have finished, speaks while it thinks, and can be interrupted at any time. The API follows the OpenAI Realtime / Chat Completions protocols, so in most cases you only change the base URL and key.

要用带人设、记忆、工具的角色，或接入硬件设备（TCP），请看 Harness 文档。

For characters with persona, memory and tools, or for hardware devices over TCP, see the Harness docs.

快速开始Quick start

1. 在 控制台 用邮箱注册，在「API Keys」页创建一把 key（只显示一次，请保存好）。
2. 选一个接入地址：国内 api.adventists.cn，海外 api.adventists.ai（两者共用账号、key 和余额）。
3. 按下面的示例连上 /sophon/v1/realtime，持续发送麦克风音频，播放返回的音频。

1. Sign up in the console and create a key on the "API keys" page (shown once — store it).
2. Pick an endpoint: api.adventists.ai (global) or api.adventists.cn (mainland China). Accounts, keys and balance are shared.
3. Connect to /sophon/v1/realtime as below, stream microphone audio continuously and play the audio you get back.

认证Authentication

Authorization: Bearer sk-sophon-xxxxxxxx

key 自带组织信息，不需要额外传 org。WebSocket 也用同样的请求头。浏览器里无法给 WebSocket 加请求头，可以用子协议 openai-insecure-api-key.<key>，但这会把 key 暴露给页面，只适合本地调试；正式产品请让你的服务端连接我们，或者等我们的短期令牌功能。已有企业账号（32 位 api-key + 6 位 org-id）的客户，用 api-key / org-id 两个请求头也可以。

Keys carry their organization; no org header is needed. WebSocket connections use the same header. Browsers cannot set WebSocket headers; the subprotocol openai-insecure-api-key.<key> works but exposes the key to the page, so use it only for local testing. In production, connect from your server (short-lived tokens are coming). Existing enterprise accounts (32-char api-key + 6-char org-id) can also send api-key / org-id headers.

模型Models

三者听、想的部分是同一个 32B 模型，只是"怎么发声"不同；接口完全一样，换 model 参数即可切换。语言：中文、英文完整支持；意、德、西、法、葡可以正常对话。模型听到什么语言就用什么语言回答。

All three share the same 32B listening/thinking model and differ only in how the reply becomes sound; the API is identical, switch with the model parameter. Languages: Chinese and English fully supported; Italian, German, Spanish, French and Portuguese are conversational. It answers in the language you speak.

内置音色（voice）：Built-in voices (voice): chelsie vivian_zh brand_zh brand_en ryan_en。也可以用几秒钟的录音克隆自己的音色（Tag / Flash），见 音色克隆。 You can also clone a voice from a few seconds of audio (Tag / Flash): see Voice cloning.

Realtime (WebSocket)

wss://api.adventists.ai/sophon/v1/realtime?model=sophon-realtime-tag

连上之后：发一次 session.update 配置会话，然后持续不断地发送麦克风音频（包括静音），不需要自己判断说话结束，也不需要发 response.create——模型自己决定什么时候接话。服务端以流式事件返回文字和音频。

After connecting, send one session.update, then stream microphone audio continuously (silence included). You do not detect end of speech and do not send response.create: the model decides when to reply. Text and audio come back as streaming events.

session.update

{"type": "session.update",
 "session": {
   "voice": "chelsie",
   "instructions": "你是小安，一家咖啡店的点单助手，说话简短热情。",
   "input_audio_format": "pcm16",
   "output_audio_format": "pcm16",
   "turn_detection": {"type": "server_vad"},
   "temperature": 0.8,
   "max_tokens": 300,
   "extra_body": {"language": "zh"}
 }}

客户端 → 服务端Client → server

服务端 → 客户端（常用）Server → client (main events)

还会有 response.listen、overlap.decision、rate_limits.updated 等内部状态事件，可以忽略。

You will also see internal state events such as response.listen, overlap.decision and rate_limits.updated; they can be ignored.

音频与打断Audio & barge-in

• 一直发音频。用户不说话时也要发（静音），模型靠连续的音频判断停顿、附和和打断。不要在客户端做 VAD 截断。
• 边收边播。收到 response.output_audio.delta 就放进播放队列；建议预留 100–200 ms 缓冲。
• 打断。用户说话时继续上传，模型会自己决定是否停下；收到 audio.cancelled 时清空播放队列。
• 回声。在免提设备上请开启回声消除（AEC），否则模型会听到自己的声音。

• Always stream. Keep sending audio (silence included); the model uses the continuous stream to judge pauses, backchannels and interruptions. Do not cut audio with a client-side VAD.
• Play as you receive. Queue each response.output_audio.delta; a 100–200 ms buffer is recommended.
• Barge-in. Keep uploading while the user talks; the model decides whether to stop. On audio.cancelled, clear the playback queue.
• Echo. On speakerphone devices enable acoustic echo cancellation, or the model hears itself.

完整示例（Python：发送一个 WAV 文件，保存回复音频）Full example (Python: send a WAV file, save the reply)

可下载的完整脚本：realtime_api.py（更多示例见 示例代码）。

Downloadable script: realtime_api.py (more in Example code).

# pip install "websockets>=14" numpy soundfile
import asyncio, base64, json, numpy as np, soundfile as sf, websockets

KEY = "sk-sophon-..."
URL = "wss://api.adventists.ai/sophon/v1/realtime?model=sophon-realtime-tag"

async def main():
    x, sr = sf.read("question.wav", dtype="int16")          # 16 kHz mono recommended
    x = np.concatenate([x, np.zeros(sr * 6, np.int16)])     # keep streaming silence after you finish
    out = []
    async with websockets.connect(URL, additional_headers={"Authorization": f"Bearer {KEY}"}, max_size=None) as ws:
        await ws.send(json.dumps({"type": "session.update", "session": {
            "voice": "chelsie", "input_audio_format": "pcm16", "output_audio_format": "pcm16",
            "turn_detection": {"type": "server_vad"}, "extra_body": {"language": "zh"}}}))

        async def receive():
            async for msg in ws:
                e = json.loads(msg)
                if e["type"] == "response.output_audio.delta":
                    out.append(np.frombuffer(base64.b64decode(e["delta"]), np.int16))
                elif e["type"] == "response.output_audio_transcript.delta":
                    print(e["delta"], end="", flush=True)
                elif e["type"] == "audio.cancelled":
                    out.clear()                              # barge-in: drop what is queued
                elif e["type"] == "error":
                    print("\nerror:", e["error"])
        rx = asyncio.create_task(receive())
        step = sr // 10                                      # 100 ms chunks, in real time
        for i in range(0, len(x), step):
            await ws.send(json.dumps({"type": "input_audio_buffer.append",
                                      "audio": base64.b64encode(x[i:i + step].tobytes()).decode(),
                                      "sample_rate_hz": sr}))
            await asyncio.sleep(0.1)
        rx.cancel()
    if out:
        sf.write("reply.wav", np.concatenate(out), 24000)

asyncio.run(main())

实时麦克风：把读文件换成麦克风回调（例如 sounddevice.InputStream(samplerate=16000, channels=1, dtype="int16", blocksize=1600)），把收到的音频写进 24 kHz 的播放流。

Live microphone: replace the file with a mic callback (e.g. sounddevice.InputStream(samplerate=16000, channels=1, dtype="int16", blocksize=1600)) and write received audio to a 24 kHz output stream.

Chat Completions

一问一答：传一段语音（或文字），返回文字和语音。适合离线处理、留言回复、测试。

Request / response: send audio (or text), get text and audio back. Good for offline processing, voice messages and testing.

POST /sophon/v1/chat/completions
{
  "model": "sophon-realtime-tag",
  "modalities": ["text", "audio"],
  "voice": "chelsie",
  "language": "zh",
  "messages": [
    {"role": "user", "content": [
      {"type": "input_audio", "input_audio": {"data": "<base64 WAV>", "format": "wav"}}
    ]}
  ]
}

• 非流式：choices[0].message.content 是文字，choices[0].message.audio.data 是 base64 WAV（24 kHz）。
• 流式（"stream": true）：SSE，每个 chunk 有 modality 字段："text" 时 delta.content 是文字，"audio" 时是一段 base64 WAV。
• 可以加 system 消息设定人设；不加则使用模型默认人设。纯文字输入也可以，只要 content 是字符串。

• Non-streaming: choices[0].message.content is the text, choices[0].message.audio.data a base64 WAV (24 kHz).
• Streaming ("stream": true): SSE chunks carry a modality field — "text": delta.content is text; "audio": a base64 WAV piece.
• Add a system message for a persona; otherwise the default persona is used. Plain text input works too (content as a string).

from openai import OpenAI
client = OpenAI(base_url="https://api.adventists.ai/sophon/v1", api_key="sk-sophon-...")
r = client.chat.completions.create(model="sophon-realtime-flash",
        messages=[{"role": "user", "content": "Introduce yourself in one sentence"}])
print(r.choices[0].message.content)

音色克隆Voice cloning

用 3–15 秒的录音克隆一个音色，约半分钟后就能在实时对话里用这个声音说话：不需要训练，也不用重新连接。克隆的音色只属于你的组织，其他账户看不到、也用不了。Tag、Flash、E2E 都支持克隆，Tag 和 E2E 最像原声，Flash 只能大致接近。也可以在 控制台 的「音色」页直接录音或上传。

Clone a voice from 3–15 s of audio and, about half a minute later, talk in that voice in real time: no training, no reconnecting. Cloned voices belong to your organization only; other accounts cannot see or use them. Tag, Flash and E2E all support cloning; Tag and E2E sound closest to the original, Flash is an approximation. You can also record or upload on the "Voices" page of the console.

录音要求Recording

• 只有一个人说话，3–15 秒（超过 12 秒的部分会截掉，首尾静音自动去掉）；安静环境，正常语速和音量。
• 格式：WAV、MP3、M4A、OGG / Opus、FLAC、WebM 都可以，文件不超过 10 MB。
• 语言不限：中文录音克隆出的声音也能说英文。

• One person speaking, 3–15 s (anything past 12 s is cut; leading and trailing silence is trimmed); a quiet room, normal pace and volume.
• Formats: WAV, MP3, M4A, OGG/Opus, FLAC or WebM, up to 10 MB.
• Any language: a voice cloned from Chinese speech can also speak English.

授权：只能克隆你本人的声音，或已获得说话人授权的声音；不得用于冒充他人、诈骗或任何违法用途。创建时必须传 consent=true，我们会记录授权时间和来源 IP。违规使用会被停用账户。

Consent: clone only your own voice or a voice whose speaker has given you permission; never use it to impersonate anyone, defraud or break the law. Creating a voice requires consent=true; we log the time and source IP of each consent. Misuse leads to account suspension.

接口Endpoints

curl https://api.adventists.ai/sophon/v1/voices \
  -H "Authorization: Bearer sk-sophon-..." \
  -F file=@my_voice.wav -F name="Support agent" -F consent=true
# {"id": "voice_3f9c2a1b7d4e", "object": "voice", "type": "custom", "name": "Support agent",
#  "status": "processing", "duration_s": 8.4, "models": ["sophon-realtime-flash", "sophon-realtime-tag"]}

使用Use it

和内置音色一样，把 voice 设成音色 ID：Realtime 写在 session.update 里（会话中途也可以再发一次换声音），Chat Completions 写在请求体的 voice 里。

Use the voice ID wherever a built-in voice goes: in session.update for Realtime (send it again mid-session to switch), or the voice field of a Chat Completions request.

{"type": "session.update", "session": {"voice": "voice_3f9c2a1b7d4e", "turn_detection": {"type": "server_vad"}}}

用量查询Usage

GET /sophon/v1/usage?from=2026-11-01&to=2026-11-30
GET /sophon/v1/models

返回按天、按模型汇总的会话数、分钟数和费用。逐个会话的明细和 CSV 导出在 控制台「用量」页。

Daily totals per model (sessions, minutes, cost). Per-session detail and CSV export are on the console "Usage" page.

错误码与限制Errors & limits

空闲 = 双方都没有声音（上传的是静音、模型也没在说话）。需要更高并发或更长会话，请联系 jiejin@adventists.ai。

Idle = no audio either way (silence uploaded and the model not speaking). For higher limits contact jiejin@adventists.ai.

计费Billing

额度按分钟计，按 Flash 计：1 分钟额度 = 用 Flash 聊 1 分钟。其他模型按下表的消耗倍数扣，例如倍数 1.5 的模型聊 10 分钟，计 15 分钟。按连接时长计费，精确到秒。注册送 3 小时免费对话（按 Flash 计）。剩余额度、每个会话的实际时长 / 倍数 / 计费分钟都在控制台查看。

Credit is counted in minutes (Flash): 1 minute of credit = 1 minute of chat with Flash. Other models use the rate in the table below, e.g. 10 minutes on a 1.5× model counts as 15 minutes. Billed by connection time to the second. New accounts get 3 hours of free voice chat (Flash). Remaining credit and each session's duration / rate / billed minutes are in the console.

倍数可能调整，以本表和控制台显示为准（本表从服务端实时读取）。

Rates may change; this table (loaded live from the server) and the console are authoritative.

即将推出：浏览器 / 端侧短期令牌、Live 接口（兼容 OpenAI GPT-Live，可委派工具调用）、在线充值。

Coming soon: short-lived tokens for browsers / devices, the Live API (OpenAI GPT-Live compatible, with tool delegation), online top-up.