Real-time
Live video & world models
Some video models are not “submit a prompt, get a clip.” An interactive world model
holds a GPU for you while you steer it — walk around, change the scene, give it instructions — and streams
the result live. /v1/live-video/sessions opens one of those sessions and bills it
per second the GPU is held.
VideoRouter is the session broker, not the video pipe. It checks your key and credits, asks the host for a short-lived credential, and returns it to you. Your browser or app then connects directly to the host with the host’s own SDK — the video never passes through our servers, and the host API key never reaches your browser. Two hosts keep us in a thin control loop so that what you pay matches what actually ran: for Decart Lucy the SDK’s signaling socket goes through us, and for Reactor you tell us which session your browser created (see How Decart and Reactor are metered). We meter the session and settle it when it ends. Prefer a client library? There are Python and JavaScript SDKs — see the SDK overview.
boson/higgs-avatar-live needs a LiveKit Cloud project (the avatar joins your room), and vidu/vidu-s2-avatar-component needs your own RTC channel (Alibaba, Tencent, Agora or Volcano Engine). runway/characters and heygen/liveavatar also run on LiveKit, but the host runs the room (and, for LiveAvatar, the whole conversation) and you just join it with the credentials we return.
connect object changed and its SDK now needs realtimeBaseUrl; Reactor sessions should be reported to us once connected. In return you are billed the seconds the host actually ran, and ending a session now stops it on the host. What changed, and how to update: Migrating.
GET /v1/live-video/models lists only models you can book, each with its price, and a model we cannot price yet is not listed and cannot be started.
Happy Oyster is served by Reactor and billed per second the session is ready, including the 30–120 s it takes to build a world inside the session — there is no separate world fee, so budget roughly build time plus travel time (adventure travel is at most 2 min, directing 3 min). A few models also carry a minimum billed time (min_billed_seconds).
Quote a session before you book with POST /v1/pricing/estimate.
The browser snippet below has not been run end to end against a live session yet.
1. Open a session (your server)
from videorouter_live import LiveVideo
lv = LiveVideo(api_key="llmr_sk_live_...") # or set VIDEOROUTER_API_KEY
# On YOUR server: open a session. The block heartbeats in the background and settles on exit.
# The world is built inside the session by your browser (see step 2); that time is billed too.
with lv.session("reactor/happy-oyster-adventure", max_duration_seconds=300) as s:
# {"jwt": "...", "base_url": "...", "model": "...", ...} - returned once; hand it to the browser
send_to_browser(s.connect)
wait_for_user_to_finish()
# Settled on exit, even if your code raised.
print(s.status, s.billed_seconds, s.cost_usd, s.end_reason) # ended 41 ... client
connect is returned once, in the response that created (or first saw ready) the session. It is a
short-lived credential and is deliberately not stored or re-served — if you lose it, end the session and open a new one.
2. Stream it (your browser)
Hand connect to the host’s SDK. For Happy Oyster that is
Reactor’s JavaScript SDK.
Each host has its own SDK and its own control vocabulary (movement, instructions, prompts, pause/rewind) — we do not wrap them.
import { Reactor } from "@reactor-team/js-sdk";
// `connect` is what your server got back from VideoRouter (connect.jwt is locked to this one model). Never ship your llmr_sk_ key.
const reactor = new Reactor({ modelName: connect.model, jwt: connect.jwt });
await reactor.connect();
// Tell VideoRouter which Reactor session this is, so ending the booking ends exactly that session and you pay
// Reactor's own record of it (POST connect.host_session_path with a live-video token).
// Then build or attach a world and steer it with Reactor's Happy Oyster commands:
// createWorld() / attachWorld(), then move / look / interact (adventure) or instruct / pause / resume / rewind (directing).
// https://docs.reactor.inc/model-api-reference/happy-oyster/overview
Lifecycle & billing
waitingThe host is building a world (Happy Oyster: roughly 30–120 s). Nothing is metered. Poll GET until it flips.readyA host session exists and you have credentials. The meter runs every second until the session ends, including idle seconds.endedSettled. billed_seconds, cost_usd and end_reason are final.- Hold, then settle. Opening a session holds
max_duration_seconds × price_per_secondagainst your credits (402if you can’t cover it). Ending it debits the real cost and releases the rest of the hold. - What you’re billed. Whole seconds, the larger of the host’s own reported duration and our measured ready→end time, never more than your cap on our side. Never the client’s claim. Vidu is the exception: its meter starts only when your browser’s control connection is acknowledged (not while the session is being set up) and is exact, so for Vidu we bill the host’s own figure — a session you opened but never started streaming costs $0.
- Heartbeat. Call
POST .../heartbeatabout every 5 s. For Happy Oyster the host key lives with us, so the beat is forwarded to the host; for Odyssey, Vidu, Boson and Reactor it is purely our liveness signal (Vidu’s own control connection is the browser’s business; Boson’s media runs in your LiveKit room). Decart needs none while its stream is connected — the relayed socket is the signal — but a session you booked and never connected to is released after about 40 s. If beats stop, we settle the session at the host’s own timeout — a forgotten tab costs at most the silence window, not the whole cap. - Hard cap. Vidu has no duration setting of its own, so for Vidu we hang the live up at your cap, from our servers, even while your browser is still connected. For Decart, we hold the signaling socket: ending the session (or closing the stream) stops Decart’s generation within about a second, and what you pay is the seconds Decart reports — a session that never streams is free. Every session ends at
max_duration_seconds(default and ceiling are per model, below; Decart needs at least 10 s). Passhost_travel_idto/endto settle on the host’s authoritative duration and to get a replay. - Reactor: tell us your session. Your browser creates the Reactor session itself with
connect.jwt, and Reactor keeps a session running (and billing) after its client disappears. Right after you connect, sendPOST /v1/live-video/sessions/{id}/host-sessionwith{"session_id": reactor.getSessionId()}(Python:reactor.session_id). Ending the session — or the cap, or silence — then ends that Reactor session from our side, and you pay Reactor’s own record of how long it ran. A booking you never connect is free. Connect within about 60 s of booking (connect.connect_within_seconds). If you run several sessions of one model at once and do not report them, we cannot tell them apart: none is ended early and each is billed its full booked duration. - A session that never became ready costs $0 and releases its whole hold — except that a world build you started still owes its world fee (the world is built and its
world_idis yours to reuse); a build that fails is free. - Replays. Where the host composes a replay video,
recording_urlis populated on the ended session and mirrored to our storage.
How Decart and Reactor are metered
Decart and Reactor hand your browser a host credential and then run the session on their own, so a plain “booked → ended” clock cannot tell a stream that ran from one that failed to connect, or stop one you walked away from. For these two hosts VideoRouter therefore follows the host’s own record of the session. The video still goes straight between you and the host.
Decart Lucy: signaling through VideoRouter
connect for the Lucy models is { api_key, base_url, model, max_session_duration_seconds }. api_key is a token for this session only — it is not a Decart credential — and base_url is VideoRouter’s signaling URL (wss://videorouter.sh/api/v1/live-video/decart). Give both to Decart’s realtime SDK; it opens its control socket with us, and we hold the real one to Decart. Camera and result video flow directly between you and Decart as before.
// JavaScript (@decartai/sdk)
const client = createDecartClient({ apiKey: connect.api_key, realtimeBaseUrl: connect.base_url });
# Python (decart)
client = DecartClient(api_key=connect["api_key"], realtime_base_url=connect["base_url"])
- You pay the seconds Decart reports. Decart sends its own meter on that socket (generation started, a tick every 5 s, generation ended). We bill those seconds, rounded up to a whole second and never past your cap. A session that never started generating — including one Decart refused — costs $0.
- Ending it stops it.
POST .../end, reaching the cap, or a lost client closes our side of the socket, and Decart stops generating within about a second. Closing your stream does the same. - The booked tier is the billed tier.
options.speedis fixed when you book; astandardsession cannot be run asfast, and the model cannot be swapped. - Connect within about 40 s of booking. A session nobody connects to is released at $0. There is no heartbeat to send while the stream is connected.
- One stream per session, no reconnect. If the socket drops, the session has ended; book a new one. A second connection to the same session is refused.
- Close codes on the signaling socket:
4401missing or invalidapi_key,4404no such session,4409the session has ended or already has a stream attached,4410the session reached its maximum duration,4502we could not reach Decart. Decart’s own messages (prompt acknowledgements, errors) pass through unchanged, except that a problem on our Decart account is reported to you only as “this model is temporarily unavailable”. - Booking is refused with
502 host_unavailable(nothing held, nothing billed) when we cannot currently start Lucy sessions.
Lucy on a second host: plain WebRTC signaling
fal/lucy-2.6-webrtc and fal/lucy-2.5-webrtc are the same Decart models at the same rate, served by a second host. Use them when you want a vendor-neutral integration (no Decart SDK) or a second path for the same model. The integration is different, so they are their own model ids rather than a silent fallback behind decart/lucy-*: connect is { url, protocol, model, max_session_duration_seconds }, and url is a WebSocket for this session only (a session token, never a host credential). You do the WebRTC offer/answer over it with JSON text frames; the video itself goes straight between your browser and the host.
const ws = new WebSocket(connect.url);
const pc = new RTCPeerConnection({ iceServers: [{ urls: "stun:stun.l.google.com:19302" }] });
const cam = await navigator.mediaDevices.getUserMedia({ video: { width: 1280, height: 704, frameRate: 25 } });
cam.getTracks().forEach((t) => pc.addTrack(t, cam));
pc.ontrack = (e) => { outputVideo.srcObject = e.streams[0]; };
pc.onicecandidate = (e) => e.candidate && ws.send(JSON.stringify({ type: "icecandidate", candidate: {
candidate: e.candidate.candidate, sdpMid: e.candidate.sdpMid, sdpMLineIndex: e.candidate.sdpMLineIndex } }));
ws.onopen = () => ws.send(JSON.stringify({ prompt: "Anime style" })); // first frame = the input; {} is valid
ws.onmessage = async ({ data }) => {
const m = JSON.parse(data);
if (m.type === "ready") {
await pc.setLocalDescription(await pc.createOffer());
ws.send(JSON.stringify({ type: "offer", sdp: pc.localDescription.sdp }));
} else if (m.type === "answer") await pc.setRemoteDescription({ type: "answer", sdp: m.sdp });
else if (m.type === "icecandidate" && m.candidate) await pc.addIceCandidate(m.candidate);
else if (m.type === "error") console.error(m.error);
};
ws.send(JSON.stringify({ prompt: "Red leather jacket" })); // change what it renders while live
- Input frame. Any JSON object with
prompt,reference_image_url(adata:URI, at least 512×512, for character swap or virtual try-on) andenable_prompt_expansion; all optional. Send one first, then again whenever you want to change the edit. - You pay the seconds the host reports. The host counts every 2 s once the answer is applied; we bill those seconds plus at most one more interval, rounded up and never past your cap. A session that never negotiated costs $0.
- Ending it stops it.
ws.close(),POST .../end, reaching the cap, or a lost client closes our side of the host socket, which stops generation and billing at once. The open socket is the heartbeat. One stream per session, no reconnect. - Close codes as for the Decart signaling socket:
4401bad token,4404no such session,4409ended or already attached,4410maximum duration reached,4502we could not reach the host.
Reactor: tell us your session
Your browser creates the Reactor session itself with connect.jwt. Reactor keeps a session running — and billing — after its client disappears, so we end it for you from our side. To know which session is yours, report it once connected:
// browser
import { Reactor } from "@reactor-team/js-sdk";
const reactor = new Reactor({ modelName: connect.model, jwt: connect.jwt });
await reactor.connect(); // connect within connect.connect_within_seconds (60 s) of booking
// tell VideoRouter which Reactor session this is (from your server, or from the browser with a live-video token)
await fetch(`https://videorouter.sh/api/v1/live-video/sessions/${session.id}/host-session`, {
method: "POST",
headers: { Authorization: `Bearer ${token}`, "Content-Type": "application/json" },
body: JSON.stringify({ session_id: reactor.getSessionId() }),
});
// ... when you are done
await reactor.disconnect(); // ends the Reactor session; then POST .../end to settle
The Python and JavaScript SDKs wrap the report call: session.report_host_session(reactor.session_id) / await session.reportHostSession(reactor.getSessionId()).
- You pay Reactor’s own record of how long the session ran (from its creation to its close, so a second or two of start-up is included), at most your cap. A booking whose browser never connected is $0.
- Ending it stops it.
POST .../end, the cap, or silence ends that Reactor session from our servers even if your browser is gone or still connected. - Report the session id. It is how we match your session to this booking. If you do not, we pick the one session of that model created since you booked. If you run several sessions of one model at the same time and do not report them, we cannot tell them apart: none is ended early, and each is billed its full booked duration. Reporting removes the ambiguity.
- Keep sending the heartbeat. There is no signaling channel for us to watch, so
POST .../heartbeatabout every 5 s is the only sign that your session is alive; if it stops, we end the session after the silence window (about 40 s) and bill what Reactor ran. The SDKs do this for you. - Connect within 60 s of booking (
connect.connect_within_seconds). The token is short-lived on purpose, and a browser that connects after you have ended the booking is disconnected. - Booking is refused with
502 host_unavailable(nothing held, nothing billed) when we cannot currently start Reactor sessions.
POST /v1/live-video/sessions/{id}/host-session answers 400 invalid_host_session when the id is not a session of this booking (wrong model, created before you booked, unknown, or already used by another booking), and 409 host_session_already_set if a different session is already attached. See the API reference.
Migrating (changed 2026-11-14)
connect.api_key was a Decart client token, base_url was Decart’s API, plus token and expires_at. Billed from booking to /end.api_key is a session token and base_url is VideoRouter’s signaling URL; token and expires_at are gone. Pass realtimeBaseUrl: connect.base_url (JS) or realtime_base_url (Python) to Decart’s SDK. Billed on Decart’s own seconds. No reconnect./end; a session could keep running after you ended it.Token valid 60 s. Report your session with host-session. Ending a booking ends the Reactor session; billed on Reactor’s record.waiting session cost nothing, even though the world was built.A world build you started still owes its world fee; a build that fails is free. Reusing a world_id pays no fee.Options
options accepts exactly these keys, per host (Decart takes speed, Vidu, Boson, Runway, Nuance, LiveAvatar, Tavus and Anam take session_params only, Odyssey takes none — its prompt and image go to the SDK in the browser); anything else is a 400.
worldObject. Not used by the Happy Oyster rows now offered (Reactor builds the world inside your session). Kept for hosts that build a world before a session: build a new world on the host first (prompt, perspective, first_frame_image_url, … — passed through to the host). The session starts waiting. Cannot be combined with the two below.world_idString. Reuse a world you built earlier; the session is ready immediately.session_paramsObject. Forwarded verbatim to the host’s session call on hosts that take one. Vidu: required — the avatar takes avatar (persona and image_uri required; voice, greeting_instruction, farewell_enabled, persona_enhance optional), call_mode (video or audio), voice_model, audio, vad, llm, idle_motion and idle_timeout_seconds; editing takes image_url (required) and editing_type; the component avatar takes image_uri (required) and rtc_info (see Component avatar). Images are http(s) URLs or data:image/ URIs. Boson Higgs Avatar: required — transport (your LiveKit room, required; see Higgs Avatar), avatar_id (a builtin_* preset, default builtin_claire_v1) and output (480x640 default, or 640x480). Runway Characters: optional — avatar (one of Runway’s preset avatars: game-character, music-superstar, game-character-man, cat-character, influencer, tennis-coach, human-resource, fashion-designer, cooking-teacher; default music-superstar), personality (how it behaves, up to 10,000 characters) and start_script (its opening line, up to 2,000 characters); custom avatars are not available. HeyGen LiveAvatar: optional — prompt (the persona, up to 10,000 characters), opening_text, language, avatar_id and voice_id (public LiveAvatar ids only), quality and llm_model (see HeyGen LiveAvatar below). Saved personas: for LiveAvatar, Tavus, Anam and Runway, persona_id loads a persona saved with POST /v1/live-avatar/personas; keys you name in the session win over it (see Live avatars). Anam Cara: optional — prompt (up to 20,000 characters), opening_text, language, avatar_id (an Anam stock avatar or one of your own), voice_id (a stock voice) and llm_id (see Live avatars). The response's connect holds a session_token that Anam's JavaScript SDK starts the call with (valid for 2 minutes); Anam ends the call at your cap and meters only while the persona streams. Tavus CVI: optional — prompt (the system prompt, up to 20,000 characters), greeting, languages (a list of codes such as ['en', 'es']), face_id (a Tavus stock face; default Luna) and audio_only. The response's connect holds conversation_url and meeting_token for a Daily client, and room_url for a page Tavus hosts; Tavus starts its meter when the room is created and closes one nobody joins after 90 s. Nuance Live: optional — persona_id (sam default, sonia or mira), system_prompt (who the persona is, up to 64,000 characters; English only), expressiveness (0–2, default 1), timezone (IANA, so the persona knows the local time) end_user_id (your id for the person; hashed before it reaches the host), green_screen (the persona over solid #00FF00) and interface_setup — Nuance’s own session settings, forwarded as minted: delegation (client, responses with your tools, or none), video.input.camera and runtime_tools (stay_quiet, end_call, call_time). The response’s connect holds the LiveKit room (livekit_url, token); join it with livekit-client, publish the microphone, and exchange Nuance Live events on the nuance_live data topic (the JavaScript SDK and Python SDK do this for you). Mint warnings (tools the session will run without) are in connect.warnings. Any other key is a 400.speedstandard (default) or fast. Decart Lucy 2.5 / VTON only — fast bills at 2× the standard rate. The tier is fixed when you book: VideoRouter pins it on the signaling socket, so a standard session cannot be run as fast.Prompts in options.world and a Vidu avatar’s persona and greeting go through the same content moderation as video and image requests. Live steering you send straight to the host afterwards is between you and the host — we cannot see it.
Component avatar (bring your own RTC)
vidu/vidu-s2-avatar-component is for teams that already run their own real-time channel (Alibaba, Tencent, Agora or Volcano Engine RTC) and their own speech pipeline.
You stream the character’s speech and text to us; Vidu renders the character and pushes the video into your channel. Nothing else about the session differs: same hold, same settle, same limits.
- Open it with
options.session_params:image_uri(required, http(s) ordata:image/) and optionallyrtc_info—provider(artc,trtc,agora,volcengine) andtokenare required,app_id,channel_id,user_idoptional strings. Withoutrtc_infoVidu hosts a channel andconnect.managed_rtccarries the credentials to join it and watch. Your join token is forwarded to Vidu and never stored in our logs. - Stream to
wss://<host>/api/v1/live-video/sessions/{id}/stream?conn_id=<random>(the path is inconnect.stream_path). Servers authenticate withAuthorization: Bearer; a browser, which cannot set headers, passes a short-lived live-video token as?token=— a long-lived API key in a URL is refused. - Frames. Text frames are JSON signals, binary frames are raw audio. First send
{"type":1,"seq_id":1,"payload":{"conn_init":{"version":1}}}and wait forconn_init_ack.success; onNOT_READYjust send it again a couple of seconds later (do not reconnect). Then stream PCM 16-bit signed little-endian, 24 kHz, mono (48 bytes per millisecond; 20 ms or 100 ms chunks suit it), plustype 9(what the user said) andtype 10(what your model replied) as{"payload":{"text_msg":{"msg_id","content","timestamp"}}},type 7to interrupt,type 5to hang up. We forward every frame verbatim in both directions.
1000Normal: you hung up, the session was ended over HTTP, or the host ended it. The session is settled.4410Your max_duration_seconds was reached. We hung the host up and settled the session.4409The session is not open, already has a stream attached, or the host refused the stream. Nothing was settled; you may retry.4401 / 4403 / 4404 / 4400Bad credentials / missing live_video scope / no such session (or another organization’s) / the model has no stream.4502We could not reach the host. Nothing was settled; you may retry.- Why it is relayed. Vidu’s own data socket admits one connection, so whoever held it could never be cut at the session cap by anyone else. The relay holds that connection for you, which is what makes the cap, the hold and the settle real. Only audio and text cross it; the video goes straight to your RTC channel.
- One stream per session, and detaching ends it. Closing the stream hangs the host up immediately and settles the session; to continue, open a new session. The stream itself counts as the heartbeat, so you do not call
POST .../heartbeat. - Billing is the host’s own meter, as for the other Vidu models: 1 credit per second from the host’s acknowledgement of
conn_initto the hang-up, with a short warm-up that is not billed.
Models
odyssey/odyssey-2-proWorld · steer in browser (pricing soon)150 s720pdecart/lucy-2.5Real-time edit of a live stream600 s720pfal/lucy-2.6-webrtcReal-time edit · plain WebRTC signaling600 s720pfal/lucy-2.5-webrtcReal-time edit · plain WebRTC signaling600 s720pdecart/lucy-2.6Real-time edit of a live stream600 s720pdecart/lucy-vton-3.5Real-time edit of a live stream600 s720pdecart/lucy-vton-3.6Real-time edit of a live stream600 s720pdecart/lucy-restyle-2Real-time edit of a live stream600 s704pdecart/oasis-3-previewWorld · server-driven (not bookable yet)600 s768x512reactor/heliosReal-time video generation1800 s768preactor/longlive-v2Real-time video generation1800 s704preactor/sana-streamingReal-time edit of a live stream1800 s704preactor/x2Real-time edit of a live stream1800 s832preactor/xmax-x2-1Real-time edit of a live stream1800 s832preactor/happy-oyster-adventureWorld · steer in browser900 s720preactor/happy-oyster-directorWorld · steer in browser900 s480p / 720preactor/lingbotWorld · steer in browser1800 s960preactor/lingbot-world-2World · steer in browser1800 s960preactor/visko-orbis-stableReal-time video generation900 s480preactor/visko-orbis-dynamicReal-time video generation900 s480preactor/fast-h3-omniReal-time video generation900 s768preactor/fast-h3Real-time video generation900 s768preactor/h3-reference-turboReal-time video generation900 s768preactor/ltx2Real-time digital character · voice chat300 s352prunway/charactersReal-time digital character · voice chat1800 s1088x704vidu/vidu-s2-avatarReal-time digital character · voice chat1800 s720pvidu/vidu-s2-avatar-componentReal-time digital character · your own RTC channel1800 s720pheygen/liveavatarConversational avatar · join over LiveKit120 s720ptavus/cviConversational video persona · join over Daily30 min720panam/caraConversational persona · Anam JS SDK3 min720pvidu/vidu-s2-editingReal-time edit of a live camera stream1800 s720pboson/higgs-avatar-liveReal-time avatar · joins your LiveKit room3600 s480x640 / 640x480nuance/nuance-liveReal-time AI persona · video call over LiveKit (pricing soon)1800 s720pLooking for a face you can talk to? See Live avatars for how to choose between HeyGen LiveAvatar, Tavus, Runway and Vidu, with an AI-interviewer recipe.
HeyGen LiveAvatar: a conversational avatar that runs the whole call
heygen/liveavatar is a managed agent: speech recognition, the language model, the voice and the lip-synced avatar all run on our side, so there is nothing to host. You give it a persona and a language, we open the session and hand you LiveKit room credentials, and your browser joins with any LiveKit client SDK. It suits an AI interviewer, tutor or support agent.
POST /v1/live-video/sessions
{
"model": "heygen/liveavatar",
"max_duration_seconds": 120,
"options": { "session_params": {
"prompt": "You are a friendly technical interviewer for a backend engineering role. Ask one question at a time.",
"opening_text": "Hi, thanks for joining. Shall we start with a quick introduction?",
"language": "en", "quality": "high", "llm_model": "gpt-4o-mini"
} }
}
// -> connect: { protocol: "livekit", server_url: "wss://...livekit.cloud", token: "..." }
session_params keys, all optional: prompt (the persona, up to 10,000 characters; without one the avatar would never answer, so a short friendly default is used), opening_text (the first thing it says), language, avatar_id and voice_id (public LiveAvatar ids only), quality (very_high 1080p, high 720p, medium 480p, low 360p) and llm_model (gpt-4o-mini, gpt-4o, gpt-5-nano, gpt-5.4-nano). Prompts are read by the host’s model, so write them as you would a system prompt.
Billing. Per second the session is live, from LiveAvatar’s published rate of 2 credits a minute. A session nobody ever joined costs the host nothing, and we bill the larger of the host’s own duration and our observed time, as for the other avatar hosts. Limits. Sessions are capped at 120 s for now; this grows with our plan with the host.
Higgs Avatar in your LiveKit room
boson/higgs-avatar-live puts a talking-head avatar into a LiveKit Cloud room you own. The avatar renders the face and lip-sync; you supply the room, the conversation and the voice. No video or audio passes through VideoRouter — we open and meter the avatar session, nothing more.
You need:
- A LiveKit Cloud project (its
wss://<project>.livekit.cloudURL, API key and API secret). Self-hosted LiveKit is not supported. LiveKit bills you for it directly. - A voice agent that joins the room and produces the avatar’s speech as 24 kHz, 16-bit, mono PCM — any TTS or speech-to-speech model works. We do not provide one.
- A front end that joins the room as a viewer and shows the avatar’s video and audio tracks.
Your LiveKit API key and secret never come to VideoRouter. You sign a short-lived token for the avatar with them, on your own server, and send us only that token.
1. Mint the tokens (your server)
Three participants join the room: the avatar, your agent, and your viewer. Each gets its own token. Give the avatar token a lifetime longer than the session (max_duration_seconds plus a few minutes) — if it expires first, the avatar drops out of the room.
# pip install livekit-api
from datetime import timedelta
from livekit import api
LK_KEY, LK_SECRET = "APIxxxx", "xxxx" # your LiveKit project's credentials - they stay on your server
ROOM, MAX_SECONDS = "avatar-demo", 600
def token(identity, *, publish, subscribe, ttl, sources=None):
grants = api.VideoGrants(room_join=True, room=ROOM, can_publish=publish, can_subscribe=subscribe,
can_publish_data=publish, can_publish_sources=sources)
return (api.AccessToken(LK_KEY, LK_SECRET).with_identity(identity)
.with_grants(grants).with_ttl(timedelta(seconds=ttl)).to_jwt())
avatar_token = token("boson-avatar", publish=True, subscribe=False, sources=["camera", "microphone"],
ttl=MAX_SECONDS + 300) # goes to VideoRouter in step 2
agent_token = token("agent", publish=True, subscribe=True, ttl=3600) # stays with your voice agent
viewer_token = token("viewer", publish=False, subscribe=True, ttl=3600) # goes to your front end
The avatar token must allow joining the room and publishing camera and microphone tracks; it does not need to subscribe. The avatar and agent identities must differ.
2. Open the session
Send the avatar token and room details in options.session_params.transport. We forward them to Boson once; the avatar is in your room when the call returns. publisher_identity must be the identity your agent joins with — the avatar accepts audio from that participant only.
POST /v1/live-video/sessions
{
"model": "boson/higgs-avatar-live",
"max_duration_seconds": 600,
"options": { "session_params": {
"avatar_id": "builtin_claire_v1", // or builtin_emma_v1 / builtin_james_v1
"transport": {
"url": "wss://my-project.livekit.cloud",
"room_name": "avatar-demo",
"participant_token": "<LiveKit JWT for the avatar identity>",
"participant_identity": "boson-avatar", // the avatar
"publisher_identity": "agent" // your voice agent (must differ)
}
} }
}
The response is the usual session object; connect carries avatar_session_id, avatar_identity, room_name, publisher_identity and expires_at — never your token. Keep calling heartbeat about every 5 s.
3. Send the avatar its speech (your voice agent)
Join the room as agent and stream your speech to the avatar over a LiveKit data stream, using the LiveKit Agents avatar audio output. The avatar publishes the lip-synced video and the audio; your agent must not publish its own audio track.
# pip install livekit-agents
from livekit import rtc
from livekit.agents.voice.avatar import DataStreamAudioOutput
to_avatar = DataStreamAudioOutput(room, destination_identity="boson-avatar", sample_rate=24000)
await to_avatar.capture_frame(rtc.AudioFrame(pcm, 24000, 1, len(pcm) // 2)) # 24 kHz, 16-bit, mono PCM
to_avatar.flush() # end of this utterance
to_avatar.clear_buffer() # the user interrupted: drop whatever the avatar has not played yet
Listen for the playback_finished event to know when the avatar has finished speaking. Check LiveKit Agents’ avatar documentation for the exact constructor and event API of the version you use. Boson’s own livekit-plugins-boson-avatar creates the session at Boson with a Boson key, so it does not go through VideoRouter — use the data stream directly as above.
4. Show it (your front end)
Join the room with viewer_token using any LiveKit client SDK and attach the video and audio tracks published by the participant whose identity is boson-avatar.
5. End it
Call POST /v1/live-video/sessions/{id}/end when the conversation is over; we release the avatar and settle the bill. The session also ends by itself at max_duration_seconds or when every other participant has left the room, so a closed browser tab does not keep billing.
Prices are live data, not a doc table: GET /v1/live-video/models returns price_per_second_usd and
price_per_hour_usd per row. Session limits are the host’s own and differ by mode.
Billing. The general rule above: whole seconds, the larger of Boson’s own created_at→ended_at and our measured ready→end time, never more than your cap. Boson does not publish its metering rule, so we bill in the safer direction. Rate: $0.07/min at 480x640 (Boson's default; other sizes, including 640x640, are not available in real time). That is the whole VideoRouter charge: LiveKit Cloud is billed to you by LiveKit, under your own project, for the avatar and your voice agent as participants in your room.
Capacity. Avatar capacity at Boson is shared and not documented; a full pool returns 429 host_capacity. Only Boson’s builtin avatars (builtin_claire_v1, builtin_emma_v1, builtin_james_v1) can be used.
Offline version. To turn a face image and an audio file (up to 60 s) into a finished video instead, use boson/higgs-avatar on POST /v1/lipsync. It is the same Boson model, billed per second of audio rather than per session second.
Preview. The real-time Higgs Avatar needs your own LiveKit Cloud project and has been exercised against test doubles only; the offline version is verified end to end. Tell us if a session misbehaves.
Limits & errors
429 concurrency_limitToo many live sessions open for your organization. Retry-After is set; ending a session frees its slot.429 host_capacityNo capacity on that model right now. Retry shortly — sessions are never queued, a queued real-time session is useless.402 insufficient_creditsThe hold for this session isn’t covered by your balance.400 max_duration_exceeds_host_limitmax_duration_seconds is above the model’s host cap.400 invalid_session_paramsVidu and Boson: options.session_params is missing, has an unsupported key, or a value is invalid (Vidu: an image that is not an http(s) / data:image/ URI; Boson: a missing or malformed transport, a transport.url that is not wss://<project>.livekit.cloud, or an avatar_id that is not a builtin_* preset).400 host_rejected_requestVidu or Boson refused the request itself — typically a Vidu character or reference image that failed its content review, or a LiveKit room or token Boson could not use.400 invalid_host_sessionReactor: the id you reported is not a session of this booking — wrong model, created before you booked, unknown, or already used by another booking.409 host_session_already_setReactor: a different session is already attached to this booking.409 session_not_openhost-session was called on a session that has already ended.WS 4401 / 4404 / 4409 / 4410 / 4502Decart signaling socket: invalid token / no such session / session ended or already attached / reached its maximum duration / could not reach Decart. See Decart Lucy.502 host_unavailableThe model cannot start sessions right now. Nothing was held or billed; retry later.502 pricing_unavailableNo verified price for that model yet (see Preview note).404No such session, or it belongs to another organization.403 missing_scopeThe API key lacks the live_video scope.Full request and response shapes: API reference below.
API reference
All endpoints need an API key with the live_video scope. Example values are placeholders; the price_per_second_usd you see is the model’s real price.
POST/v1/live-video/tokens
Mint a short-lived credential for a browser. Call it from your server with your own key and hand the result to the client: it can only call /v1/live-video session endpoints (never billable endpoints, never this one), so a leaked token is bounded by its lifetime and limits. Sessions opened with it are billed to your organization.
Parameters
modelsstring[]Optional. Restrict the token to these live-video model ids (1–20).expires_inintegerOptional. Lifetime in seconds, 1–21,600 (default 3,600). Out-of-range values are rejected, never silently clamped.max_sessionsintegerOptional. Sessions this token may hold open at once (1–20) → 429 token_session_limit.max_session_duration_secondsintegerOptional. Ceiling on a session’s max_duration_seconds (1–3,600) → 400 token_max_duration_exceeded.Example response
{
"object": "live_video.token",
"token": "llmr_sk_live_...",
"expires_at": "2026-10-13T13:00:00+00:00",
"models": ["reactor/happy-oyster-adventure"],
"max_sessions": 1,
"max_session_duration_seconds": 60
}
Using the token anywhere outside /v1/live-video returns 403 live_token_forbidden; using a model it wasn’t minted for returns 403 model_not_allowed. The caller’s key needs the live_video scope.
GET/v1/live-video/models
The catalog. Never exposes the host. A row with available: false has no verified price yet and can’t be booked.
Example response
{
"object": "list",
"data": [{
"id": "reactor/happy-oyster-adventure", "object": "live_video.model",
"kind": "world", "control": "stream", "resolutions": ["720p"],
"max_session_sec": 900, "available": true,
"price_per_second_usd": 0.0, "price_per_hour_usd": 0.0
}]
}
kind is generate/world/avatar/edit; control is stream (your browser steers over the host’s channel) today.
POST/v1/live-video/sessions
Open a session. Holds max_duration_seconds × price_per_second against your credits and mints the host credentials (immediately, or once the world is built).
Parameters
modelstringRequired. A slug from GET /v1/live-video/models.max_duration_secondsintegerOptional. 1 to the model’s max_session_sec; defaults to that cap. Above it → 400 max_duration_exceeds_host_limit.options.worldobjectBuild a new world first (host-specific fields passed through). Session starts waiting. Exclusive with world_id/session_params.options.world_idstringReuse an existing world; session is ready at once.options.session_paramsobjectForwarded verbatim to the host’s session call.Example response
{
"id": "lvs_...",
"object": "live_video.session",
"model": "reactor/happy-oyster-adventure",
"kind": "world",
"control": "stream",
"status": "ready",
"price_per_second_usd": 0.0,
"reserved_usd": 0.0,
"max_duration_seconds": 60,
"created_at": "2026-10-13T12:00:00+00:00",
"ready_at": "2026-10-13T12:00:01+00:00",
"expires_at": "2026-10-13T12:01:01+00:00",
"heartbeat_interval_seconds": 5,
"world_id": null,
"billed_seconds": null,
"cost_usd": null,
"end_reason": null,
"recording_url": null,
"connect": { "ticket": "...", "token": "...", "api_base_url": "..." }
}
connect appears only in the response that minted it. A waiting session has no connect, ready_at or expires_at yet.
GET/v1/live-video/sessions/{id}
Status and billing so far. If the session is waiting, this call checks the host and, when the world is built, mints the host session — that one response carries connect. Ownership-checked: another organization’s id returns 404.
Parameters
idstring (path)The id from POST /v1/live-video/sessions.status: waiting → ready → ended. After ending, billed_seconds, cost_usd, end_reason and (where available) recording_url are set.
POST/v1/live-video/sessions/{id}/heartbeat
Keep the host lease alive; call about every heartbeat_interval_seconds. If the session is past its cap it is settled instead (end_reason: max_duration), and if the host reports it gone, host_limit. Returns the session object.
Parameters
idstring (path)The session id.POST/v1/live-video/sessions/{id}/end
Settle the session now: debit the real cost, release the rest of the hold. Safe to repeat — a second call returns the same ended session and never debits twice. Ending a waiting session costs nothing, except the world fee of a world build you already started. For Decart and Reactor, ending also stops the host session.
Parameters
idstring (path)The session id.host_travel_idstring (body)Optional. The host’s own travel/run id. When given we ask the host for its authoritative duration (billed if larger than our clock) and fetch the replay.end_reason is one of client, max_duration, heartbeat_timeout, host_error, host_limit, or abandoned (a world that never became ready).
POST/v1/live-video/sessions/{id}/host-session
Reactor only. Report the Reactor session your browser created with connect.jwt (the SDK’s getSessionId() / session_id). We verify it against Reactor and attach it to this booking, so ending the booking ends exactly that session and you pay Reactor’s own record of it. Safe to repeat with the same id. Returns the session object. Errors: 400 invalid_host_session, 409 host_session_already_set, 409 session_not_open.
Parameters
idstring (path)The session id.session_idstring (body)Required. The Reactor session id.WS/v1/live-video/decart/v1/stream
Decart Lucy only. The signaling socket for Decart’s realtime SDK, relayed to Decart; you do not call it yourself — give connect.base_url and connect.api_key to the SDK, which opens <base_url>/v1/stream?api_key=…&model=…. One connection per session; the booked model and speed are used whatever the query string says. Closing it ends the Decart stream. Close codes are listed under Decart Lucy.
Query
api_keystring (query)Required. connect.api_key from the session that booked this stream.resolutionstring (query)Optional, passed to Decart.