Real-time

Live video & world models

Some video models are not “submit a prompt, get a clip.” An interactive world model holds a GPU for you while you steer it — walk around, change the scene, give it instructions — and streams the result live. /v1/live-video/sessions opens one of those sessions and bills it per second the GPU is held.

VideoRouter is the session broker, not the video pipe. It checks your key and credits, asks the host for a short-lived credential, and returns it to you. Your browser or app then connects directly to the host with the host’s own SDK — the video never passes through our servers, and the host API key never reaches your browser. We meter the session and settle it when it ends. Prefer a client library? There are Python and JavaScript SDKs — see the SDK overview.

1. Open a session (your server)

from videorouter_live import LiveVideo

lv = LiveVideo(api_key="llmr_sk_live_...")     # or set VIDEOROUTER_API_KEY

# On YOUR server: open a session. Entering the block waits until the world is built
# (nothing is metered while it is "waiting"), then heartbeats in the background.
with lv.session(
    "fal/happy-oyster-adventure",
    max_duration_seconds=60,
    options={"world": {"prompt": "a rainy cyberpunk city", "perspective": "first_person"}},
) as s:
    # {"ticket": "...", "token": "...", "api_base_url": "..."} - returned once; hand it to the browser
    send_to_browser(s.connect)
    wait_for_user_to_finish()

# Settled on exit, even if your code raised.
print(s.status, s.billed_seconds, s.cost_usd, s.end_reason)   # ended 41 ... client

import { LiveVideo } from "videorouter-live";

const lv = new LiveVideo({ apiKey: process.env.VIDEOROUTER_API_KEY });

// On YOUR server: open a session. withSession waits until the world is built (nothing is
// metered while it is "waiting"), heartbeats in the background, and settles in a finally.
const session = await lv.withSession(
  {
    model: "fal/happy-oyster-adventure",
    maxDurationSeconds: 60,
    options: { world: { prompt: "a rainy cyberpunk city", perspective: "first_person" } },
  },
  async (session) => {
    await sendToBrowser(session.connect);   // { ticket, token, api_base_url } - returned once
    await waitForUserToFinish();
    return session;
  },
);

console.log(session.status, session.billedSeconds, session.costUsd, session.endReason);

connect is returned once, in the response that created (or first saw ready) the session. It is a short-lived credential and is deliberately not stored or re-served — if you lose it, end the session and open a new one.

2. Stream it (your browser)

Hand connect to the host’s SDK. For Happy Oyster that is @happy-oyster/js-sdk. Each host has its own SDK and its own control vocabulary (movement, instructions, prompts, pause/rewind) — we do not wrap them.

import { HappyOysterEngine } from "@happy-oyster/js-sdk";

// `connect` is what your server got back from VideoRouter. Never ship your llmr_sk_ key to the browser.
const engine = new HappyOysterEngine({ APIHost: connect.api_base_url, model: "happyoyster-1.0-adventure" });
engine.updateToken(connect.token);

const travel = engine.createTravel({ ticket: connect.ticket, videoElement: document.getElementById("player") });
await travel.start();
travel.sendCommand({ translation: "Front", rotation: "Mouse_Left", interaction: "Jump" });
// ...and when the user leaves:
await travel.end();

import { Odyssey, credentialsFromDict } from "@odysseyml/odyssey";

// `connect` is what your server got back from VideoRouter. No API key in the browser.
const client = new Odyssey();
await client.connectWithCredentials(credentialsFromDict(connect), { /* optional handlers */ });
await client.startStream({ prompt: "A cat walking through a neon city" });   // image: imageFile is optional
// steer it with client.interact({ prompt: "..." }); stop with client.endStream()

// connect.api_key is a short-lived Decart client token, locked to the model you booked.
// Use it wherever Decart's realtime SDK takes an apiKey (or an apiKeyProvider):
// https://docs.platform.decart.ai/sdks/javascript-realtime
//
// connect.max_session_duration_seconds is enforced by Decart itself — the stream ends there.
// Keep calling POST .../heartbeat on our API every ~5 s while it runs.

// Open the session on your server with options.session_params (required for Vidu):
//   avatar : { avatar: { persona, image_uri, voice? }, call_mode: "video" | "audio" }
//   editing: { image_url, editing_type: "style_transfer" | "virtual_tryon" | "subject_replacement" | "background_replacement" }
// `connect` then holds a short-lived client_secret (never a Vidu key), the control-socket URL and the RTC credentials.
const connId = crypto.randomUUID();
const ws = new WebSocket(`${connect.ws_url}&conn_id=${connId}`);          // control channel (no media)
let seq = 1;
const send = (type, payload) => ws.send(JSON.stringify({ type, live_id: connect.live_id, conn_id: connId, seq_id: seq++, payload }));
const init = () => send(1, { conn_init: { version: 1 } });
ws.onopen = init;
ws.onmessage = (e) => {
  const ack = JSON.parse(e.data).payload?.conn_init_ack;
  if (ack?.error_code === "NOT_READY") setTimeout(init, 2000);            // the renderer is still starting: retry
};

// Media is Alibaba Cloud RTC (aliyun-rtc-sdk): publish your mic / camera, subscribe to the result.
const rtc = AliRtcEngine.getInstance();
await rtc.joinChannel(connect.rtc.token, connect.rtc.user_id);
await rtc.publishLocalAudioStream(true);                                    // avatar; editing publishes the camera instead
// editing: show only connect.render_uid — that stream is the edited picture.

send(99, { text_msg: { msg_id: "1", content: "Hello!", timestamp: Date.now() } });        // avatar: talk to the character
send(13, { switch_prompt: { editing_type: "virtual_tryon" } });                       // editing: change scenario / image (no ack on success)
send(5, { hangup: { hangup_reason: "user_end" } });                                  // when the user leaves
// Or skip all of this: the videorouter-live JS SDK ships ViduAvatarModel / ViduEditingModel.

Lifecycle & billing

• Hold, then settle. Opening a session holds max_duration_seconds × price_per_second against your credits (402 if you can’t cover it). Ending it debits the real cost and releases the rest of the hold.
• What you’re billed. Whole seconds, the larger of the host’s own reported duration and our measured ready→end time, never more than your cap on our side. Never the client’s claim. Vidu is the exception: its meter starts only when your browser’s control connection is acknowledged (not while the session is being set up) and is exact, so for Vidu we bill the host’s own figure — a session you opened but never started streaming costs $0.
• Heartbeat. Call POST .../heartbeat about every 5 s. For Happy Oyster the host key lives with us, so the beat is forwarded to the host; for Decart, Odyssey and Vidu it is purely our liveness signal (Vidu’s own control connection is the browser’s business). If beats stop, we settle the session at the host’s own timeout — a forgotten tab costs at most the silence window, not the whole cap.
• Hard cap. Vidu has no duration setting of its own, so for Vidu we hang the live up at your cap, from our servers, even while your browser is still connected. Every session ends at max_duration_seconds (default and ceiling are per model, below; Decart needs at least 10 s). Pass host_travel_id to /end to settle on the host’s authoritative duration and to get a replay.
• A session that never became ready costs $0 and releases its whole hold.
• Replays. Where the host composes a replay video, recording_url is populated on the ended session and mirrored to our storage.

Options

options accepts exactly these keys, per host (Happy Oyster takes the first three, Decart takes speed, Vidu takes session_params only, Odyssey takes none — its prompt and image go to the SDK in the browser); anything else is a 400.

Prompts in options.world and a Vidu avatar’s persona and greeting go through the same content moderation as video and image requests. Live steering you send straight to the host afterwards is between you and the host — we cannot see it.

Component avatar (bring your own RTC)

vidu/vidu-s2-avatar-component is for teams that already run their own real-time channel (Alibaba, Tencent, Agora or Volcano Engine RTC) and their own speech pipeline. You stream the character’s speech and text to us; Vidu renders the character and pushes the video into your channel. Nothing else about the session differs: same hold, same settle, same limits.

1. Open it with options.session_params: image_uri (required, http(s) or data:image/) and optionally rtc_info — provider (artc, trtc, agora, volcengine) and token are required, app_id, channel_id, user_id optional strings. Without rtc_info Vidu hosts a channel and connect.managed_rtc carries the credentials to join it and watch. Your join token is forwarded to Vidu and never stored in our logs.
2. Stream to wss://<host>/api/v1/live-video/sessions/{id}/stream?conn_id=<random> (the path is in connect.stream_path). Servers authenticate with Authorization: Bearer; a browser, which cannot set headers, passes a short-lived live-video token as ?token= — a long-lived API key in a URL is refused.
3. Frames. Text frames are JSON signals, binary frames are raw audio. First send {"type":1,"seq_id":1,"payload":{"conn_init":{"version":1}}} and wait for conn_init_ack.success; on NOT_READY just send it again a couple of seconds later (do not reconnect). Then stream PCM 16-bit signed little-endian, 24 kHz, mono (48 bytes per millisecond; 20 ms or 100 ms chunks suit it), plus type 9 (what the user said) and type 10 (what your model replied) as {"payload":{"text_msg":{"msg_id","content","timestamp"}}}, type 7 to interrupt, type 5 to hang up. We forward every frame verbatim in both directions.

• Why it is relayed. Vidu’s own data socket admits one connection, so whoever held it could never be cut at the session cap by anyone else. The relay holds that connection for you, which is what makes the cap, the hold and the settle real. Only audio and text cross it; the video goes straight to your RTC channel.
• One stream per session, and detaching ends it. Closing the stream hangs the host up immediately and settles the session; to continue, open a new session. The stream itself counts as the heartbeat, so you do not call POST .../heartbeat.
• Billing is the host’s own meter, as for the other Vidu models: 1 credit per second from the host’s acknowledgement of conn_init to the hang-up, with a short warm-up that is not billed.

Models

Prices are live data, not a doc table: GET /v1/live-video/models returns price_per_second_usd and price_per_hour_usd per row. Session limits are the host’s own and differ by mode.

Limits & errors

Full request and response shapes: API reference below.

API reference

All endpoints need an API key with the live_video scope. Example values are placeholders; the price_per_second_usd you see is the model’s real price.

{
  "object": "live_video.token",
  "token": "llmr_sk_live_...",
  "expires_at": "2026-10-13T13:00:00+00:00",
  "models": ["fal/happy-oyster-adventure"],
  "max_sessions": 1,
  "max_session_duration_seconds": 60
}

{
  "object": "list",
  "data": [{
    "id": "fal/happy-oyster-adventure", "object": "live_video.model",
    "kind": "world", "control": "stream", "resolutions": ["480p"],
    "max_session_sec": 60, "available": true,
    "price_per_second_usd": 0.0, "price_per_hour_usd": 0.0
  }]
}

{
  "id": "lvs_...",
  "object": "live_video.session",
  "model": "fal/happy-oyster-adventure",
  "kind": "world",
  "control": "stream",
  "status": "ready",
  "price_per_second_usd": 0.0,
  "reserved_usd": 0.0,
  "max_duration_seconds": 60,
  "created_at": "2026-10-13T12:00:00+00:00",
  "ready_at": "2026-10-13T12:00:01+00:00",
  "expires_at": "2026-10-13T12:01:01+00:00",
  "heartbeat_interval_seconds": 5,
  "world_id": null,
  "billed_seconds": null,
  "cost_usd": null,
  "end_reason": null,
  "recording_url": null,
  "connect": { "ticket": "...", "token": "...", "api_base_url": "..." }
}