Live translation

A translation session is a realtime session whose model is a simultaneous interpreter rather than a conversation partner. Speech goes in; the same words in another language come out. The model never answers, comments, greets, or takes a turn of its own.

When to use it

A meeting in two languages

Everyone speaks their own; everyone hears their own. Leave the source language unset and it follows whoever is talking.

Support across a language line

One agent, one caller, no interpreter scheduled in advance.

Speaking to a room

Give it your cloned voice and the room hears you in their language.

Not for recorded video

Dubbing a file is transcription plus synthesis, not a live session — transcribe it, translate the text, then synthesize.

Why this is an endpoint and not a prompt

Every conversational /v1/realtime session carries the platform persona, and the persona instructs the model to introduce itself and — in as many words — to “translate yourself only when you are asked to”. Instructions you send are appended after it, so a translation prompt arrives behind an identity that contradicts it and loses. The observable failure is not a bad translation. It is the model answering the speaker: given “Good morning everyone, where is the train station?” it replies “Buenos días. ¿Cómo te llamas?” — a greeting in the target language, and nothing of what was actually said. So interpretation is its own session type, and the persona is skipped for it.

Connect

import asyncio, base64, json, websockets

URL = (
    "wss://api.eesi.ai/v1/realtime/translations"
    "?model=nur-realtime-v1&language=es&source_language=en"
)

async def interpret(pcm_chunks):
    async with websockets.connect(
        URL, additional_headers={"Authorization": "Bearer sk-eesi-..."}
    ) as ws:
        await ws.recv()  # session.created
        for chunk in pcm_chunks:  # 16 kHz mono PCM16, ~100 ms each
            await ws.send(json.dumps({
                "type": "input_audio_buffer.append",
                "audio": base64.b64encode(chunk).decode(),
            }))
            await asyncio.sleep(0.1)  # pace in real time — VAD reads arrival timing
        async for raw in ws:
            event = json.loads(raw)
            if event["type"] == "response.output_audio_transcript.done":
                print(event["transcript"])

Read the model id from /v1/audio/models rather than hardcoding it — speech tiers are opt-in per environment, and a deployment with no realtime backend serves no realtime model at all.

Choose the languages

Either in the URL, as above, or in session.update — which is also how you re-target a session that is already running:

{
  "type": "session.update",
  "session": {
    "type": "translation",
    "audio": {
      "output": { "language": "es" },
      "input": { "language": "en" }
    }
  }
}

Field	Meaning
`audio.output.language`	Target, ISO 639-1. Required.
`audio.input.language`	Source, ISO 639-1. Omit to detect per utterance.

Omitting the source is the right default for a meeting. Speakers switch languages mid-conversation, and a pinned source turns every other speaker into gibberish; with none, each utterance is detected on its own and anything already in the target language produces no output. Both spellings work on the ordinary /v1/realtime endpoint too: setting session.type to "translation", or naming a target language, switches that session to interpreting.

Choose the voice

An interpreting session takes a voice exactly like a conversational one, in the same place — session.audio.output.voice — and cloned voices work here too:

{
  "type": "session.update",
  "session": {
    "type": "translation",
    "audio": {
      "output": { "language": "es", "voice": "ev_abc123" }
    }
  }
}

Clone your own voice and the room hears you speaking the target language, which is the version of this feature people actually want. See Voices for creating one.

Never send a session without a voice and never send "voice": null on a later update. session.update merges partially, so a null erases the voice already set — and with no speaker conditioning the TTS model samples a new random voice on every utterance, so the interpreter changes speaker mid-conversation. The gateway pins the platform default when you name none; the failure mode is only reachable by explicitly clearing it.

Text in, text out

The endpoint is not audio-only. Input and output modalities are independent, so all four combinations run on the same session and the same connection — which is what lets one integration cover both typed chat and voice chat.

	Send	Receive
Audio	`input_audio_buffer.append`	`response.output_audio.delta`, plus the transcript events
Text	`conversation.item.create` with an `input_text` part	`response.output_audio_transcript.delta` / `.done`

Sending text

{
  "type": "conversation.item.create",
  "item": {
    "type": "message",
    "role": "user",
    "content": [{ "type": "input_text", "text": "Where is the train station?" }]
  }
}

Text has no voice activity detection behind it, so adding an item does not start a turn. Follow it with {"type": "response.create"}. Audio input needs no such nudge — turn detection fires the response when you stop speaking. That is the one asymmetry between the two paths.

Asking for text only

Set it once on the session, and every turn follows it:

{
  "type": "session.update",
  "session": {
    "type": "translation",
    "output_modalities": ["text"],
    "audio": { "output": { "language": "es" } }
  }
}

Or per response, which overrides the session for that turn:

{ "type": "response.create", "response": { "output_modalities": ["text"] } }

A text-only session skips synthesis entirely rather than rendering audio nobody plays. That is the reason to ask for it: speech is the slowest stage in the turn, so text out is not a formatting preference — it is the difference between a translation that lands in a moment and one that waits for a voice to be generated first. Omitting output_modalities, or including "audio" in it, gives you audio. That is the default and the existing behaviour.

The four shapes, and when each is the one you want

Shape	Use it for
Text in → text out	Typed chat between people in different languages. No microphone, no audio pipeline, and the lowest latency of the four.
Audio in → text out	Voice chat rendered as subtitles. One person talks; everyone else reads.
Text in → audio out	A typed message spoken aloud in the listener’s language — accessibility, or someone who would rather not type in a second language.
Audio in → audio out	Live interpreting, the default.

Nothing changes between them but the two fields above, so a client can move someone from subtitles to spoken audio mid-session with one session.update.

Receive

The same events as a conversational session. What was said arrives on conversation.item.input_audio_transcription.completed; what the interpreter said arrives on response.output_audio_transcript.delta and its .done, with the spoken audio on response.output_audio.delta.

response.output_audio_transcript.done fires per segment, not once per turn. Keeping only the last one truncates every multi-segment translation — append the deltas instead.

Defaults you get for free

Default	Why
`interrupt_response: false`	An interpreter is always a beat behind, so the speaker starting again is the steady state, not an interruption. With the API default, the in-flight translation is cut off the moment the next utterance begins — it clips the tail off every sentence.
Source language as an STT hint	Improves decoding when you pinned one. The `auto` sentinel is stripped rather than forwarded, since it isn’t a language.
Length and fidelity rules	The serving backend appends “keep replies to one spoken sentence” and “correct likely mishearings” after your instructions, and trailing text wins. Unchallenged, the first truncates long sentences and the second invents words nobody said.

Set audio.input.turn_detection yourself and your choice is kept, on that frame and every later one.

Refine it

Anything you put in instructions is appended after the interpreter prompt, where trailing weight makes it an override — useful for domain vocabulary or a formality register:

{
  "type": "session.update",
  "session": {
    "type": "translation",
    "instructions": "Use formal register. Render 'stand-up' as the agile ceremony, not a comedy show.",
    "audio": { "output": { "language": "ja" } }
  }
}

Try it

The platform console has an interpreting studio at Speech → Translate: pick two languages, speak, and read both halves of each turn as they land.

Get started

Speech

Realtime

Telephony

Care

Platform

Live translation

When to use it

A meeting in two languages

Support across a language line

Speaking to a room

Not for recorded video

Why this is an endpoint and not a prompt

Connect

Choose the languages

Choose the voice

Text in, text out

Sending text

Asking for text only

The four shapes, and when each is the one you want

Receive

Defaults you get for free

Refine it

Try it

Going further

Realtime

Voices

​When to use it

A meeting in two languages

Support across a language line

Speaking to a room

Not for recorded video

​Why this is an endpoint and not a prompt

​Connect

​Choose the languages

​Choose the voice

​Text in, text out

​Sending text

​Asking for text only

​The four shapes, and when each is the one you want

​Receive

​Defaults you get for free

​Refine it

​Try it

​Going further

Realtime

Voices

When to use it

Why this is an endpoint and not a prompt

Connect

Choose the languages

Choose the voice

Text in, text out

Sending text

Asking for text only

The four shapes, and when each is the one you want

Receive

Defaults you get for free

Refine it

Try it

Going further