The EESI SDKs turn Nur into characters you can talk to. You describe who the character is, give it a voice and some tools, and open a session. Nur handles the conversation itself: listening while it speaks, knowing when a turn ends, and stopping when someone cuts in. The SDK handles everything on your side: the microphone and speaker, live state from your game, the tools the character calls, and what it remembers about each player.
The source is on GitHub: eesi-sdk-python and eesi-sdk-typescript. PyPI (eesi-sdk) and npm (@eesi/sdk) packages are on the way; the code you import is the same. The Python audio extra adds this computer’s microphone and speaker. Leave it out on a server that streams audio some other way. The TypeScript package runs on Node 22 or later, Deno, Bun and in browsers; @eesi/sdk/browser has the page’s microphone and speakers, @eesi/sdk/node file memory and WAV files. The examples below are Python; the TypeScript SDK has the same parts under camelCase names, and its README walks through each one.

A character in a few lines

start() opens a session, connects your microphone and speaker, prints the conversation, and runs until you press Ctrl+C. Kael greets you first. Talk over him and he stops. You can also try a character without writing code:
chat lets you type instead of speak.

Who the character is

Every field is optional except name. The SDK composes them into the character’s instructions, adds the rules every spoken character needs (short turns, no lists or stage directions), and sends them when a session starts. kael.instructions_text() shows exactly what was composed. A character keeps its own name and stays in role. When someone sincerely asks whether they are talking to a person, it says it is an AI.

Presets

Presets are starting points with sensible defaults. Every option still works on top of them.

Voices

voice takes a built-in voice name ("Atlas", "Nova", "Nur"), any voice id from your library ("ev_…"), or a voice described in words:
A described voice is created once in your library and reused by name after that, so Kael sounds the same in every session. python -m eesi voices lists what you have. To hear takes before keeping one, use voice design directly.

Save and share a character

A saved character is plain JSON. Tools are functions, so you pass them again when you load it.

Sessions

A character is defined once. Each conversation is a session, usually one per player:
Without audio, send microphone audio with await session.send_audio(pcm) (16-bit mono PCM, 16 kHz by default, at the pace it was recorded) and play the character’s voice from audio.output.chunk events. That is how a game server relays audio from a player’s machine. You can also type. await session.send_text("Where is the mill?") adds the line and asks for a reply; reply = await session.ask("Where is the mill?") also waits for the character’s answer to that line and returns it.

Events

Callbacks can be plain functions or async functions. on("tool.*") matches every tool event and on("*") every event. A character’s own on(...) hears every one of its sessions.

Who speaks when

turn_taking on the character decides who starts replies:
  • "natural" (the default). Nur decides, with its full conversational timing: it backchannels, waits through pauses, and yields when cut off. It also takes part on its own: when a conversation goes quiet after its reply it may follow up, and it may chime in briefly when someone pauses mid-thought. A session that sets "eesi_proactive": false does neither.
  • "app". The SDK asks for a reply about half a second after each turn ends, once fresh state and memories are in place. Tool use is most reliable here.
  • "director". The character only speaks when your code says so, with session.respond() or session.cue(). Use it for scripted scenes and several characters in one room.
To make the character speak first about something that happened:
A cue is answered as soon as nobody is talking. A moment is optional: it is paced so the character never talks over anyone and remarks at most once every few seconds, and only the newest moment of each kind waits. session.say(line) asks the character to say a line in its voice, and session.interrupt() stops it mid-sentence.

Tools

The tool’s name, description and argument schema come from the function: its name, docstring and type hints. Arguments are validated before your function runs; a bad call is answered with what was wrong so the character can try again. Plain functions run in a worker thread, so a slow tool never holds up the audio. A parameter typed ToolContext receives the calling session and player instead of a model argument:
For work that takes a while, return Started:
The character hears “Off to the cellar.” at once and keeps talking. When the work finishes, it is told and speaks about it as soon as nobody is talking. Tool descriptions never authorize anything. Check permissions in allow or in your own game code, and keep your game server the authority on what happened.

Live context

Context is what is true right now, beside the conversation. Send it whenever it changes:
The character reads the latest state on its next reply. Context never enters the conversation history and never starts a reply by itself. When it disagrees with something said earlier, the context wins. Nothing the character says changes it: your application stays the authority.
  • Updates merge field by field. None removes a field.
  • ttl=30 makes fields expire.
  • Pydantic models and dataclasses are accepted, so typed game state works as is.
  • Fields are grouped into named blocks. The character’s own context is the world block and a session’s is the scene block; pass block="combat" to start another.
  • Update as often as you like. Each block is sent at most a few times a second, and only when its text changed.
session.inspect() shows the instructions, every block as the character reads it, the tools and the memories in use. await session.set_mood("annoyed", reason="the player insulted the king") colours how the character speaks for a while without changing who it is.

Memory

Memory is a SQLite file on your machine. Nothing is hosted. With memory=True the file is .eesi/memory/memory.db. What happens in each session:
  1. When it starts, the memories that matter most for this player are chosen by relevance to the current situation, importance and recency, and sent as the memory block. A new player is introduced as one: the character knows it has not met them.
  2. While it runs, what the player says is searched against their memories and anything relevant is added to a small recall block. This happens off the audio path and usually reaches the next reply.
  3. When it ends, what was worth keeping is learned: facts about the player, their preferences, events, promises, how the relationship changed, and a summary of the conversation. Corrections replace the memory they correct, and the old one is kept as history.
Learning asks Nur (nur-v1) in text, with your key. Pass MemoryPolicy(extractor=...) to use your own model, or MemoryPolicy(extract=False) to keep only what your code writes. Learning updates the memory file. It does not train or fine-tune any model. Write memories yourself:
A player’s memories are only ever recalled into that player’s sessions. Memories without a player belong to the character and every session can use them. Inspect, correct and delete:
The same from a terminal: python -m eesi memory list --character kael. MemoryPolicy sets what is kept: retention_days (episodes expire), max_records_per_player, redact (a function applied to every text before it is stored) and the memory block’s size. memory="read" on connect recalls without learning, for a guest or a player who has not agreed to be remembered.

Preferences

Preferences apply at once and are kept for later sessions with this character. persist=False adapts only this session. pace also sets the voice’s speed. personalize=False on the character stops it from learning preferences by itself.

Several players

One character, many players. Every player gets their own session from the same character. Their memories never mix. With multiplayer_npc (or party=True), each session also sees a short log of what the character said to the other players, so its story stays consistent. Several people on one microphone. Nur labels each voice Speaker 1, Speaker 2 once it has heard more than one. Name the players and the SDK binds the labels to them from introductions (“I’m Layla”) and the order they were heard:
transcript.final events then carry each part’s player_id, and tools know who asked. It is a best effort and inherits every mistake the voice labels make. A room. When your server already receives every player’s microphone (your game’s voice chat runs through it), a room mixes them into one conversation with the character and knows who spoke from who was loudest while each turn was spoken, not from introductions:
Give players a language and the character answers each in their own.

Languages

A character follows the player by default and switches when they do. Pin one language (a code or a name) and the greeting and every reply are in it; ar-EG is Egyptian Arabic, and language_name names any variety:

Speaking first

A character can speak without being spoken to. Remarks wait for the floor, so they never talk over anyone:
After a quiet stretch a proactive character asks itself, out of band and in text, whether something is worth saying, and says it only if so.

Browsers and game engines

Never put your API key in a game client. Your server keeps it and hands each client a client secret instead:
The client opens secret.url as a WebSocket and is talking to Kael at once: the session already has his instructions, voice, tools, the player’s memories and the context you passed. It streams audio and answers tool calls by forwarding them to your server, which runs them. A secret opens one session and expires after a minute unless you set expires_in (up to ten minutes). lock decides what the client may not change. By default it is the instructions and the tools.
A locked field in a session.update is left out, and the rest of the update applies. A locked event (live state, a system message, a deleted message) is refused with an error event whose code is session_locked. This works from any engine that has a WebSocket: browsers, Unity, Unreal, Godot. The session speaks the protocol described in Nur. In a browser, the TypeScript SDK opens it for you. Send the client secret.join (in either SDK): the URL, and who greets, runs tools and asks for replies. It never holds your key.

Steer the session from your server

Mint the secret with control=True and your server can act in the session the client opened, without relaying its audio: live state, tool calls, cues, remarks and memory, as in a session of its own.
The server sees the session’s events (transcripts, replies, tool calls; never audio) over wss://api.eesi.ai/v1/realtime/control/{id}, authenticated with your key. Tool calls are the server’s to answer; control="client" leaves them to the client. What the server sends is not held to the locks, and only the organization that minted the secret can open its channel.

When something goes wrong

  • A dropped connection is resumed with the conversation intact (reconnecting, then reconnected). If it cannot be resumed, the session ends with reason="connection_lost".
  • A refused key raises AuthenticationError from connect.
  • An unknown voice raises ConfigurationError listing the voices you have.
  • Context over a limit raises ContextError saying which block and why.
  • Errors inside your event handlers are logged and never end the session.
Write a trace to see what the character heard, recalled, did and said, and how long each reply took:
A trace holds no words by default: transcripts, replies, tool arguments and memories are replaced by their length. Use Trace(path, content=True) while you debug.

Limits

  • Python has no echo canceller. On open speakers, nur.audio.local() keeps the microphone from hearing the character, and you cut in by speaking up. Use headphones and nur.audio.local(echo="off") for the most natural barge-in. Browsers and phones cancel echo themselves.
  • Latency measured in the SDK is from the end of a turn to the first audio of the reply reaching your machine. It does not include your speaker’s own buffering.
  • Everything else (session length, idle timeout, billing) is as described in Nur.