Architecture

EESI is a gateway in front of self-hosted models. Public model ids map to backends; the API speaks the OpenAI wire protocol so existing clients work against it unchanged. Knowing two things explains most of the platform: how a request finds a backend, and why every live conversation is the same kind of session.

​

The gateway

Every tier is opt-in per environment. A backend with no SPEECH_BACKEND_*_URL configured is hidden from /v1/audio/models, so a deployment only advertises what it can actually serve. Always read the model list rather than hardcoding ids.

​

One realtime session, four ways in

A realtime session is a bridged socket: the client’s frames are relayed to a model backend and its frames relayed back, with the gateway owning admission, recording, resumption and billing on the way through. The phone case is the one worth understanding, because it is the reason there is no second architecture. Carriers stream 8 kHz G.711 mu-law, base64 inside a JSON envelope, and the realtime server accepts g711_ulaw natively. So a phone leg is not a pipeline converting between a carrier and a model — it is the same session with a different socket on the near side, and the ~25k lines that used to sit between them were transcoding audio that never needed transcoding. See Telephony. Everything downstream follows from that: one concurrency pool, one recorder, one transcript shape, one per-minute meter, whatever opened the session.

​

Processes

One image, three processes. Two of them are API roles chosen with EESI_ROLE, and the third runs the queues: The api/gateway split is not cosmetic. A media socket held open for ten minutes and an HTTP request that finishes in 40 ms have opposite deployment characteristics, and a request that reaches the wrong role 404s loudly rather than quietly running media on the control plane. Postgres holds durable state, the job queue and worker heartbeats. Redis holds ephemeral concurrency leases, resumable-session checkpoints, request-rate windows and cross-worker pub/sub. S3-compatible object storage holds audio.

​

What a conversation leaves behind

Every session produces the same artifacts, so tooling built against one surface works against all of them:

• a transcript with role, text and timing per turn
• a recording per direction, where the deployment keeps audio
• reply latency per turn, from end-of-speech to first audio out
• the model that served it and the source that opened it

​

Products on top

Care is a product built on this platform rather than a system beside it: it renders a persona, stages it for one call, dials a carrier, and reads the transcript afterwards. The dependency runs one way — Care reaches into the phone service, and the phone service reaches back in exactly one place, to ask which carrier account a call was placed with so a forged webhook cannot nominate the key it wants to be verified against. That boundary is what lets a second product place calls without either of them changing.

​

Going further