The complete real-time stack for products that cannot feel bolted on.

Signaling · Media · Data · Intelligence — on your infrastructure, without ever losing access to the protocol.

Browser SDK Backend Clients SFU Routing TURN Relay Media Graph

Mesh & SFU

Mesh and SFU topologies — selected by server policy at join (RTC_TOPOLOGY).

Extensible AI

Run transcription, LLMs, and compliance bots inside the room as first-class service participants.

Observability

Live metrics, session topology graphs, heartbeats, SIEM exports, usage NDJSON, and redacted support bundles.

Hardened ICE

Dynamic token-derived credentials, relay enforcement, IPv6 support, and regional credential routing.

Provenance

Independent new work — informed by years of WebRTC platform delivery.

Created by Nathaniel Currier after serving as architect and CTO at Temasys. Clean-room implementation carrying forward operational learnings — no prior proprietary source or customer materials.

Signaling & Session Control

Room lifecycle, permissions, reconnect, and tenant-scoped policy from large-scale integration work.

Media Path

Mesh/SFU selection, simulcast, TURN/STUN relay design, and diagnostics shaped by production traffic.

SDK & Workers

Browser client APIs, Node/Python service participants, and customer onboarding patterns.

What This Is Not

Not copied from, derived from, or dependent on Temasys IP, customer-specific materials, or prior proprietary codebases.

Note

Note Slide context

See PROVENANCE.md at the repository root for the full statement. Any similarity to prior systems reflects general professional knowledge, public WebRTC standards (IETF/W3C), browser APIs, and newly authored TypeScript in this monorepo.

The Challenge

Hosted platforms are convenient — and completely opaque.

"Most teams building real-time hit the same wall: they get a call working in a day, then spend the next year fighting the platform when they need compliance, audits, or custom AI." We built this to give teams ownership over the entire stack.

Opaque Failures Cannot trace packet loss or RTT directly through the hosted cloud boundary.
Vendor Lock-in Dependent on vendor pricing models and blackbox media server configurations.
Security Audits Enterprise reviews fail when signaling, media, and recording logs cannot be audited.
Extension Limits Hard to build custom AI workers or media filters when you cannot touch the media server.

The Solution

One codebase from early prototype to enterprise rollout.

Modular · Composable · Deployable Anywhere · Protocol Agnostic. These four principles recur throughout every layer of the platform.

Ship

Browser SDK, Node/Python workers, in-app messaging, and TURN relay work out-of-the-box.

Scale

Mesh and SFU topologies (server policy at join), multi-region redirects, and inter-node SFU cascading.

Secure

Enforced tenant isolation, RS256 token validation, short-lived REST credentials, and cryptographically verified E2EE overlays.

Extend

Pluggable adapters for identity (OIDC), databases, telemetry, and AI workers. Your servers, your control.

Architecture

A modular, adapter-driven stack.

Applications → SDKs → Platform Services → Media Infrastructure → Cloud Resources. Session coordination is cleanly separated from media routing and storage adapters.

Edge Clients

Browser SDK, custom widgets, signaling helpers

Service Bots

Node clients, Python listeners, bot processes

Control plane Orchestration Server @rtc-sdk/server · Room Registry · Auth

Media Routing

Production SFU · Integrated SFU hardening · bounded PMG slices

Adapters

Auth (JWKS, OIDC) · Storage (Mongo, Postgres)

Room state, auth, audit, admin

Media routing, relay, storage, telemetry

Media Topologies

Topology as dynamic room policy, not a codebase split.

Every browser client calls the same createRtcClient() → joinRoom() flow. Mesh and SFU are the primary browser paths; cascaded-SFU and broadcast modes expose bounded routing and media-graph foundations that still require deployment-specific validation.

Deployment zones: Edge · Core Services · Persistent Services · External Integrations

Media Plane

Six stages of media flow terminated directly on the server.

For teams seeking absolute control, our integrated media plane terminates ICE, DTLS, and SRTP directly on Node UDP sockets — color-coded by ownership: Developer Owned · TomatoRTC Managed · Infrastructure Managed.

Note

Note Slide context

† First-party integrated SFU, media-plane RTP termination, pacing, and cross-browser production validation are in active hardening — available for evaluation and pilot deployments; the production SFU is the default path today.

Bandwidth Adaptation

Publishers encode once. Mesh or SFU forwards the right quality.

Simulcast sends parallel spatial layers (RIDs f, h, q) from a single encode. Whether traffic flows mesh peer-to-peer or through an SFU relay, each subscriber receives only the layer its bandwidth can sustain — no server-side transcoding.

Data path: Client → Signal Mesh → Media Services → Room State → AI Services

Bandwidth Adaptation

One bitstream, layered inside. Mesh and SFU adapt together.

SVC packs spatial and temporal layers into a single encoded stream (scalabilityMode L3T3). In SFU mode the relay drops enhancement packets via dependency descriptors; in mesh mode peers decode only the layers they need from the same P2P bitstream. Same layer-selection policy in both topologies.

The Signaling Contract

The typed protocol envelope is the platform contract.

All control messages, media parameters, chat, and telemetry flow through a single typed envelope schema. Transport adapters provide identical semantics across WebSocket, WebTransport (HTTP/3), SSE+POST, and Connect RPC.

version "2026-05-01" · client and server check to prevent stale protocol bugs

type "room.join" | "room.joined" | "ice.candidate" | "chat.message" ...

correlationId "tx-8f9a2b" · matches async requests to client handlers

tenantId Enforces complete multitenant segment partitioning

payload Strictly validated schemas preventing JSON prototype pollution

WebSocket WebTransport (HTTP/3) SSE + POST Connect RPC

System Security

Security is a system architecture decision, not a checklist item.

Every trust boundary is independently configured and verified: user authentication, room capability claims, short-lived media relay credentials, worker client scopes, and audit trail generation.

Identity Isolation

Bring your own identity provider (JWKS, OIDC, OAuth). Enforce tenant-namespaced rooms. Guest invite links auto-exchanged for short-lived participant JWTs.

Authorization

RBAC scopes from signed token claims determine capabilities. Admin API supports room queries, policy CRUD, and live participant kicking.

Media Protection

Default DTLS-SRTP key negotiation. Support for application-layer end-to-end media encryption (E2EE) transforms.

Operations Audit

Tamper-evident hash-chained logs, SIEM export adapters, tenant usage telemetry (NDJSON/HTTP), and proactive client credential redaction across diagnostic report bundles.

Transport Security

A mandatory cryptographic chain before a single packet flows.

Media security in WebRTC operates at the network packet layer. The connection sequence establishes identity, path, credentials, and encryption before routing any media. Fault domains: Client Layer · Regional Services · Media Plane · Storage Plane.

Sign

Secure tokens issued by auth

ICE

STUN/TURN verification (first-party server with metering) & RFC 7675 consent

DTLS

ECDHE key exchange & fingerprint validation

SRTP

AES-128-CM frame protection derived per session

Secure RTP

Authenticated media packets bypass intermediate eyes

Client-Side Security

The SDK acts as a security boundary, not just a media helper.

Our browser client validates capabilities locally to fail fast, isolates E2EE key transforms inside dedicated Workers, and filters credentials from telemetry before logs exit the device footprint.

Redaction Recursively strips passwords, authorization headers, JWTs, and TURN secrets from diagnostic exports.
Local Verification Denies unpermitted publishes before initiating signaling round-trips.
Worker Isolation Handles E2EE frame modifications inside a dedicated thread, preventing main-thread scripts from scraping raw frame keys.

Credential Lifecycle

Short-lived, single-purpose tokens issued per participant.

Instead of long-lived API keys or global secrets, the platform uses short-lived tokens. The SDK automatically exchanges tenant-level tokens into unique participant-scoped JWTs, refreshed transparently before expiration.

Client Auth Request

Browser fetches short-lived tenant room token from client API

Identity Validation

Signaling server validates token signatures (HS256/RS256) and claims

Credential Exchange

Server issues unique participant session JWT and first-party TURN credentials with bandwidth metering

Connection Established

Client connects WebSocket and initializes media session. Auto-rotates credentials before expiry.

Client Libraries

One protocol core, with capability-specific SDK maturity.

Browser media is the complete client path. Service and native clients share signaling and room semantics without implying media parity where interop or platform UX is still being hardened.

Shipped

Backend bots, load tests

Signaling, chat, receipts, reconnect, service workers

C# / .NET

Shipped

Desktop tooling, Unity services

Signaling, chat, receipts, reconnect, service workers

Python

Shipped

AI workers, automation pipelines

Signaling, chat, reconnect, optional worker media (aiortc)

Browser TS

Shipped

Web applications

Full WebRTC media, mesh/SFU transport, custom UI helper tools

Client libraries · Native foundations

Native signaling ships; media and room-UX parity are still hardening.

Swift, Kotlin, Flutter, and C++ paths have concrete packages, examples, and shared protocol types. Production adoption should follow the capability matrix: signaling foundations are real, while native media interop and packaging vary by platform.

Swift

Foundation

iOS & macOS

Signaling + SFU control; libwebrtc and WHIP/WHEP paths; interop proof open

Kotlin

Foundation

Android devices

Signaling + SFU control; Android PeerConnection module; interop proof open

Flutter

Foundation

Cross-platform mobile

Platform channels and native video views through Swift/Kotlin media bridges

C++

Foundation

Native and game engines

Protocol + signaling foundation; WHIP/WHEP through Rust media FFI

System Extensibility

Extendable at every boundary without SDK forks.

The core protocol handles rooms, participants, and media forwarding. Telemetry, storage backends, identity providers, and background processors are exposed through swappable interface adapters. Extensibility sequence: SDK → Hooks → Plugins → Services → Applications.

Auth Adapter

JWKS / OIDC — Rotate keys dynamically from your identity provider

Storage Adapter

Mongo / Postgres — Swap database backends for messages and audits

Effects Plugins

Worklet / WASM — Inject custom video filters or noise suppression

Worker Adapter

Transcription / Translation — Pass raw media bytes to custom LLM/NLP

Stable core Stable Platform Core Protocol & WebRTC signaling

AI Integration

Service participants join rooms exactly like human users.

We treat AI services as first-class room participants. AI workers subscribe to publisher media tracks using standard SDK protocols, analyze frames, and publish transcripts directly to the chat thread — no sideband pipelines.

Transcription

Word-level timestamps and speaker attribution. Delivered as structured chat updates.

Moderation

Audio and frame analysis. Automatically triggers mute/kick signals upon violation.

AI Integration · Patterns

Assistants and custom pipelines.

Architectural pillars: Composable · Scalable · Observable · Extensible — these four themes recur across the full platform surface.

AI Assistant Shipped pattern

Worker patterns, supervisor wiring, and runnable Node/Python examples ship. Production model providers and a packaged voice-agent runtime remain application-owned.

Custom Pipeline

Any Python/Node containerized process can consume media tracks via pluggable WorkerRoomClient.

Encoded Transforms

Application-layer transforms before transport.

Using insertable streams, the SDK gets synchronous access to encoded media frames. This enables end-to-end encryption (E2EE) overlays and sender-side redaction before packetization.

End-to-End Encryption (E2EE) Shipped

AES-GCM encryption overlay. The SFU routes encrypted packets but has no access to application keys. Key transform layer shipped; key distribution protocol proposed.

Compliant Redaction

Sender-side filter overlays redact sensitive UI fields or voice keywords before packetization.

Encoded Transforms · Effects

Client-side segmentation and audio worklets.

Background Segmenter

Integrated MediaPipe Image Segmenter blur plugin handles virtual backgrounds client-side.

Audio RNNoise Worklet

WASM noise-suppression worklet runs inside Web Audio thread with SIMD assets.

Note

Note Slide context

† MediaPipe segmentation, RNNoise, and insertable-stream transforms require supported client platforms — typically Chromium 94+ / Edge 94+ for MediaStreamTrackProcessor pipelines; run capability probes in the kitchen-sink demo before customer commitments.

Developer Experience

Clean room abstractions, zero WebRTC boilerplate.

The SDK hides raw SDP negotiations, transport creations, and trickle ICE logic, exposing a clean event-driven interface to manage media streams and data channels.

// pseudo-code — see examples/basic-call
import { createRtcClient } from "@rtc-sdk/client-browser";

const client = createRtcClient({
  signalingUrl: "wss://127.0.0.1:8080",
  authToken: "eyJhbGciOi...",
  displayName: "Alice"
});

const room = await client.joinRoom("demo-room");
await room.publishCamera();
await room.publishMicrophone();

room.on("trackSubscribed", (event) => {
  remoteGrid.addTrack(event.participantId, event.track);
});

Evaluation Tools

Test features directly with a complete kitchen-sink application.

We ship functional examples to evaluate the platform immediately. Launch the kitchen-sink demo to compare mesh vs SFU topologies, inspect stats, trigger network simulation in the Chaos Lab, and export diagnostic logs.

Chaos Lab Presets

Simulate 3G latency, 20% packet loss, or complete offline transitions directly in the browser tile UI.

Multi-Source Publish

Publish camera, microphone, screen share, and local video file sources concurrently from a single tab.

Evaluation Tools · Inspection

Deep diagnostics for multi-monitor evaluation.

Stack X-Ray

Inspect negotiated candidate-pairs, active layer representations, and ICE agent checklists on live tiles.

Popup Inspector

Pop out video tiles and session statistics graphs into separate system windows for multi-monitor inspection.

Telemetry & Diagnostics

Correlate client quality scoring with server logs instantly.

The platform calculates real-time call quality scores (1–100) aligned with ITU-T G.114. Correlation IDs map client-side dropouts directly to SFU pacing statistics and TURN server metering lines.

CallQualityAnalyzer

Calculates Excellent/Good/Fair/Poor score bands based on rolling loss, jitter, and RTT

OpenTelemetry Spans

Server actions and signaling handshakes emit standard spans directly to OpenTelemetry collectors

Telemetry · Operator Tools

Support bundles and live operator console.

Redacted Support Bundles

One-click diagnostic reports strip tokens and keys. Ready to paste directly into support tickets.

Global Operator Console

Real-time dashboard for nodes, rooms, and participants with heartbeats and kick controls (Demo-ready).

Network Resilience

Reconnection is a multi-stage state machine, not a spinner.

When networks drop, the SDK coordinates recovery in transparent phases. The UI stays synchronized with exact SDK status as transports rebuild, room history replays, and media recovery is confirmed.

Beyond Media

Primitives for a complete chat, signal, and data experience.

Live communications require more than video streams. We ship core primitives for messaging with receipts, ephemeral signaling channels, and chunked file transfer directly inside the SDK room model.

Signaling-Backed Chat

Room messaging with cursor-based replay. Durable Postgres/Mongo storage adapters. Message delivery and read receipts.

Shared Signals & Awareness

High-frequency mouse coordinate signals. Synchronized participant state sync blobs with partial patch merges.

Data Channels

Direct peer-to-peer data channels for low-latency annotations or file chunking (with compression fallback).

Recording & PMG

VP8/Opus WebM muxing ships in the synthetic SFU demo. Bounded batch/live PMG compositor slices also ship; live-room recording orchestration remains hardening work.

Deployment Tiers

Start local, cluster regional, route globally.

The same SDK integration scales across deployment profiles. As routing requirements change, you update server configurations and tenant policies — no client-side refactoring required.

Local

Fast dev & CI validation

Single container, loopback

In-process mock media

Regional

SaaS or internal clusters

Signaling node, Redis (in progress), TURN

SFU media routing, mesh default

Global

Geo-routed deployments

Multi-region signaling registry, OTel

Cascaded SFU inter-node relay links

Enterprise

Regulated customer tenants

SIEM audit exports, RS256 JWKS

Tenant-scoped E2EE key requirements

Strategic Value

Own the real-time core of your application.

Evaluating a communications framework is a business strategy decision. Owning the stack prevents vendor price escalations, secures client trust, and ensures you can add custom AI features without requesting permission.

Differentiated UX

Control exact layout, reconnection alerts, frame filters, and local device hot-swaps to fit your product's voice.

Predictable Sizing

Centralized media forwarding, dynamic peer mesh offsets, and bandwidth-limited TURN allocations prevent surprise bills.

Compliance Ready

Isolate customer tenants, redact passwords, verify E2EE bytes, and export SIEM logs for security checks.

AI Pipeline Moat

Add meeting assistants, translators, and audit checkers directly into the room. Keep raw media off foreign API boundaries.

Control the stack behind your live communications.

Simple to start. Powerful to extend. Flexible to deploy. Built for real-time systems.

Technical Evaluation
Run the kitchen-sink demo and explore mesh vs SFU topology in minutes.
Architecture Review
Bring your deployment requirements — signaling, media, compliance, and AI pipeline.
Proof of Concept
We ship a working integration on your infrastructure, your timeline.

Own the UX Own the protocol Own the media path Own the roadmap