Layer 03 · System architecture

System architecture

Traces is four running programs around two stores: a CLI on the developer's machine, a web app, an API backend, and an assistant service — backed by a relational store for accounts and metadata and a columnar analytics store for conversation bodies. This page first shows the system from the outside, then opens it up.

Perspective A: the system from outside

Everything the platform does is triggered from one of these edges: developers publishing, visitors reading, or an external system calling in. The coding agents themselves never talk to Traces — they only write files that the CLI picks up.

flowchart LR
  DEV[Developer]
  AGENTS[AI coding agents
Claude Code · Cursor · Codex · …]
  VISITOR[Web visitor]
  GH[GitHub]
  GOOG[Google]
  LLM[LLM provider
OpenAI · Anthropic]
  MODELS[models.dev catalog]
  HF[Hugging Face]
  MON[Monitoring
Sentry · SigNoz]
  PKG[npm · Homebrew]

  SYS((Traces platform))

  AGENTS -- "write session recordings
to the local disk" --> DEV
  DEV -- "uploads traces · manages orgs
HTTPS CLI" --> SYS
  VISITOR -- "browses public traces
HTTPS browser" --> SYS
  SYS -- "serves pages, feeds,
trace viewer" --> VISITOR
  GH -- "login consent · pull-request
webhooks" --> SYS
  SYS -- "posts trace links as
PR comments · REST" --> GH
  GOOG -- "login consent · OIDC" --> SYS
  SYS -- "derives titles, summaries,
assistant answers" --> LLM
  SYS -- "syncs model catalog daily" --> MODELS
  SYS -- "uploads dataset exports" --> HF
  SYS -- "errors · traces · metrics" --> MON
  SYS -- "distributes the CLI" --> PKG

Notable for what is absent: there is no inbound path from the coding agents and no account requirement for reading public content. The system's only write path of consequence is the CLI's upload; everything else is account management or derivation from already-uploaded data.

Perspective B: inside the boundary

Four deployable programs and three stores. The API backend is the only writer of shared state; the web app never touches a store directly; the assistant service reads metadata straight from the primary store but must go through the API for conversation bodies and for anything permission-checked.

flowchart LR
  subgraph DevMachine[Developer machine]
    CLI[CLI
capture · scrub · upload]
    LDB[(Local SQLite
events · sync cache · indexes)]
    CLI -- "stores parsed events" --> LDB
  end

  subgraph Cloud[Hosted platform]
    FE[Web app
Next.js · server-rendered]
    API[API backend
self-hosted Convex]
    AGENT[Assistant service
Bun · Hono]
    PG[(Primary store
Postgres via Convex)]
    TB[(Analytics store
Tinybird / ClickHouse)]
    CACHE[(Cache
Valkey behind HTTP)]
  end

  LLM2[LLM provider]
  GH2[GitHub]

  CLI -- "uploads traces · authenticates
HTTPS REST /v1" --> API
  FE -- "reads & mutates, cookies forwarded
HTTPS REST /v1" --> API
  FE -- "live trace updates · optional
WebSocket query" --> API
  FE -- "relays chat, same-origin proxy
SSE stream" --> AGENT
  API -- "stores records · functions
internal" --> PG
  API -- "appends bodies · queries pipes
HTTPS events API" --> TB
  API -- "buffers & caches hot reads
HTTP" --> CACHE
  AGENT -- "searches & reads traces as the caller
HTTPS REST /v1" --> API
  AGENT -- "reads metadata tables directly
SQL" --> PG
  AGENT -- "streams prompts & completions" --> LLM2
  API -- "generates trace analysis" --> LLM2
  GH2 -- "pull-request webhooks · HMAC
HTTPS" --> API
  API -- "reads notes · upserts comments
installation tokens" --> GH2

Communication boundaries

How the system behaves

Where state accumulates

Primary store (Convex over Postgres) owns everything permission- or freshness-critical: users, identities, sessions, namespaces, trace records, scheduling counters. The analytics store owns the bulk: message and part bodies, mirrored trace display rows, and analysis output, all append-only and collapsed to latest-wins on read. The cache holds only derived, losable state. The CLI's SQLite is a private mirror of the developer's own traces plus a sync cache of shared ones. The dual-write rule that keeps the two server stores coherent — every mutation writes the primary store first, then pushes to the analytics store idempotently — is centralized in one service module, api/convex/services/traces.ts.

What controls movement

Writes are gated by layered throttles: per-user rate limits at the API edge (a rate-limiter component), byte and item caps on upload batches that the CLI learns from server-published remote config, and a bounded worker pool for background analysis so bursts of uploads cannot stampede the LLM provider. Visibility policy gates every read. A maintenance-mode guard can freeze writes without touching reads (api/convex/lib/feature_flags.ts).

Feedback and delay, deliberately

The design accepts eventual consistency in exactly two places, and engineers around both. First, the analytics store flushes on a window of seconds, so any surface where a user just edited something (titles, visibility) resolves from the primary store and lets the analytics copy catch up. Second, AI analysis waits for uploads to go quiet — a debounced, self-deferring schedule — so an active session is summarized once, at a natural pause, not fifty times. Upload failures rely on client retry plus idempotent appends (latest version wins), so a retried batch never duplicates content; partial batch failures are healed by the same retry rather than a reconciliation job.

The goal this shape serves

Make publishing a trace a five-second, one-command act while keeping reads of a large trace library cheap — including search and analytics that would punish the transactional store. The split of stores, the debounced derivation pipeline, and the cache layer all follow from that.

Security and trust boundaries

Three credential styles, one resolver. Every API request resolves to an auth context through a single path: trk_ API keys (namespace-scoped, scope-checked), bearer session tokens (CLI), or cookie sessions with a double-submit CSRF token (browser). Tokens are stored only as SHA-256 hashes. See api/convex/lib/auth.ts and api/convex/http/shared.ts.

Public versus internal surface. The public surface is the HTTP router's /v1 REST API plus explicitly public Convex functions; everything under internal/ (scheduling, migrations, webhook processing) is unreachable from outside. The assistant service holds no credentials of its own — it forwards the caller's session to the API for permission-checked reads, so it can never see more than the person asking. Its direct SQL path reads only metadata tables, not conversation bodies.

Tenant isolation is namespace membership, checked in one policy module and re-checked at the analytics layer by scoping every pipe query to pre-authorized IDs — the columnar store holds no authorization logic of its own.

Sensitive content is scrubbed for secrets on the CLI before upload (cli/src/services/scrub-secrets.ts) and again when building derived artifacts server-side (api/convex/lib/scrub.ts). OAuth access tokens from login providers are stored only where a feature needs them (e.g. Hugging Face exports); Google provider tokens are never persisted. Webhook authenticity is HMAC-verified; GitHub App calls use short-lived installation tokens minted from a signed JWT.

Infrastructure, at the depth that matters

The backend is a self-hosted Convex deployment: the Convex backend container plus Postgres 18, pinned by image digest and run the same way locally and in production (api/docker-compose.yml). The web app is a Next.js app built for Vercel. The assistant service ships as a Docker image (agent/Dockerfile). The analytics store is Tinybird Cloud in production and a Tinybird Local container in development (tinybird/); the cache is a two-container Valkey + webdis pair (cache/). Telemetry fans out through OpenTelemetry collectors — including a small gateway that forwards to a managed SigNoz — and Sentry for errors (observability/). The CLI is distributed as compiled binaries through npm and Homebrew from cli/scripts/.