opencoworkers

opencoworkers.com 20 orgs · 380 depts
telemetry plane

Atlas — the telemetry plane

Every coworker container and session in this platform reports dimensional metrics into a Netflix Atlas standalone instance at localhost:7101 — Atlas's standalone mode is an in-memory time-series store, which is the point: it is the hot, queryable working set for cross-container/cross-session telemetry, with durable history landing in the postgres 18 warehouse the data-engineering coworker owns (see /team).

The metric contract

Two real metric families, both instrumented via Netflix Spectator semantics (counter/timer/gauge with tag maps):

1. conductor.api.calls — the Conductor worker poll/complete loop

Language-specific coworker workers (the coworkers-{py,ts,ja,rs,go} container images) interact with the Conductor server via a polling model, and every API call tracks the same dimensional contract:

tagvalues
applicationcoworkers
languagepy | ts | ja | rs | go
api_methodpoll | update | complete
statussuccess | failure

2. coworkers.api.* — the coworkers API itself

From packages/coworkers/typescript/src/metrics.ts, which adopts @opencoworkers/telemetry (the repo's nflx-spectator facade + AtlasPublishWriter, defaulting to http://localhost:7101 with 60s buckets). Publishing is explicit opt-in (ATLAS_PUBLISH=1); with no reachable Atlas the meters still record in-memory and the API keeps serving — degraded, never crashed.

metrictypetags
coworkers.api.requestscounterroute, status
coworkers.api.latencytimerroute
coworkers.ooda.tokenscounterphase
coworkers.workflow.runscounterstatus

Atlas Stack Language (ASL) examples

Atlas graphs are queried with a stack language against GET /api/v1/graph?q=... on the standalone instance. Three real queries against the contract above:

# conductor API call rate, one line per language
GET http://localhost:7101/api/v1/graph?q=name,conductor.api.calls,:eq,(,language,),:by

# failures only, still split by language
GET http://localhost:7101/api/v1/graph?q=name,conductor.api.calls,:eq,status,failure,:eq,:and,(,language,),:by

# total call volume, single summed line
GET http://localhost:7101/api/v1/graph?q=name,conductor.api.calls,:eq,:sum

The same shapes apply to the coworkers.api.* family — e.g. name,coworkers.api.requests,:eq,(,route,),:by for per-route request rates.

Honest scope note: opencoworkers.com serves the public docs + directory plane. The Atlas instance is local (localhost:7101, in-memory) and this Worker does not proxy live Atlas graphs — wiring a read-only graph proxy (so ASL queries render here) is a documented next step, not something this page pretends already works.