Telemetry

Telemetry Reference

RunEvent envelope, extending the schema with collect and custom fields, privacy rules, TELEMETRY.md disclosure, and consent / outbox reliability.

Standard envelope

Every run shares the same shape. You do not declare per-command schemas.

{
  "event": "run",
  "command": "sync",
  "durationMs": 412,
  "outcome": "success",
  "flags": { "dryRun": true, "output": true },
  "tool": { "name": "my-tool", "version": "1.0.0" },
  "env": {
    "node": "20.11",
    "ci": false,
    "provider": null,
    "tty": true,
    "agent": "cursor"   // std-env: cursor, claude, codex, … or null
  },
  "machineId": "ab3f…", // hashed; omitted in ephemeral CI
  "custom": { "itemsSynced": 42 }
}

Extending the schema

The envelope above is fixed — every tool sends the same top-level shape so parseIngestBody() and disclosure stay predictable. You do not add fields next to command or durationMs.

Your product schema lives in two extension zones:

ZoneWho sets itWhat goes there
flagscitty (auto) + collect.flagsCLI switches — booleans/numbers as values, strings only when allowlisted
customyou via telemetry.set() + collect.fieldsBusiness counters, categorical dimensions, schema version

Think of it as two contracts:

  • Transport contract (RunEvent) — owned by @evlog/telemetry, stable across tools
  • Product contract (custom + declared flags) — owned by you, declared in collect, mirrored on the server

Declare your extensions once

Everything you collect beyond the standard envelope is declared inline in the same withTelemetry() / createTelemetry() call:

src/index.ts
withTelemetry(command, {
  name: 'acme-cli',
  version: '2.1.0',
  endpoint: 'https://telemetry.acme.dev/api/ingest',
  collect: {
    flags: {
      format: ['json', 'yaml', 'table'],
      target: ['staging', 'production'],
    },
    fields: {
      product: ['cli', 'action', 'migrator'],
      plan: ['free', 'pro', 'enterprise'],
      framework: ['nuxt', 'next', 'remix'],
    },
  },
})

Inside handlers, add counters and dimensions:

src/commands/deploy.ts
await t.run('deploy', async () => {
  const result = await deployServices()
  telemetry.set({
    servicesDeployed: result.count,   // number — always allowed
    rollbackUsed: false,              // boolean — always allowed
    framework: 'nuxt',                // string — allowed (in collect.fields)
    schemaVersion: 2,                 // number — version your custom shape
  })
})

Undeclared strings are dropped at runtime, never thrown. That is the privacy guarantee — no accidental paths, tokens, or free-form PII in custom.

Typing:TelemetryHandle.set() autocomplete covers declared collect.fields keys. Use the ambient telemetry.set() for extra numeric/boolean counters in the same run — both merge into custom before the event is recorded.

Mirror on the server

Your ingest handler is the second gate. Pass the same tool name and custom keys you declared on the CLI:

app/api/telemetry/ingest/route.ts
import { parseIngestBody } from '@evlog/telemetry/ingest'

const ALLOWED_CUSTOM = [
  'servicesDeployed', 'rollbackUsed', 'framework',
  'product', 'plan', 'schemaVersion',
  'ghaAction', 'ghaEvent',
] as const

export async function POST(req: Request) {
  const raw = await req.text()
  const events = parseIngestBody(raw, {
    allowedTools: ['acme-cli'],
    allowedCustomKeys: { 'acme-cli': ALLOWED_CUSTOM },
  })
  // dedupe on idempotencyKey, then persist
}

parseIngestBody() strips any custom key you did not list — even if a client somehow sent it.

Map to your warehouse schema

After validation, shape the event for your database. The evlog envelope is the wire format; your tables are your choice:

lib/telemetry-store.ts
function toRow(event: RunEvent) {
  return {
    command: event.command,
    duration_ms: event.durationMs,
    outcome: event.outcome,
    tool_version: event.tool.version,
    format: event.flags.format,
    target: event.flags.target,
    services_deployed: event.custom.servicesDeployed,
    framework: event.custom.framework,
    plan: event.custom.plan,
    schema_version: event.custom.schemaVersion ?? 1,
    is_ci: event.env.ci,
    node: event.env.node,
    received_at: new Date(),
  }
}

This is where a company "modifies the schema" in practice — not by changing RunEvent, but by defining how custom maps into internal analytics tables.

Version breaking changes in custom

When your product counters evolve, bump a numeric schemaVersion in custom (no allowlist needed) and branch in the ingest mapper:

const version = (event.custom.schemaVersion as number | undefined) ?? 1
if (version === 1) return mapV1(event)
return mapV2(event)

Commit an updated TELEMETRY.md whenever collect changes so disclosure stays aligned with releases.

Multiple tools, one endpoint

If several CLIs POST to the same ingest URL, namespace custom keys or rely on tool.name:

// Option A — prefix keys per product line
telemetry.set({ acme_product: 'cli', deployCount: 3 })

// Option B — separate allowedCustomKeys per tool.name in parseIngestBody
allowedCustomKeys: {
  'acme-cli': ['deployCount', 'framework'],
  'acme-action': ['jobDuration', 'ghaAction', 'ghaEvent'],
}

What v1 does not support

Needv1 answerWorkaround
New top-level field (tenantId)Not on the wireFlat key in custom + allowlist, or map server-side from machineId
Nested objects (custom.user.plan)Flat records onlyFlatten: userPlan: 'pro' with collect.fields
Free-form strings (paths, emails)Blocked by designPresence-only on flags (output: true) or hash before telemetry.set()
Different schema per commandOne envelope per runConvention: only set relevant keys per command
Transform before outboxNo hook yetMap on the server (recommended) or see planned v2 below
Rule of thumb: if it is safe to print in TELEMETRY.md and fits a closed set or a number, it belongs in collect + custom. Everything else stays server-side.

Planned v2 extensions

Not shipped yet — design targets for when the v1 extension zones are not enough:

enrich hook — run after sanitization, before the outbox append. Lets you inject computed fields without forking the package:

// Planned API — not available in v1
createTelemetry({
  name: 'acme-cli',
  version: '3.0.0',
  collect: { fields: { product: ['cli', 'action'] } },
  enrich(event) {
    return {
      ...event,
      custom: {
        ...event.custom,
        schemaVersion: 2,
      },
    }
  },
})

The hook cannot bypass sanitization rules — strings still require collect.fields. Server-side allowedCustomKeys stays the source of truth.

Namespaced keys — optional acme.product convention for multi-tool endpoints without prefix collisions. Would be documented in disclosure as acme.product: cli | action.

If you need one of these before v2 lands, open a discussion on the repo — the v1 path (custom + server mapping) covers most product analytics needs.

Privacy

Raw argv is never read. Sanitization applies to citty-parsed flags only:

  • Booleans / numbers → value stored (json: true, limit: 50)
  • Strings → presence only (output: true) unless allowlisted in collect.flags
  • telemetry.set() → numbers and booleans always; strings only via collect.fields (undeclared values are dropped at runtime, never thrown)
src/index.ts
collect: {
  flags: { format: ['json', 'csv'] },           // --format json → "json"; --format yaml → true
  fields: { framework: ['nuxt', 'next'] },      // telemetry.set({ framework: 'nuxt' }) ok
}

Declare allowlists in the same withTelemetry() / createTelemetry() call as collect — no separate config file.

Disclosure

generateDisclosure() produces markdown + JSON from the standard envelope plus your collect extensions. Commit the output (e.g. TELEMETRY.md) so it stays in sync with releases:

scripts/generate-disclosure.ts
import { writeFile } from 'node:fs/promises'
import { generateDisclosure } from '@evlog/telemetry'

const { markdown } = generateDisclosure('my-tool', {
  flags: { format: ['json', 'csv'] },
})

await writeFile('TELEMETRY.md', markdown)

Users can also read it at runtime via my-tool telemetry status when you wire defineTelemetryCommands().

Opt-out priority: DO_NOT_TRACK=1EVLOG_TELEMETRY=0 → persisted preference (disableTelemetry() / telemetry disable). Opt-out purges the undelivered outbox.

Never harms the host: telemetry never throws, never blocks exit; flush() has a 500ms hard cap.

Outbox: events append locally before any network attempt. Offline machines and CI matrix jobs drain the backlog on the next invocation once your ingestion endpoint returns 2xx.

Endpoint: EVLOG_TELEMETRY_ENDPOINT env → endpoint option baked into the CLI. Omit both during local development; ship the URL when your ingest handler is live.