Telemetry Ingest

You ship a CLI. Users run it on their machines. You want to know which commands run, how often they fail, and which versions are out there — without building a separate analytics SDK or reading raw argv.

That means a server endpoint your CLI POSTs to. The local outbox is only there so short CI jobs and offline runs do not lose data before the POST succeeds.

The security question (read this first)

"I put the ingest URL in my CLI — isn't that a secret?"

No. The URL lives in your npm package or binary. Anyone can read it with strings, by installing the package, or by watching network traffic. The same applies to an API key or Authorization header baked into the CLI — extractable, not a real barrier.

"So can't anyone send fake events?"

In theory, yes — someone could curl your endpoint with forged JSON. That is why you treat the endpoint as semi-public (like a browser analytics key) and defend on the server:

What "good enough" looks like: validate every POST, store idempotently, rate-limit at the edge (CDN, API gateway, or middleware), and keep the payload boring (no PII by design). That is the same playbook as client-side analytics — not banking-grade auth, but reliable product insight.

Step 1 — Validate with parseIngestBody()

Mirror the CLI config on the server: same name, same custom keys you allow via collect / telemetry.set().

import { parseIngestBody } from '@evlog/telemetry/ingest'

export const ingestOptions = {
  allowedTools: ['my-tool'],
  allowedCustomKeys: {
    'my-tool': ['checksFailed', 'checksWarn', 'itemsSynced'],
  },
} as const

export function parseTelemetryBody(raw: string) {
  return parseIngestBody(raw, ingestOptions)
}

parseIngestBody() checks batch size, JSON shape, event: 'run', tool allowlist, envelope types, ISO timestamp, and strips custom keys you did not declare. Throws IngestValidationError on failure — return 400 from your route.

Step 2 — Wire your route

Same contract for any framework — read the raw body, validate, store, return 204 so the CLI clears its outbox:

import { defineEventHandler, readRawBody, setResponseStatus } from 'h3'
import { IngestValidationError } from '@evlog/telemetry/ingest'
import { parseTelemetryBody } from '~/lib/telemetry-ingest'
import { storeRunEvents } from '~/lib/telemetry-store'

export default defineEventHandler(async (event) => {
  const raw = await readRawBody(event, 'utf8')
  if (!raw) {
    setResponseStatus(event, 400)
    return { error: 'empty body' }
  }

  try {
    const events = parseTelemetryBody(raw)
    await storeRunEvents(events)
    setResponseStatus(event, 204)
    return null
  } catch (err) {
    setResponseStatus(event, err instanceof IngestValidationError ? 400 : 500)
    return { error: 'invalid payload' }
  }
})

import { IngestValidationError } from '@evlog/telemetry/ingest'
import { parseTelemetryBody } from '@/lib/telemetry-ingest'
import { storeRunEvents } from '@/lib/telemetry-store'

export async function POST(request: Request) {
  const raw = await request.text()
  try {
    const events = parseTelemetryBody(raw)
    await storeRunEvents(events)
    return new Response(null, { status: 204 })
  } catch (err) {
    const status = err instanceof IngestValidationError ? 400 : 500
    return Response.json({ error: 'invalid payload' }, { status })
  }
}

import { Hono } from 'hono'
import { IngestValidationError } from '@evlog/telemetry/ingest'
import { parseTelemetryBody } from '../lib/telemetry-ingest'
import { storeRunEvents } from '../lib/telemetry-store'

const app = new Hono()

app.post('/api/telemetry/ingest', async (c) => {
  const raw = await c.req.text()
  try {
    const events = parseTelemetryBody(raw)
    await storeRunEvents(events)
    return c.body(null, 204)
  } catch (err) {
    const status = err instanceof IngestValidationError ? 400 : 500
    return c.json({ error: 'invalid payload' }, status)
  }
})

export default app

Step 3 — Store idempotently

Retries and backlog drains can POST the same idempotencyKey twice. Upsert (or skip) so metrics stay correct:

import type { RunEvent } from '@evlog/telemetry'

export async function storeRunEvents(events: RunEvent[]): Promise<void> {
  for (const run of events) {
    await db.telemetryRun.upsert({
      where: { idempotencyKey: run.idempotencyKey },
      create: {
        tool: run.tool.name,
        version: run.tool.version,
        command: run.command,
        outcome: run.outcome,
        durationMs: run.durationMs,
        custom: run.custom,
        recordedAt: run.timestamp,
      },
      update: {},
    })
  }

  // Or forward into evlog's drain pipeline for Axiom / Datadog / OTLP:
  // await ingestWideEvents(events.map(run => ({ source: 'telemetry', ...run })))
}

Step 4 — Rate-limit at the edge

parseIngestBody() is your last line of schema defense, not your only one. Add rate limits where traffic enters — CDN, API gateway, or middleware — especially per IP. Optional: also bucket by machineId hash inside the handler if you need tighter control.

Non-2xx responses keep events in the user's outbox; they will retry on the next CLI invocation. Return 204 (or any 2xx) only when storage succeeded.