Precomputing Now Takes OpenTelemetry and Lets a Running File Change Its Policy

Precomputing 0.3 is out. It closes the four gaps that stood between it and a real deployment. Applications and OpenTelemetry Collectors can send logs and metrics straight to precomputing serve over OTLP/HTTP. The server speaks HTTPS. A billing stream can refuse a report from a gateway whose clock runs ahead. And a file that has been running for months can take a new version of its policy without starting over.

The last one was the biggest obstacle. Until now a file kept the policy it was made with, so adding a daily rollup or one more precompute meant a new, empty file and losing what the old one held. Most teams would want to change a policy within the first week.

OpenTelemetry In

precomputing serve now answers POST /v1/logs and POST /v1/metrics, the paths every OTLP/HTTP exporter sends to, in protobuf or JSON, gzipped or not. An OpenTelemetry Collector needs only an otlphttp exporter pointed at it, with a write token.

Log records go to the streams from logs, the same ones that read log lines. Their attributes become fields, with dots read as underscores, so http.route is http_route. The service comes from service.name, the level from the severity, and the template is learned from the body, as it is from a line. Metrics get a new kind of stream:

stream requests from metrics "http.server.request.duration" {
  key    service text
  key    http_route text
  value  count integer
  value  sum real
  rollup 1m keep 30d
}

Gauges and sums fill the stream’s value. Histograms fill values named count, sum, min and max. Most SDKs send cumulative totals, so the server keeps each series’ last point in the file and counts what changed since. A series that restarts counts from zero again. OTLP has no sequence numbers, so the server remembers every request it counted by its SHA-256 for an hour, and an exporter’s retry after a lost reply counts once.

The protobuf and JSON decoders are written by hand, so the binary still needs nothing outside Go’s standard library. To make sure they read what real exporters send, the tests decode requests written by the official OpenTelemetry protobuf classes, in both encodings, and check that both give the same records and points.

A File That Changes Its Policy

precomputing migrate --policy usage-v2.precompute --dry-run usage.db
precomputing migrate --policy usage-v2.precompute usage.db

A migration adds to a file and changes what happens from then on. New streams, rollups, precomputes and quotas start empty at the migration, and the file records when. How long raw events and windows are kept can change, and so can sample counts and anomaly thresholds. What a file has already counted can’t change. A different period for a billing stream, another key, a sketch accuracy or a removed rollup is refused with the reason, and nothing is written. The whole migration is one SQLite transaction, so one that fails part way leaves the file as it was.

The tests feed the same events through both runtimes, the compiled triggers and the Engine, migrate both files halfway, feed more, and require the two files to be identical. They also compare the old rollups with a file that was never migrated, to show a migration doesn’t disturb what was there.

Clocks That Run Ahead

A billing stream’s clock is the newest report it has counted. One gateway with a clock a year ahead used to be enough to make a month look closed, and every report after it was refused. A line ahead 10m in an exact stream now refuses any report stamped more than ten minutes past the meter’s own clock, as ahead in the refusal counts, and that report moves no clock at all. It’s the one rule that reads the wall clock, and it can be added to a running meter with migrate.

HTTPS

precomputing serve --tls-cert cert.pem --tls-key key.pem speaks HTTPS only, TLS 1.2 or later. The server checks the files at each new connection, so a certificate renewed in place is used without a restart, and a pair caught half written keeps the old one in use.

What the Review Found

An independent review went through all of it twice before the release, and most of what it found was in the OTLP decoder. The worst was a request of about a kilobyte that grew into a 33 MB attribute value, because nested arrays were quoted again at every level. A 29 KB gzipped request of 15 million empty metrics could make the server run out of memory. Both are fixed. Nested values are written as JSON once, and every message a request holds counts against one limit, so that same request is now refused in a third of a second with the server at 86 MB. The review also caught a stream added by a migration counting a series’ history from before it existed, and a new precompute that the migration called empty when it would in fact answer from the start of the file. Both are fixed, and each fix has a test.

Nothing else moved. The eight demos ran headless on the 0.3 code and the native files matched the browser’s as they did in 0.2.

The download page has the binaries for Linux, macOS and Windows, and the README has the commands.