Deployment

One configuration flag on the engine, one subscriber process per engine, and one 64 byte key file. Nothing is installed in the serving path.

How you get the binary

There is no public download, and this page is not a self-serve install. The tap is closed source and binary vended: release credentials are issued as part of an engagement rather than handed out from a download page. Read this to know what deploying it would involve and what your security review would be reviewing. It is written to be useful before you have the artifact, not only after.

What arrives is one tarball per release, infertap-<tag>-x86_64-linux.tar.gz: the binary, the systemd unit files whose headers are themselves the install instructions, and an example environment file. A deployment needs no checkout of any repository of ours. Beside it travel a CycloneDX SBOM and SHA256SUMS. A signed container image is published the same way.

Every artifact carries a cosign signature made by a key that never leaves KMS and answers only to tag builds. You verify origin first and integrity second, against a public key delivered once and out of band, never beside the artifact it verifies. That verification runs offline, with the public transparency log deliberately opted out of, which is what lets an air gapped site verify a release without reaching anything of ours at install time.

Engine contract

The engine must publish KV cache events. In vLLM that is one flag, and it forks nothing:

vllm serve MODEL \
  --kv-events-config '{"enable_kv_cache_events": true, "endpoint": "tcp://*:5557"}'

The endpoint address is the common first mistake. Binding tcp://127.0.0.1:5557 publishes to loopback only. If the subscriber runs in a different network namespace, which it does under a container runtime or an orchestrator, it connects successfully and receives nothing forever. The publisher reports no error, because from its side nothing is wrong.

Hash algorithm

The default is fine. --prefix-caching-hash-algo sha256_cbor is more reproducible across environments and vLLM's own prefix caching documentation recommends it for that reason. It is not required here: fleet wide block identity is derived from block contents rather than from the engine's hash, so it does not depend on the engine's algorithm or seed.

systemd

The shipped unit is packaging/systemd/infertap.service, and every hardening choice in it exists to enforce a claim on the trust page:

Setting	Effect
`DynamicUser`, empty capability set	Runs unprivileged with no capabilities to drop later
Read only filesystem, one writable path	`/var/lib/infertap` holds the record stream and is the only thing it can write
`LoadCredential` plus `--key-file`	The key is never on argv and never in the environment
`MemoryMax=1G`, `TasksMax=16`, `LimitNOFILE=256`	The bounds are enforced a second time at the operating system layer
`Restart=on-failure`	A restart re resubscribes. A publisher sequence reset is not treated as a drop, and an abrupt stop leaves at most one truncated final line

Container

packaging/docker/Dockerfile builds a minimal Debian image with libzmq5 as the only added library, running as a non root user. There is no EXPOSE, because there is no listener. Run it with the least privilege the runtime offers:

docker run --read-only --cap-drop=ALL --security-opt no-new-privileges \
  -v /etc/infertap/key:/run/secrets/key:ro \
  -v /var/lib/infertap:/var/lib/infertap \
  infertap --endpoints i0=tcp://engine:5557 \
           --out-dir /var/lib/infertap --key-file /run/secrets/key

Continuous integration builds the image and runs --show-payload inside it on every push, so the published image is checked against the same output the trust page describes.

Under an orchestrator

There is no manifest in the repository yet. The shape below is guidance rather than a shipped artifact, and it follows from three properties of the process rather than from a preference.

Run it as a native sidecar

One subscriber per engine process, in the engine's own pod. Kubernetes native sidecars, meaning an init container with restartPolicy: Always, are the right shape: the subscriber starts before the engine and is torn down after it, so the window where the engine publishes to nobody is as small as the platform allows. A plain second container is racier in both directions.

Sharing the pod also solves the endpoint problem above. The engine can publish to tcp://127.0.0.1:5557 and stay off the pod network entirely, because the subscriber is in the same network namespace.

Liveness only, no readiness

The subscriber serves no traffic, so it has nothing to be ready for. A readiness probe on it can only do harm: it gates the pod on a process that is not in the request path. A liveness probe is appropriate, and it should check that the process is alive rather than that it is receiving events. A quiet engine is a legitimate state, and a probe that treats silence as failure will restart the subscriber precisely when there is nothing to record.

Sizing the volume

An emptyDir is the natural home for the record directory in a pod, and it must be sized against the retention cap rather than against the expected rate. Peak disk is --max-disk-bytes plus one active segment, and orphans left by a crash sit outside the cap because the process will not delete data it never got to ship. Size the volume to the cap plus a margin, and set the cap rather than leaving it at the 16 GiB default if the volume is smaller.

An emptyDir is lost when the pod is. Records that have not shipped when a node drains are gone, which is bounded by the shipping interval rather than by the retention cap. That is the tradeoff for keeping the subscriber in the engine's pod, and it is usually the right one, because a subscriber outside the pod cannot use loopback and cannot be lifecycle bound to the engine.

A full sizeLimit evicts the whole pod, engine included. If the volume carries a sizeLimit and the record directory fills past it, the kubelet does not fail a write. It evicts the pod, and your engine restarts because a monitoring sidecar wrote too much. Keep --max-disk-bytes comfortably under sizeLimit: the gap has to hold one active segment plus any orphans from earlier crashes, so give it a couple of segments of headroom, not a few kilobytes.

Prefer medium: Memory for the volume. It removes disk contention with model loading and checkpoints, and a tap should not be doing disk I/O on a GPU node anyway. One rule comes with it: a memory backed volume counts against the pod's memory limit, so the limit must cover sizeLimit plus everything else in the pod with headroom, or filling the volume becomes an out of memory kill instead of an eviction.

Set the cap explicitly in pods rather than inheriting the default. 16 GiB is sized for a host disk that also survives long shipping outages. In a pod, where the volume competes with memory or triggers eviction, a few GiB with a working shipper is the right shape, and a shipping outage longer than that buys is better handled by fixing the shipper than by holding gigabytes of records inside an engine pod.

Provisioning the key

Records are pseudonymized before they leave the host, under a key you generate and hold. Every instance in a fleet must be provisioned from the same key file, because identities derived under different keys cannot be compared and every cross instance figure would read zero.

openssl rand -hex 64 > /etc/infertap/key
chmod 400 /etc/infertap/key

The file is exactly 64 bytes, hex or base64: a 32 byte key followed by a 32 byte nonce. Any other length is refused at startup, and so are two identical halves, which is what generating 32 bytes and pasting it twice produces. Both are fail closed rather than warnings, because a key file that is subtly wrong produces records that look correct and compare against nothing.

Distribute it the way you already distribute secrets to inference nodes. The process needs no other credential, and rotating it is covered under Operations.

First run

Before pointing it at an engine, look at what it emits:

infertap --show-payload

It prints every record the process can emit, pseudonymized under a labeled test key, with no engine required, and ends with the egress audit count. That output is byte compared against the live path by test, so what it shows is what ships. The record format section of the contract is the complete field enumeration behind it.