Docket.

Benchmark conditions

These measurements cover Docket 3B and 8B. Decision accuracy and serving speed are separate studies. The accuracy tables use the same labelled items and scoring method for each compared model. Serving measurements identify the GPU, input length, precision, concurrency, and timing boundary.

Quality reference evaluation: October 3, 2026. H100 serving study: October 4, 2026.

Enterprise decisions

A sealed test of 3,063 questions across 10 enterprise workflows. The metric is workflow-weighted exact-label accuracy. Splits were grouped, and the test was opened after model selection. Accuracy was measured on documents up to about 3,000 tokens.

The difference intervals use paired group bootstrap. These results describe the evaluated enterprise workflows; they do not establish accuracy on a new business process.

JevBench

231 public items, with the same harness for every row. No JevBench item was used in training or model selection.

Generalisation and confidence

The held-out workflow uses a decision type excluded from training. The high-confidence error metric is the share of all labelled answers that were wrong while reported confidence was at least 0.9; it is not the error rate among only high-confidence answers.

Docket can be overconfident on unfamiliar decisions. On the unseen workflow, roughly a quarter of Docket’s labelled answers were wrong despite confidence of at least 0.9. The calibrated probabilities are useful signals, not proof that a decision is correct. Choose production thresholds using independently labelled examples from your own workflow.

HTTP latency

Hardware: one NVIDIA H100 SXM 80 GB (Hopper). Workload: one choice question over approximately 1,000 input tokens; the workload mean was 987.4 tokens. Serving: FP8, CUDA graph capture up to 2,048 tokens.

Timing is end-to-end HTTP with the client on the same host. It includes server handling but excludes external network travel. Requests are unique; cached-prefix speedups are excluded from the headline results. There were zero recorded request errors in these runs.

One request in flight delivers the lowest measured latency. Increasing concurrency raises throughput and adds queueing and batching latency. These short benchmark runs do not establish a production SLA.

Latency by prompt length

Same H100 SXM 80 GB, FP8, and 2,048-token graph capture. Prefix caching is disabled. These are engine-only p50 measurements; rendering, tokenisation, HTTP, and network time are excluded. Each length/model cell contains 50 measurements. The nominal 1,000-token condition averaged 988.1 tokens.

Serving throughput

Hardware: one NVIDIA L40S 48 GB, with the serving process limited to four vCPUs. Workload: approximately 1,000 input tokens per decision. Precision: FP8.

Costs are a study estimate using a GPU rate of $2.61/hour at 70% average utilisation, not a current provider quote. The denominator is one million input tokens.

At one request in flight, the H100 measured 17.8 ms versus the L40S’s 43 ms for 3B, and 27.2 ms versus 87 ms for 8B. Hardware and serving configuration are part of the result; another GPU is not assumed to produce the same latency.

FP8 and CUDA graph improvements

At eight requests in flight on the L40S, FP8 improved throughput relative to bf16:

FP8 and bf16 accuracy matched in the checks performed. This is not a claim that every future workload or quantisation configuration will have identical accuracy.

On the H100, enlarging CUDA graph capture from the default short-prefill range to 2,048 tokens reduced the 250-token 3B bf16 engine median from 13.7 ms to 8.7 ms. For approximately 1,000 tokens, the optimized FP8 engine medians were 14.6 ms for 3B and 23.3 ms for 8B. Full-plus-piecewise graph mode added no measurable improvement beyond the larger capture size.

Several questions, one state

The original H100 FP8 in-process run scored three questions concurrently and reused their shared state prefix. Prefix caching was enabled. These numbers include preparation and engine work, but exclude HTTP and external network time. The three-question condition contains 35 measurements per model.

This is a prefix-reuse scenario, not a cold independent-request latency claim. Keep reusable instructions or shared context at the start of the state when your workload permits it.

Deployment boundaries

• Your workflow: validate on your own labelled inputs, then set confidence thresholds and review paths.

• Input length: both model sizes default to an 8,192-token total serving context in vLLM, with one answer token reserved (up to 8,191 tokens for the complete rendered prompt). The context, question, instructions, options, and chat formatting all count. Longer-document evaluations use the 8K configuration; the accuracy figures reported on this page were evaluated up to about 3,000 tokens. Headline latency was measured around 1,000 tokens.

• Language: the reported decision-quality evaluations cover English.

• Untrusted text: Docket is not a security boundary. Do not treat a model decision as a substitute for deterministic access controls.

• Timing: local engine, local HTTP, and remote end-to-end latency are different measurements. The tables label the boundary for each study.

See the API reference for request shapes, limits, and serving options.