PROJECT 001 · OPEN SOURCE · APACHE-2.0

Docket — Benchmarks.

Fast models for decision workloads.
Docket-3B for GPUs, Docket-3B-GGUF for Macs and CPUs.

Benchmark conditions

These measurements cover Docket 3B and its GGUF release. Decision accuracy and serving speed are separate studies. The accuracy tables use the same labelled items and scoring method for each compared model. Serving measurements identify the GPU, input length, precision, concurrency, and timing boundary.

Decision-quality evaluations use bf16. The separate H100 serving reference study uses weights that differ from those used in the quality tables. FP8 quality, latency and cost have not been remeasured for the models in the quality tables.

Enterprise decisions

A sealed test of 3,063 questions across 10 enterprise workflows. The metric is workflow-weighted exact-label accuracy. Splits were grouped. Adapter selection and calibration used development data, with weights and temperatures fixed before scoring. These tests include previously evaluated held-out sets; their reported results informed the choice of models presented here. The enterprise test uses documents up to about 3,000 tokens.

Model Base model accuracy Docket accuracy Difference vs. Jev
Docket 3B 55.6% 90.1% +3.6 points (95% CI +2.0 to +5.3)
Jev reference — 86.4% Reference

The difference intervals use paired group bootstrap. These results describe the evaluated enterprise workflows; they do not establish accuracy on a new business process.

JevBench

231 public items, with the same harness for every row. No JevBench item was used in training or development-time adapter selection.

Model Easy / 48 Original / 72 Hard / 111 Total / 231
Granite 3B base 48 51 54 153
Docket 3B 48 66 63 177
Jev reference 48 70 80 198

Confidence

The high-confidence error metric is the share of all labelled answers that were wrong while reported confidence was at least 0.9; it is not the error rate among only high-confidence answers.

Model Wrong at ≥0.9 confidence, enterprise test
Docket 3B 1.18%
Jev reference 1.14%

The calibrated probabilities are useful signals, not proof that a decision is correct. Choose production thresholds using independently labelled examples from your own workflow.

GGUF release

Docket-3B-GGUF is a Q8_0 quantisation of the same Docket 3B weights for llama.cpp, with the same calibration and no retraining. The weight file is 3.89 GB, versus 7.32 GB for BF16.

A paired comparison scored 1,326 fixed questions with both formats: enterprise, reasoning, logical verification, and public-dataset items. Docket-3B-GGUF gave the same answer as Docket-3B (BF16) on 98.87% of questions. This is prediction agreement with the 3B reference model, not task accuracy: the two models can agree on an incorrect answer.

Workflow-weighted accuracy against the labelled answers was 85.26% for Docket-3B-GGUF and 85.28% for Docket-3B (BF16). This subset measures the effect of quantisation; it does not replace the full evaluations above. The comparison ran on an NVIDIA GPU; Mac latency has not been measured.

Serving reference study

The following measurements examine GPU configurations and serving optimisations. They use reference weights, separate from the bf16 models in the quality tables above. They are not latency, cost, or FP8 accuracy measurements for those models. Remeasure the selected weights and calibration before using these figures as deployment estimates.

Reference rows identify the parameter size; they do not change the model-quality results.

HTTP latency

Hardware: one NVIDIA H100 SXM 80 GB (Hopper). Workload: one choice question over approximately 1,000 input tokens; the workload mean was 987.4 tokens. Serving: FP8, CUDA graph capture up to 2,048 tokens.

Timing is end-to-end HTTP with the client on the same host. It includes server handling but excludes external network travel. Requests are unique; cached-prefix speedups are excluded from the headline results. There were zero recorded request errors in these runs.

Model Requests in flight Mean input tokens p50 p95 p99 Requests/s
3B reference run 1 987.2 17.8 ms 18.5 ms 20.3 ms 55.9
3B reference run 2 987.0 26.2 ms 27.4 ms 32.8 ms 75.8
3B reference run 4 987.5 43.1 ms 43.7 ms 44.8 ms 92.9

One request in flight delivers the lowest measured latency. Increasing concurrency raises throughput and adds queueing and batching latency. These short benchmark runs do not establish a production SLA.

Latency by prompt length

Same H100 SXM 80 GB, FP8, and 2,048-token graph capture. Prefix caching is disabled. These are engine-only p50 measurements; rendering, tokenisation, HTTP, and network time are excluded. Each length contains 50 measurements. The nominal 1,000-token condition averaged 988.1 tokens.

Target prompt tokens Measured mean tokens 3B engine p50
250 249.9 8.8 ms
500 499.9 9.8 ms
1,000 988.1 14.6 ms
2,000 2000.9 23.9 ms

Serving throughput

Hardware: one NVIDIA L40S 48 GB, with the serving process limited to four vCPUs. Workload: approximately 1,000 input tokens per decision. Precision: FP8.

Costs are a study estimate using a GPU rate of $2.61/hour at 70% average utilisation, not a current provider quote. The denominator is one million input tokens.

Model Requests in flight Requests/s HTTP p50 Estimated $ / M input tokens
3B reference run 1 23.4 43 ms $0.045
3B reference run 4 32.6 123 ms $0.032
3B reference run 8 35.1 247 ms $0.030

At one request in flight, the H100 measured 17.8 ms versus the L40S’s 43 ms for 3B. Hardware and serving configuration are part of the result; another GPU is not assumed to produce the same latency.

FP8 and CUDA graph improvements

At eight requests in flight on the L40S, FP8 improved throughput relative to bf16:

Model bf16 requests/s FP8 requests/s Improvement
3B reference run 26.8 35.1 ~31%

FP8 and bf16 accuracy matched in the reference-weight checks performed; the selected weights have not had this comparison. This is not a claim that every future workload or quantisation configuration will have identical accuracy.

On the H100, enlarging CUDA graph capture from the default short-prefill range to 2,048 tokens reduced the 250-token 3B bf16 engine median from 13.7 ms to 8.7 ms. For approximately 1,000 tokens, the optimized FP8 engine median was 14.6 ms for 3B. Full-plus-piecewise graph mode added no measurable improvement beyond the larger capture size.

Several questions, one state

The original H100 FP8 in-process run scored three questions concurrently and reused their shared state prefix. Prefix caching was enabled. These numbers include preparation and engine work, but exclude HTTP and external network time. The three-question condition contains 35 measurements.

Model One question p50 Three questions, same state p50
3B reference run 17.9 ms 27.3 ms

This is a prefix-reuse scenario, not a cold independent-request latency claim. Keep reusable instructions or shared context at the start of the state when your workload permits it.

Deployment boundaries

  • Your workflow: validate on your own labelled inputs, then set confidence thresholds and review paths.

  • Input length: Docket 3B defaults to an 8,192-token total serving context in vLLM, with one answer token reserved (up to 8,191 tokens for the complete rendered prompt). The context, question, instructions, options, and chat formatting all count. The enterprise set uses documents up to about 3,000 tokens. The selected quality and broader studies use an 8K limit, without establishing accuracy at every length. Reference latency was measured around 1,000 tokens with separate weights. A configured context limit is not evidence of accuracy at every input length.

  • Language: the reported decision-quality evaluations cover English.

  • Untrusted text: Docket is not a security boundary. Do not treat a model decision as a substitute for deterministic access controls.

  • Timing: local engine, local HTTP, and remote end-to-end latency are different measurements. The tables label the boundary for each study.

See the API reference for request shapes, limits, and serving options.