PROJECT 001 · OPEN SOURCE · APACHE-2.0

Docket — API reference.

Fast models for decision workloads.
Docket-3B for GPUs, Docket-3B-GGUF for Macs and CPUs.

Request and response format

The request and response format follows the TypeSafe System One API, so System One clients work against docket-serve by changing the base URL.

Request

POST /v1/systemone (also served at /api/alpha/decisions)

{
  "model": "optional, ignored",
  "state": "any text, or a JSON object/array (rendered as indented JSON)",
  "questions": {
    "<question id>": {"type": "choice", "instructions": "...", "criteria": {"<key>": "<one-line description>", "...": "..."}},
    "<question id>": {"type": "noul",   "instructions": "...", "criteria": {"true": "optional: when to answer yes", "false": "optional"}},
    "<question id>": {"type": "score",  "instructions": "...", "criteria": ["level 0 description", "level 1 description", "..."]}
  }
}

Limits: - choice: 2 to 255 options. Descriptions are optional but improve accuracy. - score: 2 to 10 levels, ordered lowest first. - Questions: at least one per request. All questions about one state are scored concurrently and share its cached prefix, so extra questions are cheap. - Length (vLLM): Docket-3B defaults to an 8,192-token total context via --max-model-len 8192. One token is reserved for the answer, leaving at most 8,191 prompt tokens per question. Count the complete rendered prompt: state (text or structured data), question, instructions, options, and Granite chat formatting. Longer requests get HTTP 422; nothing is truncated silently. The portable Transformers (hf) backend allows 8,192 prompt tokens; this backend currently uses its own fixed prompt limit rather than --max-model-len. Quality evaluations use the 8K configuration; the enterprise test uses documents up to about 3,000 tokens. Reference GPU timings use approximately 1,000 tokens and separate weights, so they are not latency guarantees for these models.

Response

{
  "model": "docket-3b",
  "answers": {
    "<id>": {"type": "choice", "choice": "<key>", "confidence": 0.0, "probabilities": {"<key>": 0.0}},
    "<id>": {"type": "noul", "noul": 0.0},
    "<id>": {"type": "score", "score": 1.7, "confidence": 0.0, "legend": {"0": "..."}, "probabilities": {"0": 0.0}}
  },
  "usage": {"input_tokens": 512, "output_tokens": 3}
}
Field Meaning
noul P(yes)
choice the most probable key
confidence for choice: (p_max - 1/K) / (1 - 1/K) (0 = uniform, 1 = certain); for score: 1 - std / std_max
score the probability-weighted level index
probabilities calibrated with the per-type temperatures in docket_calibration.json (start the server with --no-calibration for raw probabilities)
usage.input_tokens prompt tokens processed
usage.output_tokens number of questions answered

Errors. Validation and length errors return HTTP 422 with {"detail": [{"loc": ["body"], "msg": "...", "type": "value_error"}]}.

Other endpoints

Endpoint Returns
GET /v1/models the served model name
GET /healthz, GET /health {"ok": true, "model": ..., "requests": n, "questions": n, "input_tokens": n}

The response header x-docket-latency-ms gives the server-side time per request.

Server options

docket-serve --model PATH_OR_HF_ID [--backend vllm|hf] [--port 8080] [--quantization fp8]
             [--max-model-len 8192] [--max-num-seqs 256] [--gpu-memory-utilization 0.9]
             [--cudagraph-max-tokens 2048] [--name docket-3b] [--no-calibration]

Environment equivalents: DOCKET_MODEL, DOCKET_NAME, DOCKET_BACKEND, DOCKET_QUANTIZATION, DOCKET_CUDAGRAPH_MAX_TOKENS, PORT.

Precision. Start with bf16, matching the model-quality evaluations. FP8 is supported on compatible GPUs; validate accuracy and calibration before changing precision. The selected weights have not had FP8 quality or serving economics remeasured.

Serving optimisations. CUDA graphs capture prefills up to 2,048 tokens by default. Continuous batching and shared-prefix caching can improve throughput. Concurrency adds per-request latency, so measure the tradeoff for your workload.

See the serving reference study for the H100 and L40S measurements, including tokens, precision, concurrency and timing boundaries. Those runs used separate reference weights.