ResearchMeasured on hardware · Cited to source

Research.(8)

Every figure here is cited to a measurement, an engine pull request, vendor documentation or a paper. Where no primary source exists for a claim, that is stated rather than filled in. The ideas that did not work stay published.

Anyone can call a model. Making one work reliably, at low cost, on real hardware is the part that has to be measured, and the part where a simulation is only as right as its cost model.

01

Notes(8)

From the Engine’s docs. Newest first.
  1. 01

    A serving layer that admits on predicted end-to-end time, from a cost model that never saw the output length, refuses almost everything on a slow decoder, and refuses more the longer it runs. Observed on a DGX Spark at 31.6 tok/s. Fixed with an output-aware cost model and a queue-wait SLO.

    InferenceMeasurement
  2. 02

    The first non-simulated measurement in this work. N-gram self-speculation on an RTX 5070 Ti: 556 of 598 drafted tokens accepted, 93.0% overall, and 100% at the first five positions. The acceptance curve, not the block size, is what decides the win.

    InferenceMeasurement
  3. 03

    Same machine, same model, eight prompts. SQL 3.5×, code 1.5×, and zero drafted tokens on JSON, number lists and prose. Whether to speculate is where the value is, and the drafter makes that call correctly on its own.

    InferenceMeasurement
  4. 04

    Decode on a DGX Spark runs at about 80% of its 273 GB/s memory bandwidth, inferred from tokens per second times bytes per token rather than read from a datasheet. The highest fraction of peak of any comparable machine, and the constraint that bounds every host-side optimisation to the remainder.

    InferenceMachines
  5. 05

    Only things that move fewer bytes per token help. Quantising the KV cache is the largest lever and is off by default: +48% for about 0.002 perplexity, citing the upstream measurement. Four-bit weights are solved; one popular format is worth less than nothing at decode.

    InferenceModels
  6. 06

    Every item is arithmetic about batching, not a theory about text. One request using penalties drags an entire vLLM batch onto the expensive sampling path, including every request that asked for none. Invisible from inside a single request; sharp from above it.

    Inference
  7. 07

    A rigorous simulation, with an oracle bound, seed averaging and 179 passing tests, said adaptive block sizing beats a constant by 152%. Hardware said a 16× range in block size moves throughput about 5%. A simulation is only as right as its cost model. Kept because the negative result stops the idea being rebuilt.

    Measurement
  8. 08

    A survey against llama.cpp, vLLM, SGLang and unsloth, every claim checked against source, release notes or a paper, version-specific numbers labelled as such. It decides what belongs in the layer and what belongs in a backend adapter.

    Inference

02

Method(5)

  1. 01

    Measure on hardware

    A simulation is only as right as its cost model, and the only test of a cost model is measurement. Numbers on this site are cited to the file they came from.

  2. 02

    Publish the negative result

    An idea that was measured to do nothing stays documented, so it is not rebuilt. Absence is not evidence.

  3. 03

    Refuse rather than substitute

    An unknown backend, an unlicensed drafter, a tool call outside the guard: each is an error that says what was valid, never a quiet fallback.

  4. 04

    Licences in code, not comments

    Which checkpoints an engine may load and which extensions a store may use are enforced where they are used. Both projects are Apache-2.0.

  5. 05

    Open source is complete on its own

    The Harness runs on your endpoints, your database and your storage. The Cloud is the managed side of the same app: the config gets shorter, the code does not change.

03

The code is where the notes end up.

A note that survives measurement becomes a module in Praecise Engine. One that does not is kept as a document, so the idea is not rebuilt.

praecise-engine on GitHub