Module testing/bench
Micro-benchmarking utilities.
Runs a function N times, measures elapsed nanoseconds, and returns statistics (average, min, max).
Example
{ bench } :: import "std/testing/bench";
result := bench(`sort`, u64(1000), () => { /* work */ });
println(result.to_string());
Stability
unstable — BenchResult's statistics are total/average/min/max, and that
is not the set a benchmark harness ends up wanting. A mean over a
distribution with scheduler outliers is the wrong summary; Rust's libtest
reports a median and a deviation, and criterion a confidence interval.
Adding those means adding public fields to a value struct, which is not an
additive change, so the shape has to settle before the module is frozen.
What avg_ns MEANS is the second open question. bench reads the clock
twice per iteration, so every number here includes two Instant.now()
calls of per-iteration overhead — for a trivial body that is the whole
measurement, which is exactly why bench_auto had to calibrate against
wall time instead. Timing a BATCH and dividing would be more accurate and
would change every reported figure without changing a single signature:
precisely the silent semantic shift a stability promise is meant to
prevent.
black_box is separately constrained rather than unsettled — it is
scalar-only because the aggregate route is blocked by two compiler
defects, and widening it later is additive. Freezing follows the
statistics set and the per-iteration/per-batch decision.
Types
Timing statistics returned by bench.
Fields
| Name | Type | Description |
|---|---|---|
name | String | Benchmark name. |
iterations | u64 | Total number of iterations executed. |
total_ns | i64 | Total elapsed nanoseconds across all iterations. |
avg_ns | i64 | Average nanoseconds per iteration. |
min_ns | i64 | Minimum nanoseconds for a single iteration. |
max_ns | i64 | Maximum nanoseconds for a single iteration. |
Trait Implementations
impl(generic(T : Type), where(T <: ToString), T : (ToString))
impl(generic(T : Type), where(T <: ToString), T : (ToString), Format)
format : fn(self : Self, spec : str) -> StringRender self under spec. An unrecognised spec degrades to the plain
to_string() rendering rather than failing.
Parameters
| Name | Type | Notes |
|---|---|---|
self | Self | |
spec | str |
Returns: String
impl(BenchResult, ToString(...))
to_string : (BenchResult) fn(self : BenchResult) -> StringRender self as the text a USER should read — Rust's Display::fmt,
not its Debug. Hand-written (or generated by derive(Error) from a
per-variant format string) whenever the structural form would be wrong.
Parameters
| Name | Type | Notes |
|---|---|---|
self | BenchResult |
Returns: String
Functions
Hide v from the optimizer and hand it back — Rust's black_box.
Without it a benchmark whose result is discarded measures NOTHING, because
the whole computation is dead code. Measured on this tree, 20 000 calls to
a 2 000-iteration loop at --optimize 2:
| body | time |
|---|---|
_r := work(i); |
0 ns — the loop was deleted outright |
black_box(work(i)); |
39 273 000 ns |
The barrier is an empty asm that takes the value in a REGISTER with a
"memory" clobber. A version that instead stored the value's address into
a module-level global was measured too and does NOT work — both loops came
back at 0 ns, because a store to a never-read global is itself dead, and
there is no volatile in the language to say otherwise
(issues/no-volatile-so-black-box-needs-inline-asm.md). On the wasm targets,
which have no inline assembly, this is a NO-OP and elision-sensitive
microbenchmarks there cannot be trusted.
v must be a scalar
Primitive numeric, pointer or bool — the compiler enforces it ("asm in() value type is not valid for inline assembly"), which covers the value a benchmark body normally produces. An aggregate or RC type would have to go in by ADDRESS, and that route is blocked twice over:
¶minside a generic fn emits a placeholder into the C (issues/address-of-a-parameter-in-a-generic-fn-emits-a-placeholder.md), so the address has to be taken of a local copy; and- copying an RC value into a local, taking its address, and returning it
DOUBLE-DROPS the inner buffer — found by CI's Linux ASan leg on the first
cut of this function,
heap-use-after-free in __yo_decr_rc(issues/rc-value-copied-address-taken-and-returned-is-double-dropped.md).
So this ships scalar-only rather than shipping a double free. To keep an
aggregate alive across a benchmark, black_box a scalar derived from it —
black_box(xs.len()), black_box(s.byte_at(0)).
Type Parameters
| Name | Type | Notes |
|---|---|---|
T | Type | comptime |
Parameters
| Name | Type | Notes |
|---|---|---|
v | T |
Returns: T
Run body for iterations times and return timing statistics.
Parameters
| Name | Type | Notes | Description |
|---|---|---|---|
name | String | Benchmark name. | |
iterations | u64 | Total number of iterations executed. | |
body | Impl(Fn() -> unit) |
Returns: BenchResult
bench, with the iteration count CHOSEN for you.
Escalates a pilot run until it lasts at least _PILOT_FLOOR_NS, then
extrapolates the count that should fill _AUTO_TARGET_NS. Removes the two
ways a hand-picked count goes wrong: too few iterations on a fast body, so
the result is clock granularity rather than the body; far too many on a
slow one, so the suite stalls.
println(bench_auto(`sort`, () => sort_a_thousand()).to_string());
The pilot is a real bench call timed by WALL clock, not a bare loop over
body. That matters more than it looks: bench reads the clock TWICE per
iteration, and for a trivial body those two reads are the entire cost. A
first version timed only body, measured ~0 for () => (), and so asked
for the maximum 100 000 000 iterations — a 3.4 s run on a fast machine and
minutes on a CI runner under sanitizers. Extrapolating from the wall time of
the thing actually being repeated cannot make that mistake.
Parameters
| Name | Type | Notes | Description |
|---|---|---|---|
name | String | Benchmark name. | |
body | Impl(Fn() -> unit) |
Returns: BenchResult