Calametra deterministic testing
Deterministic testing for devices, drivers and distributed systems

The bug happens in exactly one order. Make that order repeatable.

Calametra runs your real code against a stateful model of the device on a single-threaded virtual clock. Same specification, same scenario, same seed, and the trace hash comes back identical every time. A race stops being a story about a flaky rig and becomes a file you can replay.

Trace reproduction
byte-exact

A replay matches the recorded hash, or the run is reported as failed.

Fault primitives
6

Matchable at any transition your device model defines.

Failure minimisation
automatic

Delta debugging cuts a failing run down to its shortest form.

Hardware required
none

Every run is offline, isolated, and repeatable on any machine.

Fig. 1 Signal path nothing is stubbed at the boundary your code sees model side return path
device specification scenario seed fault schedule CALAMETRA CORE virtual clockevent scheduler stateful device modelinvariants and coverage delta debugging framed socket bridge CALAMETRA SANDBOX transport adapter host stack system services your real code nothing runs against your machine or its hardware observed output returned to the model and checked run manifest trace (content-addr.) semantic diff minimised failure one seed · one ordering · one trace hash
Scenario
stale-status-unplug-race
Seed
0x4F21A9
Orderings explored
48 · 1 fail
Replay
10/10 identical
01 Method

Search the orderings, then cut the failure to the bone

Exploration walks fault sites and seeded schedules until an invariant breaks. Delta debugging then removes every event that is not load bearing, until what remains is short enough to read aloud.

Input

Specification

Interfaces, commands, state transitions and a fault schedule, versioned as plain data.

Core

Virtual clock

One thread. Simultaneous events are ordered by seed, never by luck.

Device

Stateful model

Memory, configuration, readiness and output state. Transitions, not canned replies.

Bridge

Transport bridge

A framed socket carries every request and reply between the model and the sandbox.

Real code

Your software

Your driver, firmware or service runs unmodified, and its real output comes back.

Find it

$ calametra explore stale-status-unplug-race

explore  fault sites              24
explore  seeded schedules         48
hit      seed 0x4F21A9
fail     device_state_matches_model
         stale status accepted after
         the device went away

Shrink it

$ calametra minimize && calametra replay

delta    48 to 12 to 5 to 3 events
minimal  read status · disconnect
         · read status
replay   36/36 records reproduced
hash     sha256:9c41…e07 identical
02 Coverage

Bring the thing that keeps failing once a fortnight

Calametra is device agnostic. A device family is described once as a specification, and the engine, transport, faults, traces and exploration know nothing about any particular product.

You testWhat Calametra modelsWhat you get back
Device driversEnumeration, configuration, command framing, bounded memory writes and output statereplayable trace
Embedded firmwareUpdate and recovery flows, partial writes, aborted transfers and malformed repliesminimised failure
Connected hardware fleetsAttach and detach races, stale readings, retries and timeout cascadesfailing seed
Distributed servicesMessage ordering, partial delivery, delayed and duplicated responsessemantic diff
Protocol implementationsA published class specification used as ground truth, so a mismatch is a real defectconformance verdict
Regression suitesEvery past failure kept as a scenario, a seed and an expected trace hashfails every time
03 Boundary

An honest instrument names what it cannot see

These limits are how the product actually behaves. We would rather lose a sale than hand you a green result we cannot stand behind.

Guaranteed

  • Isolation. Your code runs inside a disposable sandbox, never against your machine or its hardware.
  • Reproduction. A replay is byte exact, or the run is reported as failed.
  • Refusal. When the environment cannot establish a result, Calametra records that it could not, and reaches no verdict.
  • Comparison. Observations captured from real hardware can be imported and compared against the model.

Never

  • No physical access. No device on your desk is touched, and none is passed into the sandbox.
  • No changes to your machine. Nothing is installed, loaded, or altered outside the sandbox.
  • No recorded tape. Determinism comes from the virtual clock, not from replaying a capture.
  • No certification. A passing run is evidence about your model and your code, not a hardware qualification.