How this report is presented
Each example below shows, for every detector, four things: the
expected ground-truth label, the actual detector behaviour
(fired / silent), the derived verdict, and the raw output the
engine actually emitted (the literal Warn*.csv rows). The raw
output is the truth — it is what the detector printed, not a
re-description of it.
Ground-truth label vocabulary.
FIRE — the bug class is genuinely present, the detector must
warn. SILENT — the bug is intentionally absent, the detector
must stay silent (negative example).
FALSE_POS — a documented engine limitation: a reset-blind
detector is expected to fire here, so the fire is the known, non-buggy
manifestation and is counted as a correct positive (reported separately as a
"limitation"). N/A — the detector is structurally unable to
observe this bug (e.g. Engine A has no MissingReset rule); these rows are
excluded from the metrics denominator.
Precision & recall — methodology
A detector is scored only on rows where it is in scope (it is implemented by
its engine and the label is not N/A). "Fire" is the positive
class. For each in-scope row we compare the expected behaviour to the actual
firing:
- TP — fired & expected-fire
- FP — fired & expected-silent (a genuine false alarm)
- FN — silent & expected-fire (a missed bug)
- TN — silent & expected-silent
FALSE_POS rows are treated as expected-fire, so their fire
counts as TP (not FP) — this measures each detector against its
documented as-built contract, exactly as the CI gate does. They are also
tallied in the "Limitation" column so the limitation is never hidden.
Precision = TP / (TP + FP); Recall = TP / (TP + FN).
Per-detector metrics
| Detector | Bug class | Engine | TP | FP | FN | TN | Precision | Recall | F1 | N/A excl. | Limitation |
|---|---|---|---|---|---|---|---|---|---|---|---|
| A_DoubleMeas | DoubleMeas | A | 1 | 0 | 0 | 2 | 100.0% | 100.0% | 100.0% | 7 | 0 |
| A_GhostCompose | GhostCompose | A | 1 | 0 | 0 | 1 | 100.0% | 100.0% | 100.0% | 8 | 0 |
| A_OpAfterMeas | OpAfterMeas | A | 4 | 0 | 0 | 0 | 100.0% | 100.0% | 100.0% | 6 | 1 |
| B_DoubleMeas | DoubleMeas | B | 1 | 0 | 0 | 2 | 100.0% | 100.0% | 100.0% | 7 | 0 |
| B_MissingReset | MissingReset | B | 2 | 0 | 0 | 6 | 100.0% | 100.0% | 100.0% | 2 | 0 |
| B_OpAfterMeas | OpAfterMeas | B | 3 | 0 | 0 | 1 | 100.0% | 100.0% | 100.0% | 6 | 0 |
Bug-class coverage of the eval set
| Bug class | Eval examples covering it |
|---|---|
| OpAfterMeas | basic_reset_initialised, ex01_op_after_meas_basic, ex02_op_after_meas_reset_safe, ex10_double_meas_2qubitgate |
| DoubleMeas | ex06_double_meas_basic, ex07_double_meas_reset_between, ex10_double_meas_2qubitgate |
| MissingReset | basic_reset_initialised, ex01_op_after_meas_basic, ex02_op_after_meas_reset_safe, ex06_double_meas_basic, ex07_double_meas_reset_between, ex10_double_meas_2qubitgate, ex11_missing_reset_basic, ex14_missing_reset_2qubit_gate |
| GhostCompose | ex15_ghost_compose_basic, ex16_ghost_compose_assigned |
Per-example results (raw detector output)
Basic OpAfterMeas (single qubit) OpAfterMeas
A gate (h) is applied to q0 after q0 was measured, with no reset between. Both engines should flag it.
Qiskit source (ex01_op_after_meas_basic.py)
from qiskit import QuantumCircuit # OpAfterMeas: a gate acts on a qubit after that qubit was measured, # with no reset between along the CFG path. qc = QuantumCircuit(2, 2) qc.reset(0) qc.reset(1) qc.h(0) qc.cx(0, 1) qc.measure(0, 0) qc.h(0) # BUG: gate on q0 after measure(0) -> OpAfterMeas
| Detector | Expected | Actual | Verdict | Raw detector output |
|---|---|---|---|---|
| A_OpAfterMeas | FIRE | FIRE | FIRE | 8 qc h 0 7 |
| A_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| A_GhostCompose | N/A | silent | N/A | (no warnings emitted) |
| B_OpAfterMeas | FIRE | FIRE | FIRE | 7 0 6 |
| B_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| B_MissingReset | SILENT | silent | SILENT | (no warnings emitted) |
Note: Resets at the top only initialise the qubit; the post-measurement gate is the bug.
OpAfterMeas negative: reset between measure and gate OpAfterMeas
A reset lies between the measure and the later gate, so the gate operates on a re-initialised qubit. A reset-AWARE detector (Engine B) must stay silent; the reset-BLIND Engine A falsely fires -- a documented limitation.
Qiskit source (ex02_op_after_meas_reset_safe.py)
from qiskit import QuantumCircuit # NEGATIVE for OpAfterMeas: a reset lies between the measure and the later # gate, so the gate does NOT operate on a post-measurement qubit state. # A correct (reset-aware) detector must stay silent; a reset-blind detector # (Engine A) will falsely fire here -- a documented limitation. qc = QuantumCircuit(2, 2) qc.reset(0) qc.reset(1) qc.h(0) qc.measure(0, 0) qc.reset(0) qc.h(0) # SAFE: reset(0) between measure(0) and this gate
| Detector | Expected | Actual | Verdict | Raw detector output |
|---|---|---|---|---|
| A_OpAfterMeas | FALSE_POS | FIRE | FALSE_POS(limitation) | 8 qc h 0 6 |
| A_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| A_GhostCompose | N/A | silent | N/A | (no warnings emitted) |
| B_OpAfterMeas | SILENT | silent | SILENT | (no warnings emitted) |
| B_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| B_MissingReset | SILENT | silent | SILENT | (no warnings emitted) |
Note: Key cross-engine example: discriminates reset-aware vs reset-blind OpAfterMeas.
OpAfterMeas via 2-qubit gate (cx touches measured qubit) OpAfterMeas
cx(0,1) acts on both q0 and q1, but only q1 was measured earlier. Detector must flag the gate on q1 (the measured qubit) and NOT on q0.
Qiskit source (ex03_op_reach_2qubit.py)
from qiskit import QuantumCircuit # OpAfterMeas via a 2-qubit gate: cx(0,1) touches BOTH q0 and q1, but only # q1 was measured earlier. The detector must flag the gate on q1 (the # measured qubit) and NOT on q0 (which was never measured). qc = QuantumCircuit(2, 2) qc.reset(0) qc.reset(1) qc.h(0) qc.measure(1, 1) qc.cx(0, 1) # BUG on q1: 2-qubit gate acts on measured qubit q1
| Detector | Expected | Actual | Verdict | Raw detector output |
|---|---|---|---|---|
| A_OpAfterMeas | FIRE | FIRE | FIRE | 7 qc cx 1 6 |
| A_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| A_GhostCompose | N/A | silent | N/A | (no warnings emitted) |
| B_OpAfterMeas | FIRE | FIRE | FIRE | 6 1 4 |
| B_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| B_MissingReset | SILENT | silent | SILENT | (no warnings emitted) |
Note: Tests that ActsOn propagation over 2-qubit gates does not over-report to the unmeasured control qubit.
Basic DoubleMeas (consecutive measure of same qubit) DoubleMeas
q0 is measured twice with no operation on q0 between. Both engines should flag the redundant consecutive measure.
Qiskit source (ex06_double_meas_basic.py)
from qiskit import QuantumCircuit # DoubleMeas: qubit q0 is measured twice with no operation on q0 between. qc = QuantumCircuit(2, 2) qc.reset(0) qc.reset(1) qc.h(0) qc.measure(0, 0) qc.measure(0, 0) # BUG: redundant consecutive measure of q0
| Detector | Expected | Actual | Verdict | Raw detector output |
|---|---|---|---|---|
| A_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| A_DoubleMeas | FIRE | FIRE | FIRE | 7 qc 0 6 |
| A_GhostCompose | N/A | silent | N/A | (no warnings emitted) |
| B_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| B_DoubleMeas | FIRE | FIRE | FIRE | 4 5 0 |
| B_MissingReset | SILENT | silent | SILENT | (no warnings emitted) |
Note: The canonical redundant-measurement pattern.
DoubleMeas negative: reset between the two measures DoubleMeas
A reset on q0 sits between the two measures, so the second measure is NOT redundant. Both engines must stay silent (an op on the qubit between suppresses DoubleMeas).
Qiskit source (ex07_double_meas_reset_between.py)
from qiskit import QuantumCircuit # NEGATIVE for DoubleMeas: a reset on q0 sits between the two measures, # so the second measure is NOT redundant. Both engines must stay silent. qc = QuantumCircuit(2, 2) qc.reset(0) qc.reset(1) qc.measure(0, 0) qc.reset(0) # op on q0 between the two measures -> breaks DoubleMeas qc.measure(0, 0) # SAFE: reset(0) between the two measures of q0
| Detector | Expected | Actual | Verdict | Raw detector output |
|---|---|---|---|---|
| A_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| A_DoubleMeas | SILENT | silent | SILENT | (no warnings emitted) |
| A_GhostCompose | N/A | silent | N/A | (no warnings emitted) |
| B_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| B_DoubleMeas | SILENT | silent | SILENT | (no warnings emitted) |
| B_MissingReset | SILENT | silent | SILENT | (no warnings emitted) |
Note: Confirms the OpBetween / ModifiedQubit guard works as the dual of the reset case in OpAfterMeas.
DoubleMeas suppressed but OpAfterMeas fires on the same cx OpAfterMeas+DoubleMeas
A 2-qubit gate cx(0,1) on q0 lies between two measures of q0: that op correctly SUPPRESSES DoubleMeas, yet cx is itself a gate on q0 applied after the first measure, so it TRIGGERS OpAfterMeas. One detector must fire (OpAfterMeas) while the other stays silent (DoubleMeas).
Qiskit source (ex10_double_meas_2qubitgate.py)
from qiskit import QuantumCircuit
# COMBINED / cross-detector: a 2-qubit gate cx(0,1) on q0 between two
# measures of q0 correctly SUPPRESSES DoubleMeas (an op touches q0 between),
# yet the same cx is a gate on q0 applied after the first measure, so it
# TRIGGERS OpAfterMeas. Demonstrates that one detector fires while the other
# must not.
qc = QuantumCircuit(2, 2)
qc.reset(0)
qc.reset(1)
qc.measure(0, 0)
qc.cx(0, 1) # gate on q0 between measures -> suppresses DoubleMeas,
# but is itself a gate after measure -> OpAfterMeas on q0
qc.measure(0, 0)
| Detector | Expected | Actual | Verdict | Raw detector output |
|---|---|---|---|---|
| A_OpAfterMeas | FIRE | FIRE | FIRE | 6 qc cx 0 5 6 qc cx 0 7 |
| A_DoubleMeas | SILENT | silent | SILENT | (no warnings emitted) |
| A_GhostCompose | N/A | silent | N/A | (no warnings emitted) |
| B_OpAfterMeas | FIRE | FIRE | FIRE | 4 0 3 |
| B_DoubleMeas | SILENT | silent | SILENT | (no warnings emitted) |
| B_MissingReset | SILENT | silent | SILENT | (no warnings emitted) |
Note: Cross-detector interaction: the same statement is relevant to two analyses with opposite expectations.
Basic MissingReset (first op is a gate, not a reset) MissingReset
The first operation that touches q0 is h (a gate), not a reset, so q0 is used while uninitialised. Engine B only.
Qiskit source (ex11_missing_reset_basic.py)
from qiskit import QuantumCircuit # MissingReset: the first operation that touches q0 is a gate (h), not a # reset, so q0 is used while uninitialised. Engine B only. qc = QuantumCircuit(2, 2) qc.h(0) # BUG: first op on q0 is h, not reset (MissingReset q0) qc.measure(0, 0)
| Detector | Expected | Actual | Verdict | Raw detector output |
|---|---|---|---|---|
| A_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| A_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| A_GhostCompose | N/A | silent | N/A | (no warnings emitted) |
| B_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| B_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| B_MissingReset | FIRE | FIRE | FIRE | 1 0 |
Note: Engine A has no MissingReset rule; marked N/A there.
MissingReset via a 2-qubit gate (cx uses uninitialised qubits) MissingReset
cx(0,1) is the first operation touching both q0 and q1, so both are used while uninitialised. Engine B must flag q0 and q1 (the cx is a gate, not a reset).
Qiskit source (ex14_missing_reset_2qubit_gate.py)
from qiskit import QuantumCircuit # MissingReset via a 2-qubit gate: cx(0,1) is the first operation touching # both q0 and q1, so both are used while uninitialised. Engine B must flag # q0 and q1 (the cx is a gate, not a reset). qc = QuantumCircuit(2, 2) qc.cx(0, 1) # BUG: cx uses q0 and q1 before any reset (MissingReset q0, q1)
| Detector | Expected | Actual | Verdict | Raw detector output |
|---|---|---|---|---|
| A_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| A_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| A_GhostCompose | N/A | silent | N/A | (no warnings emitted) |
| B_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| B_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| B_MissingReset | FIRE | FIRE | FIRE | 1 0 2 1 |
Note: MissingReset over a multi-target gate.
Basic GhostCompose (discarded compose return value) GhostCompose
qc.compose(sub) is non-mutating in Qiskit, but its return value is discarded, so the composed circuit is silently dropped. Engine A only.
Qiskit source (ex15_ghost_compose_basic.py)
from qiskit import QuantumCircuit # GhostCompose: qc.compose(sub) is non-mutating in Qiskit, but its return # value is discarded, so the composed circuit is silently dropped. Engine A # only. qc = QuantumCircuit(1, 1) sub = QuantumCircuit(1, 1) sub.h(0) qc.compose(sub) # BUG: compose() return value discarded (GhostCompose)
| Detector | Expected | Actual | Verdict | Raw detector output |
|---|---|---|---|---|
| A_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| A_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| A_GhostCompose | FIRE | FIRE | FIRE | 5 qc sub |
| B_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| B_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| B_MissingReset | N/A | FIRE | N/A | 1 0 |
Note: Engine B has no GhostCompose rule; marked N/A there.
GhostCompose negative: return value captured GhostCompose
qc2 = qc.compose(sub) keeps the return value, so the circuit is intentionally retained. Engine A must NOT fire (negative example).
Qiskit source (ex16_ghost_compose_assigned.py)
from qiskit import QuantumCircuit # NEGATIVE for GhostCompose: the result of compose() is captured in qc2, so # the circuit is intentionally kept. The detector must NOT fire. qc = QuantumCircuit(1, 1) sub = QuantumCircuit(1, 1) sub.h(0) qc2 = qc.compose(sub) # SAFE: return value captured -> no GhostCompose
| Detector | Expected | Actual | Verdict | Raw detector output |
|---|---|---|---|---|
| A_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| A_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| A_GhostCompose | SILENT | silent | SILENT | (no warnings emitted) |
| B_OpAfterMeas | N/A | silent | N/A | (no warnings emitted) |
| B_DoubleMeas | N/A | silent | N/A | (no warnings emitted) |
| B_MissingReset | N/A | FIRE | N/A | 1 0 |
Note: Confirms the AssignedStmt guard: only non-assigned compose() calls are flagged.