Arbiter

Agent-decided CI scope. Savings are only meaningful next to the miss count — a job a variant skipped that a full canary run proved would have failed. Recall below 98% means the variant is not safe to enforce.

Deferred safety — post-merge accountability

Deferral is safe only when every non-gating check reports and every failure is attributable to the merge that introduced it. Reverts per 100 merged PRs and time-to-detection are the primary safety signal.

repositoryposturecritical pathdeferredskippedin flightreverts / 100 merged PRsmedian detection
superbuilders/arbiter unconfigured 0m 0m 0m 0 no merged PRs yet / 100 merged PRs
0 incidents across 0 merges		 no deferred failures
superbuilders/arbiter-testbed careful 645m 240m 60m 0 40.00 / 100 merged PRs
2 incidents across 5 merges		 3.4m

Posture separation

1 of 4 posture steps changes nothing on this data (fast). Model confidence is bimodal, so a threshold in the middle of the range has almost nothing to sort. The canary-rate floor still differentiates these postures.

posturemin confidencerequired canarywould deferwould blockvs next safer
paranoid 1.01 0.0% 0 789 —
careful 0.95 10.0% 310 479 +310
balanced 0.85 15.0% 336 453 +26
fast 0.75 25.0% 342 447 +6
reckless 0.60 40.0% 366 423 +24
decision runs
30
canary full runs
6
minutes run
630
minutes if full
930
billed saved
32.3%
wall clock saved
11.7%
fallbacks to full
5

Variant scorecard

variantdecisionsselection ratebilled min savedwall-clock min saved recallmissescontract violationsescalations avg latencyspend$/min saved
deterministic 27 52.0% 396 79
of 270		 80.8%
of 26 observed		 5 late 0 1 0.0s $0.00 $0.0000
llm_single 27 60.8% 323 74
of 270		 80.8%
of 26 observed		 5 late 0 1 18.2s $0.74 $0.0023
agent_code 27 63.3% 303 74
of 270		 100.0%
of 26 observed		 0 0 3 54.9s $6.67 $0.0220

Recent runs

prreposcenariomodestatusjobsminutesfallback
#25 superbuilders/arbiter-testbed assist completed 15 40/40m the decide workflow finished (cancelled) without posting a decision; running the full suite
#24 superbuilders/arbiter-testbed assist canary completed 5 15/40m
#23 superbuilders/arbiter-testbed assist completed 15 40/40m governance change (`.arbiter.yml`): edits to Arbiter's own config, charter, or workflows always run the full suite under
#22 superbuilders/arbiter-testbed assist completed 6 11/40m
#21 superbuilders/arbiter-testbed readme-plus-api assist completed 9 31/40m
#20 superbuilders/arbiter-testbed utils-behavior-change assist completed 15 40/40m
#19 superbuilders/arbiter-testbed types-breaking assist canary completed 15 40/40m
#18 superbuilders/arbiter-testbed types-widening assist completed 14 39/40m
#17 superbuilders/arbiter-testbed schema-breaking-reachable assist completed 15 40/40m
#16 superbuilders/arbiter-testbed schema-additive-unreachable assist canary completed 11 35/40m
#15 superbuilders/arbiter-testbed mobile-only assist completed 6 13/40m
#14 superbuilders/arbiter-testbed docs-only assist completed 3 7/40m
#13 superbuilders/arbiter-testbed enforce canary completed 5 13/25m
#12 superbuilders/arbiter-testbed enforce completed 4 10/25m
#11 superbuilders/arbiter-testbed enforce completed 4 10/25m
#10 superbuilders/arbiter-testbed enforce canary completed 4 10/25m
#10 superbuilders/arbiter-testbed enforce completed 4 10/25m
#9 superbuilders/arbiter-testbed assist completed 6 11/25m
#9 superbuilders/arbiter-testbed assist completed 12 25/25m agent_code escalated: fell back to the full suite: agent produced no parseable decision object
#8 superbuilders/arbiter-testbed assist completed 12 25/25m build or dependency configuration changed (.github/workflows/arbiter-canary.yml); the import graph is not trustworthy fo
#7 superbuilders/arbiter-testbed assist completed 5 12/25m
#6 superbuilders/arbiter-testbed assist completed 6 13/25m
#5 superbuilders/arbiter-testbed assist canary completed 12 25/25m
#4 superbuilders/arbiter-testbed assist completed 12 25/25m
#3 superbuilders/arbiter-testbed assist completed 10 22/25m
#2 superbuilders/arbiter-testbed assist completed 5 10/25m
#1 superbuilders/arbiter-testbed assist completed 6 11/25m
#1 superbuilders/arbiter-testbed assist planned 6 11/25m
#1 superbuilders/arbiter-testbed assist planned 6 11/25m
#1 superbuilders/arbiter-testbed assist planned 12 25/25m dispatch of arbiter-decide.yml failed: No ref found for: 776aff26d3f6c8bf330314e497749aacdbac665e - https://docs.github.