Windows · about 12 minutes

Check it yourself instead of taking my word for it

Everything on these pages can be verified on your own laptop. No API key, no account, no Docker, nothing that costs money. If you only do step 3, you will have independently confirmed that your Chapter 5 numbers follow from your logs.

Step 1 · Install Bun

Bun runs the code. Open PowerShell — press the Windows key, type powershell, press Enter — and paste this:

powershell -c "irm bun.sh/install.ps1 | iex"

Then close PowerShell and open it again. That is not optional — the new window is what knows where Bun was installed. Check it worked:

bun --version

A version number means you are ready. An error means the window was not reopened.

Step 2 · Get the code

This downloads the branch into a folder called Dissertation inside your Documents.

cd ~\Documents
git clone https://github.com/Lagadnakul/Dissertation.git
cd Dissertation
git checkout add-implementation-and-final-document
cd recovery-bench
bun install

Step 3 · Run the tests

bun test

You should see, in roughly three seconds:

 399 pass
  34 skip
   0 fail
Ran 433 tests across 23 files.

Step 4 · See the finding, measured

bun run audit-legacy

This reads the 61 patches your model actually wrote and reports:

unified_diff     model wrote the diff      30/51 clean (58.8%)
    16  hunk_count_mismatch
     6  no_file_header
search_replace   difflib wrote the diff    10/10 clean (100%)

That is finding 1 from the findings page, computed in front of you rather than quoted at you.

Step 5 · Regenerate your Chapter 5

This is the one worth doing. It needs your original evaluation logs, which live on the repository's main branch. Copy them in:

cd ~\Documents\Dissertation
git checkout main -- 06_Methodology

New-Item -ItemType Directory -Force -Path recovery-bench\archive\legacy_runs
Copy-Item -Recurse -Force `
  06_Methodology\code\logs\run_evaluation\* `
  recovery-bench\archive\legacy_runs\

New-Item -ItemType Directory -Force -Path recovery-bench\archive\legacy_predictions
Copy-Item -Force `
  06_Methodology\code\predictions*.json `
  recovery-bench\archive\legacy_predictions\

cd recovery-bench

Now run the replay:

bun run replay

replay — 51 cells from archive/legacy_runs

  test-verified (report.json)     42
  apply-verified (no report)       9   of which APPLY_FAIL: 9

  baseline resolved            12/20
  failure pool                     8   evaluable: 7
  recovered                      3/7
    logic-class                  0/4
    format-class                 3/3

  failure pool matches declared   yes
  failure class matches hardcoded yes

And then the one that matters:

bun run report

report — 51 cells
  wrote data/chapter5_figures.json
  byte-identical to the committed master_table.md

Run bun test again now and all 248 pass, with no skips: the 34 that were waiting on your logs can finally run.

Step 6 · See the pre-flight check and the budget

This is the check that runs before any model is contacted. It needs no key and no internet:

bun run doctor

It verifies the config parses, the 300-task benchmark is intact and pinned, sampling is fixed, and — the unusual one — that git actually accepts the patches the program generates. Then it prints the budget:

  PASS  budget ledger
        no ledger at data/ledger.jsonl — nothing spent yet
        nvidia      0/8,000,000 tokens (0.00%)  0/1500 calls
        qwencloud   0/900,000 tokens (0.00%)  0/100 calls
        gemini      0/500,000 tokens (0.00%)  0/30 calls

Those ceilings are now enforced before a request is sent, rather than checked afterwards. In your original config the limit was written 8_000_000, which the config file format read as text rather than a number — so nothing ever compared against it. Phase 4 explains what changed.

The endpoint probe will report no key set, which is correct: the keys are deliberately not in the repository.

Step 7 · Prove the check can fail

A test that always passes proves nothing. Break the analysis on purpose and watch it get caught. Open this file in Notepad:

notepad packages\pipeline\analysis\aggregate.ts

Find the line containing n: BASELINE_N and change BASELINE_N to 16 — that is the survivorship error, scoring the run only over tasks that produced a result. Save, then:

bun test

Three tests fail. Undo the change (Ctrl+Z in Notepad), save, run again — all pass. The check has teeth.

Step 8 · Open the dashboard

cd packages\dashboard
bun install
bun run dev

Open the address it prints, usually http://localhost:5178. The page begins with your question and your answer, then the two numbers that are that answer, then every attempt behind them.

It opens on the submitted study, which is committed and always present. If you have not run a sweep on this machine the second tab says so in words rather than showing you zero — because no attempts and zero attempts are different things, and confusing them is the mistake this whole project was built against.

To check the colours are legible rather than take anyone's word for it:

bun run contrast

  ok  127 palette checks pass in light and dark

If something goes wrong

Starting over

Nothing was installed into Windows except Bun, and nothing outside the Dissertation folder was touched. To remove it all, delete that folder. Your dissertation files and your GitHub repository are unaffected — every command here only read them.