Studio 8: Stress-test and adjudicate
This studio is part of a book in active development and has not yet been through the author’s review. Content may change as the review advances.
What you can defend when you leave. Attack your own result the way a hostile reviewer would, and record what survived.
The milestone ahead
A result you have not tried to break is a result you do not yet know. This milestone attacks your own finding the way a hostile reviewer would: pre-listed robustness checks, negative tests with predictions written before the run, an adversarial review, and an audit of where your confidence actually comes from. What survives, you can defend. What does not, you now know before a reader finds it for you.
This studio closes with Milestone 8: Your robustness audit, a short chapter of its own after the lessons. What it asks you to produce. A pre-listed robustness grid, negative tests with their assumptions stated, diagnostics, an adversarial-review record, and your adjudication of what survived.
Milestone 9 writes down only the claims this audit licenses, at the strength it licenses them. The milestone chapter keeps the working details: the practice steps, the versioned record, and how the artifact is assessed.
The lessons in this studio
Read them in order, working each lesson’s It is your turn as you go. Each one builds a piece of Milestone 8; beside each lesson is the piece it hands you.
- Lesson 24 — Robustness and Sensitivity: your pre-listed robustness grid with its full results, direction and size, plus the named null check.
- Lesson 25 — Diagnostics and Negative Tests: your negative test and diagnostic, each with the prediction you wrote before running it.
- Lesson 26 — AI as Adversarial Reviewer: your commissioned adversarial review with every flag adjudicated against the data, confirmed or refuted.
- Lesson 27 — Recognizing False Confidence: your confidence audit: the not-yet-recomputed list, where belief in the headline number comes from, and your boundary sentence.
When the last lesson closes, the milestone chapter is next. Nothing you write in a lesson is busywork: every piece above is on the milestone’s checklist.