Milestone 7: Your first reproducible analysis
This milestone chapter is part of a book in active development and has not yet been through the author’s review. Content may change as the review advances.
This chapter closes Studio 7: Produce a reproducible first analysis. Its lessons each ended at It is your turn; what you wrote there arrives here as working material, and this is where the pieces become one artifact you can defend.
What this milestone produces. A route-specific result with an uncertainty statement, a restart-and-run-all verification, an environment record, and a claim-to-output check.
What you bring
You also carry forward the last milestone’s artifact: Milestone 6: Your data and measurement, governed. This one builds on it, and the chain assumes it is versioned and filed.
Check you are carrying each of these before you start. If one is missing, go back and work that lesson’s It is your turn first; the practice below assumes it.
- Lesson 22 — AI as Programmer: your first computed number, its delegated code read line by line, and the sentence naming what the number does not cover.
- Lesson 23 — AI as Analytical Assistant: your headline estimate with its uncertainty statement and cycle log, two independently re-derived numbers, a placebo run where licensed, the clean-restart record, the environment record, and the claim-to-output trace.
The practice
- Write the analysis you declared, and only that analysis, before looking at anything else.
- Produce the result with its uncertainty statement, in the form your Contract specified.
- Restart and run everything from a clean state; if the numbers move, the pipeline is the finding.
- Record your environment, and check that every claim you plan to make traces to a specific output.
A version, not a pass
Your milestone artifact is a dated, numbered version with the reason for the version attached. When later evidence changes it, you write the next version rather than editing the last one, because the sequence of changes is itself part of your research record.
The four rails, here
Four concerns cross every studio in this book. At this milestone they take these specific forms.
- Ethics, permissions, and data exposure. What you send to a tool during analysis is a disclosure decision each time.
- Evidence, provenance, and reproducibility. Every number in your result traces to a data cell and a line of code.
- AI activity, verification, and human decisions. Delegate the writing of code freely; verify every returned number against the data yourself.
- Uncertainty, claim boundary, and revision history. A result reported without uncertainty is not yet a result.
Where this milestone sits
You arrived carrying Milestone 6: Your data and measurement, governed. Milestone 8 exists to attack this result; the cleaner your record here, the more the attack can teach you. The chain ends at Milestone 12: Your release and next cycle, where you decide whether your finished research artifact leaves your hands.
What sends you back
Failing to reproduce your own analysis sends you back here before any stress-testing is meaningful.
How this milestone is assessed
Each row scores 0, 1, or 2. 10 points total.
| # | Criterion | 0 | 1 | 2 |
|---|---|---|---|---|
| 1 | The milestone artifact exists in full, specific to your own project | absent, or generic text that would fit any project | present but incomplete, or specific in some parts and generic in others | complete and unmistakably about your project |
| 2 | The milestone is written as a dated, numbered version with its reason | no version, or a version with no reason given | versioned, but the reason is decorative rather than an account of what changed | versioned with a reason a reader could use to reconstruct your thinking |
| 3 | Every judgment in the artifact is defended by you, not asserted by a tool | conclusions appear with no reasoning, or reproduce a tool’s wording | reasoning present but thin where the decision was hardest | each judgment carries a reason you could defend under questioning |
| 4 | The four rails are carried in the form this studio requires | one or more rails absent where this studio required them | all rails present but one is nominal | every rail addressed in the form this studio requires |
| 5 | The milestone’s own product is present and defensible: A route-specific result with an uncertainty statement, a restart-and-run-all verification, an environment record, and a claim-to-output check | the milestone’s own product is missing | present but would not survive a careful question | present and defensible as stated |
No work at or after Studio 4 proceeds past a not-authorized determination. This is not scored and cannot be averaged away.
Your workbook
Open the workbook with the badge above. It walks these steps with a cell for each one, ends by writing your milestone version with its reason, and adds the rows this studio contributes to your AI Research Ledger and your Research Dossier.