4 Specify, Delegate, Interrogate, Inspect, Verify, Document, Defend
This chapter is part of a book in active development and has not yet been through the author’s review. Content may change as the review advances.
The research decision. Every time you hand work to an AI tool, you decide how that work gets routed: what you pin down before you ask, what you open and read when the answer comes back, and what you are willing to say in your own words at the end. Your decision in this chapter is to stop improvising that route and run the same seven-step checklist every time.
4.1 Why this decision matters
The decision on the table: whether each delegation gets improvised, or run through the same fixed checklist every single time.
“Show me the comparison, not the number. Who saw each version, over which days, and measured against what? A result I cannot reproduce is a rumor.” — an analytics lead, reading a claim that a new checkout page converts better
A tool can now write the code, run it, and hand you a tidy result in one breath. That is exactly why a loose habit is dangerous. If you paste the result straight into your work, you have imported a guess and signed your name to it. That reviewer does not care how confident the output sounded. They care about one question, and it is the question every serious reader of your work will eventually ask you:
What did you set up, what did you hand off, and how did you check it before you believed it?
This chapter gives you a habit that answers that question every time, so you never have to reconstruct the answer from memory.
4.2 The concept
SDIIVDD is a checklist: seven steps you run every time you hand research work to an AI tool, in the same order, no matter how small the task looks. A protocol is just a fixed sequence of steps you run every time, so you never skip the check that matters. You already met the three-word shorthand, Ask then Verify then Document. SDIIVDD is that shorthand opened up into the steps you actually perform, and the name is only the first letter of each one: Specify, Delegate, Interrogate, Inspect, Verify, Document, Defend.
Run it the way a pilot runs a preflight checklist. Not because you have forgotten how to fly, but because the item you would have skipped is the one that bites (Degani and Wiener 1993).
The seven steps, each defined once with an example:
- Specify. Write down the exact task and your own expected answer before you ask. Example: “compare conversion on the old and new checkout pages; I expect the new page to win by a few percent, not by forty, and both versions have to be measured over the same weeks.”
- Delegate. Hand the well-specified piece to the tool. To delegate is to assign a checkable chunk of work, not your judgment. Example: “pull the consumer price index values for 2019 and 2025 and write the code that converts my prices into 2025 dollars.”
- Interrogate. Ask the draft what it assumes. Example: “when you report turnout, is the denominator registered voters or every voting-age adult?”
- Inspect. Read what the tool actually produced, not its summary of itself. Example: open the analysis script and see which wells it treats as controls, instead of trusting the comment that says controls were excluded.
- Verify. Confirm the result with a real, independent check before you trust it. Example: recompute the inflation adjustment by hand from the two published index values and see whether you land on the same number.
- Document. Log the tool, the task, the prompt, and how you checked, in a record that travels with your work. Example: one ledger row saying the tool wrote the conversion script and that you re-derived one week’s rate by hand.
- Defend. State the result in your own words, with the tool out of the room. Example: “turnout was six points higher in these precincts, among registered voters, in this one election.”
Two of these seven stay human no matter how good the tool gets: Specify (step 1) and Defend (step 7). They are the human bookends, and everything you hand off sits safely between them. That is what makes them never-delegate decisions, the calls you always own: what you are really asking, what counts as a fair test, and what your evidence licenses you to claim. A tool can run the comparison for you. Only you decide the comparison was fair enough to report.
4.2.1 The loop, and who is running it
The middle of the checklist is not a straight line. Real AI work runs in a loop: you prompt, you read the output, you interrogate it, you refine the prompt or the code, and you run it again. Steps 2 through 5 are that loop, and one task can send you around it five times. Going around is not a sign you are doing it badly. A first output that needs no second pass is rare enough that you should be a little suspicious of it.
What is new is that the tool can now run the loop without you. Agentic tools plan the task, write the code, execute it, read their own error, rewrite, and run again, many cycles deep, then hand you one clean result. That is a real gain in reach. It also means all the interrogating and inspecting you would have done between cycles never happened, unless you insisted on it.
SDIIVDD is how you keep command either way. When you run the loop, the checklist tells you what to do on each pass. When the tool runs the loop, the checklist tells you what to hold it to: a specification written before the first cycle, the intermediate work opened and read rather than summarized, and an independent check on the final number, before you say anything in your own name. The cycles can be automated. The bookends cannot.
The loop is also where AI earns its keep as a brainstorming partner: ask it for candidate questions, rival explanations, designs you had not considered. Generate widely with it; the choosing stays yours.
4.2.2 The same checklist, at 126,000 conversations
A research team at the Wharton Generative AI Labs asked whether the ordinary tactics of human persuasion work on a chatbot. Would an appeal to authority, or a reminder that everyone else has already agreed, move a model into answering a request it was built to refuse?
A model’s answer is not the same twice, so settling that question means running each prompt hundreds of times and comparing rates rather than anecdotes. The team built a tool to do the running. For their first study, in early 2025, they operated it through its web interface: write the plain version of a request, write the persuasive version, say how an answer should be judged, set how many times to run each one, then read the results. That study covered 28,000 conversations with a single model.
For the follow-up they described the experiment to an AI coding agent in plain language and let it drive the tool. The agent offered different ways to word a condition, ran pilots, sent the whole design to three models at once, and read the raw answers back to flag the ones that broke the pattern. The published study reports 126,000 conversations, and the persuasion tactics lifted compliance from about 35 percent to about 51 percent (Meincke et al. 2026).
Look at which steps changed hands. Configuring the runs, launching them in parallel, and taking the first pass over the transcripts: all delegated. Writing the plain request, writing the persuasive one, and deciding what counts as the model complying: never delegated, in either version of the study. The team’s own summary of what stayed theirs names this chapter’s first and last steps. The researcher, they write, “still makes the scientific decisions, determining what question is worth asking, what counts as a fair comparison, and whether the results are meaningful.”
One thing this case does not show is that the agent made the science better. It made the science bigger: more conversations, more models, more transcripts actually opened. Whether the comparison was fair, and whether 51 percent means what it appears to mean, still rest with the seven researchers whose names are on the paper. Scale is not rigor, and a specification you got wrong gets more expensive the faster it runs.
The study: Lennart Meincke, Dan Shapiro, Angela Duckworth, Ethan R. Mollick, Lilach Mollick, Christophe Van den Bulte, and Robert Cialdini, “Persuading large language models to comply with objectionable requests,” PNAS, 2026, doi:10.1073/pnas.2535868123. Their account of working with the coding agent: “Prompting Research Itself,” Wharton Generative AI Labs, 2026.
4.3 A worked example
You are checking whether a new checkout page on a small online store genuinely converts better than the old one. Conversion rate is the share of visits that end in a purchase, so 40 purchases out of 1,000 visits is 4 percent. Watch the checklist run.
Specify. You write your expectation first: “the new page should win by a few percent at most, and the two versions have to be measured over the same weeks, on the same kind of traffic.”
Delegate. You ask the tool to write the script that computes the conversion rate for each version from the session log and reports the difference.
Interrogate. You ask, “what does your calculation assume about who saw each version?” It admits it combined every session in the file, from both weeks.
Inspect. You read the script and then the data. The store ran a site-wide sale in the first week, which filled the site with bargain hunters who browse and leave. The new page only went live in the second week. The script charged version A an entire week of window shoppers that version B never saw.
Verify. You restrict both versions to week 2, when they ran side by side, and recompute. You also confirm both versions count a purchase the same way.
Two numbers describe that gap, and they are easy to mix up. A percentage-point difference subtracts one rate from the other: 4.24 percent minus 4.00 percent is 0.24 percentage points. A relative increase compares that difference with the rate you started from: 0.24 divided by 4.00 is 6 percent. Both describe one thing, twelve more recorded purchases for version B out of 5,000 visits.
Then you look at what the log does not hold. It does not record how a visit ended up on one version rather than the other, whether the same shopper came back, or which day a visit happened. So the fair comparison tells you what this one shared week recorded. It does not tell you that version B will do better on the next 5,000 visits.
A question that often comes up here: “The 6 percent is real arithmetic. Why can I not say the new page is better?” Because the arithmetic describes this log, and “better” is a claim about visits you have not seen yet. To get there you would need to know how visits were routed to each version, and how much a different 5,000 visits could move a gap this size. That second question is a lesson of its own, later in this book, on uncertainty. Here the honest move is to report what you measured and say where it stops.
Document. You log the prompt, the comparison you threw away, the comparison you kept, and both ways of describing the gap.
Defend. You write: “in the one week both versions were live, the old page converted 4.00 percent of visits and the new page 4.24 percent. That is 0.24 percentage points, a 6 percent relative increase, and twelve purchases. This log does not record how visits reached either version or whether shoppers came back, so it does not establish that the new page performs better. I have not tested it during a sale, and I have not looked at mobile traffic separately.”
The tool did the typing. You did the research.
Running a fixed checklist rather than trusting recall is an old and well-studied safeguard in high-stakes work (Degani and Wiener 1993).
The first block builds the session log from exact counts. Inside each version-week group it shuffles the purchase and non-purchase rows without changing either count, so the rates are fixed by the counts alone.
import numpy as np, pandas as pd
SEED = 464
rng = np.random.default_rng(SEED)
# A constructed session log. Week 1 is the site-wide sale, and only version A
# was live. Each row is one visit; version, week, visits, purchases.
cells = [("A", 1, 5000, 105), ("A", 2, 5000, 200), ("B", 2, 5000, 212)]
log = pd.concat([
pd.DataFrame({"version": v, "week": w,
"bought": rng.permutation(np.r_[np.ones(b), np.zeros(n - b)])})
for v, w, n, b in cells], ignore_index=True)The second block prints the counts before the rates, and reports the gap on both scales. The first result is what the tool’s script produced. The second is what survived Inspect and Verify.
def report(label, frame):
counts = frame.groupby("version")["bought"].agg(["sum", "size", "mean"])
a, b = counts.loc["A"], counts.loc["B"]
print(label)
print(f" A: {a['sum']:.0f}/{a['size']:.0f} = {a['mean']*100:.2f}%")
print(f" B: {b['sum']:.0f}/{b['size']:.0f} = {b['mean']*100:.2f}%")
print(f" gap: {(b['mean']-a['mean'])*100:+.2f} percentage points, "
f"{(b['mean']/a['mean']-1)*100:+.2f}% relative to A")
report("all weeks, and the two versions do not share them", log)
report("week 2, both versions live", log[log.week == 2])4.4 An AI failure case
You ask for the analysis and the tool reports, with total confidence, “the new checkout page converts 39 percent better.” The arithmetic is right and the code runs without a single error. The comparison is still wrong. Here is the trap: the script combined every session in the log. The old page’s numbers include the sale week, when the site was full of bargain hunters who never intended to buy, and the new page only existed afterward. The comparison charged one version a crowd the other never met.
You catch it at Inspect and Verify. Reading the script shows it never requires both versions to appear in the same week. Restricting the comparison to the week they actually ran side by side shrinks the 39 percent to 6. That repair removes one unfair difference between the two groups. It does not turn what is left into a finding, because the log still cannot say how a visit reached either version. A green check is not a correct result, and a corrected number is not automatically a claim you can defend. You verify the comparison and the number, not the paragraph about them.
4.5 It is your turn
You are working inside Studio 2: Set your rules, shape your question. Keep what you write here; the studio’s milestone chapter is where it joins the other lessons’ pieces into one artifact you can defend.
You have a task list that says what you are willing to delegate. Now you delegate something real, and keep command of it from the first word to the last.
The hands-on half of this section lives in the chapter’s companion notebook: open it in Colab with the badge at the top, and work the steps there.
Commit your own answer first, then delegate. Each prompt below is a checkable job, not a request for a verdict, and each is the opening move of a loop rather than the whole conversation.
Specify and Defend never leave your hands. You decide what question the comparison answers, what counts as a fair comparison, and whether the data behind it is honest for the claim you want to make. You own the final sentence, its boundary, and its uncertainty. The tool can compute the rates, but it cannot decide the test was fair or the claim was earned. Those are yours.
Pick one genuine, checkable claim — your opening move’s starting belief is the natural choice, because checking it sharpens the question you will declare at this studio’s close. Otherwise any rate, total, comparison, anything a source or a calculation could settle. Write it as a single sentence.
Specify. Before you open any tool, write your own expected answer and what a fair way of getting it would look like.
Delegate, Interrogate, Inspect. Hand the task over. Ask the output what it assumes. Then open what it actually produced, the source or the code or the table, rather than its summary of itself. Expect several passes around that loop, and note what changed on each one.
Optional depth. The milestone needs only this lesson’s core cycle; run this second angle when you want more practice.
Locate the standard method.
Act as a business-analytics assistant. I need to compare conversion rates for two versions of a checkout page. Before any code, name the standard statistical test for comparing two proportions and the exact library function that implements it, citing the official documentation section. Only name functions you are confident exist.After running, verify: open the official docs and confirm the function and its defaults exist. Counters confident fabrication (an invented function name arrives as confidently as a real one).
A second angle, optional:
Delegate, with a checklist to verify.
Write the script that computes the conversion rate for version A and version B from this session log. Report the gap twice, as a percentage-point difference and as a relative increase from version A. Then return a table of every assumption your calculation makes: which sessions are included and excluded, how you count a purchase, whether both versions ran over the same weeks, and how repeat visits are handled.After running, verify: recompute one version’s week-2 rate from the counts by hand, 200 divided by 5,000 is 4.00 percent, and check that the script agrees. Counters plausible-but-wrong-method (a calculation that quietly compares unequal conditions).
Verify. Run one independent check and write its outcome down either way — “nothing changed” is a result worth recording. If the check and the output disagree, do not average them; find out which one is wrong.
Defend. With the tool closed, write one sentence stating the result in your own words, and one sentence naming what it does not establish.
Optional depth. The milestone needs only this lesson’s core cycle; run this second angle when you want more practice.
Red-team the comparison.
Here is my claim: "in the one week both versions were live, the new checkout page converted 4.24 percent of visits against the old page's 4.00 percent, a gap of 0.24 percentage points. The log does not record how visits reached each version, so I am not claiming the new page performs better." Act as a hostile reviewer. Name every way this comparison could be unfair or these numbers could mislead. Do not rewrite the claim for me.After running, verify: trace each criticism to a column the log actually has, or mark it as something the log cannot settle. If it only praises the claim, push back and demand the single worst flaw. Counters sycophantic agreement (praise that reviews your ego, not your evidence).
Document the whole run in your AI Research Ledger, naming the verification method you used from the Verification Guide and the check’s outcome — the step where you caught something the tool got wrong, or the record that the check agreed, which is a result too. An AI reviewer may run the check with you; the decision to accept or reject stays yours.
A workflow can be technically correct and still leave authorship vague. The next lesson decides which judgments remain yours and how the assistance will be disclosed.