6 Choose Your Question’s Kind and Reach
This chapter is part of a book in active development and has not yet been through the author’s review. Content may change as the review advances.
The research decision. Place your question on the inquiry compass: what KIND of answer it wants (descriptive or causal) and what REACH the answer may claim — with prediction as its own compass position, a descriptive kind aimed at unseen cases, never a third kind. The classification fixes what your evidence will ever be allowed to say; the next lesson turns the classified question into the formal declaration.
6.1 Why this decision matters
The decision on the table: what kind of answer your question wants, and for which units that answer has to hold.
Suppose a committee member asks, “What, exactly, are you trying to learn?” In the curiosity studio you did not force an early answer; you chose a problem worth investigating. Since then you have opened the ledger, decided what may be delegated, verified one AI-assisted task, and stated what remains your responsibility. You are now ready to answer with a sentence that governs the work: a broad topic cannot tell you which evidence belongs, and a promising problem cannot tell you what would count as an answer. A formal research question can, provided its parts agree.
A thesis advisor at your first committee meeting puts it plainly: “Before I approve a single method, I make you answer two questions about your question. What kind of answer does it want, and for whom must that answer hold?” Everything downstream sits inside the box those two answers draw. Misclassify your question and every later method aims at the wrong target, because a method that describes what is there cannot tell you what an intervention would change. The compass fixes what your evidence may claim before you spend months collecting it.
6.2 The concept
Every project narrows to a research question: the single sentence evidence can answer and be wrong about, such as “does a second register shorten the line?” Before you pick a method, you classify that question on the inquiry compass, the map that sorts any question by exactly two things. The compass is adapted from Research Design in the Social Sciences (Blair, Coppock, and Humphreys, 2023, ch. 7).
The first axis is kind. A descriptive question asks what the world is or was, with no intervention needed; you look and record, as in “what fraction of first-years skip breakfast?” A causal question asks what would change if someone intervened, which forces a comparison to a world that did not happen, as in “does a free breakfast raise attendance?”
The second axis is reach: for which units the answer must hold. Data at hand means only the units you actually observed. A population means a larger group your sampling reaches beyond the data. Unseen cases means units you have not observed yet, usually future ones.
Crossing the two axes gives four familiar names. Description is descriptive about the data at hand. Generalization is descriptive about a population, and it must pay for its reach with a sampling design that lets the units you saw stand in for the ones you did not. Prediction is descriptive about unseen cases; it forecasts a new case from a pattern. Prediction is still descriptive, not causal: guessing who will drop out is not the same as knowing what makes them drop out (Shmueli 2010). Causal reasoning is the causal kind, the only position that earns the word “because.”
Those four names are labels, not boxes that answer anything for you. Causal questions have reach as well, and the reach changes what you are asking. You might want the effect for the units you actually studied, the effect for a wider population your sample stands in for, or the effect inside one named subgroup. Those are three different questions with three different answers, so say which one you want. Example: the effect of a second register on the twelve lunch periods you timed is not the same quantity as its effect on every weekday lunch this term.
Now the rule that keeps the whole compass honest, and it is the one most often broken. Your data never change what kind of question you asked (Blair et al. 2023). A question about what an intervention would change stays causal even when your evidence cannot answer it. When your design cannot isolate that answer, the honest status is currently unidentified, meaning the question is still causal and this design does not reach it yet. Example: you want to know whether a second register cuts waiting, but managers open one only when the line is already long, so crowding and registers move together. When that happens, you still have something to report, and keeping three labels apart tells you what. The question is what you asked, and its kind is fixed by its words: would a second register cut waiting? The warrant is the design’s reason your evidence can answer it, and here the warrant is missing, because crowding decides which counters get the register. The result is what this evidence supports anyway: in the periods you observed, two-register counters had longer waits. That result is descriptive, real, and worth stating. It sits beside your causal question; it does not replace it. In some designs the missing warrant is partial rather than absent, and the evidence can still narrow the answer, bounding it or fixing its sign, without pinning it down.
From there you have two legitimate moves. Strengthen the design until it can carry a causal answer, or deliberately declare a new descriptive question and record the swap as a decision you made. Sliding quietly between the two is how a project ends up claiming more than it earned.
A question that often comes up here: if my data cannot answer a causal question, should I just rewrite it as a descriptive one?
Only if a descriptive question is genuinely what you now want to ask. Your current result may well be descriptive while your causal question sits unanswered beside it. Write “currently unidentified” next to the causal one, then decide whether to redesign or to change questions. Both are respectable. Quietly swapping the question and keeping the causal wording is not.
One last hazard. A double-barreled question is one sentence that secretly asks two questions of different kinds, such as “how common is burnout, and does night work cause it?” It cannot be answered as one, and the fix is to split it into two clean questions, each with its own position.
6.3 A worked example
Start with an everyday business curiosity: “why is the campus coffee line so slow at noon?” Give it edges and it becomes a topic, waiting time at campus dining counters. That one topic hides four different questions, one at each compass position. A standard result in operations research says waiting time climbs sharply once customers arrive nearly as fast as the counter can serve them. Against that backdrop you time how long twelve customers wait to be served.
- “Among these twelve customers, what was the average wait?” This asks only what your recorded numbers say. It is descriptive with reach data at hand. Position: Description.
- “Across every customer at this counter during weekday lunch, what is the mean wait?” Same descriptive kind, but the answer must hold for a population you did not measure, paid for with a sampling design that makes your twelve stand in for the rest. Position: Generalization.
- “A customer walks up tomorrow at noon; how long will they wait?” Still descriptive, because you only need a pattern that travels to a case you have not seen. Position: Prediction. You forecast the wait without knowing why any particular line runs fast or slow.
- “Does opening a second register shorten the wait?” Now the kind flips. To answer it you must compare the same lunch hour in two worlds, one with a second register and one without, and only one of those worlds actually happens. That imagined comparison is the signature of a causal question. Position: Causal reasoning.
One move finishes the skill. A project partner offers what they call one question: “how long are our lines, and does a second register shorten them?” It is double-barreled, fusing the descriptive average of question 1 with the causal comparison of question 4, and no single design answers both. You split it, and each half keeps its own position.
That predicting well and explaining why are different goals, not stages of one goal, is argued in detail in the statistics literature (Shmueli 2010).
The block below builds this example’s data and prints the numbers this section quotes. Run it, then change one input and watch which sentence above stops being true.
import numpy as np, pandas as pd
SEED = 464
rng = np.random.default_rng(SEED)
# Twelve timed waits (minutes) at one register during weekday lunch.
waits = np.round(rng.gamma(shape=2.0, scale=1.8, size=12), 1)
print(pd.DataFrame({"customer": range(1, 13),
"wait (min)": waits}).to_string(index=False))
print(f"\naverage wait of these twelve : {waits.mean():.1f} min")
print("that number answers question 1 and nothing else: it describes the")
print("twelve you recorded, not the counter, not tomorrow, not a second register")6.4 An AI failure case
Suppose your project only records wait times at counters that already run two registers. Nothing is ever assigned; you measure counters as you find them. You ask a chatbot to classify “is a second register an effective way to cut waiting?” It reads the question as causal, which is right, and then reports the difference between busy two-register counters and quieter one-register counters as the effect. That is fluent, well-formatted, and wrong, because the counters that got a second register are the crowded ones. Crowding drives both the register and the wait, so the comparison carries no causal answer.
The tool’s error was not the label. It was skipping the step between a causal question and a causal answer: showing why this comparison isolates the register’s effect and nothing else. You catch it by asking what makes one counter differ from another, and whether that same thing also moves waiting time. Here it does. So your honest status is causal and currently unidentified, and you now choose openly. You can look for a design that carries the answer, such as opening periods assigned at random, or a schedule fixed in advance for reasons that have nothing to do with expected crowding. A schedule that quietly tracks the lunch rush is the same confounding wearing a clock. Or you can declare a descriptive question you can actually answer, write down that you changed it, and stop using the word “effective.”
6.5 It is your turn
You are working inside Studio 2: Set your rules, shape your question. Keep what you write here; the studio’s milestone chapter is where it joins the other lessons’ pieces into one artifact you can defend.
You arrive with a committed problem, a working agreement, an AI Research Ledger, and one verified delegation. This step classifies: kind and reach, fixed by the question’s own words. AI is welcome to challenge a placement; the final call is a recorded research decision of yours, and the next lesson turns the classified question into the formal declaration.
The hands-on half of this section lives in the chapter’s companion notebook: open it in Colab with the badge at the top, and work the steps there.
Commit your own answer first in every case. Write your classification down before you open the tool, so its reply has something of yours to disagree with. And treat a disagreement as the start of a loop rather than a verdict: if the tool assigns a different position, ask it which word in your sentence drove that reading, rewrite the sentence, and ask again. Two or three passes usually show whether the argument is about your wording or about your design.
Three decisions stay yours: which problem you will study, how you word your question, and which compass position it belongs to. A tool can offer a label, but only you can decide what answer your question truly wants, and only you can defend the placement when someone challenges it. If you cannot classify your question cleanly yet, that is a finding, not a failure; a question that will not sit in one box is usually still double-barreled, and naming that is progress.
Write your research problem out as questions, three or four of them, each a sentence evidence could answer and be wrong about. Problems almost always hide more than one.
Classify each question on both axes, kind and reach, and name the position. Beside each, write one line of reasoning based on the answer the question wants, not on the words it happens to use.
When you are ready to delegate this step:
Act as a research-methods tutor. I have already classified my question: [paste your question and your own kind + reach]. For my question, give its KIND (descriptive or causal) and REACH (data at hand, a population, or unseen cases), name the single word that most influenced you, and say whether answering it needs an intervention or only a pattern. Use a table.After running, verify (counters silent scope change): reread the position it returns against the two definitions, using the words of your question and not the data you happen to have. If it calls your question causal, ask whether your design can isolate that answer. If it cannot, the status is causal and currently unidentified, never descriptive.
Hunt the double-barrels. Any sentence with an “and” hiding a second question of a different kind gets split into two clean questions, each with its own position.
When you are ready to delegate this step:
Here is a question I think secretly asks two things: [paste it]. Split it into exactly two clean questions. For each half name its KIND and REACH. Do not add any unit, outcome, or population that was not in my sentence.After running, verify (counters illusion of completeness): a tidy split can quietly drop a half or smuggle in a new variable. Confirm both halves together cover your whole sentence and that neither added anything you did not write.
Pick your lead question, the one your project will be built around. Write one sentence on why it and not the others, and one sentence naming a claim its position forbids you even if the answer comes out beautifully.
If your lead question is causal, describe the comparison world it needs: the version of events that did not happen and that your design will have to stand in for. Then say whether your design can stand in for it. If it cannot yet, write “causal, currently unidentified” and keep the question. That is a design finding, not a reason to relabel what you asked.
Log the classification in your AI Research Ledger, and verify it with a named method from the Verification Guide. A classification is a judgment, so use peer reasoning: walk someone through the two definitions and your argument, and see whether it survives their hardest question. If your position is causal, add a second, independent check by drawing a causal diagram and confirming the effect is actually identified. An AI reviewer may run either check with you; the decision to accept or reject stays yours.
When you are ready to delegate this step:
Act as a hostile methodologist. Here is my question and the compass position I assigned it: [paste both]. Argue the strongest case that I classified it wrong. If you claim it is causal, name the method you would use and the one assumption that method needs, in a table.After running, verify (counters sycophantic agreement and plausible-but-wrong method): praise with no objection is a red flag, so force the objection. For any method it names, check whether your data can actually meet the assumption it lists. An assumption you cannot meet turns a confident method into a wrong one.
Your lead question now knows what kind of answer it wants and whom it may speak for. The next lesson writes it down for keeps: the formal declaration, under your rules.