Making Recommendations From Imperfect Evidence

Useful Does Not Mean Certain

Most analytical work contains uncertainty: observational data can be confounded, records can be missing, samples can be small, outcomes can be noisy, and models can make mistakes. The goal is not to eliminate every uncertainty before speaking. It is to say what the evidence supports, what it does not support, and what action is proportionate to the risk.

This is a practical skill in case interviews, take-homes, and project reviews. A confident claim unsupported by the evidence can send a team in the wrong direction. Excessive hedging can leave a team unable to act. Strong analysts do both: they make a recommendation and clearly describe its boundary.

Five Levels of Claim

Keep these levels separate when presenting a result.

Observation: What the data directly shows.
Interpretation: What that pattern may mean.
Hypothesis: A possible explanation that needs testing.
Recommendation: An action justified by the available evidence.
Causal claim: A stronger statement that requires an appropriate design or evidence.

Consider a churn analysis. The observation is: "Customers with three or more support tickets had a higher churn rate in this dataset." An interpretation is: "High support burden is associated with churn risk." A hypothesis is: "Poor support experience may contribute to churn." A reasonable recommendation is: "Prioritize this segment for qualitative investigation or a targeted retention experiment." The unsupported causal jump is: "Support tickets cause churn."

The wording changes, but so does the responsibility of the claim. A correlation can justify investigation or a reversible test; it does not by itself identify the cause.

Use Calibrated, Specific Language

Useful phrases include:

"We observed..."
"The pattern is consistent with..."
"The data suggests..."
"We cannot determine causality from this analysis."
"Given this evidence, I recommend..."
"The largest uncertainty is..."

These phrases are not substitutes for a conclusion. Pair each one with the relevant evidence and next action. "More data is needed" is weak if it does not say what data, why it matters, or what decision it would change.

Decide Under Uncertainty

Evidence need not be perfect for every decision. Consider the cost of action, cost of inaction, reversibility, risk, and expected benefit. A low-cost, reversible email experiment may be reasonable when evidence is suggestive but incomplete. A high-cost, irreversible pricing change needs stronger validation.

For example, imagine a marketing campaign shows a four-percent conversion increase for new subscribers but flat results overall. A weak response is: "Roll it out to everyone." A stronger response is: "The increase appears limited to new subscribers, and we should confirm sample size, channel mix, and whether the segment was preselected. I would extend or replicate the campaign for that segment while monitoring conversion and downstream quality before a broad rollout." The recommendation is actionable without pretending the observed association is universal.

Communicate Model Outputs Honestly

If a retention model flags customers as high risk, do not say, "These customers will churn." Say: "The model assigns higher estimated risk based on patterns in historical data." Then connect the score to a business action, capacity, expected errors, and limitations.

The right action might be to rank customers for limited outreach, not automatically grant discounts. A threshold is a business choice as well as a modeling choice: lower thresholds may reach more customers and create more unnecessary contacts; higher thresholds may miss some at-risk customers. The output should guide a decision process, not become an unquestioned fact about an individual.

Mention the Limitations That Matter

Do not append a generic list of every possible limitation. Prioritize the limitations that could change the recommendation: selection bias, missing data, temporal mismatch, confounding, inconsistent measurement, tiny segments, or model error. If a campaign was shown only to customers who were already highly engaged, that selection can materially affect how the result should be interpreted. If a customer history ends before the outcome window, the evidence may be incomplete.

An honest limitation can lead directly to the next validation step. For example: "Because this comparison is observational and engagement differs across groups, I would test the intervention on a comparable eligible population before attributing the difference to the campaign."

A Recommendation Format That Stays Grounded

Use this compact sequence in a case answer or executive summary:

Evidence -> interpretation -> recommendation -> risk or uncertainty -> next validation step

For instance: "Repeat purchase declined most in first-time mobile buyers after the checkout release. This is consistent with a mobile checkout issue, although traffic mix may also have changed. I recommend investigating the release and running targeted funnel checks before broader merchandising changes. The key uncertainty is whether the pattern persists after controlling for acquisition channel."

This structure makes the chain of reasoning visible and prevents recommendations from floating free of their evidence.

Failure Signals

Common Mistakes

Converting an association directly into a causal claim.
Saying nothing because the evidence is imperfect.
Burying the recommendation beneath analysis detail.
Presenting a model score without explaining the business action or error risk.
Using generic uncertainty language instead of naming the material limitation.
Recommending an action that does not follow from the finding.

Applied Rehearsal

For each result, identify the observation, one unsupported claim, a reasonable recommendation, a material limitation, and a next validation step.

Customers on annual plans churn less often than monthly customers.
A product feature is used more frequently by high-spending customers.
A fraud model gives some transactions a high-risk score.
Delivery time rose in one city during a holiday week.
A campaign improved clicks but not purchases overall.

Interview Perspective

What the interviewer is testing: Whether you can make a decision-oriented recommendation without overstating causality or hiding uncertainty. A likely follow-up is: "What evidence would make you more confident in this recommendation?"

Key Takeaway

Key Takeaways

Separate what the data shows from what it might mean. Make recommendations proportional to evidence and risk, name the limitation that matters most, and propose a concrete way to validate the next decision.

Next Lesson

Next, turn completed analytical work into a clear project story for interviews and portfolio conversations.