Structuring an Analytics Case Study

Start With the Decision, Not the Tool

An analytics case often begins with a prompt such as: "Customer retention has declined. What should we do?" That is not yet an analysis plan. It does not define retention, say which customers changed, identify a time period, or name the decision that the analysis should support.

Jumping straight to SQL, charts, or a churn model is premature. Those are possible tools, not an answer to the framing problem. A useful opening response is: "Before choosing an analysis, I would clarify the retention definition, the affected period and segments, the size of the change, and which decision the team needs to make." This shows momentum without pretending the prompt is already precise.

A Flexible Case Structure

Use this sequence to make a vague question tractable. It is a guide, not a script to recite in every interview.

1. Business decision: What action might change after the analysis? For example, should the team investigate onboarding, adjust outreach, or pause a product change?
2. Define the outcome: Does retention mean an active subscription after 30 days, a repeat purchase, or no cancellation within a quarter?
3. Set the unit and time: Is one row a customer, order, session, or cohort? What dates are being compared?
4. Segment the problem: Which customer segments, products, regions, acquisition channels, or cohorts could hide different behavior?
5. Choose metrics: Identify one primary measure and supporting measures that explain it.
6. Form hypotheses: State plausible explanations that need evidence, rather than conclusions.
7. Identify data: Name the records, definitions, and quality checks needed to test those hypotheses.
8. Prioritize analysis: Start with the checks most likely to change the decision.
9. Synthesize and recommend: Connect evidence, uncertainty, and a justified next action.

The order matters. A beautiful chart of the wrong metric can be less useful than a small, well-framed comparison.

Use a Metric Tree to Break Down a Vague Result

Suppose an e-commerce team says, "Revenue declined last month." Revenue is an outcome, not a diagnosis. A compact decomposition can turn it into measurable questions:

Revenue
-> number of purchasing customers
-> purchase frequency
-> average order value

Each branch suggests a different investigation. Fewer purchasing customers may point to traffic, acquisition, or conversion. Lower frequency may point to retention or product availability. Lower order value may be related to discounting, product mix, or a reporting change. The purpose is not to prove a causal tree; it is to avoid treating one headline metric as one undifferentiated problem.

For retention, a useful breakdown might compare cohort retention by acquisition channel, plan type, region, and time since signup. This makes it possible to ask whether a broad decline is actually concentrated in new customers or one recent cohort.

Hypotheses Are Not Findings

"An onboarding change caused churn" is a hypothesis. It becomes a finding only after the relevant timing, affected cohorts, behavior changes, and alternative explanations have been checked. A stronger phrasing is: "The timing makes onboarding a plausible explanation. I would compare eligible cohorts before and after the change, inspect completion behavior, and check whether a channel or plan shift explains the pattern instead."

This distinction protects against a common case-study failure: offering a confident recommendation before the evidence exists. It also keeps an analysis open to disconfirming evidence.

Mini-Case: Conversion Fell 12%

Imagine the prompt: "E-commerce conversion fell 12% last month. What would you investigate?"

Start with focused clarifications: Is this visitor-to-purchase conversion, and is the comparison month adjusted for seasonality or traffic mix? Did the decline occur across all devices and markets? Was there a site release, pricing change, or tracking change?

Then define a first pass. Confirm that the event definitions and denominator are stable. Break conversion into key funnel steps, segment by device, channel, geography, and new versus returning visitors, and locate when the decline began. Plausible hypotheses could include a mobile checkout issue, lower-intent campaign traffic, an unavailable product category, or an instrumentation change. The first analyses should favor large, falsifiable breaks over a list of twenty disconnected charts.

A recommendation boundary matters here. If the decline is concentrated on mobile immediately after a release, it may justify an urgent engineering investigation. If the evidence only shows a small shift in channel mix, it may justify validating acquisition quality before changing the product. Neither conclusion should be claimed before the checks are made.

Prioritize for Decision Value

Good case answers do not propose every analysis that could possibly be run. Start with the largest affected segment, major structural breaks, and checks that can quickly falsify an important explanation. Ask: "If this result were true, would it change the action?" If not, it is rarely a first-priority analysis.

This is especially important under interview time pressure. Name two or three high-value checks, explain why they come first, and say what you would investigate next if the result points somewhere useful.

Applied Rehearsal

For each prompt, state one key clarification, a primary metric, two hypotheses, the first analysis, and the decision it would support. Do not try to solve the case fully.

1. A subscription app reports that trial conversion has fallen.
2. A retailer wants to know why repeat purchases are lower in one region.
3. A marketplace sees more customer-support contacts after a product launch.
4. A bank asks which customers are "most valuable."
5. A marketing lead says a campaign "did not work."

Interview Perspective

What the interviewer is testing: Whether you can turn ambiguity into a decision-oriented plan before producing analysis. A likely follow-up is: "Which metric would you investigate first, and why?"

Next Lesson

Next, use that judgment to deliver a coherent analysis when time, data, and instructions are limited.