Data scientist interview guide
SQL, statistics and experimentation, product metrics cases, modelling, and behavioral. What data science interviewers listen for and how to structure a metrics answer.
Data scientist loops vary more than most. An analytics-leaning role will be heavy on SQL, experiment design, and product metric cases. A modelling-leaning role adds ML fundamentals and a coding round. Almost every loop includes a “what would you do” case on a real product metric, and that is where candidates who know the statistics still fall down.
The skill being tested in the case round is judgement under ambiguity: can you turn a vague business question into a metric, an experiment, and a recommendation, and say what could be wrong with your answer?
What the rounds look like
SQL is usually two or three questions in a shared editor: joins across event tables, window functions for retention or ranking, and a “find the bug in this query” variant. Speed matters less than getting the grain of each table right.
Statistics and experimentation is a conversation: sample size and power, what to do with p = 0.06, novelty effects, multiple comparisons, guardrail metrics, and when an A/B test is the wrong tool.
The product case gives you a scenario (“DAU dropped 10% week over week”) and watches how you decompose it. The modelling round, if present, looks like a lighter version of the ML engineer fundamentals round.
What interviewers listen for
- A precise metric definition before any analysis: numerator, denominator, time window, and who is excluded.
- Decomposition of a drop: by platform, by geography, by new versus returning users, by funnel step; instrumentation and calendar effects checked first.
- Experiment design rigour: the unit of randomisation, the minimum detectable effect, the run length, and the guardrails.
- Knowing when a result is not significant and saying so without hedging into nothing.
- Awareness of paradoxes and traps: Simpson’s paradox, survivorship, peeking, and network effects between test and control.
- Translation for a non-technical audience: what the number means and what you recommend.
How to structure an answer
For a product case: clarify the goal, define the metric, propose a north star and two guardrails, lay out how you would decompose or test, describe the analysis, and end with the decision and its main risk. Say the structure out loud at the start so the interviewer can follow along.
For a statistics question: give the direct answer first, then the reasoning, then the caveat. “I would not ship on p = 0.06 alone; I would look at the effect size and confidence interval, check whether we were underpowered, and if the effect is directionally large I’d extend the test rather than call it.”
Questions to practise
- An A/B test on checkout conversion comes back at p = 0.06. Do you ship?
- How do you decide the sample size for an experiment on a metric with 2% baseline conversion?
- Daily active users fell 10% week over week. Walk me through how you would find out why.
- Write a SQL query for 7-day retention by signup cohort.
- Explain a p-value to a product manager in two sentences.
- Give an example of Simpson’s paradox in a product metric and how you would catch it.
- You cannot run an A/B test on this change. How else would you measure its effect?
- When would you use logistic regression over a gradient-boosted model, and vice versa?
- How do you handle missing data, and how does the answer depend on why it is missing?
- Describe a model you built and how you explained its output to a non-technical stakeholder.