Statistics & Probability: lesson 2 of 5
Statistics & Probability
Measures of Center, Spread, and Outliers
Choose and interpret mean, median, spread measures, and outliers in real Data Science datasets.
Concept
Measures of center describe a typical value; measures of spread describe how much values vary. Outliers are observations far from the rest of the data. These ideas help turn a list of salaries, spends, or delivery times into an interpretable summary.
Why It Matters
The “average” can mislead when data is skewed or contains extreme values. A model feature with very high spread may need investigation, while a sudden outlier can be a data-entry error or a meaningful fraud signal.
Center: Mean, Median, and Mode
The mean is the arithmetic average: add values and divide by their count. The median is the middle value after sorting. The mode is the most frequent value, often useful for categories.
Consider salaries: 45k, 48k, 50k, 52k, 900k. The mean is pulled toward the executive salary, while the median better represents the typical employee in this small group. The executive salary is not necessarily wrong; it answers a different question about total compensation.
Spread: Range, Variance, Standard Deviation, and IQR
The range is maximum minus minimum. Variance measures average squared distance from the mean. Standard deviation is the square root of variance, so it returns to the original unit. In words: low standard deviation means values cluster tightly; high standard deviation means they are more spread out.
For values 8, 9, 10, the mean is 9 and deviations are small. For 2, 9, 16, the mean is also 9 but values vary much more. The interquartile range (IQR) is the distance from the first quartile to the third quartile; it describes the middle half of the data and is less influenced by extremes.
Outliers
An outlier is unusually distant from other observations. A common IQR rule flags values below Q1 - 1.5 * IQR or above Q3 + 1.5 * IQR. This is a review rule, not an automatic deletion rule. A value may be a misplaced decimal, a legitimate enterprise order, or the very event a fraud analysis needs to study.
Real-World Example
Two stores each average 10,000 in daily revenue. Store A is consistently near that value; Store B alternates between 2,000 and 18,000. Equal means hide different planning risk. Standard deviation and distribution inspection explain the difference.
Interpretation and Use
Use mean for roughly symmetric values without dominating extremes, median for skewed monetary or duration data, and mode for common categories. Use standard deviation to compare consistency in the same unit, and IQR for robust spread and outlier review.
Failure Signals
Common Mistakes
- Deleting every outlier automatically.
- Reporting only a mean for strongly skewed data.
- Treating variance as easily interpretable in the original units.
- Assuming high variation means bad data rather than real heterogeneity.
Best Practices
Report center and spread together. Investigate outliers against source records and business context, document any removal rule, and compare summaries across meaningful segments. In modeling, decide whether outliers should be corrected, capped, transformed, or retained based on the objective.
Data Science Perspective
Center and spread guide EDA, cleaning, anomaly detection, scaling, and feature review. A value that is an outlier globally may be ordinary within a customer segment, so segmentation often matters before modeling.
Interview Perspective
Question: When is median preferable to mean? A strong answer: when the distribution is skewed or contains influential extremes, such as salaries or transaction amounts.
Practice Questions
- Which center measure would you report for executive-influenced salaries, and why?
- Two stores have equal mean revenue but different standard deviations. What does that tell you?
- Give one legitimate and one erroneous possible explanation for an extreme transaction.
Quick Quiz
- What unit does standard deviation use? Answer: the original data unit.
- What does IQR describe? Answer: the middle 50% of values.
- Are outliers always errors? Answer: no.
Worked Interpretation: Two Stores
Two stores each average 10,000 in daily revenue. Store A usually earns between 9,700 and 10,300; Store B ranges from 3,000 to 17,000. Their means match, but Store B has much greater spread, making staffing and inventory planning less predictable. Standard deviation describes that typical variation, while the range shows the full observed span. For a skewed salary dataset, the median may be more representative than a mean pulled up by one executive; the mode can help identify the most common defect type.
A Careful Outlier Workflow
When a value looks unusual, validate it against the source: check units, input rules, duplicate rows, and dates. Then compare it with domain knowledge. A million-dollar transaction may be a misplaced decimal in retail but a valid B2B order. Document whether you keep, correct, cap, or exclude it, and compare important results with and without it where practical. The IQR rule is a screening tool, not a verdict: it identifies observations worth investigating.
When presenting results, pair the chosen center with a spread measure and say what population it represents. A single average without variation can hide the operational risk that matters most.
Key Takeaway
Key Takeaways
Choose a center measure that matches the distribution, pair it with spread, and investigate outliers before changing data.
Next Lesson
Next, learn why samples vary and how the Central Limit Theorem supports later estimation.
Finish this lesson on your terms
Mark it complete when you have worked through the material and are ready to move on.