GroupBy and Aggregation

Concept

Grouping summarizes values by category. Pandas groupby() follows a useful mental model: split rows into groups, apply a calculation to each group, then combine the results. This is how analysts answer questions such as revenue by region or average order value by segment.

Why It Matters

A total across all customers can hide important differences. Grouping may show that one region drives revenue, one campaign has unusually low conversion, or one product category has many orders but small values.

Real-World / Data Science Example

import pandas as pd

orders = pd.DataFrame({
    "region": ["North", "South", "North", "South"],
    "segment": ["New", "New", "Returning", "Returning"],
    "revenue": [120, 80, 220, 150],
    "order_id": [1, 2, 3, 4],
})

revenue_by_region = orders.groupby("region")["revenue"].sum()
print(revenue_by_region)

Expected output:

region
North    340
South    230
Name: revenue, dtype: int64

Read it left to right: split orders by region, choose the revenue column, then sum each group's revenue.

Count, Mean, Min, and Max

summary = orders.groupby("segment")["revenue"].agg(
    order_count="count",
    total_revenue="sum",
    average_order_value="mean",
    smallest_order="min",
    largest_order="max",
)
print(summary)

agg() calculates several metrics in one readable table. Make sure the selected column matches the metric: counting order_id answers a different question from summing revenue.

Grouping by More Than One Column

region_segment_sales = (
    orders.groupby(["region", "segment"])["revenue"]
    .sum()
    .reset_index()
)
print(region_segment_sales)

Multiple group columns produce a multi-level index by default. reset_index() turns those group labels back into ordinary columns, which is often easier to inspect, merge, or chart.

Technical Explanation

groupby() does not calculate anything until you apply an aggregation such as sum() or mean(). The output has one row per group, not one row per original order. Missing group labels are normally excluded by default, so inspect whether null categories should be handled separately.

Interpreting Grouped Results

Grouping changes the unit of analysis. Before grouping, each row in orders is one order. After groupby("region")["revenue"].sum(), each row in the result is one region. This is why a grouped total can answer “which region generated the most revenue?” but cannot identify a specific order without returning to the original table.

For a region-and-month report, group by both columns, then check whether every region-month combination is represented. A missing combination may mean zero sales, absent source data, or a category that was not recorded; those cases should not be treated as identical without context.

region_summary = (
    orders.groupby("region")
    .agg(order_count=("order_id", "count"), revenue_total=("revenue", "sum"))
    .reset_index()
)

This form makes the source column for every metric explicit and produces a table ready to merge or visualize.

When to Use It

Use grouping for descriptive analysis, KPI reports, cohort comparisons, campaign summaries, and feature creation. It is not a substitute for checking distribution and sample size: a segment average from two records may be unstable.

Best Practices

Name output metrics clearly, inspect group sizes, and use reset_index() when the result needs normal columns. Check whether each group represents a comparable population and decide how missing categories should be reported.

Data Science Perspective

GroupBy is central to exploratory analysis and business analytics. The same summaries can reveal feature patterns before ML, such as average churn by plan, but a grouped association is not automatically a causal explanation.

Interview Perspective

Question: What does df.groupby("region")["revenue"].sum() do? A strong answer: it partitions rows by region, selects revenue, and returns the revenue total for each region.

Practice Questions

1. Calculate order count by product category.
2. Calculate mean and maximum revenue by marketing channel with agg().
3. Why might an average order value be misleading for a segment with only two orders?
4. Explain why grouped output should be interpreted at its new row granularity.

Quick Quiz

1. What are the three GroupBy stages? Answer: split, apply, combine.
2. Which method can return several named metrics? Answer: agg().
3. Why use reset_index() after grouping? Answer: to turn group labels back into columns.
4. What does a group with a small count require? Answer: cautious interpretation because its summary may be unstable.

Next Lesson

Next, combine related tables while checking that joins do not silently duplicate your analysis.