pandas groupby: How to Group and Aggregate Data in Python

Data · Published 2026-10-26

Bins of radishes, beetroot, celery and spring onions arranged side by side on a shop shelf — Produce sorted into bins, one kind per bin.

To total or average a column per category, call df.groupby("key")["column"].sum() (or .mean(), .count(), .max()). For several statistics with readable names, use named aggregation: df.groupby("key").agg(total=("column", "sum"), orders=("column", "size")). Use transform when you need the group result back on every row, and filter to keep or drop whole groups.

Split, apply, combine

A groupby call does three things: it splits the rows into groups by the values of one or more keys, applies a function to each group, and combines the results into a new Series or DataFrame. The pandas user guide on group by uses the same three words. Calling groupby on its own computes nothing; it returns a DataFrameGroupBy object that waits for the "apply" step.

The examples below use a small, made-up sales table and were run with pandas 3.0.5. One row has a missing units value on purpose, because that is where group statistics most often surprise people.

import pandas as pd

sales = pd.DataFrame({
    "region": ["North", "South", "North", "East", "South", "North", "East", "South"],
    "product": ["tea", "tea", "coffee", "tea", "coffee", "tea", "coffee", "tea"],
    "units": [12, 7, 5, 9, 14, 3, 8, None],
    "price": [2.5, 2.5, 4.0, 2.5, 4.0, 2.5, 4.0, 2.5],
})
sales["revenue"] = sales["units"] * sales["price"]
print(sales)

  region product  units  price  revenue
0  North     tea   12.0    2.5     30.0
1  South     tea    7.0    2.5     17.5
2  North  coffee    5.0    4.0     20.0
3   East     tea    9.0    2.5     22.5
4  South  coffee   14.0    4.0     56.0
5  North     tea    3.0    2.5      7.5
6   East  coffee    8.0    4.0     32.0
7  South     tea    NaN    2.5      NaN

If your own data comes from a CSV export that will not load, fix the encoding first; the guide to UnicodeDecodeError in read_csv covers the usual causes.

One key, one statistic

Select the column after groupby and call a reduction. The group labels become the index, sorted alphabetically by default:

print(sales.groupby("region")["revenue"].sum())

region
East     54.5
North    57.5
South    73.5
Name: revenue, dtype: float64

Pass a list of columns to get a DataFrame back, or a list of function names to agg for several statistics at once:

print(sales.groupby("region")[["units", "revenue"]].mean())
print(sales.groupby("region")["revenue"].agg(["sum", "mean", "max"]))

            units    revenue
region
East     8.500000  27.250000
North    6.666667  19.166667
South   10.500000  36.750000
         sum       mean   max
region
East    54.5  27.250000  32.0
North   57.5  19.166667  30.0
South   73.5  36.750000  56.0

Missing values are skipped, as in the rest of pandas: South's mean revenue is (17.5 + 56.0) / 2, not divided by three.

count() or size()?

The two look alike but answer different questions. count counts non-missing values in a column; size counts rows, missing or not:

print(sales.groupby("region")["units"].agg(["count", "size"]))

        count  size
region
East        2     2
North       3     3
South       2     3

Use size for "how many orders" and count for "how many orders have a quantity recorded". A gap between the two is a quick data-quality check.

Grouping by several columns

A list of keys gives one group per combination, with a MultiIndex on the result. Add as_index=False when you want ordinary columns instead, for example before writing to CSV or merging:

print(sales.groupby(["region", "product"], as_index=False)["units"].sum())

  region product  units
0   East  coffee    8.0
1   East     tea    9.0
2  North  coffee    5.0
3  North     tea   15.0
4  South  coffee   14.0
5  South     tea    7.0

To see the same numbers as a cross-table, unstack the inner level. pivot_table produces an identical result and reads more naturally when that layout is the goal:

print(sales.groupby(["region", "product"])["units"].sum().unstack(fill_value=0))
# same output:
# sales.pivot_table(index="region", columns="product", values="units", aggfunc="sum", fill_value=0)

product  coffee   tea
region
East        8.0   9.0
North       5.0  15.0
South      14.0   7.0

Named aggregation with agg()

For a summary table, named aggregation is the clearest form. Each keyword becomes an output column, and its value is a (column, function) pair:

summary = sales.groupby("region").agg(
    orders=("units", "size"),
    units=("units", "sum"),
    avg_price=("price", "mean"),
    revenue=("revenue", "sum"),
)
print(summary)

        orders  units  avg_price  revenue
region
East         2   17.0       3.25     54.5
North        3   20.0       3.00     57.5
South        3   21.0       3.00     73.5

The function can be a string such as "sum", "median", "nunique" or "first", or any callable that reduces a Series to one value, such as lambda s: s.quantile(0.9). Built-in names are much faster than lambdas on large frames because they run in optimised code. To rank groups, sort the result: summary.sort_values("revenue", ascending=False). If large totals start printing as 1.2e+09, see how to suppress scientific notation in pandas.

transform: the group result on every row

A reduction returns one row per group. transform returns a result with the same index as the original frame, which is what you need to compare each row with its group, for example each sale's share of its region's revenue:

region_total = sales.groupby("region")["revenue"].transform("sum")
sales["region_share"] = sales["revenue"] / region_total
print(sales[["region", "revenue", "region_share"]].round(3))

  region  revenue  region_share
0  North     30.0         0.522
1  South     17.5         0.238
2  North     20.0         0.348
3   East     22.5         0.413
4  South     56.0         0.762
5  North      7.5         0.130
6   East     32.0         0.587
7  South      NaN           NaN

The same pattern fills gaps with a group statistic (df["x"].fillna(df.groupby("g")["x"].transform("median"))) or centres values on their group mean.

filter: keep or drop whole groups

filter takes a function that receives each group as a DataFrame and returns True or False. Rows of groups that return True are kept, in their original order:

big = sales.groupby("region").filter(lambda g: g["revenue"].sum() > 55)
print(big[["region", "product", "revenue"]])

  region product  revenue
0  North     tea     30.0
1  South     tea     17.5
2  North  coffee     20.0
4  South  coffee     56.0
5  North     tea      7.5
7  South     tea      NaN

East, with 54.5 in revenue, is dropped. To inspect a single group instead, use sales.groupby("region").get_group("East"), or loop with for name, group in sales.groupby("region"): when you need per-group side effects such as one file per region.

Grouping by dates

To aggregate by month, group by a period derived from a datetime column, or use pd.Grouper with a frequency:

d = pd.DataFrame({
    "date": pd.to_datetime(["2026-01-05", "2026-01-20", "2026-02-03", "2026-02-28", "2026-03-15"]),
    "amount": [100, 50, 80, 20, 60],
})
print(d.groupby(d["date"].dt.to_period("M"))["amount"].sum())
print(d.groupby(pd.Grouper(key="date", freq="MS"))["amount"].sum())

date
2026-01    150
2026-02    100
2026-03     60
Freq: M, Name: amount, dtype: int64
date
2026-01-01    150
2026-02-01    100
2026-03-01     60
Freq: MS, Name: amount, dtype: int64

The period version gives tidy labels for tables; the Grouper version keeps real timestamps, which plotting libraries handle well. The property-price tutorial uses exactly this kind of grouping, by district and month, on a public dataset of real transactions.

Defaults that trip people up

Missing keys are dropped. Rows whose group key is NaN or None vanish from the result. Pass dropna=False to keep them as their own group.
Text columns break .mean() on the whole frame. sales.groupby("region").mean() raises TypeError in pandas 3 because product is a string column. Select the numeric columns first, or pass numeric_only=True.
Categorical keys show only observed categories. Since pandas 3.0, observed=True is the default, so an unused category (say, a "West" region with no sales) does not appear. Use observed=False to list it with a zero or NaN, which is useful for complete reports.
Groups are sorted. sort=True is the default; sort=False keeps the order in which keys first appear and can be faster on large data.
apply is the slow, general tool. Reach for agg, transform or filter first. Use apply only when the function must see the whole group and none of those fit, such as taking the top row per group with nlargest.

The full list of parameters is in the DataFrame.groupby reference.

Summary

groupby splits a frame by key values, applies a function and combines the results. Use a column selection plus a reduction for one statistic, named aggregation in agg for a summary table, transform to put group results back on each row, and filter to keep whole groups. Remember that missing keys are dropped unless dropna=False, that count ignores missing values while size does not, and that pandas 3 shows only observed categories by default.