CTE + GROUP BY — Aggregate a Daily Sales Summary for an API
A CTE (Common Table Expression) is a temporary named query defined as WITH name AS (...). In batch processing, a standard pattern is to split the three stages—aggregation, filtering, and formatting—into CTEs.
WITH aggregate_name AS ( -- ① Define the aggregation (GROUP BY) here SELECT col1, SUM(col2) AS total FROM table_name GROUP BY col1 -- Aggregate rows for each col1 value ) -- ② Filter and sort the aggregated result SELECT * FROM aggregate_name WHERE total > 10000 -- A CTE lets WHERE reference the aggregated total ORDER BY col1; -- Sort using the default ASC order
WHERE SUM(...) and must use HAVING. Moving the aggregation into a CTE lets you treat its result as an ordinary column. As batch SQL grows, naming and separating processing stages becomes increasingly valuable.From the orders table below, calculate the total sales, order count, and average order amount for each order_date.
Return only dates whose total sales are at least 50,000 yen, sorted by total sales in descending order.
| order_id | order_date | amount |
|---|---|---|
| 1 | 2024-04-01 | 20000 |
| 2 | 2024-04-01 | 35000 |
| 3 | 2024-04-02 | 60000 |
| 4 | 2024-04-02 | 15000 |
| 5 | 2024-04-03 | 80000 |
| 6 | 2024-04-03 | 40000 |
| 7 | 2024-04-04 | 12000 |
| 8 | 2024-04-04 | 18000 |
| order_date | total_sales | order_count | avg_order |
|---|---|---|---|
| 2024-04-03 | 120000 | 2 | 60000 |
| 2024-04-02 | 75000 | 2 | 37500 |
| 2024-04-01 | 55000 | 2 | 27500 |
-- WITH: split processing into a CTE WITH daily_stats AS ( -- ① Aggregate orders by date SELECT order_date, SUM(amount) AS total_sales, COUNT(*) AS order_count, AVG(amount) AS avg_order FROM orders GROUP BY order_date ) -- ② Filter and sort the CTE result SELECT order_date, total_sales, order_count, avg_order FROM daily_stats WHERE total_sales >= 50000 ORDER BY total_sales DESC; /* Evaluation order: 1. CTE daily_stats → aggregate by date (SUM/COUNT/AVG) 2. Outer WHERE → filter by the total_sales threshold 3. ORDER BY total_sales DESC → return rows in descending order */
LEGEND
① FROM
FROM ordersRead all eight rows from orders. In the next step, GROUP BY collects rows by date.| order_id | order_date | amount |
|---|---|---|
| 1 | 2024-04-01 | 20,000 |
| 2 | 2024-04-01 | 35,000 |
| 3 | 2024-04-02 | 60,000 |
| 4 | 2024-04-02 | 15,000 |
| 5 | 2024-04-03 | 80,000 |
| 6 | 2024-04-03 | 40,000 |
| 7 | 2024-04-04 | 12,000 |
| 8 | 2024-04-04 | 18,000 |
COUNT(*) counts every row, including rows containing NULL, while COUNT(col) excludes NULL values in that column. Use the former when counting row existence itself.WHERE SUM(...) fails because of SQL evaluation order. Use HAVING or an outer WHERE around a CTE.LAG() Window Function — Calculate Prior-Month Differences and Growth Rates
LAG() is a window function that brings a value from one row earlier (or n rows earlier) onto the current row. It is essential for month-over-month and day-over-day calculations.
LAG(value_column, 1, 0) OVER ( PARTITION BY group_column -- Reference the previous row independently within each group ORDER BY sort_column -- This order determines which row is previous ) AS previous_value
The arguments are LAG(column, offset, default). The offset defaults to 1 (one row earlier), and the default value is used when no previous row exists, as on the first row.
From the monthly_sales table below, calculate each month’s previous-month sales, difference from the previous month, and month-over-month growth rate (%).
| month | sales |
|---|---|
| 2024-01 | 100000 |
| 2024-02 | 130000 |
| 2024-03 | 120000 |
| 2024-04 | 160000 |
| 2024-05 | 145000 |
| month | sales | prev_sales | diff | growth_rate |
|---|---|---|---|---|
| 2024-01 | 100000 | 0 | NULL | NULL |
| 2024-02 | 130000 | 100000 | +30000 | 30.00 |
| 2024-03 | 120000 | 130000 | -10000 | -7.69 |
| 2024-04 | 160000 | 120000 | +40000 | 33.33 |
| 2024-05 | 145000 | 160000 | -15000 | -9.38 |
-- WITH: split processing into a CTE WITH sales_with_prev AS ( -- ① Use LAG to place previous-month sales on the current row SELECT month, sales, LAG(sales, 1, 0) OVER ( ORDER BY month ) AS prev_sales FROM monthly_sales ) -- ② Calculate the difference and growth rate SELECT month, sales, prev_sales, CASE WHEN prev_sales = 0 THEN NULL ELSE sales - prev_sales END AS diff, CASE WHEN prev_sales = 0 THEN NULL ELSE ROUND((sales - prev_sales)::numeric / prev_sales * 100, 2) END AS growth_rate FROM sales_with_prev ORDER BY month; /* Evaluation order: 1. CTE sales_with_prev → attach previous-month sales with LAG 2. Outer CASE → calculate the difference and growth rate 3. ORDER BY month → return rows in ascending month order */
LEGEND
① FROM
FROM monthly_salesRead all five rows from monthly_sales. In the next step, LAG adds the sales value from one row earlier to each row.| month | sales |
|---|---|
| 2024-01 | 100,000 |
| 2024-02 | 130,000 |
| 2024-03 | 120,000 |
| 2024-04 | 160,000 |
| 2024-05 | 145,000 |
LAG(sales, 1, 0) prevents NULL on the first row. This is useful when an API response should avoid NULL.::numeric or an equivalent type before calculating.CASE WHEN prev_sales = 0 THEN NULL to prevent division-by-zero errors.LAG(sales, 7), which helps remove day-of-week noise, is another common pattern.SUM() OVER (ROWS BETWEEN) — Calculate a Running Sales Total and Moving Average
The ROWS BETWEEN clause of a window function precisely specifies which range of rows to aggregate for each current row.
SUM(sales) OVER ( ORDER BY month ROWS BETWEEN UNBOUNDED PRECEDING -- From the first row AND CURRENT ROW -- Through the current row → running total ) AS running_total AVG(sales) OVER ( ORDER BY month ROWS BETWEEN 2 PRECEDING -- From two rows earlier AND CURRENT ROW -- Through the current row → three-month moving average ) AS moving_avg_3m
From the monthly_sales table below, calculate each month’s running sales total (running_total) and moving average for the latest three months (moving_avg_3m).
| month | sales |
|---|---|
| 2024-01 | 80000 |
| 2024-02 | 120000 |
| 2024-03 | 100000 |
| 2024-04 | 150000 |
| 2024-05 | 130000 |
| 2024-06 | 170000 |
| month | sales | running_total | moving_avg_3m |
|---|---|---|---|
| 2024-01 | 80000 | 80000 | 80000.0 |
| 2024-02 | 120000 | 200000 | 100000.0 |
| 2024-03 | 100000 | 300000 | 100000.0 |
| 2024-04 | 150000 | 450000 | 123333.3 |
| 2024-05 | 130000 | 580000 | 126666.7 |
| 2024-06 | 170000 | 750000 | 150000.0 |
SELECT month, sales, SUM(sales) OVER ( -- ① Running sales total: sum from the first row through the current row ORDER BY month ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW ) AS running_total, ROUND( -- ② Three-month moving average: average from two rows earlier through the current row AVG(sales) OVER ( ORDER BY month ROWS BETWEEN 2 PRECEDING AND CURRENT ROW )::numeric, 1 ) AS moving_avg_3m FROM monthly_sales ORDER BY month; /* Evaluation order: 1. FROM monthly_sales → read rows 2. SUM() OVER (...) → calculate the running sales total 3. AVG() OVER (...) → calculate the three-month moving average 4. ORDER BY month → return rows in ascending month order */
LEGEND
① FROM
FROM monthly_salesRead all six rows from monthly_sales. In the next step, SUM OVER adds the running total from the first row through the current row.| month | sales |
|---|---|
| 2024-01 | 80,000 |
| 2024-02 | 120,000 |
| 2024-03 | 100,000 |
| 2024-04 | 150,000 |
| 2024-05 | 130,000 |
| 2024-06 | 170,000 |
ROWS BETWEEN defines a window by physical row count, while RANGE BETWEEN uses a range of values. Prefer ROWS when its clearer and typically faster semantics match the requirement.SUM() OVER() returns the total across all rows. A running total requires ORDER BY and ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW.ROW_NUMBER() + CTE — Build an API for Top-N Products by Category
APIs often need only the top N rows in each category. A standard pattern assigns each row a rank within its group using ROW_NUMBER() OVER (PARTITION BY ... ORDER BY ...), isolates that result in a CTE, and then filters with WHERE rn <= N.
WITH ranked AS ( SELECT *, ROW_NUMBER() OVER ( PARTITION BY category -- Number rows independently within each category ORDER BY sales DESC, product_id -- Break ties by product_id ascending ) AS rn FROM products ) SELECT * FROM ranked WHERE rn <= 2; -- Keep only the top two rows in each category
From the product_sales table below, extract the top two products by sales within each category.
| product_id | category | product_name | sales |
|---|---|---|---|
| P01 | Food | Apple | 85000 |
| P02 | Food | Banana | 62000 |
| P03 | Food | Mandarin | 62000 |
| P04 | Food | Grape | 41000 |
| P05 | Refreshments | Green tea | 95000 |
| P06 | Refreshments | Coffee | 78000 |
| P07 | Refreshments | Juice | 78000 |
| P08 | Refreshments | Water | 55000 |
| category | product_name | sales | rn |
|---|---|---|---|
| Food | Apple | 85000 | 1 |
| Food | Banana | 62000 | 2 |
| Refreshments | Green tea | 95000 | 1 |
| Refreshments | Coffee | 78000 | 2 |
-- WITH: split processing into a CTE WITH ranked_products AS ( -- ① Rank sales within each category SELECT category, product_name, sales, ROW_NUMBER() OVER ( PARTITION BY category ORDER BY sales DESC, product_id ) AS rn FROM product_sales ) -- ② Extract the top two rows in each category SELECT category, product_name, sales, rn FROM ranked_products WHERE rn <= 2 ORDER BY category, rn; /* Evaluation order: 1. CTE ranked_products → rank sales within each category 2. Outer WHERE rn <= 2 → extract the top two rows per category 3. ORDER BY category, rn → sort and return the result */
LEGEND
① FROM
FROM product_salesRead all eight rows from product_sales. In the next step, PARTITION BY forms one group for each category.| product_id | category | product_name | sales |
|---|---|---|---|
| P01 | Food | Apple | 85,000 |
| P02 | Food | Banana | 62,000 |
| P03 | Food | Mandarin | 62,000 |
| P04 | Food | Grape | 41,000 |
| P05 | Refreshments | Green tea | 95,000 |
| P06 | Refreshments | Coffee | 78,000 |
| P07 | Refreshments | Juice | 78,000 |
| P08 | Refreshments | Water | 55,000 |
ORDER BY created_at DESC are essential data-mart patterns.CASE WHEN + SUM — Pivot Counts by Status
Pivot aggregation, which expands row-oriented data into columns, is common in dashboard APIs. Because many SQL dialects lack a dedicated PIVOT statement, it can be implemented with CASE WHEN + SUM.
SELECT category, SUM(CASE WHEN status = 'done' THEN 1 ELSE 0 END) AS completed, SUM(CASE WHEN status = 'pending' THEN 1 ELSE 0 END) AS pending FROM tasks GROUP BY category;
COUNT(*) FILTER (WHERE status = 'done') produces the same result. CASE WHEN is more portable because some databases, including BigQuery, do not support this FILTER syntax.From the tasks table below, return one row per assignee containing counts by status (done, in progress, and not started) plus the total count.
| task_id | assignee | status |
|---|---|---|
| T01 | Baker | done |
| T02 | Baker | done |
| T03 | Baker | in progress |
| T04 | Baker | not started |
| T05 | Clark | done |
| T06 | Clark | in progress |
| T07 | Clark | in progress |
| T08 | Adams | not started |
| T09 | Adams | not started |
| T10 | Adams | done |
| assignee | done | in_progress | not_started | total |
|---|---|---|---|---|
| Adams | 1 | 0 | 2 | 3 |
| Baker | 2 | 1 | 1 | 4 |
| Clark | 1 | 2 | 0 | 3 |
SELECT assignee, SUM(CASE WHEN status = 'done' THEN 1 ELSE 0 END) AS done, -- ① Count done tasks SUM(CASE WHEN status = 'in progress' THEN 1 ELSE 0 END) AS in_progress, -- ② Count tasks in progress SUM(CASE WHEN status = 'not started' THEN 1 ELSE 0 END) AS not_started, -- ③ Count tasks not started COUNT(*) AS total -- ④ Count all tasks FROM tasks GROUP BY assignee ORDER BY assignee; /* Evaluation order: 1. FROM tasks → read rows 2. GROUP BY assignee → group by assignee 3. SUM(CASE...) → aggregate counts by status (pivot) 4. ORDER BY assignee → return rows in ascending order */
LEGEND
① FROM
FROM tasksRead all ten rows from tasks. In the next step, GROUP BY groups the data by assignee.| task_id | assignee | status |
|---|---|---|
| T01 | Baker | done |
| T02 | Baker | done |
| T03 | Baker | in progress |
| T04 | Baker | not started |
| T05 | Clark | done |
| T06 | Clark | in progress |
| T07 | Clark | in progress |
| T08 | Adams | not started |
| T09 | Adams | not started |
| T10 | Adams | done |
SUM(CASE WHEN ... THEN 1 ELSE 0 END) is an idiom for counting rows that satisfy a condition.ELSE 0 to prevent unintended behavior.