CTEs and the WITH Clause — Learn Query Organization and Refactoring from the Basics
BasicCTEs (WITH clause)SQL refactoringReadabilityPostgreSQL-ready5 questions
QUESTION 1
WITH Clause Basics — Build a Temporary Data Set with a CTE
WITHCTEFundamentalsTemporary table
Background

A CTE (Common Table Expression) is syntax that uses a WITH clause to define a “temporary table” at the beginning of a SQL statement.

It dramatically improves SQL readability by splitting up complex subqueries and allowing the code to be read naturally from top to bottom.

WITH cte_name AS (         -- 1. Define a temporary table named cte_name here
  SELECT id, name
  FROM   users
  WHERE  status = 'active'
)
SELECT *                   -- 2. Use the defined cte_name in the main query
FROM cte_name;
Why use a CTE: Unlike subqueries whose nested parentheses can become deeply layered, CTEs let you build SQL with the programming-like idea of “assigning something to a variable and using it later.”
Problem

Define a CTE named active_users, a temporary table containing only users whose status is 'active' from the users table.

Then retrieve user_id and name from active_users in the main query.

Table used
▸ users
user_idnamestatus
1Aliceactive
2Bobinactive
3Carolactive
Expected Output
user_idname
1Alice
3Carol
Model Answer
-- 1. Define a CTE (temporary table) named active_users with WITH
WITH active_users AS (
  SELECT
    user_id,
    name
  FROM  users
  WHERE status = 'active'  -- keep only active rows
)

-- 2. Main query: treat the CTE like a regular table
SELECT
  user_id,
  name
FROM active_users;  -- name the CTE in the FROM clause

/*
  Execution order (fundamentals):
  1. CTE definition: filter rows with status='active' from users,
                     creating the virtual table active_users in memory
  2. Main query: read from active_users and return the required columns
*/
Explanation (table transitions & key points)
WITH active_users AS ( SELECT user_id, name FROM users WHERE status = 'active' ) SELECT user_id, name FROM active_users;
LEGEND
Columns / keys under evaluation
Excluded / hidden data
1. Evaluate the Query Inside the CTE
FROM users WHERE status = 'active'First, the SELECT inside the WITH clause is evaluated. Rows that do not satisfy status = 'active' are removed from the entire users table, leaving only the required data.
1 / 3
user_idnamestatus
1Aliceactive
2Bobinactive
3Carolactive
3 rows → 2 rows
LEARNING POINTS
CTE (WITH clause) basics: SQL normally begins with SELECT, but a CTE lets you define a component—a virtual table—at the start of the query as WITH name AS ( query ). This maps the thought process “first carve out only the data needed from the large source table” directly into code.
Scope: A defined CTE is available only within the single main statement that immediately follows it, such as SELECT, UPDATE, or DELETE. It is discarded when that statement finishes.
Field Notes
In production, you would not use a CTE for work that a simple WHERE clause can handle, as in this exercise. Once complex aggregation and joins are involved, however, extracting meaningful units such as “active users” or “this month’s sales” into CTEs makes the code’s intent much clearer in team development.
QUESTION 2
Define Multiple CTEs — Chain Virtual Tables with Commas
Multiple CTEs,JOINReadability
Background

You can define multiple CTEs by separating them with commas. This is especially useful when you need to preprocess several tables—by filtering or aggregating them—before a JOIN.

WITH cte_A AS (
  SELECT ... -- first CTE
),
cte_B AS (
  SELECT ... -- second CTE (connect with a comma; do not repeat WITH)
)
SELECT * FROM cte_A JOIN cte_B ON ... ;
Do not forget the comma: A common beginner syntax error is to omit the comma before the second or later CTE, or to write WITH again.
Problem

Create a CTE named premium_users that selects users with rank = 'premium' from users.

Next, create a CTE named high_orders that selects orders with amount >= 5000 from orders.

Finally, INNER JOIN the two CTEs on user_id in the main query and return the user name (name) and order amount (amount).

Tables used
▸ users
user_idnamerank
1Alicepremium
2Bobnormal
3Carolpremium
▸ orders
order_iduser_idamount
118000
2212000
333000
439000
Expected Output
nameamount
Alice8000
Carol9000
Model Answer
-- First CTE: select premium users
WITH premium_users AS (
  SELECT user_id, name
  FROM users
  WHERE rank = 'premium'
),                        -- connect CTEs with a comma (essential)
high_orders AS (          -- Second CTE: select high-value orders of 5000 or more
  SELECT user_id, amount
  FROM orders
  WHERE amount >= 5000
)

-- Main query: join the two CTEs
SELECT
  p.name,
  h.amount
FROM premium_users p
JOIN high_orders h
  ON p.user_id = h.user_id;

/*
  Execution order:
  1. Evaluate premium_users → hold it temporarily in memory
  2. Evaluate high_orders   → hold it temporarily in memory
  3. JOIN in the main query → match on user_id
  */
Explanation (table transitions & key points)
WITH premium_users AS ( SELECT user_id, name FROM users WHERE rank = 'premium' ), high_orders AS ( SELECT user_id, amount FROM orders WHERE amount >= 5000 ) SELECT p.name, h.amount FROM premium_users p JOIN high_orders h ON p.user_id = h.user_id;
LEGEND
Columns / keys under evaluation
Excluded / hidden data
1. Build the First CTE
CTE: premium_usersFirst, select the specified users from users to build the first temporary table, premium_users. Unneeded data is removed.
1 / 4
user_idnamerank
1Alicepremium
2Bobnormal
3Carolpremium
3 rows → 2 rows
LEARNING POINTS
Filter before joining (pre-filtering): In some cases, removing unnecessary rows from each table in CTEs before joining performs better than joining large tables first and filtering afterward with WHERE, although the result depends on the database optimizer.
ANTI-PATTERNS
Writing WITH before the second CTE: WITH cte_A AS (...), WITH cte_B AS (...) is a syntax error. Write WITH only once at the beginning of the query and connect subsequent CTEs with commas.
Field Notes
Large SQL statements in data engineering and analytics platforms such as BigQuery or Snowflake commonly chain ten or more CTEs. CTEs are indispensable for separating processing into explicit steps such as “aggregate A,” “reshape B,” “join A and B,” and “format the final result.”
QUESTION 3
CTEs and Aggregation — Join Aggregate Results to User Information
CTE + GROUP BYJOINAggregationUser analysis
Background

One of the most useful CTE patterns is joining results aggregated with GROUP BY to master data such as user information.

Forcing aggregation (GROUP BY) and joining (JOIN) into one SELECT can make the code complicated. Extracting the aggregation into a CTE keeps the statement clean.

Problem

Create a CTE named user_sales that calculates each user’s total order amount (total_amount) from orders.

Then JOIN users and user_sales in the main query and return the user name (name) and total order amount.

Sort the output by total order amount in descending order, highest first.

Tables used
▸ users
user_idname
1Alice
2Bob
3Carol
▸ orders
order_iduser_idamount
115000
213000
3210000
Expected Output
nametotal_amount
Bob10000
Alice8000
Model Answer
-- 1. CTE: aggregate total order amount per user from orders
WITH user_sales AS (
  SELECT
    user_id,
    SUM(amount) AS total_amount
  FROM orders
  GROUP BY user_id
)

-- 2. Main query: join the aggregate CTE to users to retrieve names
SELECT
  u.name,
  s.total_amount
FROM users u
JOIN user_sales s
  ON u.user_id = s.user_id
ORDER BY s.total_amount DESC;

/*
  Execution order:
  1. Evaluate user_sales         → aggregate orders by user_id
  2. JOIN users and user_sales   → match on user_id
  3. ORDER BY                    → sort total amount descending
  */
Explanation (table transitions & key points)
WITH user_sales AS ( SELECT user_id, SUM(amount) AS total_amount FROM orders GROUP BY user_id ) SELECT u.name, s.total_amount FROM users u JOIN user_sales s ON u.user_id = s.user_id ORDER BY s.total_amount DESC;
LEGEND
Grouping keys / aggregation targets
Group classification
1. Group Aggregation in the CTE
GROUP BY user_id → SUM(amount)Group the orders table by user—a vertical compression—and calculate total amounts. The aggregate result becomes a temporary table named user_sales.
1 / 3
user_idtotal_amount
18000
210000
3 rows → 2 groups
LEARNING POINTS
Separate aggregation from joining: It is confusing to calculate totals per group—a vertical compression—while simultaneously attaching another table—a horizontal expansion. Finishing the vertical compression in a CTE and creating a simple one-to-one mapping first makes the horizontal expansion (JOIN) in the main query highly intuitive.
CTEs as an alternative to subqueries: This SQL could also use a subquery inside FROM, as in FROM users u JOIN (SELECT user_id, SUM(amount)...) s ON .... Extracting it to a top-level CTE instead makes what is being aggregated immediately clear from the name user_sales and dramatically improves the code’s structure and readability.
QUESTION 4
Referencing Other CTEs — Build Logic Step by Step
CTE chainsStaged processingRefactoringPipeline
Background

When multiple CTEs are defined in a comma-separated chain, a later CTE can reference a CTE defined before it.

WITH cte_1 AS (
  SELECT ... -- Step A (for example, remove unnecessary data)
),
cte_2 AS (
  SELECT ... FROM cte_1  -- Step B (further aggregate Step A’s result)
)
SELECT * FROM cte_2;     -- final output

Chaining processing in this way lets you read and write SQL like a data pipeline flowing from top to bottom.

Problem

Implement the following steps as a chain of CTEs over the orders table.

  1. CTE valid_orders: select only orders whose status is not 'cancelled'.
  2. CTE user_totals: aggregate each user’s total amount (total) from valid_orders.
  3. Main query: from user_totals, retrieve user_id and total for users whose total is at least 10000.
Table used
▸ orders
order_iduser_idamountstatus
1115000completed
215000cancelled
328000completed
423000completed
Expected Output
user_idtotal
115000
211000
Model Answer
-- 1. First CTE: exclude canceled orders
WITH valid_orders AS (
  SELECT user_id, amount
  FROM orders
  WHERE status != 'cancelled'
),
user_totals AS (               -- 2. Second CTE: aggregate by referencing valid_orders
  SELECT
    user_id,
    SUM(amount) AS total
  FROM valid_orders            -- reference the previously defined CTE
  GROUP BY user_id
)

-- 3. Main query: filter by referencing the second CTE, user_totals
SELECT
  user_id,
  total
FROM user_totals
WHERE total >= 10000;  -- only users whose total is at least 10000

/*
  Execution order:
  1. Exclude cancelled from orders → build valid_orders
  2. Group by user_id              → build user_totals
  3. Filter on the total threshold → keep qualifying rows
  */
Explanation (table transitions & key points)
WITH valid_orders AS ( SELECT user_id, amount FROM orders WHERE status != 'cancelled' ), user_totals AS ( SELECT user_id, SUM(amount) AS total FROM valid_orders GROUP BY user_id ) SELECT user_id, total FROM user_totals WHERE total >= 10000;
LEGEND
Columns / keys under evaluation
Excluded / hidden data
1. Step One: Remove Unneeded Data
CTE: valid_ordersFirst, exclude canceled orders. Performing this preprocessing in the first CTE keeps the subsequent aggregation logic simple.
1 / 3
order_iduser_idamountstatus
1115000completed
215000cancelled
328000completed
423000completed
4 rows → 3 rows
LEARNING POINTS
The code follows the reasoning process: The greatest strength of a CTE chain is that you can write SQL in the natural order of thought: first remove unnecessary data, next aggregate it, and finally keep only what is needed.
ANTI-PATTERNS
Subquery nesting hell: Without CTEs, this SQL becomes something like SELECT * FROM (SELECT user_id, SUM(amount) FROM (SELECT ... WHERE status != 'cancelled') GROUP BY ...) WHERE total >= 10000;. The matryoshka-like nesting makes it extremely difficult to read.
QUESTION 5
Applied Pattern: Coordinate with Aggregation — Retrieve Each User’s Latest Order
MAX + JOINLatest rowAvoid N+1User analysis
Background

A very common production web-application task—for example, on an account page—is retrieving each user’s latest record, such as their newest order or login history entry.

Several techniques can solve this problem. One of the most general and understandable is to calculate each user’s latest timestamp with MAX in a CTE, then JOIN that result to the source table to identify the corresponding row.

Problem

Retrieve each user’s latest order—the row with the greatest ordered_at—from the orders table.

First, in a CTE named latest_dates, aggregate the greatest ordered_at for each user and name it max_date.
Then JOIN the original orders table to the CTE on both user_id and ordered_at = max_date. Return order_id, user_id, amount, and ordered_at for each latest order.

Table used
▸ orders
order_iduser_idamountordered_at
1130002024-01-10
2150002024-02-15
32100002024-01-20
4280002024-03-01
Expected Output
user_idorder_idamountordered_at
1250002024-02-15
2480002024-03-01
Model Answer
-- 1. CTE: identify each user’s newest order date
WITH latest_dates AS (
  SELECT
    user_id,
    MAX(ordered_at) AS max_date  -- use MAX to obtain the latest date
  FROM orders
  GROUP BY user_id
)

-- 2. Main query: join the source table to the CTE and keep only the latest rows
SELECT
  o.user_id,
  o.order_id,
  o.amount,
  o.ordered_at
FROM orders o
JOIN latest_dates ld
  ON o.user_id = ld.user_id       -- composite JOIN: same user_id and order date equal to max_date
  AND o.ordered_at = ld.max_date
ORDER BY o.user_id;

/*
  Execution order:
  1. Build latest_dates
  2. Join orders (o) to latest_dates (ld)
  */
Explanation (table transitions & key points)
WITH latest_dates AS ( SELECT user_id, MAX(ordered_at) AS max_date FROM orders GROUP BY user_id ) SELECT o.user_id, o.order_id, o.amount, o.ordered_at FROM orders o JOIN latest_dates ld ON o.user_id = ld.user_id AND o.ordered_at = ld.max_date ORDER BY o.user_id;
LEGEND
Grouping keys / aggregation targets
Group classification
1. Calculate Latest Dates in the CTE
GROUP BY user_id → MAX(ordered_at)First, calculate only the newest order date (MAX) for each user and build a temporary table named latest_dates. Other columns such as amount are not available yet.
1 / 3
user_idmax_date
12024-02-15
22024-03-01
4 rows → 2 groups (CTE complete)
LEARNING POINTS
Why the JOIN is necessary: SQL does not allow SELECT user_id, order_id, MAX(ordered_at) inside the CTE because order_id is not aggregated. A two-stage process is therefore required: calculate only the latest date in the CTE, then use that date to look up the source row with a JOIN.
Field Notes
In a web application backend or ORM, displaying 100 users and attaching each user’s latest login often creates the N+1 problem: 101 database queries. This pattern retrieves everyone’s latest history in one database query, so it is frequently used as a decisive performance improvement. Modern PostgreSQL also supports a more advanced solution with the ROW_NUMBER() window function, covered in Q8.