Subqueries — Learn Scalar, IN, EXISTS, FROM-Clause, Correlated, and CTE Comparison from the Basics
BasicSubqueryScalar subqueryIN / NOT IN / EXISTSFROM clause / correlatedCTE comparisonPostgreSQL-compatible5 questions
QUESTION 1
Scalar Subquery — Attach the Overall Average to the SELECT Clause and Compare Each Row
Scalar SQSELECT clauseAVG aggregateKPI analysis
Background

A subquery is another SELECT statement written inside a SQL statement. A scalar subquery is a subquery that "always returns a single value (one row, one column)," and it can be freely embedded anywhere a value is allowed, such as the SELECT clause or the WHERE clause.

SELECT
  col,
  (SELECT AVG(col) FROM tbl) AS avg_val  -- attach the same aggregate value to every row
FROM tbl;
Key point of a scalar subquery:before the outer query runs, the inner subquery is evaluated first and returns a single value. Because that value is expanded across every row, the process of comparing each row against the overall average is achieved in a single query.
Problem

From the orders table, for orders whose status is 'completed', retrieve order_id, user_id, amount, and additionally attach the average amount of all completed orders and the difference from that average as columns. Sort by amount in descending order.

Tables used
▸ orders
order_iduser_idamountstatus
10118000completed
102112000completed
10323500pending
10439500completed
10536000completed
10642000cancelled
Expected Output

Expected output (completed only, amount descending):

order_iduser_idamountavg_amountdiff_from_avg
10211200088753125
104395008875625
101180008875-875
105360008875-2875

※ avg_amount = (8000+12000+9500+6000)÷4 = 8875. 103 and 106 are excluded.

Model Answer
SELECT
  order_id,
  user_id,
  amount,
  (SELECT ROUND(AVG(amount))    -- scalar SQ: always returns a single value
   FROM   orders
   WHERE  status = 'completed'
  ) AS avg_amount,
  amount -
  (SELECT ROUND(AVG(amount))    -- can also be used inside an arithmetic expression
   FROM   orders
   WHERE  status = 'completed'
  ) AS diff_from_avg
FROM   orders
WHERE  status = 'completed'     -- filter in the outer query
ORDER BY amount DESC;

/*
  Execution order (SQL's logical evaluation order):
  1. Scalar SQ evaluated first
       → SELECT ROUND(AVG(amount)) FROM orders WHERE status='completed'
       → Result: 8875
  2. FROM orders              → read all rows
  3. WHERE status='completed' → narrow to 4 rows
  4. SELECT                   → attach avg_amount=8875 and diff_from_avg to each row
  5. ORDER BY amount DESC     → sort by amount descending
*/
Explanation (table transitions & key points)
SELECT order_id, user_id, amount, ( SELECT ROUND(AVG(amount)) FROM orders WHERE status = 'completed' ) AS avg_amount, amount - ( SELECT ROUND(AVG(amount)) FROM orders WHERE status = 'completed' ) AS diff_from_avg FROM orders WHERE status = 'completed' ORDER BY amount DESC;
LEGEND
Columns / keys under evaluation
Excluded / hidden data
✓ pass
✗ excluded
① Run scalar SQ
SELECT ROUND(AVG(amount)) FROM orders WHERE status='completed'The subquery is evaluated first: from the 4 rows with status='completed' it computes the average (8000+12000+9500+6000)÷4 = 8875 and returns it as a single value. This value is attached to every row of the outer query.
1 / 3
order_idamountstatusSQ target?
1018000completed✓ AVG target
10212000completed✓ AVG target
1033500pending✗ excluded
1049500completed✓ AVG target
1056000completed✓ AVG target
1062000cancelled✗ excluded
→ returns ROUND(AVG) = 8875
LEARNING POINTS
The essence of a scalar SQ:it is a "SELECT statement that returns a single value (one row, one column)." It can be written anywhere a numeric literal or a function's return value can go. An aggregate value such as AVG does not depend on the row, so it is expanded across all rows as a "constant."
Practical pattern – KPI comparison queries:comparisons of individual-row data vs an aggregate value, such as "each order amount vs the overall average" or "this month's sales vs last month's average," are extremely common in queries for API dashboards. With a scalar SQ they fit into a single query without a JOIN.
Timing of execution:the scalar SQ is evaluated before the outer query, and the result is cached and applied to every row. In other words, (SELECT AVG(amount)...) scans the table only once and passes the same value 8875 to every row.
Can be refactored with a CTE:if you write the same subquery in multiple places, some databases evaluate it twice. Writing WITH avg_val AS (...) SELECT ..., avg_val.v FROM orders, avg_val lets it be evaluated only once (see Q9).
ANTI-PATTERNS
A scalar SQ that returns multiple rows:(SELECT user_id FROM users WHERE plan='premium') returns multiple rows, so it cannot be used as a scalar SQ and causes a runtime error. A scalar SQ must always be reduced to a single value with an aggregate function or LIMIT 1.
Repeating the same expression in the SELECT clause:writing the same scalar SQ in multiple places causes some databases to evaluate it twice. If performance concerns you, consolidate it with a CTE (WITH clause) so it is evaluated only once.
Field note: choosing between a scalar SQ and a window function
Computing "the difference from the overall average" with a scalar SQ is essentially equivalent to the window function AVG(amount) OVER (). However, in older DB environments where window functions are unavailable (e.g. MySQL 5.x), the scalar SQ is the only option. On PostgreSQL, MySQL 8.0+, and the like, window functions are recommended, but a scalar SQ is a portable way of writing that works on any RDBMS.
QUESTION 2
WHERE IN Subquery — Build a Dynamic List with a Subquery to Filter Orders
WHERE INSubqueryFilteringPer-plan analysis
Background

WHERE col IN (subquery) filters rows using the list of values that the subquery returns. The key point is that instead of a fixed list IN (1, 3), you can generate a list that changes dynamically according to the state of the table.

SELECT * FROM orders
WHERE  user_id IN (
  SELECT user_id FROM users
  WHERE  plan = 'premium'   -- the inner query generates a dynamic list
);
The inside of IN must return a "single column." Because it is compared against the outer user_id, the inner query must always SELECT exactly one column.
Problem

From the orders table, retrieve the orders of users whose plan is 'premium' as order_id, user_id, amount, status, ordered_at. Sort by ordered_at in descending order.

Tables used
▸ users
user_idnameplan
1Tanaka Taropremium
2Sato Hanakofree
3Suzuki Ichiropremium
4Yamada Jirostandard
5Ito Saburofree
▸ orders
order_iduser_idamountstatusordered_at
10118000completed2024-05-01
102112000completed2024-05-20
10323500pending2024-05-10
10439500completed2024-05-15
10536000completed2024-05-25
10642000cancelled2024-05-08
Expected Output

Expected output (premium users only, ordered_at descending):

order_iduser_idamountstatusordered_at
10536000completed2024-05-25
102112000completed2024-05-20
10439500completed2024-05-15
10118000completed2024-05-01

※ 103 (free) and 106 (standard) are excluded

Model Answer
SELECT
  order_id, user_id, amount, status, ordered_at
FROM   orders
WHERE  user_id IN (        -- IN: keep rows matching the subquery result list
  SELECT user_id            -- always return exactly one column
  FROM   users
  WHERE  plan = 'premium'  -- dynamically generate the list of premium user_ids
)
ORDER BY ordered_at DESC;

/*
  Execution order (SQL's logical evaluation order):
  1. Evaluate the subquery
       → SELECT user_id FROM users WHERE plan='premium'
       → Result list: [1, 3]
  2. FROM orders              → read all rows
  3. WHERE user_id IN (1, 3)  → narrow to 4 rows
  4. SELECT ...               → select the needed columns
  5. ORDER BY ordered_at DESC → sort by date descending
*/
Explanation (table transitions & key points)
SELECT order_id, user_id, amount, status, ordered_at FROM orders WHERE user_id IN ( SELECT user_id FROM users WHERE plan = 'premium' ) ORDER BY ordered_at DESC;
LEGEND
Columns / keys under evaluation
Excluded / hidden data
✓ pass
✗ excluded
① Run subquery
SELECT user_id FROM users WHERE plan = 'premium'The subquery runs first. From users it takes the user_ids where plan='premium', generating the value list [1, 3].
1 / 3
user_idnameplanin list?
1Tanaka Taropremium✓ added to [1,3]
2Sato Hanakofree✗ excluded
3Suzuki Ichiropremium✓ added to [1,3]
4Yamada Jirostandard✗ excluded
5Ito Saburofree✗ excluded
→ generate IN list: [1, 3]
LEARNING POINTS
How the IN subquery works:the "list of values" returned by the inside of IN is finalized before the outer query. It is logically equivalent to a fixed list IN (1, 3), but the important point is that it changes dynamically according to the state of the table.
Equivalence with JOIN:WHERE user_id IN (SELECT user_id FROM users WHERE plan='premium') returns the same result as INNER JOIN users ON orders.user_id = users.user_id WHERE users.plan='premium'. If you do not need to output the joined table's columns in SELECT, the IN subquery states the intent more clearly.
Watch out for NULL contamination:if the IN list contains NULL, non-matching rows become UNKNOWN rather than FALSE. For a column where NULL might appear, add WHERE col IS NOT NULL in the inner query.
ANTI-PATTERNS
Returning multiple columns inside IN:returning multiple columns like SELECT user_id, name FROM users causes a syntax error. SELECT only the single column that matches the IN comparison target.
A large IN list can be slow:when the IN list grows to tens of thousands of entries, the evaluation cost rises. For large data, consider EXISTS or an INNER JOIN.
Field note: criteria for choosing among IN / JOIN / EXISTS
All three are used to "filter by a condition from another table." Use an IN subquery when the inner list is small to medium and the intent is clear. Use a JOIN when you also want to output the joined table's columns in SELECT. Use EXISTS when you only need to confirm "whether it exists" and do not care about duplicates. These are the practical rules of thumb.
QUESTION 3
WHERE NOT IN Subquery — Detect Users Who Have Never Placed an Order
NOT INSubqueryDormant detectionNULL caution
Background

WHERE col NOT IN (subquery) returns the rows not contained in the subquery's result. This is the pattern for finding rows that do not exist in another table, such as "users who have never ordered" or "targets of an undelivered campaign."

SELECT * FROM users
WHERE user_id NOT IN (
  SELECT DISTINCT user_id FROM orders
  WHERE user_id IS NOT NULL   -- prevent NULL contamination (important)
);
The NULL trap (important):if the inner list of NOT IN contains even a single NULL, every outer row is excluded and you get 0 rows. Always add WHERE col IS NOT NULL to the inner query, or use NOT EXISTS (Q6), which is recommended.
Problem

From the users table, retrieve the users who have never placed an order. Return user_id, name, plan in ascending user_id order.

Tables used
▸ users
user_idnameplan
1Tanaka Taropremium
2Sato Hanakofree
3Suzuki Ichiropremium
4Yamada Jirostandard
5Ito Saburofree
▸ orders (list of user_id)
order_iduser_id
1011
1021
1032
1043
1053
1064
Expected Output

Expected output (users with no orders only):

user_idnameplan
5Ito Saburofree

※ DISTINCT user_id = [1,2,3,4]. Only user_id=5 is outside the list.

Model Answer
SELECT
  user_id, name, plan
FROM   users
WHERE  user_id NOT IN (   -- keep only rows not contained in the list
  SELECT DISTINCT user_id  -- remove duplicates with DISTINCT (efficiency)
  FROM   orders
  WHERE  user_id IS NOT NULL -- prevent NULL contamination (avoid the NOT IN trap)
)
ORDER BY user_id;

/*
  Execution order (SQL's logical evaluation order):
  1. Evaluate the subquery
       → SELECT DISTINCT user_id FROM orders WHERE user_id IS NOT NULL
       → Result list: [1, 2, 3, 4]
  2. FROM users              → read all rows
  3. WHERE user_id NOT IN (1,2,3,4)
                             → only the row with user_id=5 remains
  4. SELECT user_id, name, plan  → select the needed columns
  5. ORDER BY user_id        → sort by user_id ascending
*/
Explanation (table transitions & key points)
SELECT user_id, name, plan FROM users WHERE user_id NOT IN ( SELECT DISTINCT user_id FROM orders WHERE user_id IS NOT NULL ) ORDER BY user_id;
LEGEND
Columns / keys under evaluation
Excluded / hidden data
✓ pass
① Run subquery
SELECT DISTINCT user_id FROM orders WHERE user_id IS NOT NULLFrom orders, get the user_ids that have orders, without duplicates. The result list [1,2,3,4] is generated. user_id=5 has no row in orders.
1 / 3
order_iduser_id (after DISTINCT)added to list
1011✓ added to list
1021(duplicate → ignored)
1032✓ added to list
1043✓ added to list
1053(duplicate → ignored)
1064✓ added to list
→ NOT IN list: [1, 2, 3, 4]
LEARNING POINTS
NOT IN = set difference:this is the set-difference operation of "take the rows that are in table A but not in table B." It directly maps to tasks like "extract targets for email delivery to users who have not purchased."
Remove duplicates with DISTINCT:if a single user has multiple orders, user_id is duplicated, but duplicates are not a problem for IN evaluation. Still, adding DISTINCT keeps the list small and lowers the evaluation cost.
NOT EXISTS is a safe alternative:if you would rather not worry about the NULL trap, use WHERE NOT EXISTS (SELECT 1 FROM orders WHERE orders.user_id = users.user_id). NOT EXISTS works as expected even when NULL is present (detailed in Q6).
ANTI-PATTERNS
If NULL is present, every row is excluded:suppose orders.user_id had even one NULL row. The NOT IN list would become [1, NULL, 2, 3, 4]. In SQL's three-valued logic, 5 NOT IN (1, NULL, 2, 3, 4) is evaluated as UNKNOWN, so user_id=5 is also excluded and the result becomes 0 rows.
NOT IN vs NOT EXISTS choice:for production data where NULL contamination is a risk, it is safer to make NOT EXISTS your first choice as a habit.
Field note: detecting dormant users and unperformed actions
Queries that detect "has not done ~," such as "users who have never ordered," "employees who have not answered a survey," or "users with no login this month," are common in operational batches. Master the three patterns: NOT IN / NOT EXISTS / LEFT JOIN + IS NULL. For production data where NULL contamination is a risk, it is important to make NOT EXISTS your first choice as a habit.
QUESTION 4
FROM-Clause Subquery (Derived Table) — Filter an Aggregated Result Further with WHERE
FROM-clause SQDerived tableGROUP BYSales aggregation
Background

When you want to apply a WHERE condition to a result aggregated with GROUP BY, use a "derived table (inline view)"—a subquery written in the FROM clause. Because a normal WHERE clause is evaluated before GROUP BY, you cannot use post-aggregation columns directly in WHERE. A derived table removes this restriction.

SELECT *
FROM (
  SELECT col, SUM(amount) AS total  -- aggregate on the inside
  FROM   table
  GROUP BY col
) AS sub                             -- always give it an alias with AS
WHERE  total >= 10000;              -- can filter by the post-aggregation value
Difference from HAVING:HAVING can only be used within the same query as GROUP BY. Because a derived table treats the aggregated result as a "virtual table," it allows further flexible processing such as JOINing with other tables.
Problem

From the orders table, aggregate the total amount per user targeting only orders whose status is 'completed', and return only the users whose total amount is 15,000 or more, in descending order of total amount. The columns to retrieve are user_id, total_amount.

Tables used
▸ orders
order_iduser_idamountstatus
10118000completed
102112000completed
10323500pending
10439500completed
10536000completed
10642000cancelled
Expected Output

Expected output (completed total ≥ 15000, total descending):

user_idtotal_amount
120000
315500

※ user_id 2 (pending) and 4 (cancelled) are excluded by the inner WHERE. The post-aggregation outer WHERE does the final filtering.

Model Answer
SELECT
  user_id,
  total_amount
FROM (                           -- subquery in the FROM clause (derived table)
  SELECT
    user_id,
    SUM(amount) AS total_amount  -- name the aggregate column with AS (referenced from outside)
  FROM   orders
  WHERE  status = 'completed'    -- filter in the inner query before aggregating
  GROUP BY user_id
) AS user_totals                 -- always give the derived table an AS alias
WHERE  total_amount >= 15000     -- can filter by the aggregate column name (impossible in a normal WHERE)
ORDER BY total_amount DESC;

/*
  Execution order (SQL's logical evaluation order):
  1. Evaluate the inner query (derived table)      → aggregate completed by user_id
  2. FROM ... AS user_totals       → reference the aggregated result
  3. WHERE total_amount threshold      → filter by the condition
  4. SELECT user_id, total_amount  → select columns
  5. ORDER BY total_amount DESC    → total descending
  */
Explanation (table transitions & key points)
SELECT user_id, total_amount FROM ( SELECT user_id, SUM(amount) AS total_amount FROM orders WHERE status = 'completed' GROUP BY user_id ) AS user_totals WHERE total_amount >= 15000 ORDER BY total_amount DESC;
LEGEND
Grouping keys / aggregation targets
Group classification
Excluded / hidden data
① Inner query (GROUP BY)
SELECT user_id, SUM(amount) FROM orders WHERE status='completed' GROUP BY user_idThe inner subquery runs first. It groups the 4 completed rows by user_id and computes the totals. user_id=2 (pending) and 4 (cancelled) are excluded by the inner WHERE.
1 / 4
order_iduser_idamountstatusgroup
10118000completeduser1 group
102112000completeduser1 group
10323500pendingexcluded
10439500completeduser3 group
10536000completeduser3 group
10642000cancelledexcluded
→ derived table: 2 rows generated
LEARNING POINTS
What a derived table is:a subquery written in the FROM clause is called a "derived table" or "inline view." From the outside it can be treated like an ordinary table. It is mandatory to give it an AS alias (omitting it causes an error).
Solving the "WHERE cannot reference aggregate columns" problem:because a normal WHERE clause is evaluated before GROUP BY, you cannot write WHERE SUM(amount) >= 15000 (error). By moving it into a derived table and filtering in the outer WHERE, you achieve the same thing, and you can also handle flexible processing such as JOINing with other tables.
Rewriting as a CTE improves readability:writing WITH user_totals AS (SELECT user_id, SUM(amount) AS total_amount FROM orders WHERE status='completed' GROUP BY user_id) SELECT * FROM user_totals WHERE total_amount >= 15000 reduces the nesting depth (see Q9).
ANTI-PATTERNS
Forgetting the AS alias:if you do not add AS user_totals to the FROM-clause subquery, many databases raise an error (required on both MySQL and PostgreSQL). Always give a derived table an alias.
Referencing the inner base table directly from the outside:trying to use the derived table's inner table name in the outer query, like WHERE orders.status = ..., causes an error. From the outside you can only access the column names the derived table returned (such as user_totals.total_amount).
Field note: HAVING vs derived table — which should you use
For a simple "filter after aggregation," HAVING SUM(amount) >= 15000 is shorter to write. On the other hand, for complex cases such as JOINing the aggregated result with another table, filtering by combining multiple aggregate columns, or aggregating the aggregated result further (double aggregation), a derived table (or CTE) is needed. In practice, a natural judgment is to "first try HAVING, and migrate to a derived table/CTE once it gets complex."
QUESTION 5
EXISTS Subquery — Filter by "Whether It Exists," the Most Efficient Pattern
EXISTSCorrelated SQExistence checkActive detection
Background

WHERE EXISTS (subquery) keeps a row if the subquery "returns even one row." What it returns does not matter—it only checks whether a row exists—so SELECT 1 is enough on the inside.

SELECT * FROM users u
WHERE EXISTS (
  SELECT 1                        -- the value can be anything; only row existence is checked
  FROM   orders o
  WHERE  o.user_id = u.user_id  -- a "correlated subquery" referencing an outer column
);
Characteristics of EXISTS:the moment the inner query finds even one row, it immediately returns TRUE and stops the rest of the search (short-circuit evaluation). It works efficiently even on large data, and it is a safe form that is unaffected by NULL.
Problem

From the users table, retrieve the users who have at least one order with status 'completed'. Return user_id, name, plan in ascending user_id order.

Tables used
▸ users
user_idnameplan
1Tanaka Taropremium
2Sato Hanakofree
3Suzuki Ichiropremium
4Yamada Jirostandard
5Ito Saburofree
▸ orders
order_iduser_idstatus
1011completed
1021completed
1032pending
1043completed
1053completed
1064cancelled
Expected Output

Expected output (has completed, user_id ascending):

user_idnameplan
1Tanaka Taropremium
3Suzuki Ichiropremium

※ user_id 2 (pending only), 4 (cancelled only), and 5 (no orders) are excluded

Model Answer
SELECT
  user_id, name, plan
FROM   users u                    -- alias u for the outer query
WHERE  EXISTS (                   -- TRUE if the SQ returns one or more rows
  SELECT 1                       -- the value can be anything (1, *, 'x' are all equivalent)
  FROM   orders o
  WHERE  o.user_id = u.user_id  -- correlated SQ referencing the outer u.user_id
    AND  o.status  = 'completed' -- passes if there is at least one completed
)
ORDER BY user_id;

/*
  Execution order (SQL's logical evaluation order):
  1. FROM users u                → process one row at a time
  2. EXISTS(...)                 → run the subquery per user
  3. SELECT user_id, name, plan  → select the passing rows
  4. ORDER BY user_id            → user_id ascending
  */
Explanation (table transitions & key points)
SELECT user_id, name, plan FROM users u WHERE EXISTS ( SELECT 1 FROM orders o WHERE o.user_id = u.user_id AND o.status = 'completed' ) ORDER BY user_id;
LEGEND
Rows read / loaded
① FROM users
FROM users uProcess all 5 rows of the users table one at a time. For each row, evaluate the EXISTS subquery.
1 / 3
user_idnameplan
1Tanaka Taropremium
2Sato Hanakofree
3Suzuki Ichiropremium
4Yamada Jirostandard
5Ito Saburofree
read all 5 rows
LEARNING POINTS
EXISTS is an operator dedicated to "existence checking":EXISTS only checks whether the subquery returns even one row. It is conventional to write SELECT 1 to make clear that "returning a value has no meaning."
A "correlated subquery" referencing the outer query's columns:a form like o.user_id = u.user_id inside EXISTS, where the inner query references an outer column (u.user_id), is called a correlated subquery. The inner query runs each time the outer query processes a row.
Safe against NULL:EXISTS does not treat NULL specially. It is a simple two-valued evaluation: TRUE if the subquery returns a row, FALSE if not. The NOT IN NULL problem (Q3) does not occur with EXISTS.
Efficiency through short-circuit evaluation:EXISTS immediately returns TRUE and stops searching the moment even one row is found. Even when "searching 100,000 orders for a specific user's order," it does not scan the remaining 99,999 once the first is found.
ANTI-PATTERNS
Using SELECT * inside EXISTS:writing SELECT * works, but since EXISTS uses no value at all it is wasteful. SELECT 1 or SELECT NULL is conventional and clarifies the intent.
Confusing EXISTS and IN:if you also want to output the joined table's columns in SELECT, use a JOIN instead of EXISTS. Use EXISTS purely for "existence checking," and understand that only the outer columns appear in the result set.
Field note: how to think about IN vs EXISTS performance
You may see the explanation that "IN is slow because it generates the whole list first, while EXISTS is fast because it evaluates row by row," but modern RDB optimizers often internally convert IN into EXISTS or a JOIN to optimize it, so this cannot be stated categorically. What matters is whether indexes are in place, the number of rows in the tables, and the selectivity of the join. Tune by checking the execution plan (EXPLAIN).