The first step in making SQL faster is learning to read the execution plan. EXPLAIN shows how the database plans to execute a query, while EXPLAIN ANALYZE actually executes it and returns measured values as well. The representative nodes are Seq Scan (scanning every row) and Index Scan (accessing rows through an index); for a query with a low selectivity, Index Scan is dramatically faster.
-- EXPLAIN: plan only / EXPLAIN ANALYZE: plan + measurements (the query executes) EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM orders WHERE order_id = 50000; -- How to read the plan node: -- Index Scan using orders_pkey on orders -- (cost=0.29..8.31 rows=1 width=64) -- (actual time=0.018..0.020 rows=1 loops=1)
WHERE primary_key = value, the planner chooses Index Scan.Retrieve the one row with order_id = 50000 from the orders table (100,000 rows; order_id is the primary key). Write SQL that outputs the execution plan with EXPLAIN ANALYZE, and read the difference between Seq Scan and Index Scan.
| order_id (PK) | customer_id | amount | status |
|---|---|---|---|
| 1 | 101 | 1200 | completed |
| 2 | 102 | 800 | completed |
| 3 | 103 | 2000 | cancelled |
| 4 * | 104 | 3000 | completed |
| 5 | 105 | 1500 | completed |
| 6 | 106 | 500 | completed |
※ * marks the target row (in the real table it corresponds to order_id = 50000). The visualization treats it as “the fourth row” for clarity.
Illustrative EXPLAIN ANALYZE output (comparison with and without an index):
| Scenario | Node | cost | actual time | Rows read |
|---|---|---|---|---|
| With primary key (PK) | Index Scan | 0.29..8.31 | 0.02 ms | 1 row |
| No index | Seq Scan | 0..1834 | 12.4 ms | 100,000 rows |
Even when the query returns the same one row, different plans can make the measured cost differ by hundreds of times.
EXPLAIN (ANALYZE, BUFFERS) -- Get the plan, measured values, and buffer usage SELECT * FROM orders WHERE order_id = 50000; -- Primary-key equality → Index Scan is selected /* Read path: 1. The planner confirms that the orders_pkey index exists 2. It chooses Index Scan because equality has extremely low selectivity (1/100000) 3. It follows the B-tree to the target row in log2(100000) ≈ 17 steps 4. It fetches one block and finishes (hundreds of times faster than Seq Scan) Reading the example output: Index Scan using orders_pkey on orders (cost=0.29..8.31 rows=1 width=64) ← estimated values (actual time=0.018..0.020 rows=1 loops=1) ← measured values Planning Time: 0.10 ms Execution Time: 0.04 ms */
LEGEND
1. Target table — orders (100,000 rows, 6 shown)
FROM ordersThis is the state before reading the entire orders table. order_id is the primary key, so a B-tree index is created automatically. There is only one target row: order_id = 4 (50000 in the real table).| order_id | customer_id | amount | status |
|---|---|---|---|
| 1 | 101 | 1200 | completed |
| 2 | 102 | 800 | completed |
| 3 | 103 | 2000 | cancelled |
| 4 | 104 | 3000 | completed |
| 5 | 105 | 1500 | completed |
| 6 | 106 | 500 | completed |
EXPLAIN only displays costs estimated from statistics. EXPLAIN ANALYZE actually runs the query and returns actual time and actual rows. A large gap between estimated and actual rows is a sign that the statistics need updating with ANALYZE table_name.EXPLAIN (ANALYZE, BUFFERS) displays shared hit / read, revealing cache hits and reads from the actual disk. Comparing I/O as well as execution time prevents a cache-only “speedup” from looking better than it is.EXPLAIN ANALYZE UPDATE ... actually performs the update. Accidentally running it in production changes data. To inspect only the plan for a write, use EXPLAIN without ANALYZE, or use BEGIN; EXPLAIN ANALYZE ...; ROLLBACK; for a safe check.A condition that can use an index is called Sargable (Search ARGument-able). Conversely, wrapping the column on the left side in a function or expression disables the index and falls back to Seq Scan. For LIKE, only a prefix match such as ('abc%') can use an index; a wildcard at both ends ('%abc%') cannot.
-- ✗ Non-Sargable: apply a function to the column → index disabled WHERE EXTRACT(YEAR FROM created_at) = 2024 -- ✓ Sargable: rewrite as a range condition → Index Range Scan WHERE created_at >= '2024-01-01' AND created_at < '2025-01-01'
The created_at column in the orders table has a B-tree index. Write a Sargable SQL query that uses the index to retrieve orders created during 2024.
| order_id | created_at | amount |
|---|---|---|
| 1 | 2023-11-20 | 1200 |
| 2 | 2024-01-15 | 800 |
| 3 | 2024-05-03 | 2000 |
| 4 | 2024-09-30 | 3000 |
| 5 | 2024-12-31 | 1500 |
| 6 | 2025-01-02 | 500 |
| 7 | 2025-03-10 | 700 |
※ The 2024 rows are ids 2–5, four rows. The standard boundary is >= '2024-01-01' AND < '2025-01-01', expressed as “before New Year's Day of the following year.”
Expected output:
| order_id | created_at | amount |
|---|---|---|
| 2 | 2024-01-15 | 800 |
| 3 | 2024-05-03 | 2000 |
| 4 | 2024-09-30 | 3000 |
| 5 | 2024-12-31 | 1500 |
The same four rows can come from either an Index Range Scan or a Seq Scan depending on how the condition is written.
SELECT order_id, created_at, amount FROM orders WHERE created_at >= '2024-01-01' -- Keep the column bare and compare it with a constant (Sargable) AND created_at < '2025-01-01' -- Express the boundary as “before New Year's Day” (half-open range) ORDER BY order_id; /* Execution order: 1. Binary-search the index for the start of the range 2. Follow the leaves to scan continuously until the end boundary 3. Read the selected columns from the table (or use Index Only Scan) 4. Sort by order_id */
LEGEND
1. FROM orders (B-tree index on created_at)
FROM ordersFour of the 7 rows are in 2024: ids 2, 3, 4, and 5. The created_at index can be used or not depending on how the predicate is written.| order_id | created_at | amount |
|---|---|---|
| 1 | 2023-11-20 | 1200 |
| 2 | 2024-01-15 | 800 |
| 3 | 2024-05-03 | 2000 |
| 4 | 2024-09-30 | 3000 |
| 5 | 2024-12-31 | 1500 |
| 6 | 2025-01-02 | 500 |
| 7 | 2025-03-10 | 700 |
EXTRACT(YEAR FROM created_at), UPPER(name), LOWER(email), and CAST(col AS ...) all transform the column on the left, so the B-tree cannot look up that value. Leave the column bare and make the right side a constant—that is the Sargability rule.BETWEEN '2024-01-01' AND '2024-12-31' has the trap that it can miss the time portion of December 31 (23:59:59.999...). >= '2024-01-01' AND < '2025-01-01' handles time, time zones, and leap seconds without depending on a last representable instant, and it naturally supports Index Range Scan.name LIKE 'Tan%' can use a B-tree range scan, but name LIKE '%Tan%' and '%Tan' cannot. When middle or suffix matching is required, the proper solution is a separate full-text search index such as GIN/pg_trgm.WHERE varchar_col = 12345, can make the database insert a CAST on one side and disable the index. Pass the value with the application-side type, or write the literal as '12345', and Index Scan can return.WHERE a = 1 OR UPPER(b) = 'X' can become a Seq Scan for the whole predicate because one side is non-Sargable. Make both sides Sargable, or split the query into two with UNION ALL.CREATE INDEX ON users (LOWER(email)); lets WHERE LOWER(email) = ... use an index. Still, first ask whether the query can be rewritten to be Sargable. Expression indexes increase maintenance cost and write load, so the practical order is rewrite the query > expression index > full-text search index.Query cost is proportional to “columns read × rows read.” SELECT * reads every column, increasing disk I/O and network transfer, and it prevents a covering index or Index Only Scan from helping. LIMIT returns only the required number of rows; with an index that already supplies the requested order, it can finish early. Combined with ORDER BY, even when the input must be fully examined, it enables the efficient specialized algorithm called Top-N sort.
-- ✗ Every column and every row: unnecessary I/O and sorting SELECT * FROM orders ORDER BY amount DESC; -- ✓ Required columns only + LIMIT: less I/O + Top-N sort SELECT order_id, amount FROM orders ORDER BY amount DESC LIMIT 3;
ORDER BY ... LIMIT K lets the planner choose a Top-N heap sort (O(N log K)) instead of a full sort. Only K rows need to be kept in memory, so the benefit grows as N grows. Simply declaring that only the needed rows should be returned can change the plan itself.Retrieve the top three orders by amount from orders, returning order_id and amount. Minimize the columns read and limit the number of rows with LIMIT.
| order_id | customer_id | amount | status | created_at | note |
|---|---|---|---|---|---|
| 1 | 101 | 1200 | completed | 2024-01-15 | ... |
| 2 | 102 | 800 | completed | 2024-02-20 | ... |
| 3 | 103 | 5000 | completed | 2024-03-10 | ... |
| 4 | 104 | 3000 | cancelled | 2024-04-05 | ... |
| 5 | 105 | 1500 | completed | 2024-05-12 | ... |
| 6 | 106 | 4200 | completed | 2024-06-30 | ... |
※ The top three by descending amount are id=3 (5000), 6 (4200), and 4 (3000). In production, note is assumed to be long TEXT.
Expected output (amount descending, top 3):
| order_id | amount |
|---|---|
| 3 | 5000 |
| 6 | 4200 |
| 4 | 3000 |
Reducing the result to 2 columns × 3 rows makes the transferred result about (2/6 columns) × (3/6 rows) = 1/6 of the original. ORDER BY + LIMIT also enables Top-N sort.
SELECT order_id, amount -- Explicit columns: only the required 2; preserve I/O, network, and Index Only Scan opportunities FROM orders ORDER BY amount DESC -- Sort by amount descending LIMIT 3; -- Top-N: keep only the top 3; use a heap rather than retaining a full sort /* Execution order (logical SQL evaluation order): 1. FROM orders → scan 6 rows 2. ORDER BY amount DESC → Top-N sort (keep only the leading rows) 3. LIMIT 3 → finalize the retained top 3 (an ordered index could allow early termination) 4. SELECT order_id, amount → return only the required 2 columns Plan comparison (conceptual): ✗ SELECT * ORDER BY amount DESC; → read all 6 columns → fully sort 6 rows → return all 6 rows ✓ SELECT order_id, amount ORDER BY amount DESC LIMIT 3; → read 2 columns → Top-N sort with heap size 3 → return 3 rows */
LEGEND
1. FROM orders — source table (6 rows × 6 columns)
FROM ordersOnly 2 of the 6 columns are needed: order_id and amount. note is assumed to be long TEXT, so SELECT * would transfer a great deal of unnecessary data.| order_id | customer_id | amount | status | created_at | note |
|---|---|---|---|---|---|
| 1 | 101 | 1200 | completed | 2024-01-15 | ... |
| 2 | 102 | 800 | completed | 2024-02-20 | ... |
| 3 | 103 | 5000 | completed | 2024-03-10 | ... |
| 4 | 104 | 3000 | cancelled | 2024-04-05 | ... |
| 5 | 105 | 1500 | completed | 2024-05-12 | ... |
| 6 | 106 | 4200 | completed | 2024-06-30 | ... |
(amount DESC, order_id) INCLUDE (...), allows Index Only Scan without touching the table itself. SELECT * loses this benefit completely, so explicit columns pair naturally with this optimization.SELECT * FROM huge_table are common. It works while the table is small, then stops with a timeout or out-of-memory error as data grows. Add “pagination = LIMIT + OFFSET (or key-set pagination)” from the beginning.LIMIT 20 OFFSET 1000000 internally reads and discards 1 million rows before returning 20. For deep pagination, switch to a key-range condition (WHERE id > the last id from the previous page LIMIT 20). Use OFFSET only for shallow pages.IN and EXISTS are predicates for checking whether something exists in another table. EXISTS stops evaluating as soon as it finds one match, while IN can be understood as collecting the subquery result and then comparing against it (the optimizer may transform either form in practice). Also, if a subquery contains NULL, NOT IN has the famous trap of making every row evaluate to FALSE/UNKNOWN; NOT EXISTS is safer.
-- ✓ EXISTS: stop after finding one match; safe around NULL SELECT c.* FROM customers c WHERE EXISTS ( SELECT 1 FROM orders o WHERE o.customer_id = c.customer_id );
SELECT 1 is conventional; (2) its Boolean result is not affected by NULL; and (3) correlated conditions can use an index effectively. If you only need to know whether something exists, EXISTS is the first choice.From customers and orders, retrieve customers who have placed at least one order. Use a correlated subquery with EXISTS in a NULL-safe form. Return customer_id, name, ordered by customer_id ascending.
| customer_id | name |
|---|---|
| 101 | Alice |
| 102 | Bob |
| 103 | Carol |
| 104 | Dan |
| order_id | customer_id | amount |
|---|---|---|
| 1 | 101 | 1200 |
| 2 | 101 | 800 |
| 3 | 103 | 2000 |
| 4 | NULL | 500 |
| 5 | 103 | 1500 |
Customers with orders are 101 (Alice) and 103 (Carol). 102 and 104 have no orders. Order id=4 has a NULL customer_id.
Expected output (customer_id ascending):
| customer_id | name |
|---|---|
| 101 | Alice |
| 103 | Carol |
Trying to find customers without orders with NOT IN would return zero rows because of the NULL (the trap is explained below).
SELECT c.customer_id, c.name FROM customers c WHERE EXISTS ( -- Existence check: TRUE if even one matching orders row exists SELECT 1 -- The contents do not matter; conventionally use “1” (minimal) FROM orders o WHERE o.customer_id = c.customer_id -- Correlation: match the outer c.customer_id (NULL does not match with =) ) ORDER BY c.customer_id; -- Sort by customer_id ascending /* Execution order (logical SQL evaluation order): 1. FROM customers c → scan 4 outer rows 2. Evaluate EXISTS for each c → stop when a match is found 3. ORDER BY customer_id → sort ascending Why NOT IN is unsafe: WHERE customer_id NOT IN (SELECT customer_id FROM orders) → the subquery result is {101,101,103,NULL,103} → when NOT IN contains NULL, every row becomes UNKNOWN (not TRUE), so the result is 0 rows. → with NOT EXISTS, NULL does not match the correlated condition and is safe. */
LEGEND
1. Read 2 tables — customers and orders
FROM customers c, orders oThere are 4 customers and 5 orders. Note the order with a NULL customer_id (id=4); this creates the difference in behavior from NOT IN.| src | key1 | key2 | val |
|---|---|---|---|
| customers | 101 | Alice | — |
| customers | 102 | Bob | — |
| customers | 103 | Carol | — |
| customers | 104 | Dan | — |
| orders | o.id=1 | cust=101 | 1200 |
| orders | o.id=2 | cust=101 | 800 |
| orders | o.id=3 | cust=103 | 2000 |
| orders | o.id=4 | cust=NULL | 500 |
| orders | o.id=5 | cust=103 | 1500 |
SELECT 1 is conventional. SELECT * would also produce the same result, but reviewers often flag it. EXISTS is the most direct and fast form when you only need to know whether something exists.x NOT IN (1, 2, NULL) is UNKNOWN. WHERE requires TRUE, so it excludes every row. If even one NULL enters orders, using NOT IN to find customers without orders can make the result suddenly become zero rows. With NOT EXISTS, NULL does not match the correlated condition and is safe.o.customer_id = c.customer_id runs for each outer-loop value, so an index on orders(customer_id) can make each loop an immediate Index Scan decision. JOIN + DISTINCT can express the same idea, but it often costs more because duplicates must be removed after the join.WHERE customer_id NOT IN (SELECT customer_id FROM orders) to find customers without orders. One NULL in orders makes the result always zero rows. For a set difference, use NOT EXISTS or LEFT JOIN ... WHERE o.id IS NULL.WHERE EXISTS (SELECT * FROM orders ORDER BY ...) adds unnecessary work. EXISTS stops after one row, so ORDER BY has no meaning, and the SELECT list does not affect the result. Standardize on SELECT 1 and keep needless work out of the inner query.IN ('A','B','C') is clearest. 2. If you are testing membership in a set returned by a subquery, use EXISTS (NULL-safe with early exit). 3. If you also need columns from the child table, use JOIN (but duplicates may require DISTINCT or GROUP BY). In practice, mechanically choosing “EXISTS for existence only, JOIN when you need columns” stabilizes both performance and readability. For a NOT-style set difference, make NOT EXISTS the first choice; NOT IN always hides a NULL land mine.This final quiz in the performance fundamentals set combines aggregation with Top-N, a common production pattern. Build the grain with GROUP BY, then take only the top results with ORDER BY ... LIMIT—this two-stage approach is standard for reporting queries, and its exact form changes I/O, memory, and execution time. We will also organize the differences among DISTINCT and GROUP BY, and COUNT(*) and COUNT(column).
-- Standard pattern: aggregate with GROUP BY, then take the top rows with ORDER BY + LIMIT SELECT category, SUM(amount) AS total FROM orders GROUP BY category ORDER BY total DESC LIMIT 3;
COUNT(*) = all rows versus COUNT(column) = excluding NULL versus COUNT(DISTINCT column) = unique values. (2) DISTINCT and GROUP BY often have equivalent internal implementations, but GROUP BY is natural when adding aggregate functions. (3) ORDER BY ... LIMIT K becomes a Top-N heap sort instead of a full sort.From orders, retrieve the top 3 categories by total sales and the number of orders in each category. Return category, order_count, total_amount, sorted by total_amount descending and then category ascending for ties.
| order_id | category | amount |
|---|---|---|
| 1 | Books | 1200 |
| 2 | Books | 800 |
| 3 | Toys | 5000 |
| 4 | Toys | 3000 |
| 5 | Food | 500 |
| 6 | Games | 2500 |
| 7 | Games | 1500 |
| 8 | Music | 700 |
※ Return the top 3 of the 5 categories by total. Even if the real table has 1 million rows, Top-N sort speed after aggregation depends on the number of categories, which is small.
Expected output (total_amount descending, top 3):
| category | order_count | total_amount |
|---|---|---|
| Toys | 2 | 8000 |
| Games | 2 | 4000 |
| Books | 2 | 2000 |
Food (500) and Music (700) are excluded. After aggregation, a Top-3 heap keeps only the leading results.
SELECT category, COUNT(*) AS order_count, -- All rows in the group (including NULLs) SUM(amount) AS total_amount -- Total amount in the group FROM orders GROUP BY category -- Aggregate at category grain ORDER BY total_amount DESC, category -- Total descending; category ascending breaks ties LIMIT 3; -- Top-N: keep only the top 3 (a full sort is unnecessary) /* Execution order (logical SQL evaluation order): 1. FROM orders → read 8 rows 2. GROUP BY category → aggregate by category 3. ORDER BY total DESC → Top-N sort (keep only the leading rows) 4. LIMIT 3 → finalize the retained top 3 COUNT variants (reference): COUNT(*) … all rows (including NULL; often the fastest) COUNT(amount) … rows where amount is not NULL COUNT(DISTINCT customer_id) … unique customers (requires hash deduplication) */
LEGEND
1. FROM orders — read 8 rows
FROM ordersRead 8 rows across 5 categories. Although the final result needs only the top 3 categories, every category must first be aggregated.| order_id | category | amount |
|---|---|---|
| 1 | Books | 1200 |
| 2 | Books | 800 |
| 3 | Toys | 5000 |
| 4 | Toys | 3000 |
| 5 | Food | 500 |
| 6 | Games | 2500 |
| 7 | Games | 1500 |
| 8 | Music | 700 |
SELECT DISTINCT category FROM orders and SELECT category FROM orders GROUP BY category produce the same execution plan in many databases. The difference is whether you want to add aggregate functions. Use GROUP BY with aggregation and DISTINCT for unique values only, matching the intent.COUNT(*) is a simple row count and often the lightest form, while COUNT(amount) excludes rows where amount is NULL. Optimizers treat COUNT(1) as equivalent to COUNT(*), so there is no performance difference. Choose mechanically based on whether the column being counted can be NULL.SELECT ... ORDER BY total DESC and then applying list[:3] in application code is the worst pattern. The database still fully sorts every row, and the network still transfers every row. Remember that the presence or absence of LIMIT can change the entire plan.SELECT DISTINCT is a symptomatic fix that avoids the cause (an incomplete JOIN condition). Prefer GROUP BY or EXISTS without creating the inflation. DISTINCT performs hash deduplication internally, so using it blindly can create hidden performance degradation.GROUP BY, then take the top rows with ORDER BY + LIMIT—is the standard form used most often in dashboards and reporting features. The key design idea is to fold data down as far as possible in the database (aggregation) and return only the required number of rows (Top-N) to the application. If the application receives all data and aggregates and sorts it in a programming language, the network and memory fail as the data grows. Also, instead of abusing DISTINCT just to remove duplicates, use an intentional GROUP BY to make the query's purpose clear. Fully using the database's powerful aggregation engine is a core part of performance optimization in practice.