A composite index (a, b, c) is ordered on the B-tree as “first in dictionary order by a, then by b when a is the same, and so on.” A continuous sequence of columns from the left can be used by the WHERE clause, and columns to the right of the “first range condition (<, >, BETWEEN, LIKE)” cannot be narrowed using the index's sort order. The design rule is to arrange columns in the order “equality → range → sort.”
-- ✓ (customer_id, created_at) ← equality → range order WHERE customer_id = 101 AND created_at >= '2024-06-01' ORDER BY created_at DESC; -- ✗ (created_at, customer_id) ← range first: cannot narrow by customer_id alone
For the orders table (1 million rows), a frequently executed query is “the latest orders for a specific customer (descending by date, top 3)”. Design the optimal composite index and write a SELECT statement that makes use of it.
| order_id | customer_id | created_at | amount |
|---|---|---|---|
| 1 | 100 | 2024-05-01 | 1200 |
| 2 | 101 | 2024-06-10 | 800 |
| 3 | 101 | 2024-07-22 | 2000 |
| 4 | 101 | 2024-08-30 | 3000 |
| 5 | 102 | 2024-04-15 | 500 |
| 6 | 101 | 2024-09-05 | 1500 |
| 7 | 103 | 2024-07-01 | 700 |
| 8 | 101 | 2024-05-20 | 900 |
※ For customer_id=101, created_at >= '2024-06-01' matches the four rows order_id=2, 3, 4, and 6. Retrieve the latest three.
Expected output (created_at descending, top 3):
| order_id | customer_id | created_at | amount |
|---|---|---|---|
| 6 | 101 | 2024-09-05 | 1500 |
| 4 | 101 | 2024-08-30 | 3000 |
| 3 | 101 | 2024-07-22 | 2000 |
With the proper column order, an Index Range Scan + Backward Scan reads only three rows and stops. With the reverse index order, customer_id cannot be used for equality filtering, so the scan range expands across the entire period.
CREATE INDEX idx_orders_cust_date ON orders (customer_id, created_at DESC); -- [Index design] arrange equality (customer_id) → range (created_at) -- [Query] SELECT that fully uses the index SELECT order_id, customer_id, created_at, amount FROM orders WHERE customer_id = 101 -- Equality: jump directly to the matching block in the B-tree AND created_at >= '2024-06-01' -- Range: scan the range within the narrowed block ORDER BY created_at DESC -- Same order as the index → no extra sort LIMIT 3; -- Stop scanning as soon as three rows are read (Top-N) /* Execution order (planner behavior): 1. Reach customer_id directly in the idx B-tree → reach the target through the index 2. Binary-search the starting position for created_at → locate the range start 3. Take the required rows from the newest side and stop → stop at the required count 4. Fetch the SELECT columns from the table → fetch the columns */
LEGEND
1. Target table — orders (composite index planned for customer_id, created_at)
FROM ordersSearches combining customer_id (many customers) and created_at (date) are frequent in orders. There are five rows for customer_id=101, order_id=2, 3, 4, 6, and 8; four of them are on or after 2024-06-01.| order_id | customer_id | created_at | amount |
|---|---|---|---|
| 1 | 100 | 2024-05-01 | 1200 |
| 2 | 101 | 2024-06-10 | 800 |
| 3 | 101 | 2024-07-22 | 2000 |
| 4 | 101 | 2024-08-30 | 3000 |
| 5 | 102 | 2024-04-15 | 500 |
| 6 | 101 | 2024-09-05 | 1500 |
| 7 | 103 | 2024-07-01 | 700 |
| 8 | 101 | 2024-05-20 | 900 |
CREATE INDEX ... (col DESC), or run ORDER BY col DESC as a Backward Scan from the index's ascending order, to eliminate the sort stage itself. Check whether a Sort node appears in EXPLAIN.(a, b, c) works whether WHERE uses a alone, a+b, or a+b+c, but it does not work for b alone, c alone, or b+c. The only criterion is whether columns are used continuously from the left. Keeping this in mind avoids proliferating similar single-column indexes.(created_at, customer_id), the following customer_id cannot be filtered by the index, and the query ends up reading “all customers in the target period.” The rule is to move columns that can be filtered by equality farther left.pg_stat_statements or the slow query log to extract frequently executed WHERE / ORDER BY patterns, then create a composite index matching their column order and sort direction. An index satisfying “equality → range → sort” often speeds up multiple frequent queries by itself. On the other hand, write-side cost increases with the number of indexes, so professional design aims to “cover many queries with one index.” Q7 shows how the same index-design principle applies to JOINs across multiple tables.JOIN is an operation that relates two tables, and the planner mainly chooses between two algorithms. Nested Loop searches the inner table once for each row of the driving table (outer table), and is fastest when the outer side is small and the inner side has an index. Hash Join builds a hash table from one side and scans the other side once, making it advantageous for large-scale equality joins.
-- A typical case where the planner chooses Nested Loop (customers small, orders large + indexed) SELECT * FROM customers c -- ← Driving table (outer: the loop runs once per filtered row) JOIN orders o -- ← Inner table (inner: Index Scan runs for each outer row) ON c.customer_id = o.customer_id; -- Without indexes between large tables, Hash Join is more likely to be selected SELECT * FROM huge_table_A a -- ← Build the hash table in memory from the smaller side JOIN huge_table_B b -- ← Scan the larger side once and probe the hash table ON a.id = b.a_id;
Inner-join customers and orders on customer_id and retrieve a list of each order with its customer name. orders.customer_id has an index. The output columns must be order_id, customer_name, amount, ordered by order_id ascending.
| customer_id | name |
|---|---|
| 101 | Alice |
| 102 | Bob |
| 103 | Carol |
| order_id | customer_id | amount |
|---|---|---|
| 1 | 101 | 1200 |
| 2 | 103 | 800 |
| 3 | 101 | 2000 |
| 4 | 102 | 3000 |
| 5 | 103 | 1500 |
※ Because customers is small and orders.customer_id is indexed, Nested Loop (with customers as the driving table) is the typical planner choice.
Expected output (order_id ascending):
| order_id | customer_name | amount |
|---|---|---|
| 1 | Alice | 1200 |
| 2 | Carol | 800 |
| 3 | Alice | 2000 |
| 4 | Bob | 3000 |
| 5 | Carol | 1500 |
SELECT o.order_id, c.name AS customer_name, o.amount FROM customers c -- Put the smaller side first as the explicit driving table INNER JOIN orders o -- Inner: indexed on customer_id → one jump per loop ON o.customer_id = c.customer_id -- Equality join condition ORDER BY o.order_id; -- Sort the output /* Execution order (Nested Loop plan): 1. Scan customers one row at a time → outer = driving table 2. Index Scan orders for each c → join through the index 3. Sort the joined result by order_id → reorder the result */
LEGEND
1. Scan the driving table — customers (outer)
FROM customers cThe planner chooses the smaller customers table as the driving table (outer table) and scans all three rows one at a time.| customer_id | name |
|---|---|
| 101 | Alice |
| 102 | Bob |
| 103 | Carol |
outer × inner with Nested Loop. If work_mem is too small, the hash spills to disk and becomes slow, so memory settings are part of performance.ON c.customer_id = o.customer_id_text with mismatched types introduces an implicit cast on one side and can disable the index. Keeping both sides the same type and size, such as BIGINT on both sides or VARCHAR on both sides, matters for data integrity as well as performance.INNER JOIN instead of LEFT JOIN. OUTER JOIN can narrow the planner's choices because it allows NULL rows; simply replacing it with INNER can cause Hash Join to be selected. Asking “is INNER sufficient?” every time is a good habit.A JOIN B and B JOIN A are equivalent, and the planner automatically swaps the driving table after comparing costs (within the scope of join_collapse_limit). “The table written on the left of FROM becomes the outer table” is an old myth. The only important points are (1) whether the join key has an index, (2) which side can be filtered earlier, and (3) whether statistics are current. If the planner seems to choose a different driving table than expected, first update statistics with ANALYZE; if that does not help, review the join key and the selectivity of WHERE. Q8 covers the typical case where a hidden N+1 occurs in a subquery.A scalar subquery in the SELECT clause (a subquery that returns one row and one column) is convenient, but it may be re-evaluated for every outer row, causing the subquery to run N times for N outer rows (the SQL-level N+1 problem). If the engine groups first and then performs a LEFT JOIN, the aggregation is done only once.
-- ✗ Scalar subquery: 4 customers → SUM runs 4 times SELECT c.customer_id, c.name, (SELECT SUM(amount) FROM orders WHERE customer_id = c.customer_id) AS total FROM customers c; -- ✓ Pre-aggregation: SUM runs once (one pass over orders), then LEFT JOIN combines the result
Produce a list containing every customer and that customer's total order amount (0 when the customer has no orders). Use the pre-aggregation + LEFT JOIN form and do not use a scalar subquery. The output columns must be customer_id, name, total, ordered by customer_id ascending.
| customer_id | name |
|---|---|
| 101 | Alice |
| 102 | Bob |
| 103 | Carol |
| 104 | Dan |
| order_id | customer_id | amount |
|---|---|---|
| 1 | 101 | 1200 |
| 2 | 101 | 800 |
| 3 | 103 | 2000 |
| 4 | 103 | 1500 |
| 5 | 101 | 500 |
※ Alice=2500, Carol=3500, while Bob and Dan have no orders → display 0.
Expected output (customer_id ascending):
| customer_id | name | total |
|---|---|---|
| 101 | Alice | 2500 |
| 102 | Bob | 0 |
| 103 | Carol | 3500 |
| 104 | Dan | 0 |
WITH agg AS ( -- [Pre-aggregation] calculate SUM once per customer_id SELECT customer_id, SUM(amount) AS total FROM orders GROUP BY customer_id -- Aggregate orders in one pass ) SELECT c.customer_id, c.name, COALESCE(a.total, 0) AS total -- [NULL → 0] Fill customers with no orders with 0 FROM customers c LEFT JOIN agg a -- [LEFT JOIN] keep customers with no orders ON a.customer_id = c.customer_id ORDER BY c.customer_id; /* Execution order: 1. CTE agg: scan orders in one pass and SUM by customer_id 2. Read the 4 customer rows (driving table) 3. LEFT JOIN each c to agg by customer_id 4. Replace NULL with 0 using COALESCE 5. Sort by customer_id ascending */
LEGEND
1. Read two tables — customers and orders
FROM customers, ordersThere are 4 customer rows and 5 order rows. The desired result is “every customer × total amount (0 when absent),” so how orders is scanned is the key performance issue.| src | customer_id | val1 | val2 |
|---|---|---|---|
| customers | 101 | Alice | — |
| customers | 102 | Bob | — |
| customers | 103 | Carol | — |
| customers | 104 | Dan | — |
| orders | 101 | o.order_id=1 | 1200 |
| orders | 101 | o.order_id=2 | 800 |
| orders | 103 | o.order_id=3 | 2000 |
| orders | 103 | o.order_id=4 | 1500 |
| orders | 101 | o.order_id=5 | 500 |
WITH or a derived table and then join the result sharply reduces access to large tables such as orders.LEFT JOIN and fill the NULL aggregate with COALESCE(aggregate result, 0). An INNER JOIN would create a bug by removing customers with no orders, so always choose LEFT JOIN.WITH agg AS (... GROUP BY customer_id) makes access to orders happen once.(SELECT ... FROM ...) expressions in the SELECT clause, each subquery runs once per outer row. The planner may rewrite them as joins, but that is not guaranteed across all databases and syntaxes; it is safer to write a CTE or derived table from the start.FROM c LEFT JOIN agg a ON ... WHERE a.total > 0 makes the LEFT JOIN effectively degrade to an INNER JOIN (NULL is FALSE for > 0). To retain NULLs, rewrite the condition to explicitly handle them, such as WHERE a.total IS NULL OR a.total > 0 or COALESCE(a.total,0) > 0.MATERIALIZED / NOT MATERIALIZED. In practice, mechanically applying “aggregate with a CTE, join with LEFT JOIN, and handle NULL with COALESCE” produces clean report queries in most cases. Q9 extends this aggregation pattern to “Top K per group” with window functions.A normal ORDER BY ... LIMIT can retrieve only the “top K overall.” To retrieve the “top K per group,” use the window function ROW_NUMBER() OVER (PARTITION BY ... ORDER BY ...), number rows within each group, and then filter with WHERE rn <= K. Unlike aggregation, the defining feature of a window function is that it does not collapse rows.
-- Number rows independently within each group (= add a column without collapsing rows) ROW_NUMBER() OVER ( PARTITION BY category -- Reset for each category ORDER BY amount DESC -- Order within the same category ) AS rn
ROW_NUMBER = different numbers even for ties (1,2,3,4); (2) RANK = ties share a rank and the next rank skips (1,2,2,4); (3) DENSE_RANK = ties share a rank without skipping (1,2,2,3). When you want exactly K rows in Top-N, ROW_NUMBER is the standard choice.Retrieve the top two sales orders per category from orders. The output columns must be category, order_id, amount, rn, ordered by category ascending and then rn ascending.
| order_id | category | amount |
|---|---|---|
| 1 | Books | 1200 |
| 2 | Books | 800 |
| 3 | Books | 2000 |
| 4 | Toys | 5000 |
| 5 | Toys | 3000 |
| 6 | Toys | 1500 |
| 7 | Food | 700 |
| 8 | Food | 500 |
※ Books keeps the top two of its three rows, Toys also keeps its top two, and Food has only two rows, so all of them remain.
Expected output:
| category | order_id | amount | rn |
|---|---|---|---|
| Books | 3 | 2000 | 1 |
| Books | 1 | 1200 | 2 |
| Food | 7 | 700 | 1 |
| Food | 8 | 500 | 2 |
| Toys | 4 | 5000 | 1 |
| Toys | 5 | 3000 | 2 |
WITH ranked AS ( -- [Numbering] add sales-descending rank as a column within each category SELECT category, order_id, amount, ROW_NUMBER() OVER ( PARTITION BY category -- Reset the row number for each category ORDER BY amount DESC -- Sales descending within the category ) AS rn FROM orders ) SELECT category, order_id, amount, rn FROM ranked WHERE rn <= 2 -- Keep only the top K=2 in each category ORDER BY category, rn; -- Category ascending, rank ascending within category /* Execution order (SQL logical evaluation order): 1. FROM orders → scan 2. Window evaluation → assign numbers with PARTITION+ORDER+ROW_NUMBER 3. Extract the top rows with WHERE rn → keep the top two rows per category 4. ORDER BY category, rn → reorder for display */
LEGEND
1. FROM orders — read 8 rows
FROM ordersRead 8 rows across 3 categories. Because the desired result is the top two per category, each category needs an independent ranking.| order_id | category | amount |
|---|---|---|
| 1 | Books | 1200 |
| 2 | Books | 800 |
| 3 | Books | 2000 |
| 4 | Toys | 5000 |
| 5 | Toys | 3000 |
| 6 | Toys | 1500 |
| 7 | Food | 700 |
| 8 | Food | 500 |
WITH ranked AS (...) SELECT ... WHERE rn <= K, wrapping it once in a CTE or derived table.PARTITION BY determines where numbering resets, while ORDER BY determines the order inside the window. Omitting both treats all rows as one window; using only ORDER BY creates a cumulative rank over all rows. Remembering “window = group + order” makes the whole family of functions much easier to learn.WHERE (SELECT COUNT(*) FROM orders o2 WHERE o2.category = o.category AND o2.amount > o.amount) < 2 tends toward N outer rows × N inner rows = N² complexity. One pass with ROW_NUMBER is dramatically faster and easier to read.amount contains ties and “both tied rows should remain,” ROW_NUMBER arbitrarily cuts one of the tied rows. Use RANK or DENSE_RANK when tied ranks should be respected. Decide how ties should behave at the specification level before choosing the function.SUM(...) OVER (PARTITION BY ... ORDER BY ... ROWS BETWEEN ...) support cumulative totals, moving averages, and references to preceding or following rows within a window. Common reporting metrics such as cumulative sales (no partition + ORDER BY date), cumulative sales per customer (PARTITION BY customer_id + ORDER BY date), and a 3-month moving average (ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) can almost all be written with one window function. For performance, an index matching the combination of partition and order_by can eliminate the window's sort. Q10's final problem addresses pagination at its root with “the OFFSET trap → Keyset Pagination.”The standard LIMIT ... OFFSET ... pagination pattern is fine for shallow pages, but it has the fatal property of becoming linearly slower as OFFSET grows. With OFFSET 10000, it reads and discards the first 10000 rows, then returns the next 20, so the cost grows with page depth. Keyset Pagination (the seek method) instead takes LIMIT rows starting after the last value from the previous page, so its time is constant regardless of page position.
-- ✗ OFFSET method: linearly slower on deep pages SELECT ... FROM orders ORDER BY order_id LIMIT 20 OFFSET 10000; -- ✓ Keyset method: fetch directly after the previous page's last order_id SELECT ... FROM orders WHERE order_id > 10000 -- Last order_id on the previous page ORDER BY order_id LIMIT 20;
Page through orders (1 million rows) in ascending order_id, 20 rows at a time. Assuming “the last order_id on the previous page is 10000”, retrieve the next 20 rows with Keyset Pagination. The output columns must be order_id, customer_id, amount.
| order_id | customer_id | amount |
|---|---|---|
| 9998 | 301 | 1100 |
| 9999 | 302 | 900 |
| 10000 ← previous page end | 303 | 1300 |
| 10001 | 304 | 800 |
| 10002 | 305 | 2200 |
| ... | ... | ... |
| 10020 | 323 | 1700 |
| 10021 | 324 | 950 |
※ The target is order_id=10001–10020, 20 rows. It returns the same result as OFFSET 10000, but with a vastly lower scan cost.
Expected output (order_id ascending, 20 rows; 6 shown here as an excerpt):
| order_id | customer_id | amount |
|---|---|---|
| 10001 | 304 | 800 |
| 10002 | 305 | 2200 |
| 10003 | 306 | 1400 |
| ... | ... | ... |
| 10019 | 322 | 1600 |
| 10020 | 323 | 1700 |
-- [Keyset Pagination] fetch directly through the index after the previous page's last order_id SELECT order_id, customer_id, amount FROM orders WHERE order_id > 10000 -- Last order_id on the previous page (retained by the client) ORDER BY order_id -- Same order as the index → no extra sort LIMIT 20; -- Stop after 20 rows /* Execution order (planner behavior): 1. Reach the leaf immediately after order_id with orders_pkey → binary-search to the target 2. Follow the leaves to the right and fetch the required rows → stop at the required count 3. Fetch the SELECT columns from the table → fetch the columns */
LEGEND
1. Target table — orders (1 million rows, PK index on order_id)
FROM ordersorders is a 1-million-row table with order_id as its primary key. This view shows the area around order_id=10000. When paging 20 rows at a time, OFFSET becomes linearly slower as the page gets deeper.| order_id | customer_id | amount |
|---|---|---|
| 9998 | 301 | 1100 |
| 9999 | 302 | 900 |
| 10000 (previous page end) | 303 | 1300 |
| 10001 | 304 | 800 |
| 10002 | 305 | 2200 |
| … | … | … |
| 10020 | 323 | 1700 |
| 10021 | 324 | 950 |
LIMIT K OFFSET N internally follows N index rows, discards them all, and then returns K rows. A small OFFSET is fine, but perceptible delay begins once N reaches tens of thousands, and at several million the query effectively dies. Keyset should be the default design for screens expected to support deep pagination.WHERE sort_column > previous_page_last_value ORDER BY sort_column LIMIT K. An index on the sort column is required for this to become an Index Range Scan. For composite ordering, create a composite index with the same order and form the boundary correctly with tuple comparison such as (created_at, order_id) > (...) or an OR expansion.OFFSET large_value directly for an admin screen's “last page” link will suddenly die as the data grows. If the last page is truly required, an alternative is to reverse ORDER BY and fetch from OFFSET 0 (the last K rows in one order are the first K rows in the reverse order).ORDER BY created_at LIMIT 20, rows can be omitted or displayed twice at the boundary. The rule is to add a tie-breaker column (usually the primary key) to ORDER BY and use the same tuple comparison in WHERE. Remembering that “ORDER BY should be unique” prevents many bugs.SELECT COUNT(*) to display the number of pages. On tables with millions of rows, merely counting every row matching the search condition can take seconds. When Keyset Pagination changes the UI to “load more” or “next,” the total page count is no longer needed. The practical best practice is: “if a page displays 20 rows, fetch LIMIT 21 from the database.” If row 21 exists, determine that another page exists and show a “next” button, while displaying only 20 rows in the actual screen. Combining this LIMIT K+1 technique with Keyset produces an extremely fast list screen with minimal database load.