Performance Optimization — Learn Composite Indexes, JOIN Strategies, and Keyset Pagination from the Basics
BasicComposite IndexJOIN / Nested Loop / Hash JoinPre-aggregationKeyset PaginationPostgreSQL5 questions
QUESTION 6
Composite Index Column Order — Arrange Columns as Equality → Range
Composite INDEXColumn orderLeftmost matchIndex design
Background

A composite index (a, b, c) is ordered on the B-tree as “first in dictionary order by a, then by b when a is the same, and so on.” A continuous sequence of columns from the left can be used by the WHERE clause, and columns to the right of the “first range condition (<, >, BETWEEN, LIKE)” cannot be narrowed using the index's sort order. The design rule is to arrange columns in the order “equality → range → sort.”

-- ✓ (customer_id, created_at) ← equality → range order
WHERE customer_id = 101
  AND created_at >= '2024-06-01'
ORDER BY created_at DESC;

-- ✗ (created_at, customer_id) ← range first: cannot narrow by customer_id alone
The column-order rule: (1) put columns filtered by equality on the left, (2) put only one range-condition column immediately after them, and (3) match the ordering to ORDER BY so no extra sort is needed (Index Order Scan). Remembering “equality → range → sort” as a mantra removes most of the uncertainty from column-order design.
Problem

For the orders table (1 million rows), a frequently executed query is “the latest orders for a specific customer (descending by date, top 3)”. Design the optimal composite index and write a SELECT statement that makes use of it.

Input table (conceptual view, 8 rows)
- orders (real table: 1 million rows / 8 example rows)
order_idcustomer_idcreated_atamount
11002024-05-011200
21012024-06-10800
31012024-07-222000
41012024-08-303000
51022024-04-15500
61012024-09-051500
71032024-07-01700
81012024-05-20900

※ For customer_id=101, created_at >= '2024-06-01' matches the four rows order_id=2, 3, 4, and 6. Retrieve the latest three.

Expected Output

Expected output (created_at descending, top 3):

order_idcustomer_idcreated_atamount
61012024-09-051500
41012024-08-303000
31012024-07-222000

With the proper column order, an Index Range Scan + Backward Scan reads only three rows and stops. With the reverse index order, customer_id cannot be used for equality filtering, so the scan range expands across the entire period.

Model Answer
CREATE INDEX idx_orders_cust_date ON orders (customer_id, created_at DESC);  -- [Index design] arrange equality (customer_id) → range (created_at)

-- [Query] SELECT that fully uses the index
SELECT   order_id, customer_id, created_at, amount
FROM     orders
WHERE    customer_id = 101                -- Equality: jump directly to the matching block in the B-tree
  AND    created_at >= '2024-06-01'       -- Range: scan the range within the narrowed block
ORDER BY created_at DESC                  -- Same order as the index → no extra sort
LIMIT    3;                              -- Stop scanning as soon as three rows are read (Top-N)

/*
  Execution order (planner behavior):
  1. Reach customer_id directly in the idx B-tree  → reach the target through the index
  2. Binary-search the starting position for created_at             → locate the range start
  3. Take the required rows from the newest side and stop                  → stop at the required count
  4. Fetch the SELECT columns from the table              → fetch the columns
  */
Explanation (table transitions & key points)
SELECT order_id, customer_id, created_at, amount FROM orders WHERE customer_id = 101 AND created_at >= '2024-06-01' ORDER BY created_at DESC LIMIT 3;
LEGEND
Rows read / loaded
1. Target table — orders (composite index planned for customer_id, created_at)
FROM ordersSearches combining customer_id (many customers) and created_at (date) are frequent in orders. There are five rows for customer_id=101, order_id=2, 3, 4, 6, and 8; four of them are on or after 2024-06-01.
1 / 5
order_idcustomer_idcreated_atamount
11002024-05-011200
21012024-06-10800
31012024-07-222000
41012024-08-303000
51022024-04-15500
61012024-09-051500
71032024-07-01700
81012024-05-20900
8 rows (real table: 1 million rows)
LEARNING POINTS
“Equality → range → sort” is the golden order for a composite index: Always put columns filtered by equality at the left edge, and put only one range condition immediately after them. Because columns to the right of a range condition cannot be filtered in index order, the sort column should double as the range column or come immediately after it. Following this rule turns most queries into an Index Range Scan.
Align ORDER BY direction with the index to eliminate the extra sort: Use CREATE INDEX ... (col DESC), or run ORDER BY col DESC as a Backward Scan from the index's ascending order, to eliminate the sort stage itself. Check whether a Sort node appears in EXPLAIN.
The leftmost-match principle — “(a,b,c) works for (a), (a,b), and (a,b,c)”: A composite index (a, b, c) works whether WHERE uses a alone, a+b, or a+b+c, but it does not work for b alone, c alone, or b+c. The only criterion is whether columns are used continuously from the left. Keeping this in mind avoids proliferating similar single-column indexes.
ANTI-PATTERNS
“Just add a single-column index to every column”: Creating separate indexes for each column usually means that only one can be used by a query (Bitmap Index Scan is an exception, but is less efficient than a composite index). Because write cost grows with the number of indexes, the right approach is composite index design based on the combinations in the query's WHERE clause.
Put the range-condition column at the left edge: If the range column is placed first, as in (created_at, customer_id), the following customer_id cannot be filtered by the index, and the query ends up reading “all customers in the target period.” The rule is to move columns that can be filtered by equality farther left.
Field Notes: Design indexes “for the query”
The most important principle of index design is to design by looking at “queries,” not “tables.” Use pg_stat_statements or the slow query log to extract frequently executed WHERE / ORDER BY patterns, then create a composite index matching their column order and sort direction. An index satisfying “equality → range → sort” often speeds up multiple frequent queries by itself. On the other hand, write-side cost increases with the number of indexes, so professional design aims to “cover many queries with one index.” Q7 shows how the same index-design principle applies to JOINs across multiple tables.
QUESTION 7
JOIN Basics — Nested Loop vs. Hash Join and Choosing the Driving Table
INNER JOINNested LoopHash JoinDriving table
Background

JOIN is an operation that relates two tables, and the planner mainly chooses between two algorithms. Nested Loop searches the inner table once for each row of the driving table (outer table), and is fastest when the outer side is small and the inner side has an index. Hash Join builds a hash table from one side and scans the other side once, making it advantageous for large-scale equality joins.

-- A typical case where the planner chooses Nested Loop (customers small, orders large + indexed)
SELECT *
FROM   customers c      -- ← Driving table (outer: the loop runs once per filtered row)
JOIN   orders o         -- ← Inner table (inner: Index Scan runs for each outer row)
  ON c.customer_id = o.customer_id;

-- Without indexes between large tables, Hash Join is more likely to be selected
SELECT *
FROM   huge_table_A a   -- ← Build the hash table in memory from the smaller side
JOIN   huge_table_B b   -- ← Scan the larger side once and probe the hash table
  ON a.id = b.a_id;
Driving table (outer) and inner table: In a Nested Loop, the table placed on the outside of the loop is the driving table. In principle, the ideal arrangement is to make the side with fewer rows after filtering the driving table (outer), and the side with an index on the join key the inner table. The planner automatically chooses this regardless of the SQL text's left/right order (FROM versus JOIN), but if EXPLAIN shows a different driving table than expected, statistics may need updating or indexes may need review.
Problem

Inner-join customers and orders on customer_id and retrieve a list of each order with its customer name. orders.customer_id has an index. The output columns must be order_id, customer_name, amount, ordered by order_id ascending.

Input tables
- customers (3 rows, driving-table candidate)
customer_idname
101Alice
102Bob
103Carol
- orders (5 rows, indexed on customer_id)
order_idcustomer_idamount
11011200
2103800
31012000
41023000
51031500

※ Because customers is small and orders.customer_id is indexed, Nested Loop (with customers as the driving table) is the typical planner choice.

Expected Output

Expected output (order_id ascending):

order_idcustomer_nameamount
1Alice1200
2Carol800
3Alice2000
4Bob3000
5Carol1500
Model Answer
SELECT   o.order_id,
         c.name AS customer_name,
         o.amount
FROM     customers c                     -- Put the smaller side first as the explicit driving table
INNER JOIN orders o                      -- Inner: indexed on customer_id → one jump per loop
       ON o.customer_id = c.customer_id  -- Equality join condition
ORDER BY o.order_id;                     -- Sort the output

/*
  Execution order (Nested Loop plan):
  1. Scan customers one row at a time          → outer = driving table
  2. Index Scan orders for each c              → join through the index
  3. Sort the joined result by order_id        → reorder the result
  */
Explanation (table transitions & key points)
SELECT o.order_id, c.name AS customer_name, o.amount FROM customers c INNER JOIN orders o ON o.customer_id = c.customer_id ORDER BY o.order_id;
LEGEND
Rows read / loaded
1. Scan the driving table — customers (outer)
FROM customers cThe planner chooses the smaller customers table as the driving table (outer table) and scans all three rows one at a time.
1 / 5
customer_idname
101Alice
102Bob
103Carol
Driving table: 3 rows
LEARNING POINTS
Nested Loop is fastest with “small outer side × indexed inner side”: Narrowing the row count on the outer side and repeating one index jump on the inner side creates a lightweight plan that performs one Index Scan per outer row. Its cost falls linearly as the outer side is filtered more aggressively. Thinking “filter, then join” naturally leads to a good plan.
Hash Join for “both large × equality join”: When joining large tables on equality, the planner chooses Hash Join to build a hash table from the smaller side and scan the larger side once. It can be dramatically lighter than looping over outer × inner with Nested Loop. If work_mem is too small, the hash spills to disk and becomes slow, so memory settings are part of performance.
Always index the join key: Without an index on a column used in a JOIN condition, Nested Loop is less likely to be chosen; at large scale, Hash Join may be used, and if even that is too costly the result can be a devastating Seq Scan × Seq Scan plan. The primary-key side already has an index automatically, so the rule is to index the foreign-key side as well.
ANTI-PATTERNS
The data types of the join condition do not match: A join such as ON c.customer_id = o.customer_id_text with mismatched types introduces an implicit cast on one side and can disable the index. Keeping both sides the same type and size, such as BIGINT on both sides or VARCHAR on both sides, matters for data integrity as well as performance.
Unnecessary OUTER JOIN: Unless you truly need rows that exist on only one side, use INNER JOIN instead of LEFT JOIN. OUTER JOIN can narrow the planner's choices because it allows NULL rows; simply replacing it with INNER can cause Hash Join to be selected. Asking “is INNER sufficient?” every time is a good habit.
Field Notes: Rewriting JOIN order does not (basically) change performance
In SQL standard semantics, A JOIN B and B JOIN A are equivalent, and the planner automatically swaps the driving table after comparing costs (within the scope of join_collapse_limit). “The table written on the left of FROM becomes the outer table” is an old myth. The only important points are (1) whether the join key has an index, (2) which side can be filtered earlier, and (3) whether statistics are current. If the planner seems to choose a different driving table than expected, first update statistics with ANALYZE; if that does not help, review the join key and the selectivity of WHERE. Q8 covers the typical case where a hidden N+1 occurs in a subquery.
QUESTION 8
The Scalar Subquery Trap — Compute Aggregates Once Up Front
Scalar subqueryPre-aggregationLEFT JOINN+1
Background

A scalar subquery in the SELECT clause (a subquery that returns one row and one column) is convenient, but it may be re-evaluated for every outer row, causing the subquery to run N times for N outer rows (the SQL-level N+1 problem). If the engine groups first and then performs a LEFT JOIN, the aggregation is done only once.

-- ✗ Scalar subquery: 4 customers → SUM runs 4 times
SELECT c.customer_id, c.name,
       (SELECT SUM(amount) FROM orders
        WHERE  customer_id = c.customer_id) AS total
FROM   customers c;

-- ✓ Pre-aggregation: SUM runs once (one pass over orders), then LEFT JOIN combines the result
The SQL version of “N+1”: Repeating the same aggregation N times for N outer rows has the same structure as the ORM N+1 problem. Following “aggregate once up front, join once at the end” changes the number of passes over orders from N to 1, making the query dramatically lighter. Replacing NULL with 0 using COALESCE is also a standard technique.
Problem

Produce a list containing every customer and that customer's total order amount (0 when the customer has no orders). Use the pre-aggregation + LEFT JOIN form and do not use a scalar subquery. The output columns must be customer_id, name, total, ordered by customer_id ascending.

Input tables
- customers (4 rows)
customer_idname
101Alice
102Bob
103Carol
104Dan
- orders (5 rows)
order_idcustomer_idamount
11011200
2101800
31032000
41031500
5101500

※ Alice=2500, Carol=3500, while Bob and Dan have no orders → display 0.

Expected Output

Expected output (customer_id ascending):

customer_idnametotal
101Alice2500
102Bob0
103Carol3500
104Dan0
Model Answer
WITH agg AS (                            -- [Pre-aggregation] calculate SUM once per customer_id
  SELECT   customer_id,
           SUM(amount) AS total
  FROM     orders
  GROUP BY customer_id                   -- Aggregate orders in one pass
)
SELECT   c.customer_id, c.name,
         COALESCE(a.total, 0) AS total   -- [NULL → 0] Fill customers with no orders with 0
FROM     customers c
LEFT JOIN agg a                          -- [LEFT JOIN] keep customers with no orders
       ON a.customer_id = c.customer_id
ORDER BY c.customer_id;

/*
  Execution order:
  1. CTE agg: scan orders in one pass and SUM by customer_id
  2. Read the 4 customer rows (driving table)
  3. LEFT JOIN each c to agg by customer_id
  4. Replace NULL with 0 using COALESCE
  5. Sort by customer_id ascending
  */
Explanation (table transitions & key points)
WITH agg AS ( SELECT customer_id, SUM(amount) AS total FROM orders GROUP BY customer_id ) SELECT c.customer_id, c.name, COALESCE(a.total, 0) AS total FROM customers c LEFT JOIN agg a ON a.customer_id = c.customer_id ORDER BY c.customer_id;
LEGEND
Rows read / loaded
1. Read two tables — customers and orders
FROM customers, ordersThere are 4 customer rows and 5 order rows. The desired result is “every customer × total amount (0 when absent),” so how orders is scanned is the key performance issue.
1 / 5
srccustomer_idval1val2
customers101Alice
customers102Bob
customers103Carol
customers104Dan
orders101o.order_id=11200
orders101o.order_id=2800
orders103o.order_id=32000
orders103o.order_id=41500
orders101o.order_id=5500
customers=4, orders=5
LEARNING POINTS
“Aggregate once up front, join once at the end” is the key to faster SELECTs: Scalar subqueries are easy to write, but they often have a structure that re-evaluates N times for N outer rows. Rewriting them to aggregate once up front with WITH or a derived table and then join the result sharply reduces access to large tables such as orders.
LEFT JOIN + COALESCE is the standard pattern for including a value present on only one side: To add “the number if present, 0 otherwise” to a customer list, keep every customer with LEFT JOIN and fill the NULL aggregate with COALESCE(aggregate result, 0). An INNER JOIN would create a bug by removing customers with no orders, so always choose LEFT JOIN.
CTEs and derived tables are powerful when the same aggregate feeds multiple columns: If you place “total amount,” “count,” and “latest order date” in separate scalar subqueries, scans increase by the number of subqueries. Creating several columns at once with WITH agg AS (... GROUP BY customer_id) makes access to orders happen once.
ANTI-PATTERNS
Produce scalar subqueries “just because”: If you put three or four (SELECT ... FROM ...) expressions in the SELECT clause, each subquery runs once per outer row. The planner may rewrite them as joins, but that is not guaranteed across all databases and syntaxes; it is safer to write a CTE or derived table from the start.
Compare a LEFT JOIN's right-side column with = in WHERE: Writing FROM c LEFT JOIN agg a ON ... WHERE a.total > 0 makes the LEFT JOIN effectively degrade to an INNER JOIN (NULL is FALSE for > 0). To retain NULLs, rewrite the condition to explicitly handle them, such as WHERE a.total IS NULL OR a.total > 0 or COALESCE(a.total,0) > 0.
Field Notes: Derived tables, CTEs, and subqueries — how to choose
There are three ways to write the same “pre-aggregate → join” pattern. (1) A WITH clause (CTE) has a name and is good for reuse and readability. (2) A derived table in the FROM clause is equivalent to a CTE and convenient when the logic is simple. (3) A scalar subquery should be used only when it is “one row, one column, and independent of the outer query”; avoid it when correlated. In PostgreSQL, CTEs in older versions were “optimization fences” and were materialized, but current versions let you control this with MATERIALIZED / NOT MATERIALIZED. In practice, mechanically applying “aggregate with a CTE, join with LEFT JOIN, and handle NULL with COALESCE” produces clean report queries in most cases. Q9 extends this aggregation pattern to “Top K per group” with window functions.
QUESTION 9
Group-wise Top-N with Window Functions — ROW_NUMBER + PARTITION BY
ROW_NUMBERPARTITION BYWINDOWTop-N per group
Background

A normal ORDER BY ... LIMIT can retrieve only the “top K overall.” To retrieve the “top K per group,” use the window function ROW_NUMBER() OVER (PARTITION BY ... ORDER BY ...), number rows within each group, and then filter with WHERE rn <= K. Unlike aggregation, the defining feature of a window function is that it does not collapse rows.

-- Number rows independently within each group (= add a column without collapsing rows)
ROW_NUMBER() OVER (
  PARTITION BY category   -- Reset for each category
  ORDER BY     amount DESC   -- Order within the same category
) AS rn
The three RANK siblings: (1) ROW_NUMBER = different numbers even for ties (1,2,3,4); (2) RANK = ties share a rank and the next rank skips (1,2,2,4); (3) DENSE_RANK = ties share a rank without skipping (1,2,2,3). When you want exactly K rows in Top-N, ROW_NUMBER is the standard choice.
Problem

Retrieve the top two sales orders per category from orders. The output columns must be category, order_id, amount, rn, ordered by category ascending and then rn ascending.

Input table
- orders (8 rows, 3 categories)
order_idcategoryamount
1Books1200
2Books800
3Books2000
4Toys5000
5Toys3000
6Toys1500
7Food700
8Food500

※ Books keeps the top two of its three rows, Toys also keeps its top two, and Food has only two rows, so all of them remain.

Expected Output

Expected output:

categoryorder_idamountrn
Books320001
Books112002
Food77001
Food85002
Toys450001
Toys530002
Model Answer
WITH ranked AS (                         -- [Numbering] add sales-descending rank as a column within each category
  SELECT category, order_id, amount,
         ROW_NUMBER() OVER (
           PARTITION BY category         -- Reset the row number for each category
           ORDER BY     amount DESC      -- Sales descending within the category
         ) AS rn
  FROM   orders
)
SELECT   category, order_id, amount, rn
FROM     ranked
WHERE    rn <= 2                         -- Keep only the top K=2 in each category
ORDER BY category, rn;                   -- Category ascending, rank ascending within category

/*
  Execution order (SQL logical evaluation order):
  1. FROM orders                  → scan
  2. Window evaluation            → assign numbers with PARTITION+ORDER+ROW_NUMBER
  3. Extract the top rows with WHERE rn  → keep the top two rows per category
  4. ORDER BY category, rn         → reorder for display
  */
Explanation (table transitions & key points)
WITH ranked AS ( SELECT category, order_id, amount, ROW_NUMBER() OVER ( PARTITION BY category ORDER BY amount DESC ) AS rn FROM orders ) SELECT category, order_id, amount, rn FROM ranked WHERE rn <= 2 ORDER BY category, rn;
LEGEND
Rows read / loaded
1. FROM orders — read 8 rows
FROM ordersRead 8 rows across 3 categories. Because the desired result is the top two per category, each category needs an independent ranking.
1 / 5
order_idcategoryamount
1Books1200
2Books800
3Books2000
4Toys5000
5Toys3000
6Toys1500
7Food700
8Food500
8 rows (3 categories)
LEARNING POINTS
Window functions are “aggregation without collapsing rows”: While GROUP BY collapses N rows → number of groups rows, a window function produces N rows → N rows (with columns added). This is powerful when you want to keep individual rows and aggregate values together, for example for group-wise ranking, cumulative totals, or moving averages. Top-N per group is a representative use case.
Window functions are evaluated in SELECT, so wrap them in a CTE: In SQL's logical evaluation order, a window function is calculated in the SELECT clause, so it cannot be filtered directly in WHERE in the same SELECT. The standard pattern is WITH ranked AS (...) SELECT ... WHERE rn <= K, wrapping it once in a CTE or derived table.
Define a “window” with (PARTITION BY ... ORDER BY ...) in OVER: PARTITION BY determines where numbering resets, while ORDER BY determines the order inside the window. Omitting both treats all rows as one window; using only ORDER BY creates a cumulative rank over all rows. Remembering “window = group + order” makes the whole family of functions much easier to learn.
ANTI-PATTERNS
Force group-wise Top-N with a correlated subquery: A form such as WHERE (SELECT COUNT(*) FROM orders o2 WHERE o2.category = o.category AND o2.amount > o.amount) < 2 tends toward N outer rows × N inner rows = N² complexity. One pass with ROW_NUMBER is dramatically faster and easier to read.
Use ROW_NUMBER when ties should be preserved: If amount contains ties and “both tied rows should remain,” ROW_NUMBER arbitrarily cuts one of the tied rows. Use RANK or DENSE_RANK when tied ranks should be respected. Decide how ties should behave at the specification level before choosing the function.
Field Notes: The “window” in window functions has unlimited applications
Beyond ROW_NUMBER / RANK, expressions such as SUM(...) OVER (PARTITION BY ... ORDER BY ... ROWS BETWEEN ...) support cumulative totals, moving averages, and references to preceding or following rows within a window. Common reporting metrics such as cumulative sales (no partition + ORDER BY date), cumulative sales per customer (PARTITION BY customer_id + ORDER BY date), and a 3-month moving average (ROWS BETWEEN 2 PRECEDING AND CURRENT ROW) can almost all be written with one window function. For performance, an index matching the combination of partition and order_by can eliminate the window's sort. Q10's final problem addresses pagination at its root with “the OFFSET trap → Keyset Pagination.”
QUESTION 10
Keyset Pagination — Solve the OFFSET Trap with the Seek Method
Keyset Paginationseek methodOFFSET trapPagination
Background

The standard LIMIT ... OFFSET ... pagination pattern is fine for shallow pages, but it has the fatal property of becoming linearly slower as OFFSET grows. With OFFSET 10000, it reads and discards the first 10000 rows, then returns the next 20, so the cost grows with page depth. Keyset Pagination (the seek method) instead takes LIMIT rows starting after the last value from the previous page, so its time is constant regardless of page position.

-- ✗ OFFSET method: linearly slower on deep pages
SELECT ... FROM orders ORDER BY order_id LIMIT 20 OFFSET 10000;

-- ✓ Keyset method: fetch directly after the previous page's last order_id
SELECT ... FROM orders
WHERE  order_id > 10000          -- Last order_id on the previous page
ORDER BY order_id LIMIT 20;
Keyset prerequisites: (1) the ORDER BY column has an index, (2) the column is unique (or becomes unique with “last value + a tie-breaker column”), and (3) the client remembers the last value on the previous page. When these conditions are met, it can be many orders of magnitude faster than OFFSET.
Problem

Page through orders (1 million rows) in ascending order_id, 20 rows at a time. Assuming “the last order_id on the previous page is 10000”, retrieve the next 20 rows with Keyset Pagination. The output columns must be order_id, customer_id, amount.

Input table (conceptual view, 10 rows)
- orders (real table: 1 million rows / excerpt around order_id=9998–10025)
order_idcustomer_idamount
99983011100
9999302900
10000 ← previous page end3031300
10001304800
100023052200
.........
100203231700
10021324950

※ The target is order_id=10001–10020, 20 rows. It returns the same result as OFFSET 10000, but with a vastly lower scan cost.

Expected Output

Expected output (order_id ascending, 20 rows; 6 shown here as an excerpt):

order_idcustomer_idamount
10001304800
100023052200
100033061400
.........
100193221600
100203231700
Model Answer
-- [Keyset Pagination] fetch directly through the index after the previous page's last order_id
SELECT   order_id, customer_id, amount
FROM     orders
WHERE    order_id > 10000              -- Last order_id on the previous page (retained by the client)
ORDER BY order_id                       -- Same order as the index → no extra sort
LIMIT    20;                            -- Stop after 20 rows

/*
  Execution order (planner behavior):
  1. Reach the leaf immediately after order_id with orders_pkey  → binary-search to the target
  2. Follow the leaves to the right and fetch the required rows                    → stop at the required count
  3. Fetch the SELECT columns from the table              → fetch the columns
  */
Explanation (table transitions & key points)
SELECT order_id, customer_id, amount FROM orders WHERE order_id > 10000 ORDER BY order_id LIMIT 20;
LEGEND
Rows read / loaded
1. Target table — orders (1 million rows, PK index on order_id)
FROM ordersorders is a 1-million-row table with order_id as its primary key. This view shows the area around order_id=10000. When paging 20 rows at a time, OFFSET becomes linearly slower as the page gets deeper.
1 / 5
order_idcustomer_idamount
99983011100
9999302900
10000 (previous page end)3031300
10001304800
100023052200
100203231700
10021324950
1 million rows (order_id=10000 is the previous page end)
LEARNING POINTS
OFFSET slows linearly with page depth: LIMIT K OFFSET N internally follows N index rows, discards them all, and then returns K rows. A small OFFSET is fine, but perceptible delay begins once N reaches tens of thousands, and at several million the query effectively dies. Keyset should be the default design for screens expected to support deep pagination.
Keyset requires an index on the ORDER BY column: The essence of Keyset Pagination is WHERE sort_column > previous_page_last_value ORDER BY sort_column LIMIT K. An index on the sort column is required for this to become an Index Range Scan. For composite ordering, create a composite index with the same order and form the boundary correctly with tuple comparison such as (created_at, order_id) > (...) or an OR expansion.
Infinite-scroll UIs and Keyset are a perfect match: A “next 20” or “load more” UI only needs the client to retain the last order_id from the previous page, so Keyset Pagination fits naturally. Conversely, a UI that jumps to page N needs OFFSET and is inherently slow on deep pages; it is worth asking at the UX design stage whether random access is truly necessary.
ANTI-PATTERNS
Leave “jump to the last page” implemented as OFFSET 1 million: A design that generates OFFSET large_value directly for an admin screen's “last page” link will suddenly die as the data grows. If the last page is truly required, an alternative is to reverse ORDER BY and fetch from OFFSET 0 (the last K rows in one order are the first K rows in the reverse order).
Use Keyset when ORDER BY is not unique: If created_at contains duplicates in ORDER BY created_at LIMIT 20, rows can be omitted or displayed twice at the boundary. The rule is to add a tie-breaker column (usually the primary key) to ORDER BY and use the same tuple comparison in WHERE. Remembering that “ORDER BY should be unique” prevents many bugs.
Field Notes: Fast pagination starts by abandoning the total COUNT
Alongside OFFSET latency, the biggest cause of slow pagination screens is SELECT COUNT(*) to display the number of pages. On tables with millions of rows, merely counting every row matching the search condition can take seconds. When Keyset Pagination changes the UI to “load more” or “next,” the total page count is no longer needed. The practical best practice is: “if a page displays 20 rows, fetch LIMIT 21 from the database.” If row 21 exists, determine that another page exists and show a “next” button, while displaying only 20 rows in the actual screen. Combining this LIMIT K+1 technique with Keyset produces an extremely fast list screen with minimal database load.