When the types of a column and a literal (or parameter) do not match in a WHERE clause, what happens is up to the DBMS. PostgreSQL raises an error for a varchar = integer comparison, while MySQL and Oracle implicitly cast the column before comparing it. The moment the column is cast, it is equivalent to wrapping the column in a function—the index is removed from consideration (a Sargable violation). Because the cast was inserted even though it is not written in the code, this trap is harder to spot than an explicit function wrapper.
-- ✗ varchar column × numeric literal → PostgreSQL errors immediately / MySQL etc. implicitly cast the column WHERE member_code = 250 -- ✗ Cast the "column side" to avoid the error → convert every row + runtime error on non-numeric data WHERE member_code::int = 250 -- ✓ Align the "literal side" with the column type → a straightforward varchar comparison that uses the index WHERE member_code = '250'
From the members table (1 million actual rows, with an index on member_code), retrieve the member whose member code is 250. member_code is a varchar type inherited from a legacy system; most values are numeric, but legacy codes such as 'A-100' also exist. Write the query in a form that can use the index and has no risk of a runtime error. The output columns must be member_id, member_code, name.
| member_id | member_code | name |
|---|---|---|
| 1 | 100 | Sato |
| 2 | 250 | Suzuki |
| 3 | A-100 | Tanaka |
| 4 | 30 | Ito |
| 5 | 1200 | Watanabe |
※ Also consider what happens to the 'A-100' row if you write member_code::int = 250.
Expected output:
| member_id | member_code | name |
|---|---|---|
| 2 | 250 | Suzuki |
The column-cast version cannot use the index and the query is aborted by a runtime error the moment it reads 'A-100' (the read order depends on the plan, so the incident is not reproducible).
SELECT member_id, member_code, name FROM members WHERE member_code = '250' -- Align the literal side with the column type (varchar) → index equality lookup ORDER BY member_id; -- ✗ member_code = 250 … PostgreSQL has no comparison operator and errors -- ✗ member_code::int = 250 … Casts every row (no index) + runtime error on 'A-100' /* Logical evaluation order: 1. FROM members → choose the source 2. WHERE member_code = '250' → index equality lookup with no cast 3. SELECT → read only the matching row 4. ORDER BY member_id → effectively no extra cost */
LEGEND
1. Source table — member_code is varchar (legacy-format codes are mixed in)
FROM members (index on member_code)member_code is a string type (varchar) inherited from a legacy system. The contents are mostly numeric, but legacy formats such as 'A-100' also exist. A “numeric-looking string” is the gateway to the trap of wanting to search with a numeric literal.| member_id | member_code (varchar) | name |
|---|---|---|
| 1 | '100' | Sato |
| 2 | '250' | Suzuki |
| 3 | 'A-100' | Tanaka |
| 4 | '30' | Ito |
| 5 | '1200' | Watanabe |
DATE_TRUNC(ordered_at)) is visible in the code, but an implicit cast is inserted by the DBMS, making the code look innocent. The clue is in EXPLAIN: if a condition contains a cast you do not remember writing, such as (member_code)::integer, suspect a type mismatch.setString).::int cast occurs when a row that cannot be converted to a number is actually read. The planner is free to choose the evaluation and scan order, so the query works in a test environment without 'A-100' and fails in production with it—a type of incident that slips past basic verification. Guarantee type correctness through the query shape instead of relying on accidental data.::int fails, you may add CASE WHEN member_code ~ '^[0-9]+$' THEN member_code::int END = 250 and keep layering defensive code. The query becomes heavier and harder to read. If the value is numeric, fix the schema to use a numeric type. If that is impossible, the next-best choice is to compare it as a string.a.user_id (int) and b.user_code (varchar) with ON a.user_id::text = b.user_code pays for a cast of every row on one side on every join. This is common and hard to notice at system-integration and data-migration boundaries. It is a schema designer's job to standardize keys across master data, including their types.'250' and '0250' are different values, while numeric 250 and 0250 are equal. String ordering also gives '1200' < '30' (lexicographic order), which conflicts with numeric intuition. In other words, type choice defines not only index performance but the very meaning of equality and ordering. The usual rule is “use strings for digits that are not calculated, such as codes and phone numbers; use numeric types for numbers that are calculated.” Either way, the principle of aligning both sides of a comparison does not change. Q7 moves to “the latest row per group” and introduces the window function as a new tool replacing the correlated subquery from Q1.Getting the latest row within each group (greatest-n-per-group), such as “the latest login for each user,” is one of the most common requirements in practice. If you write logged_at = (SELECT MAX(...)) with a correlated subquery, you get not only N+1 (Q1) but also a correctness problem where a timestamp tie returns two rows. With the window function ROW_NUMBER(), PARTITION BY divides the groups and the within-group ordering assigns a number to every row—the whole table can be processed in a single scan (one pass).
-- ✗ Search for MAX row by row (N+1) + both rows in a timestamp tie are returned WHERE logged_at = (SELECT MAX(logged_at) FROM logins x WHERE x.user_id = l.user_id) -- ✓ Number each partition → keep only rn = 1 outside (always exactly one row) ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY logged_at DESC, login_id DESC) AS rn
From the logins table, retrieve the latest login for each user (timestamp and device). If multiple logins have the same timestamp, choose the one with the larger login_id, guaranteeing exactly one row per user. Write it so the table is scanned only once. The output columns must be user_id, logged_at, device (user_id ascending).
| login_id | user_id | logged_at | device |
|---|---|---|---|
| 1 | 1 | 2026-06-01 09:00 | mobile |
| 2 | 2 | 2026-06-01 10:00 | pc |
| 3 | 1 | 2026-06-03 12:00 | pc |
| 4 | 3 | 2026-06-02 08:00 | pc |
| 5 | 2 | 2026-06-04 18:00 | mobile |
| 6 | 3 | 2026-06-02 08:00 | tablet |
| 7 | 1 | 2026-06-02 20:00 | tablet |
※ Before writing it, predict how many rows user_id=3 would produce with a correlated MAX subquery.
Expected output (user_id ascending):
| user_id | logged_at | device |
|---|---|---|
| 1 | 2026-06-03 12:00 | pc |
| 2 | 2026-06-04 18:00 | mobile |
| 3 | 2026-06-02 08:00 | tablet |
user_id=3 has login_id 4 and 6 at the same time. The tie-breaker (login_id DESC) selects tablet (id=6) as “the latest row.” The MAX comparison version returns both rows.
SELECT user_id, logged_at, device FROM ( SELECT user_id, logged_at, device, ROW_NUMBER() OVER (PARTITION BY user_id -- one partition per user ORDER BY logged_at DESC, login_id DESC) AS rn -- tie-breaker is required FROM logins ) t WHERE rn = 1 -- a window function cannot be written directly in WHERE; filter outside ORDER BY user_id; -- Alternative (PostgreSQL only, concise): -- SELECT DISTINCT ON (user_id) user_id, logged_at, device -- FROM logins ORDER BY user_id, logged_at DESC, login_id DESC; /* Logical evaluation order: 1. Inner FROM logins → one scan 2. Window → partition by user_id and assign ROW_NUMBER 3. Outer WHERE rn = 1 → one row per partition 4. SELECT → finalize the three columns 5. ORDER BY user_id → sort ascending */
LEGEND
1. Source table — Login history for three users
FROM loginsWe want the latest one row per user. Notice that user_id=3 has two logins at 08:00 on June 2 (pc / tablet)—this same-time tie tests whether the query is correct.| login_id | user_id | logged_at | device |
|---|---|---|---|
| 1 | 1 | 2026-06-01 09:00 | mobile |
| 2 | 2 | 2026-06-01 10:00 | pc |
| 3 | 1 | 2026-06-03 12:00 | pc |
| 4 | 3 | 2026-06-02 08:00 | pc |
| 5 | 2 | 2026-06-04 18:00 | mobile |
| 6 | 3 | 2026-06-02 08:00 | tablet |
| 7 | 1 | 2026-06-02 20:00 | tablet |
ORDER BY logged_at DESC, rn for equal timestamps may change from run to run (non-deterministic). Add a unique key (login_id) last to make the ranking unique and structurally prevent “the result changed when I reran it.” In contrast to the MAX correlated-subquery version returning two rows on a tie, ROW_NUMBER's fundamental strength is that it guarantees “exactly one” at the query-shape level.WHERE rn = 1 cannot be written at the same level. Wrapping it in a subquery/CTE is not a detour; it is required by the evaluation order. PostgreSQL also offers DISTINCT ON, and systems that support it offer QUALIFY as a shorthand.MAX(logged_at), MAX(device) next to GROUP BY user_id, the timestamp comes from one row while device comes from another—you have created a Frankenstein row that never existed. It does not error and returns plausible-looking values, making it a nasty correctness bug. Always ask whether the values are guaranteed to come from the same row.DISTINCT ON is concise for one row in PostgreSQL, and when there are few groups with many rows per group, LATERAL + LIMIT 1 can probe an index once per partition. If you also align an index as (user_id, logged_at DESC, login_id DESC), the previously learned “remove the sort step” helps the window operation too. Q8 moves to another standard list-screen problem—the pagination trap.The standard pagination pattern LIMIT 20 OFFSET 100000 internally reads 100,020 rows and throws away 100,000 of them. Because the cost of OFFSET is proportional to the number of discarded rows, later pages become slower; inserting a row midway can also cause page drift (duplicates and omissions). Keyset pagination (the seek method) remembers “the key of the last row on the previous page” and reads the index from there—so every page has a constant cost.
-- ✗ Page 3: read 6 rows and discard 4 (imagine page 100,000...) ORDER BY published_at DESC, article_id DESC LIMIT 2 OFFSET 4 -- ✓ Keyset: read only the continuation after the last key on the previous page WHERE (published_at, article_id) < (TIMESTAMP '2026-05-05 13:00', 5) ORDER BY published_at DESC, article_id DESC LIMIT 2
(a, b) < (x, y) means lexicographically a < x OR (a = x AND b < y). The rule is to write the cursor condition with the same columns and same directions as ORDER BY; including a unique key keeps pages stable even when timestamps tie.Display an article list from the articles table (1 million actual rows, with a composite index on (published_at, article_id)) in newest-first order (published_at DESC, article_id DESC), two rows per page. The last row on page 2 was (published_at, article_id) = ('2026-05-05 13:00', 5). Without using OFFSET, retrieve the two rows on page 3. The output columns must be article_id, title, published_at.
| article_id | title | published_at |
|---|---|---|
| 1 | Spring feature | 2026-05-01 09:00 |
| 2 | New feature introduction | 2026-05-02 10:00 |
| 3 | Incident report | 2026-05-03 11:00 |
| 4 | Interview article | 2026-05-04 12:00 |
| 5 | Careers | 2026-05-05 13:00 |
| 6 | Engineering blog | 2026-05-05 13:00 |
| 7 | Summer feature | 2026-05-06 14:00 |
| 8 | Announcement | 2026-05-07 15:00 |
※ The newest-first order is 8→7→6→5→4→3→2→1. Page 1=8,7 / page 2=6,5. Preventing tied id=6 from leaking into page 3 is the keyset design challenge.
Expected output (page 3, newest first):
| article_id | title | published_at |
|---|---|---|
| 4 | Interview article | 2026-05-04 12:00 |
| 3 | Incident report | 2026-05-03 11:00 |
id=6 has (13:00, 6), which is ahead in the sort order of cursor (13:00, 5) because id DESC puts 6 first; it was already shown on page 2. The tuple comparison excludes it correctly.
SELECT article_id, title, published_at FROM articles WHERE (published_at, article_id) < (TIMESTAMP '2026-05-05 13:00', 5) -- key of the last row on the previous page ORDER BY published_at DESC, article_id DESC -- same columns and directions as the cursor LIMIT 2; -- Assumed index: CREATE INDEX ON articles (published_at, article_id); (scan backward for DESC) /* Logical evaluation order: 1. FROM articles → choose the source 2. WHERE row-value comparison → convert to an index seek 3. ORDER BY → use index order with no sort 4. LIMIT → stop after 2 rows */
LEGEND
1. Source table — Paginate the newest-first list two rows at a time
FROM articles (newest first = published_at DESC, article_id DESC)The newest-first order is 8→7→6→5→4→3→2→1. Ids 5 and 6 tie on timestamp, so article_id DESC decides their order. Page 1=8,7 and page 2=6,5 have already been shown.| Order | article_id | title | published_at |
|---|---|---|---|
| 1 | 8 | Announcement | 2026-05-07 15:00 |
| 2 | 7 | Summer feature | 2026-05-06 14:00 |
| 3 | 6 | Engineering blog | 2026-05-05 13:00 |
| 4 | 5 | Careers | 2026-05-05 13:00 |
| 5 | 4 | Interview article | 2026-05-04 12:00 |
| 6 | 3 | Incident report | 2026-05-03 11:00 |
| 7 | 2 | New feature introduction | 2026-05-02 10:00 |
| 8 | 1 | Spring feature | 2026-05-01 09:00 |
LIMIT 20 OFFSET 100000 reads 100,020 rows and throws away 100,000. Page 1 is fast, then the list slows linearly with depth—the classic “it was fine at first, then the list became heavy as data grew” incident. Inserts and deletes while displaying the list also shift rows, causing duplicate displays and omissions. Deep-page OFFSET is structurally unfavorable for both performance and correctness.(a, b) < (x, y) is the same lexicographic comparison as a < x OR (a = x AND b < y). A hand-written OR expansion works, but row-value syntax states the intent more clearly and is easier for the planner to treat as a range condition on a composite index. The way id=6 (a same-time tie) is correctly excluded in the second-column comparison is exactly why the unique key belongs in the cursor.ORDER BY published_at DESC but only article_id < ? in the cursor—or without the tie-breaker column in the cursor—this mismatch causes duplicate or missing rows when timestamps tie. Make “all ordering columns, in the same directions, compared as a tuple” a mechanical checklist.next_cursor, and the client includes it in the next request—a thin, fast design that maps directly to a row-value comparison in the database. Put every ordering key into the cursor, and invalidate the cursor when the sort changes. Q9 asks an even smaller question than pagination: what is the shortest path when you only need to know whether something exists?When you want to know “does this user have an order?”, writing COUNT(*) > 0 makes the database count every row. What you actually want is a Boolean “yes/no,” not a count. EXISTS stops as soon as the inner query returns even one row—its internal representation is a Semi Join, the same tool as Q4 (avoid fan-out without DISTINCT). The difference between asking for a count and asking for existence can change the amount of scanning by a factor of one hundred or one thousand.
-- ✗ Cannot decide until the count is complete (worst case: read every row) SELECT CASE WHEN (SELECT COUNT(*) FROM orders WHERE user_id = 42) > 0 THEN 'yes' ELSE 'no' END; -- ✓ Stop the moment one row is found (Semi Join) SELECT EXISTS (SELECT 1 FROM orders WHERE user_id = 42);
From users (100,000 actual rows) and orders (5 million actual rows, with an index on user_id), build a list of users who have at least one order and users with no orders, with an existence flag for each. Do not count orders per user for the judgment (the count is not needed and the scan would be wasteful). The output columns must be user_id, name, has_order ('yes' / 'no', user_id ascending).
| user_id | name |
|---|---|
| 1 | Sato |
| 2 | Suzuki |
| 3 | Tanaka |
| 4 | Ito |
| order_id | user_id |
|---|---|
| 101 | 1 |
| 102 | 1 |
| 103 | 1 |
| 104 | 2 |
| 105 | 3 |
| 106 | 3 |
※ Even if Sato has 1 million orders, this judgment should be settled the moment the first row is read.
Expected output (user_id ascending):
| user_id | name | has_order |
|---|---|---|
| 1 | Sato | yes |
| 2 | Suzuki | yes |
| 3 | Tanaka | yes |
| 4 | Ito | no |
The COUNT version counts all 1 million of Sato's orders before deciding '> 0'. The EXISTS version uses idx(user_id) and is decided the moment it finds one row.
SELECT u.user_id, u.name, CASE WHEN EXISTS ( SELECT 1 FROM orders o WHERE o.user_id = u.user_id -- stop immediately when one row is found ) THEN 'yes' ELSE 'no' END AS has_order FROM users u ORDER BY u.user_id; -- ✗: CASE WHEN (SELECT COUNT(*) FROM orders WHERE user_id = u.user_id) > 0 … -- → Count every order per user (run the idx(user_id) Range Scan to the end) /* Logical evaluation order: 1. FROM users → scan 2. EXISTS in SELECT → stop after one row via the index (Semi Join) 3. CASE → convert TRUE/FALSE to 'yes'/'no' 4. ORDER BY user_id → sort ascending */
LEGEND
1. Source table — Decide whether each user has an order
FROM users u (query orders for each user)We want a Boolean “yes/no”: one match is enough to settle the answer. Even if Sato has 1 million orders, there is no reason to read the rest after finding the first one.| user_id | name | Order count (reference) |
|---|---|---|
| 1 | Sato | Many (up to 1 million in the worst case) |
| 2 | Suzuki | 1 row |
| 3 | Tanaka | Many |
| 4 | Ito | 0 rows |
SELECT 1, SELECT *, and SELECT NULL all perform the same. The convention of writing SELECT 1 is a signal to the reader that column names and values do not matter; only existence is being asked. There is no need for aggregation or ORDER BY inside it.EXISTS (Semi Join), and whether it “does not exist” with NOT EXISTS (Anti Join, learned previously). With an index, both can stop when a row is found—or when the search proves there is none. IN / NOT IN are semantically close, but NOT IN has a NULL trap and does not communicate early-exit intent as clearly as EXISTS. EXISTS-family forms are the standard choice for existence checks.if (count > 0), but the SQL returns COUNT—this is the most common production anti-pattern. If the count is used only for the judgment and never displayed, it can be replaced with EXISTS immediately. Ask: “Do we really use the counted number in the output?”EXISTS (SELECT 1 FROM orders ORDER BY ordered_at DESC LIMIT 1), but ordering and limiting inside EXISTS do not affect the result (only existence matters). It is redundant and can misleadingly suggest an intent such as “is there a latest order?” Keep the inside of EXISTS to the WHERE conditions.As a capstone for this part, build a practical admin-screen query. “Among unprocessed tasks, show only the latest one for each assignee, then page through them newest-first with a cursor”—five foundational techniques divide the work cleanly.
① A partial index (Q5) indexes only the minority of 'pending' rows, ② ROW_NUMBER (Q7) selects the latest row per assignee, ③ keyset pagination (Q8) pages without OFFSET, ④ avoiding implicit type conversion (Q6) keeps the index usable, and ⑤ EXISTS (Q9) expresses the filter as existence. The Part 3 fundamentals—communicate intent / do not wrap columns in functions / operate on the set once / stop at Top-N—all crystallize in one query.
-- The overall shape (pseudocode) WITH ranked AS ( -- ① latest one per assignee with ROW_NUMBER SELECT ..., ROW_NUMBER() OVER (PARTITION BY assignee_id ORDER BY created_at DESC, task_id DESC) AS rn FROM tasks WHERE status = 'pending' -- ② shape that lets the partial index work (align the string type) ) SELECT ... FROM ranked WHERE rn = 1 AND (created_at, task_id) < (?, ?) -- ③ continue with keyset pagination AND EXISTS (SELECT 1 FROM task_tags ...) -- ⑤ existence check with EXISTS ORDER BY created_at DESC, task_id DESC LIMIT ?;
Build the admin screen for a task-management SaaS showing the newest unprocessed task for each assignee, newest first. The requirements are:
- Show only the latest pending task for each assignee (tie-breaker: task_id DESC)
- Keep only tasks with the “important” tag (there is a
tag = 'important'row intask_tags) - Use cursor pagination in newest-first order (
created_at DESC, task_id DESC), two rows per page - Previous-page cursor:
(created_at, task_id) = ('2026-06-04 11:00', 4)(retrieve the next page from this cursor position) - Do not use OFFSET; also provide a partial index to avoid scanning every table row (status is varchar)
| task_id | assignee_id | title | status | created_at |
|---|---|---|---|---|
| 1 | 1 | Review invoices | pending | 2026-06-08 10:00 |
| 2 | 1 | Organize meeting minutes | pending | 2026-06-05 09:00 |
| 3 | 2 | Hiring interview | pending | 2026-06-07 14:00 |
| 4 | 2 | Design review | pending | 2026-06-04 11:00 |
| 5 | 3 | Audit response | pending | 2026-06-03 16:00 |
| 6 | 3 | Revise contract | pending | 2026-06-02 09:00 |
| 7 | 4 | Order supplies | pending | 2026-05-30 13:00 |
| 8 | 5 | Arrange cleaning | done | 2026-06-09 09:00 |
| task_id | tag |
|---|---|
| 1 | important |
| 2 | internal |
| 3 | important |
| 4 | important |
| 5 | important |
| 6 | internal |
| 7 | important |
※ Latest pending per assignee: A1→task1, A2→task3, A3→task5, A4→task7. Filtering by important leaves 1, 3, 5, 7 (all four match). Newest first is 1→3→5→7. Page 1=1,3 / page 2=5,7, but the cursor (2026-06-04 11:00, 4) is placed at task=4 rather than on a page boundary.
Expected output (the next page from the cursor position, newest first):
| task_id | assignee_id | title | created_at |
|---|---|---|---|
| 5 | 3 | Audit response | 2026-06-03 16:00 |
| 7 | 4 | Order supplies | 2026-05-30 13:00 |
Of the four targets (tasks 1, 3, 5, 7), tasks older than cursor (2026-06-04 11:00, 4) are 5 and 7. Display those two as the next page from the cursor position.
CREATE INDEX idx_tasks_pending_keyset -- Partial index: index only 'pending' rows in latest-per-assignee order (Q5 + Q7) ON tasks (assignee_id, created_at DESC, task_id DESC) WHERE status = 'pending'; CREATE INDEX idx_task_tags_task_tag ON task_tags (task_id, tag); -- Supporting index: existence lookup for the important tag (Q9 EXISTS) WITH ranked AS ( SELECT task_id, assignee_id, title, created_at, ROW_NUMBER() OVER (PARTITION BY assignee_id -- ① one partition per assignee ORDER BY created_at DESC, task_id DESC) AS rn -- tie-breaker is required FROM tasks WHERE status = 'pending' -- ② the partial index is implied (string-to-string comparison, Q6) ) SELECT task_id, assignee_id, title, created_at FROM ranked WHERE rn = 1 -- keep only the latest row per assignee AND (created_at, task_id) < (TIMESTAMP '2026-06-04 11:00', 4) -- ③ keyset pagination (Q8) AND EXISTS ( -- ⑤ existence check (Q9) SELECT 1 FROM task_tags tt WHERE tt.task_id = ranked.task_id AND tt.tag = 'important' ) ORDER BY created_at DESC, task_id DESC -- same columns and directions as the cursor LIMIT 2; /* Logical evaluation order: 1. CTE FROM/WHERE → use the partial index for status='pending' 2. CTE window → partition and assign ROW_NUMBER (use per-assignee index order) 3. Outer WHERE rn = 1 → keep the first row per assignee 4. Keyset row-value comparison → seek toward older rows 5. EXISTS → check for important with one lookup (Semi Join) 6. ORDER BY + LIMIT → order the remaining candidates and stop at 2 rows */
LEGEND
1. Raw data — 1 million tasks + task_tags (many-to-many)
FROM tasks (pending is about 0.5%; skewed distribution)Only about 0.5% of the 1 million rows are pending. At this scale, starting to read every row including done means losing immediately. task_tags is a many-to-many junction table (previously learned), a typical structure for managing the important flag in a separate table.| task_id | assignee_id | title | status | created_at |
|---|---|---|---|---|
| 1 | 1 | Review invoices | pending | 2026-06-08 10:00 |
| 2 | 1 | Organize meeting minutes | pending | 2026-06-05 09:00 |
| 3 | 2 | Hiring interview | pending | 2026-06-07 14:00 |
| 4 | 2 | Design review | pending | 2026-06-04 11:00 |
| 5 | 3 | Audit response | pending | 2026-06-03 16:00 |
| 6 | 3 | Revise contract | pending | 2026-06-02 09:00 |
| 7 | 4 | Order supplies | pending | 2026-05-30 13:00 |
| 8 | 5 | Arrange cleaning | done | 2026-06-09 09:00 |
(assignee_id, created_at DESC, task_id DESC) in this question directly expresses the per-assignee PARTITION BY + ORDER BY for the window. The global order (created_at DESC, task_id DESC) has no assignee key, so it is a different ordering and is applied after reducing to the latest row per assignee. Do not assume that one index must serve every ordering requirement; design it by explaining the trade-offs among the required sequences. An index is not just a set of columns but an ordered sequence of columns.UPPER(status) = 'PENDING'”—stacking isolated solutions produces function wrappers + fan-out + full scans + fabricated rows all at once. The right answer is to design one route as a data flow, not merely assemble disconnected parts.