Batch Processing — Learn DISTINCT, NULL, LIKE, EXISTS, window functions, and UPSERT from the Basics
BasicSQL fundamentalsDISTINCT / NULL / LIKEIN / EXISTS / date functionsWINDOW / UNIONTransactions / UPSERTPostgreSQL-ready5 questions
QUESTION 1
DISTINCT + COUNT(DISTINCT) — Understand real usage with deduplication and unique counts
DISTINCTCOUNT(DISTINCT)DeduplicationAnalytics API
Background

DISTINCT removes duplicate rows from a SELECT result. COUNT(DISTINCT column) counts unique non-NULL values. It is the foundation for answering “how many kinds of values are there?” rather than simply “how many rows are there?”

SELECT DISTINCT col1, col2       -- Remove duplicate col1+col2 combinations
FROM   table_name;

SELECT COUNT(DISTINCT col1)      -- Count unique col1 values (NULL is excluded)
FROM   table_name;
How it differs from COUNT(*): COUNT(*) counts all rows, COUNT(DISTINCT col) counts unique values, and COUNT(col) without DISTINCT counts non-NULL rows. Understand all three precisely.

Measuring unique users visiting a site or the number of different products purchased is common in every kind of API and batch aggregation.

Problem

From the access_logs table, return in one row (1) the number of unique users in May 2024 (deduplicated user_id count) and (2) the total number of accesses. In a separate query, return the deduplicated list of user IDs that accessed the site, ordered by user_id ascending.

Tables used
▸ access_logs
log_iduser_idpageaccessed_at
1U01/home2024-05-01
2U02/items2024-05-03
3U01/items2024-05-05
4U03/home2024-05-10
5U01/cart2024-05-12
6U02/home2024-05-20
7U04/items2024-06-01
Expected Output

1. Expected output for the aggregate query (May 2024 only; June's U04 is excluded):

unique_userstotal_access
36

2. User ID list (deduplicated, ascending):

user_id
U01
U02
U03

U04 is excluded because its access was in June. U01 accessed three times but counts as one distinct user.

Model Answer
-- 1. Aggregate unique users and total accesses
SELECT
  COUNT(DISTINCT user_id) AS unique_users,  -- Number of distinct user_id values
  COUNT(*)               AS total_access   -- All rows (total accesses)
FROM  access_logs
WHERE accessed_at >= '2024-05-01'           -- From May 1
  AND accessed_at <  '2024-06-01';          -- Before June 1

-- 2. Deduplicated, ascending list of accessing user IDs
SELECT DISTINCT user_id             -- Remove duplicate user_id values
FROM   access_logs
WHERE  accessed_at >= '2024-05-01'  -- From May 1
  AND  accessed_at <  '2024-06-01'  -- Before June 1
ORDER BY user_id ASC;

/*
  Logical execution order (1. aggregate query):
  1. FROM access_logs        → Read rows
  2. WHERE accessed_at ...   → Filter rows
  3. SELECT                  → Evaluate aggregate functions (unique_users, total_access)

  Logical execution order (2. DISTINCT query):
  1. FROM access_logs        → Read rows
  2. WHERE accessed_at ...   → Filter rows
  3. SELECT DISTINCT user_id → Remove duplicates
  4. ORDER BY user_id ASC    → Sort and return
*/
Explanation (table transitions & key points)
SELECT COUNT(DISTINCT user_id) AS unique_users, COUNT(*) AS total_access FROM access_logs WHERE accessed_at >= '2024-05-01' AND accessed_at < '2024-06-01';
LEGEND
Rows read / loaded
① FROM
FROM access_logsRead all seven rows from access_logs. The next WHERE step keeps only rows from May.
1 / 3
log_iduser_idpageaccessed_at
1U01/home2024-05-01
2U02/items2024-05-03
3U01/items2024-05-05
4U03/home2024-05-10
5U01/cart2024-05-12
6U02/home2024-05-20
7U04/items2024-06-01
All 7 rows read
SELECT DISTINCT user_id FROM access_logs WHERE accessed_at >= '2024-05-01' AND accessed_at < '2024-06-01' ORDER BY user_id ASC;
LEGEND
Rows read / loaded
① FROM + WHERE
FROM access_logs WHERE MayFilter access_logs with WHERE to keep May rows. U04's June access is excluded, leaving six rows.
1 / 3
log_iduser_idaccessed_at
1U012024-05-01
2U022024-05-03
3U012024-05-05
4U032024-05-10
5U012024-05-12
6U022024-05-20
After WHERE: 6 rows
LEARNING POINTS
Core idea: DISTINCT is a filter that collapses rows with the same value into one row. With multiple columns, it removes duplicate column combinations; for example, SELECT DISTINCT user_id, page deduplicates by user-and-page pair.
Three uses of COUNT: COUNT(*) = all rows including NULL, COUNT(col) = non-NULL rows, and COUNT(DISTINCT col) = unique values. Analytics and KPI dashboard APIs may use all three.
Choosing between DISTINCT and GROUP BY: Use SELECT DISTINCT user_id when you only need a user ID list. Use SELECT user_id, COUNT(*) ... GROUP BY user_id when you also need each user’s access count.
Performance note: DISTINCT and COUNT(DISTINCT) often require internal sorting and can be expensive on large datasets. Indexes on the target columns can help, and analytics systems sometimes use approximate counts such as HyperLogLog.
ANTI-PATTERNS
Overusing SELECT DISTINCT *: DISTINCT across every column makes it unclear which combination should be deduplicated and can produce unintended results. Name only the required columns, such as SELECT DISTINCT user_id.
Using COUNT(*) for unique counts: COUNT(*) includes duplicates, so “unique users = COUNT(*)” is wrong. If U01 accessed three times, it contributes three rows. Always use COUNT(DISTINCT user_id) for unique users.
Practical column: Why unique-user measurement is difficult
A web service usually identifies unique users with cookies or device IDs, while COUNT(DISTINCT user_id) gives an accurate count when only logged-in users are included. Guests and users on multiple devices can cause overcounting or undercounting. Keep the KPI definition and SQL implementation together and review them as one specification.
QUESTION 2
COALESCE / IS NULL / NULLIF — Handle NULL safely without breaking API responses
COALESCEIS NULLNULLIFNULL-safe
Background

In SQL, NULL represents a missing or unknown value. Arithmetic with NULL and string concatenation with NULL both produce NULL. Because a NULL in an API response can cause unexpected behavior, explicit NULL handling is essential.

COALESCE(col, 'default value')   -- Evaluate left to right and return the first non-NULL value
col IS NULL                       -- Test for NULL (= NULL is invalid; always use IS NULL)
col IS NOT NULL                   -- Test that a value is not NULL
NULLIF(col, 0)                   -- Return NULL when col is 0 (for example, to prevent division by zero)
NULL = NULL is FALSE: WHERE col = NULL is always false and matches no rows. Use IS NULL or IS NOT NULL for NULL comparisons. This is one of SQL’s most important rules.
Problem

From the users table, return all users. Display 'Not registered' when phone is NULL, and use name as a fallback when display_name is NULL. In a separate query, return the number of users whose phone is NULL.

Tables used
▸ users
user_idnamedisplay_namephone
1Taro TanakaTanaka090-1111-2222
2Hanako SatoNULL080-3333-4444
3Ichiro SuzukiSuzukiNULL
4Jiro YamadaNULLNULL
Expected Output

1. Expected output (all users, NULLs handled):

user_idshown_namephone_display
1Tanaka090-1111-2222
2Hanako Sato080-3333-4444
3SuzukiNot registered
4Jiro YamadaNot registered

2. Number of users whose phone is NULL:

null_phone_count
2
Model Answer
-- 1. Return all users with safe defaults for NULLs
SELECT
  user_id,
  COALESCE(display_name, name) AS shown_name,  -- Use name when display_name is NULL
  COALESCE(phone, 'Not registered')    AS phone_display  -- Use 'Not registered' when NULL
FROM  users
ORDER BY user_id;

-- 2. Count users whose phone is NULL
SELECT
  COUNT(*) AS null_phone_count  -- Count after filtering with IS NULL
FROM  users
WHERE phone IS NULL;            -- Test NULL with IS NULL (= NULL is invalid)

/*
  Logical execution order (1. NULL-default query):
  1. FROM users          → Read rows
  2. SELECT              → Evaluate columns (correct NULLs with COALESCE)
  3. ORDER BY user_id    → Sort and return

  Logical execution order (2. NULL-count query):
  1. FROM users          → Read rows
  2. WHERE phone IS NULL → Filter rows
  3. SELECT null_phone_count → Evaluate the aggregate
*/
Explanation (table transitions & key points)
SELECT user_id, COALESCE(display_name, name) AS shown_name, COALESCE(phone, 'Not registered') AS phone_display FROM users ORDER BY user_id;
LEGEND
Rows read / loaded
① FROM
FROM usersRead all four rows from users. display_name and phone each contain NULL values.
1 / 3
user_idnamedisplay_namephone
1Taro TanakaTanaka090-1111-2222
2Hanako SatoNULL080-3333-4444
3Ichiro SuzukiSuzukiNULL
4Jiro YamadaNULLNULL
All 4 rows read
SELECT COUNT(*) AS null_phone_count FROM users WHERE phone IS NULL;
LEGEND
Rows read / loaded
① FROM
FROM usersRead all four rows from users. WHERE will identify the rows whose phone is NULL (user_id=3,4).
1 / 3
user_idnamephone
1Taro Tanaka090-1111-2222
2Hanako Sato080-3333-4444
3Ichiro SuzukiNULL
4Jiro YamadaNULL
All 4 rows read
LEARNING POINTS
Core idea: NULL is special: NULL means “no value,” so NULL cannot be compared with ==. NULL = NULL is not TRUE; it is NULL (unknown). Always use IS NULL or IS NOT NULL for NULL checks.
COALESCE accepts three or more arguments: You can write COALESCE(nickname, display_name, name, 'Anonymous'). It evaluates left to right and returns the first non-NULL value; the final fixed value is used if all arguments are NULL.
Using NULLIF to prevent division by zero: SUM(amount) / NULLIF(COUNT(*), 0) makes NULLIF return NULL when COUNT(*) is 0, preventing a division-by-zero error and safely calculating an average for an empty database.
NULL and aggregate functions: SUM, AVG, and COUNT(col) ignore NULLs. If a phone column has five rows and two are NULL, COUNT(phone) returns 3. You can use this to count non-NULL values.
ANTI-PATTERNS
Filtering with WHERE col = NULL: This is the most common NULL mistake. = NULL asks whether an unknown value equals NULL, so the result is never TRUE and no rows match. Always use IS NULL.
Forgetting NULL in calculations: If either side of price * quantity is NULL, the result is NULL. Returning a NULL total to an API can break frontend display or calculations. Protect it with defaults such as COALESCE(price, 0) * COALESCE(quantity, 0).
Practical column: When to use NULL and when not to
NULL is useful for “missing” or “not registered” values, but overusing it forces NULL guards throughout the application and makes code complex. A practical rule is to allow NULL for optional fields and enforce NOT NULL for required fields at the database level. NULL and the empty string ('') are different, so standardize how “not entered” is represented during database design.
QUESTION 3
LIKE / ILIKE — Implement a search API with string pattern matching
LIKEILIKESearch APIPattern matching
Background

LIKE performs partial, prefix, and suffix string matching. ILIKE (a PostgreSQL extension) is LIKE without case sensitivity. It is a basic building block for a search API such as GET /products?q=keyword.

WHERE col LIKE  '%keyword%'     -- Partial match (% = any string of zero or more characters)
WHERE col LIKE  'keyword%'      -- Prefix match (starts with keyword)
WHERE col LIKE  '%keyword'      -- Suffix match (ends with keyword)
WHERE col LIKE  'k_yword'       -- _ = any one character (one-character wildcard)
WHERE col ILIKE '%Keyword%'     -- Case-insensitive partial match (PostgreSQL)
Performance note: LIKE '%keyword%' (with a leading %) cannot use a normal index and scans every row. For serious full-text search on large tables, consider a pg_trgm GIN index or a search engine such as Elasticsearch.
Problem

From the products table, return products whose name contains 'coffee', ordered by price from low to high (case-insensitive). In a separate query, count products whose SKU code (sku) starts with 'BEV-'.

Tables used
▸ products
product_idnameskuprice
1Iced CoffeeBEV-001350
2Green TeaBEV-002250
3Hot CoffeeBEV-003380
4Chocolate CakeFOOD-001480
5Coffee JellyFOOD-002320
6Orange JuiceBEV-004280
Expected Output

1. Products whose name contains 'coffee' (price ascending):

product_idnameskuprice
5Coffee JellyFOOD-002320
1Iced CoffeeBEV-001350
3Hot CoffeeBEV-003380

2. Number of products whose SKU starts with 'BEV-':

bev_count
4
Model Answer
-- 1. Return products whose name contains 'coffee', ordered by price
SELECT
  product_id,
  name,
  sku,
  price
FROM   products
WHERE  name ILIKE '%coffee%'  -- Case-insensitive partial match (PostgreSQL)
ORDER BY price ASC;

-- 2. Count products whose sku starts with 'BEV-'
SELECT
  COUNT(*) AS bev_count
FROM   products
WHERE  sku LIKE 'BEV-%';  -- Prefix match (starts with 'BEV-')

/*
  Logical execution order (1. ILIKE partial-match query):
  1. FROM products            → Read rows
  2. WHERE name ILIKE ...     → Filter rows
  3. SELECT                   → Evaluate columns
  4. ORDER BY price ASC       → Sort and return

  Logical execution order (2. LIKE prefix-count query):
  1. FROM products            → Read rows
  2. WHERE sku LIKE 'BEV-%'   → Filter rows
  3. SELECT bev_count         → Evaluate the aggregate
*/
Explanation (table transitions & key points)
SELECT product_id, name, sku, price FROM products WHERE name ILIKE '%coffee%' ORDER BY price ASC;
LEGEND
Rows read / loaded
① FROM
FROM productsRead all six rows from products. The next WHERE keeps rows whose name contains 'coffee'.
1 / 3
product_idnameskuprice
1Iced CoffeeBEV-001350
2Green TeaBEV-002250
3Hot CoffeeBEV-003380
4Chocolate CakeFOOD-001480
5Coffee JellyFOOD-002320
6Orange JuiceBEV-004280
All 6 rows read
SELECT COUNT(*) AS bev_count FROM products WHERE sku LIKE 'BEV-%';
LEGEND
Rows read / loaded
① FROM
FROM productsRead all six rows from products. The next WHERE keeps SKU codes beginning with 'BEV-'.
1 / 3
product_idnameskuprice
1Iced CoffeeBEV-001350
2Green TeaBEV-002250
3Hot CoffeeBEV-003380
4Chocolate CakeFOOD-001480
5Coffee JellyFOOD-002320
6Orange JuiceBEV-004280
All 6 rows read
LEARNING POINTS
Core idea: % versus _: % matches any string of zero or more characters, while _ matches exactly one character. For example, CAT and CUT match C_T, but COAT does not.
Typical search API pattern: A server receives a query parameter such as ?q=coffee and binds it with a placeholder, for example WHERE name ILIKE % || $1 || %. Placeholders (bound variables) are mandatory for SQL-injection protection.
Choosing LIKE or ILIKE: ILIKE is useful when English data such as email addresses or SKU codes should ignore case. It is a PostgreSQL extension, so other databases often use LOWER(col) LIKE LOWER(%key%).
When indexes can help: A prefix pattern such as prefix% may use a B-tree index. A partial-match pattern with % on both sides, %keyword%, usually cannot. Consider pg_trgm for large searches.
ANTI-PATTERNS
Building SQL by concatenating strings (SQL injection): Directly embedding input in a LIKE string is dangerous. Input such as DROP TABLE products can execute arbitrary SQL. Always use placeholders such as $1.
Using a two-sided % LIKE on large data: LIKE %keyword% over millions of rows can force a full scan and time out an API. Use pg_trgm plus a GIN index or a dedicated full-text search engine.
Practical column: When LIKE should give way to full-text search
LIKE is sufficient for small services with tens of thousands of rows. Once data reaches hundreds of thousands of rows, partial matching is likely to become a slow query. Try pg_trgm first; if it is still not enough, run a dedicated engine such as Elasticsearch or Meilisearch alongside the database. Revisit whether LIKE is sufficient as user and record counts grow.
QUESTION 4
IN / NOT IN / EXISTS — Efficiently filter batch targets with set operators
INNOT INEXISTSBatch filtering
Background

IN tests whether a column value appears in a specified list or subquery result set. NOT IN returns rows that are not included. EXISTS checks whether a subquery returns at least one row.

WHERE col IN ('a', 'b', 'c')      -- col matches one of 'a', 'b', or 'c'
WHERE col IN (SELECT id FROM t)  -- Included in the subquery result set
WHERE col NOT IN ('x', 'y')      -- col is neither 'x' nor 'y'
WHERE EXISTS (                    -- Check whether the subquery finds one or more rows
  SELECT 1 FROM t
  WHERE  t.id = outer.id           -- Correlate with the outer query
)
The NOT IN and NULL trap: If a NOT IN subquery contains even one NULL, every comparison becomes UNKNOWN and no rows match. Add WHERE col IS NOT NULL to the subquery or use NOT EXISTS.
Problem

1. From the orders table, return orders whose status is 'completed' or 'pending'. 2. From the same table, return orders whose purchase_id does not exist in the purchases table (unprocessed orders).

Tables used
▸ orders
order_iduser_idamountstatus
1U013000completed
2U025000pending
3U018000cancelled
4U032000completed
5U021200pending
▸ purchases (processed orders)
purchase_idprocessed_at
12024-05-10
42024-05-11
Expected Output

1. Orders with completed or pending status (order_id ascending):

order_iduser_idamountstatus
1U013000completed
2U025000pending
4U032000completed
5U021200pending

2. Unprocessed orders not present in purchases (order_id: 2, 3, 5):

order_iduser_idamountstatus
2U025000pending
3U018000cancelled
5U021200pending
Model Answer
-- 1. Return completed or pending orders (using IN)
SELECT
  order_id,
  user_id,
  amount,
  status
FROM   orders
WHERE  status IN ('completed', 'pending')  -- Match either value concisely instead of OR
ORDER BY order_id;

-- 2. Orders not present in purchases (using NOT EXISTS)
SELECT
  o.order_id,
  o.user_id,
  o.amount,
  o.status
FROM   orders o
WHERE  NOT EXISTS (                  -- No subquery row means the order is absent
  SELECT 1                           -- The value is irrelevant for an existence test
  FROM   purchases p
  WHERE  p.purchase_id = o.order_id  -- Correlate with the outer query
)
ORDER BY o.order_id;

/*
  Logical execution order (1. IN query):
  1. FROM orders           → Read rows
  2. WHERE status IN (...) → Filter rows
  3. SELECT                → Evaluate columns
  4. ORDER BY order_id     → Sort and return

  Logical execution order (2. NOT EXISTS query):
  1. FROM orders o         → Read rows
  2. WHERE NOT EXISTS      → Evaluate the subquery and filter rows
  3. SELECT                → Evaluate columns
  4. ORDER BY o.order_id   → Sort and return
*/
Explanation (table transitions & key points)
SELECT order_id, user_id, amount, status FROM orders WHERE status IN ('completed', 'pending') ORDER BY order_id;
LEGEND
Rows read / loaded
① FROM
FROM ordersRead all five rows from orders. status has three values: completed, pending, and cancelled.
1 / 3
order_iduser_idamountstatus
1U013000completed
2U025000pending
3U018000cancelled
4U032000completed
5U021200pending
All 5 rows read
SELECT o.order_id, o.user_id, o.amount, o.status FROM orders o WHERE NOT EXISTS ( SELECT 1 FROM purchases p WHERE p.purchase_id = o.order_id ) ORDER BY o.order_id;
LEGEND
Rows read / loaded
① FROM orders
FROM orders oRead all five rows from the outer orders query. For each row, the subquery checks whether it exists in purchases.
1 / 3
order_iduser_idamountstatus
1U013000completed
2U025000pending
3U018000cancelled
4U032000completed
5U021200pending
All 5 rows read
LEARNING POINTS
Core idea: an alternative to IN: WHERE status IN (completed, pending) means the same as two OR predicates. IN remains readable as the list grows.
Finding unprocessed records in a batch: NOT EXISTS is ideal for selecting records that have not yet been processed. It efficiently finds rows absent from the processed table.
Why prefer NOT EXISTS to NOT IN: A NOT IN subquery returns zero rows if purchases contains NULL. NOT EXISTS is unaffected by NULL, so it is safer; LEFT JOIN + IS NULL is another practical option.
Why SELECT 1 is used with EXISTS: EXISTS only checks whether the subquery finds at least one row, so the selected value does not matter. SELECT 1 and SELECT NULL are conventional and equivalent.
ANTI-PATTERNS
A NULL in a NOT IN subquery: If purchases contains NULL, WHERE order_id NOT IN (SELECT purchase_id FROM purchases) becomes an unknown comparison and returns no rows. Add WHERE purchase_id IS NOT NULL or use NOT EXISTS.
Passing a huge list to IN: Writing tens of thousands of values directly in WHERE id IN (1,2,3,...,10000) makes SQL parsing expensive. Insert the IDs into a temporary table and join it instead.
Practical column: Using NOT EXISTS in delta batch processing
A common nightly-batch pattern is to process only records that were not processed last time. Maintain a table such as processed_ids and use NOT EXISTS to extract pending rows. Insert the ID after processing so reruns do not process the same record twice; this creates an idempotent batch.
QUESTION 5
Date and time functions — Build daily and monthly aggregation APIs with DATE_TRUNC, EXTRACT, and NOW()
DATE_TRUNCEXTRACTNOW()Daily aggregation
Background

Date and time functions are essential for aggregating logs over periods and scheduling batch jobs. Learn the main PostgreSQL functions.

NOW()                             -- Current timestamp (with time zone)
CURRENT_DATE                      -- Today’s date (without a time)
DATE_TRUNC('month', col)          -- Truncate col to the first instant of its month (year/day/hour also work)
EXTRACT(YEAR FROM col)           -- Extract only the year (MONTH, DAY, DOW, and others also work)
col ::DATE                        -- Cast a timestamp to a date
col + INTERVAL '7 days'          -- Timestamp seven days later
col - INTERVAL '1 month'         -- Timestamp one month earlier
Specifying date ranges: To select a whole month, use DATE_TRUNC('month', NOW()) as the inclusive lower bound and DATE_TRUNC('month', NOW()) + INTERVAL '1 month' as the exclusive upper bound.
Problem

From the orders table, aggregate the number of orders and total amount per month. Display the month in 'YYYY-MM' format and show the newest month first. Also write a separate query that returns only orders from the past 30 days.

Tables used
▸ orders
order_iduser_idamountordered_at
1U0130002024-04-05
2U0250002024-04-20
3U0180002024-05-10
4U0320002024-05-15
5U0212002024-05-28
6U0145002024-06-03
Expected Output

1. Monthly aggregation (newest month first):

monthorder_counttotal_amount
2024-0614500
2024-05311200
2024-0428000

2. Orders from the past 30 days (reference date: 2024-06-03, ordered_at descending):

order_iduser_idamountordered_at
6U0145002024-06-03
5U0212002024-05-28
4U0320002024-05-15
3U0180002024-05-10
Model Answer
-- 1. Aggregate order count and total amount by month (newest first)
SELECT
  TO_CHAR(DATE_TRUNC('month', ordered_at), 'YYYY-MM') AS month,  -- Truncate to month start, then format as YYYY-MM
  COUNT(*)    AS order_count,
  SUM(amount) AS total_amount
FROM     orders
GROUP BY DATE_TRUNC('month', ordered_at)   -- Group by month
ORDER BY DATE_TRUNC('month', ordered_at) DESC;  -- Newest month first

-- 2. Return only orders from the past 30 days
SELECT
  order_id,
  user_id,
  amount,
  ordered_at
FROM   orders
WHERE  ordered_at >= NOW() - INTERVAL '30 days'  -- The last 30 days
ORDER BY ordered_at DESC;

/*
  Logical execution order (1. monthly aggregation):
  1. FROM orders             → Read rows
  2. GROUP BY DATE_TRUNC     → Group rows
  3. SELECT                  → Evaluate COUNT and SUM
  4. ORDER BY ... DESC       → Sort and return

  Logical execution order (2. past-30-days query):
  1. FROM orders             → Read rows
  2. WHERE ordered_at >= ... → Filter rows
  3. SELECT                  → Evaluate columns
  4. ORDER BY ordered_at DESC → Sort and return
*/
Explanation (table transitions & key points)
SELECT TO_CHAR(DATE_TRUNC('month', ordered_at), 'YYYY-MM') AS month, COUNT(*) AS order_count, SUM(amount) AS total_amount FROM orders GROUP BY DATE_TRUNC('month', ordered_at) ORDER BY DATE_TRUNC('month', ordered_at) DESC;
LEGEND
Rows read / loaded
① FROM
FROM ordersRead all six rows from orders. ordered_at contains data from April through June 2024.
1 / 4
order_iduser_idamountordered_at
1U0130002024-04-05
2U0250002024-04-20
3U0180002024-05-10
4U0320002024-05-15
5U0212002024-05-28
6U0145002024-06-03
All 6 rows read
SELECT order_id, user_id, amount, ordered_at FROM orders WHERE ordered_at >= NOW() - INTERVAL '30 days' ORDER BY ordered_at DESC;
LEGEND
Rows read / loaded
① FROM
FROM ordersRead all six rows from orders. WHERE will keep only rows within 30 days of the current time.
1 / 3
order_iduser_idamountordered_at
1U0130002024-04-05
2U0250002024-04-20
3U0180002024-05-10
4U0320002024-05-15
5U0212002024-05-28
6U0145002024-06-03
All 6 rows read
LEARNING POINTS
Core idea: how DATE_TRUNC truncates: DATE_TRUNC month applied to 2024-05-15 becomes 2024-05-01 00:00:00. All dates in the same month become the same key, so it works well in GROUP BY. It also supports year, week, day, and hour.
Monthly KPI dashboard API: This pattern generates chart data such as monthly sales or monthly active users in one query. It is frequently passed to frontend chart libraries such as Chart.js.
Choosing EXTRACT or DATE_TRUNC: EXTRACT retrieves a value as a number, for example EXTRACT(MONTH FROM ordered_at) → 5. DATE_TRUNC creates a timestamp key for grouping, so it is usually better for GROUP BY.
Relative dates with INTERVAL: NOW() - INTERVAL 30 days avoids hard-coded dates and is ideal for daily batches. Intervals such as 1 month, 1 year, and 6 hours are also available.
ANTI-PATTERNS
Applying a function to a column and losing an index: WHERE EXTRACT(YEAR FROM ordered_at) = 2024 applies a function to ordered_at and can disable its index. Use a range instead: ordered_at >= 2024-01-01 AND ordered_at < 2025-01-01.
Comparing timestamps without considering time zones: If the server and database use different time zones, “after midnight today” may not mean what you expect. Use TIMESTAMPTZ and explicit comparisons such as AT TIME ZONE Asia/Tokyo.
Practical column: Drift between batch time and NOW()
For a nightly batch that aggregates the previous day, a relative date such as CURRENT_DATE - INTERVAL 1 day is standard. But a batch crossing midnight can see NOW change to a new day and shift its target range. A robust design passes the batch start time as a processing-reference timestamp and uses that parameter in WHERE clauses.