Anomaly Detection — Learn Moving Averages, z-Scores, and IQR-Based Outlier Detection from the Basics
BasicAnomaly DetectionMoving Average & Standard Deviationz-score / IQRROWS BETWEENPostgreSQL Compatible5 questions
QUESTION 1
Moving Average — Calculate a three-day sales average with AVG() OVER (ROWS BETWEEN)
AVG OVERROWS BETWEENMoving AverageTime-Series Analysis
Background

The first step in anomaly detection is to establish a baseline for the normal range. Instead of using a simple average over the entire period, a moving average over the most recent N days provides a dynamic baseline that accounts for trends.

AVG(amount) OVER (
  ORDER BY dt
  ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
)
-- ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
-- = current row + two preceding rows → the latest three rows (three days)
-- The first two rows have smaller denominators because fewer rows are available
ROWS BETWEEN vs. RANGE BETWEEN: ROWS BETWEEN defines the frame by the physical number of rows. RANGE BETWEEN defines it by differences between values, so its behavior changes when multiple rows share the same dt. Row-based ROWS BETWEEN is generally used for time-series anomaly detection.
Problem

From the daily_sales table, calculate each date's sales amount and the moving average over the most recent three days, including the current day, as moving_avg. Return dt, amount, moving_avg in ascending dt order. Round moving_avg to one decimal place.

Table
► daily_sales (7 rows)
dtamount
2024-01-01100
2024-01-02120
2024-01-03110
2024-01-04130
2024-01-05500
2024-01-06115
2024-01-07125

The value 500 on 2024-01-05 is an obvious anomaly (a spike). The moving average reveals the surrounding "normal band."

Expected Output

Expected output (ascending dt):

dtamountmoving_avg
2024-01-01100100.0
2024-01-02120110.0
2024-01-03110110.0
2024-01-04130120.0
2024-01-05500246.7
2024-01-06115248.3
2024-01-07125246.7

Because only one or two rows are available for the first two frames, their averages use one and two rows, respectively.

Model Answer
SELECT
  dt,
  amount,
  ROUND(
    AVG(amount) OVER (          -- Window aggregate function
      ORDER BY dt               -- Sort by date to determine the frame
      ROWS BETWEEN 2 PRECEDING  -- From two rows before
           AND CURRENT ROW      -- Through the current row = latest three rows
    ),
    1  -- Round to one decimal place
  ) AS moving_avg
FROM  daily_sales
ORDER BY dt;

/*
  Execution order (logical SQL evaluation order):
  1. FROM daily_sales        → Read the rows
  2. AVG(amount) OVER (...)  → Evaluate the window function (preserve row count)
  3. ROUND(..., 1)           → Format the value
  4. SELECT                  → Evaluate the columns (moving_avg)
  5. ORDER BY dt             → Sort and return the output
*/
Explanation (table transitions & key points)
SELECT dt, amount, ROUND( AVG(amount) OVER ( ORDER BY dt ROWS BETWEEN 2 PRECEDING AND CURRENT ROW ), 1 ) AS moving_avg FROM daily_sales ORDER BY dt;
LEGEND
Rows read / loaded
① FROM daily_sales (7 rows)
FROM daily_salesRead the seven rows from daily_sales. The amount=500 on 2024-01-05 is an obvious spike. The goal is to quantify how far it lies outside the normal pattern with a moving average.
1 / 5
dtamount
01-01100
01-02120
01-03110
01-04130
01-05500
01-06115
01-07125
7 rows read
LEARNING POINTS
ROWS BETWEEN frame variations: ROWS BETWEEN 6 PRECEDING AND CURRENT ROW produces a seven-day moving average, while ROWS BETWEEN 29 PRECEDING AND CURRENT ROW produces a 30-day moving average. Changing N alone adjusts the degree of smoothing.
A moving average provides the anomaly-detection baseline: A moving average represents the normal trend. Defining residual = observed value − moving average removes the trend and quantifies how far today's value lies from the average. The z-score in Q3 extends this idea.
A spike contaminates subsequent moving averages: The spike on 01-05 keeps moving_avg elevated on 01-06 and 01-07. In practice, methods less sensitive to outliers include a moving median based on the median using PERCENTILE_CONT, or a combination with AVG FILTER that excludes outliers.
ANTI-PATTERNS
Using RANGE BETWEEN can behave unexpectedly with duplicate dates: RANGE BETWEEN 2 PRECEDING AND CURRENT ROW means that dt differs from the current row's value by no more than 2. When dates are duplicated, unintended rows may enter the frame. Always use ROWS BETWEEN for a row-count-based frame.
Be careful when mixing GROUP BY and OVER: When GROUP BY dt and OVER (ORDER BY dt) appear in the same query, GROUP BY executes first. For a table with multiple events per day, create a daily aggregate CTE before applying the window function.
IN PRACTICE: Choosing a moving-average window size
A moving average's window size trades off noise reduction against responsiveness. A small window reacts quickly to recent changes but produces more false positives; a large window smooths more strongly but detects genuine anomalies later. A seven-day window is often used for e-commerce sales because it absorbs day-of-week effects. In machine-learning pipelines, passing moving_avg and moving_stddev as features is standard practice.
QUESTION 2
Moving Standard Deviation — Establish a variability baseline with STDDEV() OVER (ROWS BETWEEN)
STDDEV OVERROWS BETWEENMoving Standard DeviationVariability Detection
Background

A moving average alone does not tell you how much variation is normal. Combining it with a moving standard deviation, moving_stddev, creates a dynamic threshold band: normal range = moving average ± k × moving standard deviation.

STDDEV(amount) OVER (
  ORDER BY dt
  ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
)
-- STDDEV = sample standard deviation (denominator N-1)
-- A one-row frame → NULL (standard deviation requires at least two points)
-- STDDEV_POP = population standard deviation (denominator N)
STDDEV is the sample standard deviation (denominator N-1): PostgreSQL's STDDEV() is the unbiased sample standard deviation with Bessel's correction. A one-row frame makes the denominator zero, so it returns NULL. It is important either to replace NULL with 0 using COALESCE(STDDEV(...), 0) or to exclude the first row from the decision logic.
Problem

From the daily_sales table, calculate each date's sales amount, the moving average over the latest three days as moving_avg, and the moving standard deviation as moving_stddev. Return dt, amount, moving_avg, moving_stddev in ascending dt order. Round the results to two decimal places.

Table
► daily_sales (7 rows)
dtamount
2024-01-01100
2024-01-02120
2024-01-03110
2024-01-04130
2024-01-05500
2024-01-06115
2024-01-07125
Expected Output

Expected output (ascending dt):

dtamountmoving_avgmoving_stddev
2024-01-01100100.00NULL
2024-01-02120110.0014.14
2024-01-03110110.0010.00
2024-01-04130120.0010.00
2024-01-05500246.67219.62
2024-01-06115248.33218.08
2024-01-07125246.67219.45

dt=2024-01-01 is NULL because the STDDEV frame contains only one row. Compared with stddev≈10 during the normal period, it expands more than twentyfold to ≈220 during the spike period.

Model Answer
SELECT
  dt,
  amount,
  ROUND(AVG(amount)    OVER w, 2) AS moving_avg,
  ROUND(STDDEV(amount) OVER w, 2) AS moving_stddev  -- Sample standard deviation (N-1)
FROM  daily_sales
WINDOW w AS (                     -- Share a named window definition
  ORDER BY dt
  ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
)
ORDER BY dt;

/*
  Execution order (logical SQL evaluation order):
  1. FROM daily_sales        → Read the rows
  2. WINDOW w AS (...)       → Define the named window
  3. AVG(amount) OVER w      → Evaluate the window function (preserve row count)
  4. STDDEV(amount) OVER w   → Evaluate the window function (preserve row count)
  5. ROUND(..., 2)           → Format the values
  6. SELECT / ORDER BY dt    → Evaluate the columns, sort, and return the output
*/
Explanation (table transitions & key points)
SELECT dt, amount, ROUND(AVG(amount) OVER w, 2) AS moving_avg, ROUND(STDDEV(amount) OVER w, 2) AS moving_stddev FROM daily_sales WINDOW w AS ( ORDER BY dt ROWS BETWEEN 2 PRECEDING AND CURRENT ROW ) ORDER BY dt;
LEGEND
Rows read / loaded
① FROM daily_sales (7 rows)
FROM daily_salesUse the same table as Q1. This time, calculate variability (standard deviation) as well as the moving average to express the lower and upper bounds of the normal range numerically.
1 / 5
dtamount
01-01100
01-02120
01-03110
01-04130
01-05500
01-06115
01-07125
7 rows read
LEARNING POINTS
Keep the code DRY by reusing a WINDOW clause: Repeating the same frame as OVER (ORDER BY dt ROWS BETWEEN ...) is redundant and error-prone. Define it once with WINDOW w AS (...) and reference it with OVER w, so changing the frame requires an edit in only one place.
Choosing between STDDEV and STDDEV_POP: STDDEV, an alias for STDDEV_SAMP, is the unbiased sample standard deviation with denominator N-1. Use it when estimating a population from a sample. STDDEV_POP uses denominator N and applies when the observed values constitute the entire population, such as all transactions from one day. By convention, anomaly detection commonly uses STDDEV.
An expanding stddev signals the loss of a normal range: When stddev surges after a spike enters the frame, the normal band of avg ± 2σ becomes so wide that the next anomaly is easier to miss. To avoid this contamination, production systems often use a two-pass approach that excludes the spike before calculating stddev.
ANTI-PATTERNS
Letting a NULL from STDDEV flow into later calculations: STDDEV returns NULL for a one-row frame. The calculation (amount - avg) / stddev then becomes NULL, preventing the anomaly flag from being set correctly. Guard with NULLIF(STDDEV(...), 0) or COALESCE, or exclude the first row with WHERE.
Beware division by zero when STDDEV=0: STDDEV equals 0 when all values in the frame are identical. A subsequent z-score calculation would evaluate amount / 0 and raise an error. Always use NULLIF(STDDEV(...), 0) so that a zero denominator becomes NULL.
IN PRACTICE: How Bollinger Bands relate to anomaly detection
The upper and lower bands defined by a moving average ± 2 × moving standard deviation are the same concept as financial Bollinger Bands. A value outside the bands is considered statistically unusual. In data analysis, this pattern can be applied to any metric, including sales, error rates, and latency. Machine-learning anomaly detectors such as Isolation Forest also use this residual, or deviation, as an input feature.
QUESTION 3
Anomaly Scoring with z-scores — Standardize deviations with (value - avg) / stddev
z-scoreNULLIFAnomaly ScoringStandardization
Background

A z-score, or standardized score, is a dimensionless measure of how many standard deviations the current value lies from the average. In anomaly detection, |z-score| > 2–3 is commonly used as an anomaly threshold.

-- Definition of a z-score
z = (current value - average) / standard deviation

-- SQL implementation based on a moving window
(amount - AVG(amount) OVER w)
  / NULLIF(STDDEV(amount) OVER w, 0)  -- Guard division by zero with NULL

-- |z| > 2 → outer 4.6% of a normal distribution → investigate
-- |z| > 3 → outer 0.3% of a normal distribution → rule-of-thumb anomaly
Guard division by zero with NULLIF(expr, 0): NULLIF(x, 0) returns NULL when x is 0. Simply wrap the denominator of a potentially zero-dividing expression in NULLIF to prevent a division-by-zero error. Because any value divided by NULL yields NULL, a subsequent CASE WHEN can assign the flag.
Problem

From the daily_sales table, use the moving average and moving standard deviation over the latest five days to calculate each day's z-score, then assign anomaly_flag as 'anomaly' when |z| ≥ 2 and 'normal' otherwise. Return dt, amount, moving_avg, moving_stddev, z_score, anomaly_flag in ascending dt order. Round z_score to two decimal places.

Table
► daily_sales (8 rows)
dtamount
2024-01-01100
2024-01-02105
2024-01-0398
2024-01-04102
2024-01-05110
2024-01-06400
2024-01-07108
2024-01-08103
Expected Output

Expected output (ascending dt):

dtamountmoving_avgmoving_stddevz_scoreanomaly_flag
2024-01-01100100.00NULLNULLnormal
2024-01-02105102.503.540.71normal
2024-01-0398101.003.61-0.83normal
2024-01-04102101.252.990.25normal
2024-01-05110103.004.691.49normal
2024-01-06400163.00132.561.79normal
2024-01-07108163.60132.24-0.42normal
2024-01-08103164.60131.64-0.47normal

While the spike remains in the five-day window, stddev expands sharply, making it difficult for z_score to reach 2. This is the spike-contamination problem. In practice, a LAG-style frame that uses the window before the spike is effective; see the explanation.

Model Answer
WITH stats AS (
  SELECT
    dt, amount,
    ROUND(AVG(amount)    OVER w, 2) AS moving_avg,
    ROUND(STDDEV(amount) OVER w, 2) AS moving_stddev
  FROM  daily_sales
  WINDOW w AS (
    ORDER BY dt
    ROWS BETWEEN 4 PRECEDING AND CURRENT ROW  -- Latest five days
  )
)
SELECT
  dt, amount, moving_avg, moving_stddev,
  ROUND(
    (amount - moving_avg)
      / NULLIF(moving_stddev, 0),  -- Convert division by zero to NULL
    2
  ) AS z_score,
  CASE
    WHEN ABS(
           (amount - moving_avg) / NULLIF(moving_stddev, 0)
         ) >= 2
    THEN 'anomaly'
    ELSE 'normal'
  END AS anomaly_flag
FROM  stats
ORDER BY dt;

/*
  Execution order (logical SQL evaluation order):
  1. Define CTE stats       → Evaluate window functions (preserve row count)
  2. Evaluate outer query   → Evaluate columns (z_score, anomaly_flag)
  3. ORDER BY dt            → Sort and return the output
*/
Explanation (table transitions & key points)
WITH stats AS ( SELECT dt, amount, ROUND(AVG(amount) OVER w, 2) AS moving_avg, ROUND(STDDEV(amount) OVER w, 2) AS moving_stddev FROM daily_sales WINDOW w AS ( ORDER BY dt ROWS BETWEEN 4 PRECEDING AND CURRENT ROW ) ) SELECT dt, amount, moving_avg, moving_stddev, ROUND( (amount - moving_avg) / NULLIF(moving_stddev, 0), 2 ) AS z_score, CASE WHEN ABS(...)>= 2 THEN 'anomaly' ELSE 'normal' END AS anomaly_flag FROM stats ORDER BY dt;
LEGEND
Rows read / loaded
① FROM daily_sales + WINDOW (8 rows)
CTE stats: FROM + AVG/STDDEV OVER wRead the eight rows from daily_sales.
1 / 6
dtamount
01-01100
01-02105
01-0398
01-04102
01-05110
01-06400
01-07108
01-08103
8 rows read
LEARNING POINTS
The fundamental fix for spike contamination — use only a prior window: Including the current day lets the spike itself raise the average and standard deviation, reducing its z-score. In practice, the standard pattern is ROWS BETWEEN 5 PRECEDING AND 1 PRECEDING, the five preceding days excluding the current day, so the current value is evaluated against the prior normal range.
How z-score thresholds map to a normal distribution: Assuming a normal distribution, |z| > 2 occurs for only about 4.6% of values, and |z| > 3 for about 0.3%. Adjust the threshold between 2 and 3 according to the false-positive rate the business can tolerate. A threshold of 2 is common for e-commerce order spikes, while medical sensors often use 3 or higher.
The two cases that require NULLIF: First, for a one-row frame, STDDEV = NULL because denominator N-1=0. Second, when every value in the frame is identical, STDDEV = 0. NULLIF(STDDEV(...), 0) safely guards both cases, producing NULL.
ANTI-PATTERNS
Using fixed AVG/STDDEV values over the entire period: Calculating a z-score from AVG and STDDEV over the whole table includes future observations, creating look-ahead bias, or data leakage. Always use moving statistics from a prior window in a real-time detection pipeline.
Repeating calculations inside SELECT instead of using a CTE: SELECT (amount - AVG(amount) OVER w) / NULLIF(STDDEV(amount) OVER w, 0) produces the same result, but the identical expression must then be repeated in CASE WHEN. Make it a habit to calculate stats in a CTE first and reference them from the outer query.
IN PRACTICE: Real-time z-scores in a data pipeline
To run z-score anomaly detection in a production pipeline, use the prior frame ROWS BETWEEN N PRECEDING AND 1 PRECEDING, evaluating today's value against the normal range through yesterday. A common design runs this query daily in a dbt (data build tool) or Apache Airflow batch and sends rows where anomaly_flag = 'anomaly' to Slack or email. Start with this simple z-score baseline before moving to a more complex model.
QUESTION 4
Outlier Detection with IQR — Calculate the interquartile range with PERCENTILE_CONT
PERCENTILE_CONTWITHIN GROUPIQR Outlier DetectionRobust Statistics
Background

The interquartile range, or IQR, is a robust anomaly-detection method that resists outliers. While a z-score is contaminated by the outlier itself, IQR uses only the middle 50% of the data, enabling it to calculate a stable threshold even for real-world data with many spikes.

-- Calculate the first quartile (Q1) and third quartile (Q3)
PERCENTILE_CONT(0.25) WITHIN GROUP (ORDER BY amount) AS q1
PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY amount) AS q3

-- IQR = Q3 - Q1
-- Outlier bounds (Tukey's fences)
-- Lower bound: Q1 - 1.5 * IQR
-- Upper bound: Q3 + 1.5 * IQR
PERCENTILE_CONT is an ordered-set aggregate: Unlike ordinary aggregate functions such as SUM and AVG, it requires WITHIN GROUP (ORDER BY col). PERCENTILE_CONT(0.5) is equivalent to the median. PERCENTILE_DISC, the discrete form, returns an observed value, while PERCENTILE_CONT, the continuous form, returns a linearly interpolated value.
Problem

From the daily_sales table, calculate Q1, Q3, and IQR over the entire period, then assign outlier_flag as 'outlier' when an amount lies outside [Q1 − 1.5 × IQR, Q3 + 1.5 × IQR] and 'normal' when it lies inside. Return dt, amount, q1, q3, iqr, lower_fence, upper_fence, outlier_flag in ascending dt order.

Table
► daily_sales (8 rows)
dtamount
2024-01-01100
2024-01-02105
2024-01-0398
2024-01-04102
2024-01-05110
2024-01-06400
2024-01-07108
2024-01-08103
Expected Output

Expected output (ascending dt):

dtamountq1q3iqrlower_fenceupper_fenceoutlier_flag
2024-01-01100101.50108.507.0091.00119.00normal
2024-01-02105101.50108.507.0091.00119.00normal
2024-01-0398101.50108.507.0091.00119.00normal
2024-01-04102101.50108.507.0091.00119.00normal
2024-01-05110101.50108.507.0091.00119.00normal
2024-01-06400101.50108.507.0091.00119.00outlier
2024-01-07108101.50108.507.0091.00119.00normal
2024-01-08103101.50108.507.0091.00119.00normal

q1, q3, and iqr have the same value on every row because they are aggregates over the entire period. The value 400 is an outlier because it greatly exceeds upper_fence=119. Although the spike itself affects Q3, its influence is limited.

Model Answer
WITH fences AS (
  SELECT
    PERCENTILE_CONT(0.25) WITHIN GROUP (ORDER BY amount) AS q1,
    PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY amount) AS q3
  FROM  daily_sales
)
SELECT
  s.dt,
  s.amount,
  f.q1,
  f.q3,
  f.q3 - f.q1                     AS iqr,
  f.q1 - 1.5 * (f.q3 - f.q1)     AS lower_fence,  -- Tukey's lower fence
  f.q3 + 1.5 * (f.q3 - f.q1)     AS upper_fence,  -- Tukey's upper fence
  CASE
    WHEN s.amount < f.q1 - 1.5 * (f.q3 - f.q1)
      OR  s.amount > f.q3 + 1.5 * (f.q3 - f.q1)
    THEN 'outlier'
    ELSE 'normal'
  END                              AS outlier_flag
FROM  daily_sales s
CROSS JOIN fences f            -- Join the same q1/q3 to every row (one-row CTE)
ORDER BY s.dt;

/*
  Execution order (logical SQL evaluation order):
  1. Define CTE fences       → Evaluate aggregate functions (Q1, Q3)
  2. FROM daily_sales s      → Read the rows
  3. CROSS JOIN fences f     → Join them (Cartesian product)
  4. SELECT                  → Evaluate columns (lower_fence, upper_fence, flag)
  5. ORDER BY s.dt           → Sort and return the output
*/
Explanation (table transitions & key points)
WITH fences AS ( SELECT PERCENTILE_CONT(0.25) WITHIN GROUP (ORDER BY amount) AS q1, PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY amount) AS q3 FROM daily_sales ) SELECT s.dt, s.amount, f.q1, f.q3, f.q3 - f.q1 AS iqr, f.q1 - 1.5 * (f.q3 - f.q1) AS lower_fence, f.q3 + 1.5 * (f.q3 - f.q1) AS upper_fence, CASE WHEN s.amount < lower OR s.amount > upper THEN 'outlier' ELSE 'normal' END AS outlier_flag FROM daily_sales s CROSS JOIN fences f ORDER BY s.dt;
LEGEND
Rows read / loaded
① CTE fences — Sort ascending and prepare to aggregate
WITHIN GROUP (ORDER BY amount)Sort all eight daily_sales rows in ascending order. The 25th percentile, Q1, lies between the second and third values; the 75th percentile, Q3, lies between the sixth and seventh values.
1 / 5
Rankdtamount (ascending)
101-0398
201-01100
301-04102
401-08103
501-02105
601-07108
701-05110
801-06400
8 rows (sorted ascending)
LEARNING POINTS
Why IQR is more resistant to outliers than a z-score: A z-score's average and standard deviation are pulled by the outlier itself and depend on values across all four quartiles. IQR uses only the first through third quartiles, the middle 50%, so when outliers make up less than 25% of the data, their effect on Q1 and Q3 is minor. This property is called robustness.
PERCENTILE_CONT vs. PERCENTILE_DISC: PERCENTILE_CONT linearly interpolates between two adjacent values and can return a non-integer such as 101.25. PERCENTILE_DISC returns an observed value such as 102. As a rule, use PERCENTILE_CONT for continuous quantities such as sales and prices, and PERCENTILE_DISC for discrete quantities such as ranks and rating scores.
Why the coefficient is 1.5 (Tukey's fences): The statistician John Tukey proposed 1.5 for box plots. For a normal distribution, approximately 99.3% of the data falls within these bounds. Use 3.0 for stricter detection of extreme outliers and fewer false positives.
ANTI-PATTERNS
Confusing the aggregate with a windowed PERCENTILE_CONT using OVER: PERCENTILE_CONT(0.25) WITHIN GROUP (...) is an aggregate that collapses an entire group into one row. A windowed form such as PERCENTILE_CONT(0.25) WITHIN GROUP (...) OVER (PARTITION BY ...) is also possible, but omitting WITHIN GROUP's ORDER BY causes an error.
Reaggregating after removing outliers in the wrong order: After finding IQR outliers, you may calculate an average after filtering them with WHERE outlier_flag = 'normal'. In that case, separate the table used to calculate the fences CTE from the table used to calculate the average, and preserve the order in which the data is reaggregated after outlier removal.
IN PRACTICE: Choosing between z-scores and IQR
In practice, the two methods are often combined. Use a z-score when: the data is approximately normally distributed, such as body temperature or manufacturing dimensional error, and you need prior-window, real-time detection over a continuous time series. Use IQR when: the data is skewed, as with the roughly log-normal distributions of sales and page views; many outliers may be present; or a batch process needs a stable threshold over the full period. An AND combination that alerts only when both methods flag an anomaly can also reduce false positives.
QUESTION 5
Detect Time-Series Spikes and Drops — Calculate day-over-day change with LAG() and create threshold alerts
LAGCASE WHENRate-of-Change DetectionTime-Series Alerts
Background

Production anomaly detection must monitor not only absolute magnitude but also the percentage change from the previous day, or rate of change. A change from 100 → 500 is a +400% spike: the percentage change is large even when the absolute value is small. Conversely, a large absolute value can be normal when it remains stable.

-- Calculate day-over-day percentage change
(amount - LAG(amount) OVER (ORDER BY dt))
  / NULLIF(LAG(amount) OVER (ORDER BY dt), 0)
  * 100.0  -- Convert to a percentage

-- If the previous value is 0, NULLIF converts it to NULL → prevents division by zero
-- * 100.0 (a floating-point value) prevents integer division
Choosing LEAD or LAG and the rate-of-change pattern: Use LAG, a past reference, for day-over-day change and LEAD, a future reference, for change against the following day. The standard formula is rate of change = (current − previous) / |previous| × 100. To detect a sharp drop, either take the absolute value with ABS() or define a separate negative threshold, such as −50% or lower.
Problem

From the daily_sales table, calculate each day's percentage change from the previous day as pct_change, then assign alert_type as 'spike' for a change of +50% or more, 'drop' for −30% or less, and 'normal' otherwise. Return dt, amount, prev_amount, pct_change, alert_type in ascending dt order. Round pct_change to one decimal place.

Table
► daily_sales (8 rows)
dtamount
2024-01-01100
2024-01-02110
2024-01-03105
2024-01-04108
2024-01-05400
2024-01-0660
2024-01-07112
2024-01-08115

01-05: +270% spike. 01-06: −85% drop. 01-07: +87% spike.

Expected Output

Expected output (ascending dt):

dtamountprev_amountpct_changealert_type
2024-01-01100NULLNULLnormal
2024-01-0211010010.0normal
2024-01-03105110-4.5normal
2024-01-041081052.9normal
2024-01-05400108270.4spike
2024-01-0660400-85.0drop
2024-01-071126086.7spike
2024-01-081151122.7normal
Model Answer
WITH with_prev AS (
  SELECT
    dt,
    amount,
    LAG(amount) OVER (ORDER BY dt) AS prev_amount  -- Get the previous day's amount
  FROM  daily_sales
)
SELECT
  dt,
  amount,
  prev_amount,
  ROUND(
    (amount - prev_amount)
      / NULLIF(prev_amount, 0) * 100.0,  -- Percentage change (division-by-zero guard)
    1
  ) AS pct_change,
  CASE
    WHEN (amount - prev_amount) / NULLIF(prev_amount, 0) >= 0.5   -- +50% or more
      THEN 'spike'
    WHEN (amount - prev_amount) / NULLIF(prev_amount, 0) <= -0.3  -- -30% or less
      THEN 'drop'
    ELSE 'normal'
  END AS alert_type
FROM  with_prev
ORDER BY dt;

/*
  Execution order (logical SQL evaluation order):
  1. Define CTE with_prev   → Evaluate the window function (preserve row count)
  2. Evaluate outer query   → Evaluate columns (pct_change, flag)
  3. ORDER BY dt            → Sort and return the output
*/
Explanation (table transitions & key points)
WITH with_prev AS ( SELECT dt, amount, LAG(amount) OVER (ORDER BY dt) AS prev_amount FROM daily_sales ) SELECT dt, amount, prev_amount, ROUND( (amount - prev_amount) / NULLIF(prev_amount, 0) * 100.0, 1 ) AS pct_change, CASE WHEN ... >= 0.5 THEN 'spike' WHEN ... <= -0.3 THEN 'drop' ELSE 'normal' END AS alert_type FROM with_prev ORDER BY dt;
LEGEND
Rows read / loaded
① FROM daily_sales (8 rows)
FROM daily_salesRead the eight rows from daily_sales.
1 / 6
dtamount
01-01100
01-02110
01-03105
01-04108
01-05400
01-0660
01-07112
01-08115
8 rows read
LEARNING POINTS
Combining rate-of-change and absolute-value detection is most effective: A rate alone treats 100 → 150 and 10,000 → 15,000 as the same +50%. In production, combining both conditions—rate of change (day-over-day ≥ 50%) AND absolute deviation (|amount - moving_avg| ≥ threshold)—substantially reduces false positives.
Calculating rates of change across multiple segments such as products or stores: If the table has a segment_id column, use LAG(amount) OVER (PARTITION BY segment_id ORDER BY dt) so the previous-day reference does not cross segment boundaries. PARTITION BY creates a wall between segments and guarantees independent rate-of-change calculations.
Prevent integer division with * 100.0, a floating-point literal: In PostgreSQL, 108 / 108 performs integer division and produces 1. * 100.0 forces a cast to floating point, which is necessary to obtain decimal values such as 0.2963.... Casting with ::float or ::numeric has the same effect.
ANTI-PATTERNS
Processing a table with mixed categories without PARTITION BY: LAG(amount) OVER (ORDER BY dt) orders every row as one sequence even when categories are mixed. The row after category A's final row can be category B's first row and flow into prev_amount, producing a meaningless rate of change. Always add PARTITION BY category when there are multiple categories.
Using the wrong CASE WHEN evaluation order: With CASE WHEN pct >= 0.5 THEN 'spike' WHEN pct <= -0.3 THEN 'drop', +270% is captured by the first WHEN. If the spike and drop definitions overlap—for example, both use |pct| >= 0.5—the earlier WHEN wins. Design the condition order to reflect your intent.
IN PRACTICE: Integrating rate-of-change alerts into a data pipeline
A standard production design defines this query as a dbt model, runs it every morning at 6:00, and updates an alert table. Send rows selected by WHERE alert_type != 'normal' to a Slack webhook, PagerDuty, or a Datadog monitor, and the data team no longer needs to inspect a dashboard manually every morning. A layered anomaly-detection pipeline combining rate of change, z-score, and IQR is a standard architecture for production data platforms.