What is an Outlier?
An outlier is a data point that sits far away from the rest. But "far" is subjective — there is no universal threshold that separates normal values from outliers. A delivery time of 90 minutes is an extreme outlier in a fast-food context but completely normal for a sit-down restaurant order placed online.
This is the first lesson of outlier analysis: context matters more than the number itself.
Data entry mistakes, sensor failures, system bugs. A delivery time of −5 minutes or 9,999 minutes is almost certainly wrong. These should be investigated and corrected or removed.
Real but rare events. A single customer who ordered 13 times is genuinely unusual — but real. An order taking 90 minutes happened. These are often the most interesting data points.
Why Outliers Matter
Outliers can quietly break your analysis in several ways:
- They distort the mean. One order costing $500 (a data error) would drag the average cost far above what a typical customer pays. The median is immune to this — outliers don't affect it.
- They inflate variance and standard deviation. A single extreme value makes the spread look enormous, hiding the true consistency of normal values.
- They can break model assumptions. Many statistical models assume normally distributed data. A few extreme outliers can invalidate that assumption.
- They can also BE the insight. In fraud detection, the outlier IS the fraud. In medical research, the unusual patient response might be the breakthrough.
The IQR Method — Step by Step
The IQR (Interquartile Range) method was developed by the statistician John Tukey. It defines "normal" as the middle 50% of your data and flags anything dramatically outside that range.
- 1Sort all values and find Q1 and Q3.
Q1 (the 25th percentile) is the value below which 25% of data falls. Q3 (the 75th percentile) is the value below which 75% falls. The middle 50% of your data sits between Q1 and Q3. - 2Calculate the IQR.
IQR = Q3 − Q1. This is simply the width of the middle 50% range. A small IQR means data is tightly clustered. A large IQR means it's spread out. - 3Calculate the fences.
Lower fence = Q1 − 1.5 × IQR
Upper fence = Q3 + 1.5 × IQR
These are the boundaries of "normal." - 4Flag everything outside the fences.
Any value below the lower fence or above the upper fence is a potential outlier. It doesn't mean you delete it — it means you look at it.
Why 1.5 × IQR?
Tukey chose 1.5 as a rule of thumb. For data that follows a normal (bell curve) distribution, the fences at 1.5 × IQR will flag approximately 0.7% of values as outliers — a reasonable sensitivity for most datasets.
If you want to catch only the most extreme outliers, use 3 × IQR instead (this flags roughly 0.01% of normally distributed data). This is sometimes called the "far outlier" threshold.
What To Do With Outliers
Finding an outlier is step one. Deciding what to do with it requires judgment:
Only if you can confirm it's a data error — wrong units, impossible value, clear entry mistake. Deleting real data because it's inconvenient is a form of dishonesty.
Replace values beyond the fence with the fence value itself. Preserves the row but limits the distortion. Useful when you need all data but want to limit extreme influence.
If the outlier is real, keep it and note it. Report both mean and median so readers can see its influence. This is often the most honest approach.
Build one analysis for typical data and another specifically about the outliers. Sometimes the outlier segment IS the most interesting finding.
Alternatives to the IQR Method
Z-Score Method
Flag values more than 3 standard deviations from the mean (z > 3 or z < −3). Problem: the mean and standard deviation are themselves sensitive to outliers — a method that uses them to detect outliers is somewhat circular.
Isolation Forest
A machine learning approach that randomly partitions data and measures how few splits it takes to isolate a point. Outliers are isolated quickly because they're "different." Ideal for high-dimensional data where IQR per-column misses multivariate outliers.
The IQR method was applied to delivery_time, food_preparation_time, and cost_of_the_order. Outliers were found in delivery time (some orders taking well above the upper fence). The decision was to keep them — a delivery taking 90 minutes is a real operational event, not a data entry error. It's exactly the kind of problem FoodHub needs to identify and fix. Removing it would have hidden the business problem from view. See the Data Cleaning section →