Anomaly detection: validate alerts before calling them fraud

Anomaly detection ranks cases that look unusual under selected features and a reference population. Fraud is a labelled behavioural and legal concept. The two can overlap, but an unsupervised score cannot establish fraud by itself.

Separate fitting from evaluation evidence

Our synthetic fixture has 1,000 standard-normal points and 20 shifted points across three features. Isolation Forest fits all 1,020 rows without reading generator labels. We negate its sample score so larger values rank as more unusual.

Only after fitting do we use generator labels to assess review budgets. In the top ten, all ten rows are authored anomalies. In the top 20, 19 are anomalies. In the top 40, all 20 anomalies appear alongside 20 normal rows.

These clean results come from a deliberately separated construction. They do not imply that real unusual behaviour is wrongdoing.

The IsolationForest documentation describes isolation through random feature and split selection. The score reflects the fitted feature distribution.

from unsupervised_cases import anomaly_case

r = anomaly_case()
assert r['rows'] == 1020 and r['authored_anomalies'] == 20
assert r['budgets'][0]['reviewed_true_anomalies'] == 10
assert r['budgets'][1]['review_precision'] == .95
assert r['budgets'][2]['anomaly_recall'] == 1
print(r)

Run it in the unsupervised lab. Generator labels evaluate code mechanics and are unavailable in a real unsupervised deployment.

Define what review can confirm

Before modelling, define the alert unit, reference period, available features and review outcome taxonomy. Reviewers may classify data errors, benign rare events, policy exceptions and confirmed harmful events separately. Preserve uncertainty instead of forcing every case into fraud/not-fraud.

Feature scaling and encoding shape anomaly scores. Seasonal peaks, new products and regional differences can appear unusual without being harmful. Fit or segment the reference population accordingly and test known benign extremes.

Sample alerts across score ranges, not only the top tail, to estimate how precision changes. Include randomly selected non-alerts when estimating recall or missed-event rates. Use mature independent outcomes where possible.

Do not retrain immediately on reviewer decisions without checking selection bias: reviewers see model-selected cases, so their labels do not represent the full population. Maintain audit samples and log model version, score, rank, reason and disposition.

Exercise: relabel half the shifted points as benign rare cases and add subtle labelled fraud within the normal cloud. Compare anomaly recall and review precision. Explain why supervised learning or rules may be better once reliable labels exist.

NeuraPath's Data Science course treats anomaly scores as queue signals. Independent review evidence determines what an unusual case actually means.

Continue learning

This article is part of the Clustering, reduction and recommendations sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Build a nearest-neighbour recommendation explanation.
• Continue with Isolation forests with an analyst-review budget.