Optimize recall subject to a minimum precision constraint

“Maximize recall” is incomplete. Predicting every row positive gives recall one and usually creates unacceptable false-positive workload. A more useful rule can maximize recall among validation thresholds whose observed precision meets a minimum.

Write the constraint before searching scores

Our synthetic policy requires validation precision of at least 40%. We enumerate the distinct validation probabilities, compute selected count, true positives, false positives and recall at each threshold, and retain feasible candidates. Among them, the rule chooses maximum recall; ties prefer fewer selected rows and then a higher threshold.

The selected threshold is 0.25148. On 1,500 validation rows it selects 62 cases, including 25 of 113 positives. Precision is 40.32% and recall is 22.12%.

The threshold is then frozen. On the 1,500-row test period it selects 78 cases, with 23 true positives and 55 false positives. Precision falls to 29.49%, below the validation constraint, while recall is 24.21%.

That is not a software failure. The 40% rule was enforced on a finite validation sample. It was not a guarantee for future populations.

The scikit-learn precision-recall example defines the operating-point trade-off. A threshold chosen from that curve is a fitted policy and must be assessed out of sample.

from imbalance_cases import precision_constraint_case

r = precision_constraint_case()
assert r['minimum_validation_precision'] == .4
assert r['chosen_validation']['precision'] >= .4
assert r['chosen_validation']['recall'] > .22
assert r['test_at_fixed_threshold']['precision'] < .4
print(r)

Run the search in the imbalanced-model lab. Keeping the adverse test result is central to the lesson.

Decide whether the constraint is empirical or probabilistic

An observed precision floor accepts sampling uncertainty. If falling below the target is costly, require a lower confidence bound above 40%, use a larger validation set, or add a conservative buffer. The interval method must handle the selected threshold; naïve post-selection intervals can be optimistic.

Precision also depends on prevalence. A threshold validated at 7.53% positives may miss its target when prevalence falls, even if class-conditional score distributions remain stable. Report the validation prevalence and monitor it after deployment.

Capacity can conflict with the precision floor. Define which is hard, whether abstention is allowed and what happens when no threshold is feasible. Avoid selecting on test data after a failed result; revise the protocol, acquire new evaluation data and label the original test as development evidence.

Exercise: require a Wilson or bootstrap lower confidence bound for precision above 40% rather than the point estimate. Compare the selected workload and recall, then explain how threshold search affects the interpretation of that bound.

NeuraPath's Data Science course links operating-point optimization to uncertainty and drift. A validation constraint is evidence for a candidate policy, not a permanent promise.

Continue learning

This article is part of the Imbalance, calibration and decision thresholds sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Choose a fraud-review threshold from analyst capacity.
• Continue with Evaluate rare-event models with confidence intervals.