Bayes' theorem: update a probability without ignoring the base rate

Bayes' theorem combines prior prevalence with the probability of observing evidence under different states. For a record flagger, the chance that a flagged record is erroneous depends on both its detection behavior and how common errors are in the population.

Detecting 90% of erroneous records does not mean that 90% of flagged records are erroneous. Those are different conditional probabilities.

Build the example from counts

The original synthetic population contains 10,000 records, of which 100 are erroneous and 9,900 are valid. A fictional flagger marks ninety erroneous records and 495 valid records.

Its sensitivity is 90 / 100 = 90%. Its false-positive rate is 495 / 9,900 = 5%. Among the 585 flags, only ninety are errors: 90 / 585 = 2/13, approximately 15.38%.

These are authored counts for a mathematical demonstration, not measured performance of a deployed detector or a medical diagnostic example.

Apply Bayes' formula

Let E mean error and F mean flag. The required probability is P(E | F). Its numerator is P(F | E) × P(E). The denominator adds flags from both erroneous and valid records: P(F | E) × P(E) + P(F | not E) × P(not E).

The Bayes theorem proof derives the conditional relationship. The following calculation uses the original record example and exact fractions.

from fractions import Fraction
from probability_core import flagger_probabilities,bayes_flag

r = flagger_probabilities()
assert r['prevalence']==Fraction(1,100)
assert r['sensitivity']==Fraction(9,10)
assert r['false_positive_rate']==Fraction(1,20)
posterior = bayes_flag(r['prevalence'],r['sensitivity'],r['false_positive_rate'])
assert posterior==r['error_given_flag']==Fraction(2,13)
assert r['flagged']==585 and r['errors']==100
higher_prevalence = bayes_flag(Fraction(1,10),Fraction(9,10),Fraction(1,20))
assert higher_prevalence==Fraction(2,3)
print({'error_given_flag_percent':float(posterior)*100,
       'same_assumed_rates_at_10_percent_prevalence':float(higher_prevalence)*100})

The lab fixture and verification preserve the full table, including the valid records that are easy to overlook when focusing on detected errors.

Explain the base-rate effect

Errors are rare in the first population. Even a 5% false-positive rate produces 495 flags from the much larger valid group, compared with ninety true error flags.

If prevalence rises to 10% while sensitivity and false-positive rate remain fixed, the posterior becomes two-thirds, or 66.67%. That calculation is conditional on the rates remaining stable. A real population change may alter the flagger's behavior too, so verify that assumption before transferring performance estimates.

The result is not a paradox. It follows from counting how many opportunities each type of flag has to occur.

Connect the probability to a decision carefully

A 15.38% error share among flags may or may not justify manual review, depending on review capacity, consequences and available alternatives. The table alone does not determine the action threshold.

Do not replace a cost-sensitive decision with a single accuracy headline. Also inspect missed errors, the volume of false alerts and whether performance differs across relevant slices or time periods.

If no flags are possible under a specified model, conditioning on a flag has a zero denominator. The lab returns an explicit undefined result rather than inventing a posterior.

Keep the condition visible in communication

Say “15.38% of flags are errors under this synthetic table,” and separately state “90% of errors are flagged.” Keeping both populations visible prevents a sensitivity figure from being misread as the reliability of an individual alert.

Exercise: hold prevalence at 1% and reduce the false-positive rate from 5% to 1%. Recalculate the posterior and the expected flag counts, then explain why the valid-record population still matters.

NeuraPath's Data Science course connects probability with classification and decision evaluation. Bayes' theorem is useful when the base rate and both sources of evidence remain part of the calculation.

Continue learning

This article is part of the Mathematics and statistical foundations sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Conditional probability with an explicit contingency table.
• Continue with Central limit theorem: simulate what actually converges.