Probability distributions: choose a model for the data-generating process

Choose a probability distribution by asking what outcome is measured, how observations arise and which assumptions are plausible. A histogram's shape is useful evidence, but it does not by itself identify the process that generated the data.

Distinguish a model chosen for explanation from a model fitted and evaluated on real observations. This tutorial uses original synthetic examples to make the assumptions and calculations visible.

Start with the outcome and exposure

For a binary outcome on one trial, a Bernoulli model describes success or failure. For the number of successes in a fixed number of independent trials with a common success probability, a binomial model is a natural candidate.

A Poisson model describes a nonnegative count with a specified mean; a homogeneous Poisson-process interpretation additionally relies on assumptions about event occurrence over exposure. Unequal observation windows or changing rates need explicit treatment.

A normal model is continuous and symmetric with support over the real line. It can be useful for some measurements or approximations, but it can assign impossible negative values when used carelessly for a strictly positive quantity.

Compare two count models

Ten independent trials with success probability 0.2 have expected count two. A Poisson model with mean two also has expected count two. Their probabilities of exactly two outcomes differ.

import numpy as np
from probability_core import distribution_examples

r = distribution_examples()
assert np.isclose(r['binomial_n10_p02_exactly2'],.301989888)
assert np.isclose(r['poisson_mean2_exactly2'],.2706705664732254)
assert r['binomial_n10_p02_exactly2']!=r['poisson_mean2_exactly2']
assert r['normal_sd02_density_at0']>1
assert 0<r['normal_sd02_probability_minus01_to01']<1
assert r['authored_counts_mean']==3
assert np.isclose(r['authored_counts_variance_ddof0'],19.2)
print(r)

The lab executes these calculations with SciPy 1.18.0. The primary binomial and Poisson references describe the corresponding probability functions.

Matching the mean does not make two distributions interchangeable. The binomial count cannot exceed ten in this example, while the Poisson distribution has no such upper bound.

Distinguish density from probability

For a normal distribution with mean zero and standard deviation 0.2, the density at zero is about 1.995. That does not violate probability rules: density is not the probability of an exact point.

The probability of falling between -0.1 and 0.1 is approximately 0.3829, calculated from the difference in cumulative probabilities. For a continuous distribution, an exact single point has probability zero under the model.

SciPy's normal-distribution reference provides separate density and cumulative functions. Keep those operations distinct when interpreting a model output.

Inspect assumptions with diagnostics

The authored count vector [0,0,0,0,1,1,2,4,8,14] has mean three and descriptive variance 19.2 using denominator n. A Poisson distribution has equal theoretical mean and variance, so this discrepancy motivates investigating heterogeneity, dependence, exposure or another count model.

Ten synthetic observations do not establish a definitive distribution or a formal overdispersion test. The diagnostic is a question to pursue, not proof that one named alternative must be correct.

Check whether zeros, bounds, censoring or mixtures have substantive meaning. A model that fits the center but badly misses a decision-relevant tail may be unsuitable for the intended use.

Evaluate the model where it will be used

State the quantity the distribution supports: an interval probability, a simulation, a likelihood or an uncertainty calculation. Compare predictions with suitable held-out observations or diagnostic expectations when real data is available.

Avoid choosing a distribution solely because its name appears in a syllabus or its curve looks smooth. Document the assumptions and the evidence that would make you reconsider it.

Exercise: describe the difference between counting successes among twenty eligible users and counting arrivals during one hour. Identify the exposure, support and assumptions you would inspect before choosing a model for each.

NeuraPath's Data Science course connects probability models with statistical reasoning and evaluation. The useful skill is selecting and checking assumptions that fit the process, not merely recognizing a distribution's shape.

Continue learning

This article is part of the Mathematics and statistical foundations sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Partial derivatives: understand a model's local sensitivity.
• Continue with Conditional probability with an explicit contingency table.