Partial derivatives: understand a model's local sensitivity

A partial derivative describes how a function changes locally when one input changes while the others are held fixed. The gradient collects those derivatives into a vector. It can help explain a model's local numerical sensitivity, but it does not automatically describe the causal effect of changing a real-world variable.

Use an explicit function to see both the usefulness and the limitation of the approximation.

Differentiate a two-input function

Let f(x1, x2) = x1² + 3x2. Its partial derivative with respect to x1 is 2x1; with respect to x2 it is three. At (2, 1), the function value is seven and the gradient is (4, 3).

Near that point, a small input change (delta1, delta2) gives a first-order output approximation of 4 × delta1 + 3 × delta2.

If x1 increases by 0.1 while x2 stays fixed, the approximation predicts an increase of 0.4. The actual increase is 0.41 because the squared term contributes an additional 0.1².

Verify the derivative numerically

import numpy as np
from math_core import surface,central_gradient

x = np.array([2.,1.])
analytic = np.array([4.,3.])
numerical = central_gradient(surface,x,h=1e-6)
assert surface(x)==7
assert np.allclose(numerical,analytic,rtol=1e-8,atol=1e-8)
delta = np.array([.1,0.])
approximate_change = analytic@delta
actual_change = surface(x+delta)-surface(x)
assert np.isclose(approximate_change,.4)
assert np.isclose(actual_change,.41)
assert np.isclose(actual_change-approximate_change,.01)
smaller = np.array([.01,0.])
smaller_error = surface(x+smaller)-surface(x)-analytic@smaller
assert np.isclose(smaller_error,.0001)
print({'gradient':numerical.tolist(),'approximate_change':approximate_change,
       'actual_change':actual_change,'smaller_step_error':smaller_error})

The mathematics lab provides the function and central-difference helper. The helper perturbs one coordinate at a time and compares nearby function values. This is an original deterministic example, not a sensitivity result from a fitted business model.

Understand the local approximation

The derivative applies at the specified point. For the x1 direction, sensitivity changes with x1 because the derivative is 2x1. At a different input, the same 0.1 perturbation has a different first-order effect.

The x2 contribution is exactly linear in this function, so its derivative stays three. A finite change in x2 is captured exactly when x1 remains fixed. More complex functions need not behave this way.

Reducing the x1 perturbation from 0.1 to 0.01 reduces this example's approximation error from 0.01 to 0.0001. That illustrates the second-order remainder; it is not a guarantee of a particular error rate for every model.

Keep units in the interpretation

A derivative's units are output units per input unit. Comparing derivative magnitudes across features measured in very different units can be misleading. A one-unit change in age is not directly comparable to a one-unit change in monetary amount without a meaningful scale choice.

If a model uses standardized or transformed features, account for that transformation when interpreting sensitivity in original units. The numerical gradient in transformed coordinates answers a different question from the derivative with respect to the raw input.

Do not confuse sensitivity with causality

Holding one feature fixed while changing another may be mathematically possible but unrealistic for the data-generating process. Correlated features, constraints and selection effects can make a proposed perturbation implausible.

A model may use a feature as a proxy for something else. Its derivative describes the model function, not proof that intervening on the feature will change the real outcome by that amount.

Use causal language only when the required design and assumptions support it. For ordinary model inspection, describe the result as local model sensitivity and state the input context.

Use finite differences carefully

An excessively large step can blur local behavior, while an excessively small step can amplify floating-point cancellation. Check several sensible perturbation sizes and compare with an analytic derivative when available.

At nondifferentiable points, a symmetric finite difference may not represent a unique derivative. Inspect the function and the relevant directional behavior rather than assuming the helper always returns a definitive sensitivity.

Exercise: evaluate the gradient at (-2, 1) and compare it with the gradient at (2, 1). Explain why the function values match while the x1 sensitivities have opposite signs.

NeuraPath's Data Science course connects calculus with optimization and model interpretation. Partial derivatives are useful when their locality, units and limits remain part of the explanation.

Continue learning

This article is part of the Mathematics and statistical foundations sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Gradient descent with a visible loss calculation.
• Continue with Probability distributions: choose a model for the data-generating process.