High-cardinality categories: compare hashing and encoding

One-hot encoding keeps a distinct training column per category. Feature hashing bounds dimensionality and handles unseen strings without a stored vocabulary, at the cost of collisions and reduced traceability.

Count collisions rather than hiding them

Our fixture has 200 unique merchant strings. One-hot encoding creates 200 dimensions. An unseen merchant_new maps to an all-zero block under ignore mode.

Hashing into 32 non-negative dimensions occupies all 32 buckets. Relative to one distinct bucket per category, 168 categories collide, and the largest bucket contains 12 categories. The hash is deterministic, but a bucket no longer identifies one merchant.

The FeatureHasher documentation describes its stateless signed or unsigned hashing, while OneHotEncoder stores learned categories.

from feature_cases import hashing_case

r = hashing_case()
assert r['categories'] == 200
assert r['hash_dimensions'] == r['occupied_hash_buckets'] == 32
assert r['colliding_categories'] == 168
assert r['largest_bucket'] == 12
assert r['onehot_unknown_sum'] == 0
print(r)

Run the comparison in the feature-engineering lab. Collision count is structural; predictive impact requires a held-out task.

Choose from constraints and failure modes

One-hot supports transparent category coefficients but grows with vocabulary and needs an unknown policy. Rare grouping can control size. Target encoding is compact but supervised and leakage-prone. Learned embeddings require more data and introduce model complexity.

Hashing is useful for streaming or very large vocabularies. Increase dimensions and measure collision sensitivity. With signed hashing, collisions can cancel; with non-negative hashing, contributions accumulate. Preserve the hash function and dimension with the model artifact.

Identifiers such as merchant or device can memorize entities and fail on new ones. Split by entity or time where deployment requires it. Report warm and cold-category performance.

Exercise: repeat hashing at 32, 128 and 512 dimensions, train the same model and compare memory, collisions and held-out metrics. Add a new-merchant test slice and an explanation requirement that may favour a different encoder.

NeuraPath's Data Science course connects categorical encoding to serving, leakage and interpretability. Bounded memory is a trade-off, not a free replacement for vocabulary design.

Continue learning

This article is part of the Feature engineering and data quality sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Handle outliers before deciding to cap or remove them.
• Continue with Text features: control vocabulary and fit boundaries.