Analyze search behaviour using no-result queries

A no-result search is an observed request that returns no matching items under the recorded search configuration. It can reveal missing content, vocabulary mismatch, indexing problems or restrictive filters. It does not prove that the user had no useful answer available elsewhere.

Measure both unsuccessful requests and affected users. Repeated attempts by one frustrated user can dominate request counts without representing the same number of affected people.

Inspect the search population

The synthetic search fixture contains seven requests. One belongs to an internal account and one has an empty query. After excluding those, five valid external requests remain:

The two U2 rows have different search IDs and represent two intentional requests in this fixture. They are not replayed copies of the same event. Deduplicating by query text would remove real attempts and change the question.

Calculate request and user rates

SELECT COUNT(*) AS valid_searches,
 SUM(result_count=0) AS no_result_searches,
 COUNT(DISTINCT s.user_id) AS searching_users,
 COUNT(DISTINCT CASE WHEN result_count=0 THEN s.user_id END) AS affected_users
FROM searches s JOIN users u USING(user_id)
WHERE u.user_kind='external' AND u.is_bot=0
 AND trim(s.query_text)<>'';

Three of five requests return zero results: 60%. Two of four searching users experience at least one no-result request: 50%. Those percentages differ because U2 searches unsuccessfully twice.

from build_and_verify import database

db = database()
rows = db.execute('''SELECT s.search_id,s.user_id,s.query_text,s.result_count
 FROM searches s JOIN users u USING(user_id)
 WHERE u.user_kind='external' AND u.is_bot=0 AND trim(s.query_text)<>'' ''').fetchall()
db.close()
failed = [r for r in rows if r[3] == 0]
searchers = {r[1] for r in rows}
affected = {r[1] for r in failed}
assert len(rows) == 5 and len(failed) == 3
assert len(searchers) == 4 and len(affected) == 2
assert len({r[2] for r in rows}) == 4
assert len({r[2] for r in failed}) == 2
print({'request_no_result_rate':len(failed)/len(rows),
       'user_no_result_incidence':len(affected)/len(searchers),
       'distinct_failed_query_texts':len({r[2] for r in failed})})

This fixture is a fixed extract without search timestamps, so it demonstrates population arithmetic rather than a daily trend. A production report needs request time, extract coverage and a defined observation period.

Diagnose the failed query before adding content

For “python env,” inspect whether useful environment documentation already exists but uses the phrase “virtual environment.” A synonym or ranking improvement may help more than another near-duplicate article.

For “pandas copy on write,” check the index, content availability, spelling handling and active filters. A result count of zero under a restrictive category filter means something different from zero matches across the entire corpus.

Preserve the original query for diagnosis while using a documented normalization for aggregation. Lowercasing and whitespace normalization may be reasonable for grouping, but aggressive stemming or phrase merging can combine distinct intents. Search queries may contain personal information, so avoid unnecessary raw-query exposure in broad dashboards.

Measure whether the change helps

A lower no-result rate is not sufficient if the search engine now returns irrelevant items for every query. Add an appropriate relevance check and task outcome, such as successful access to a useful document, when those can be measured honestly.

Clicks alone can reward misleading titles. Repeated reformulation, rapid return to search or an evaluated relevance set can provide additional evidence. Define the success contract before tuning the system against the metric.

When comparing versions, keep the query population and filters visible. A change in searcher mix can alter the rate even if retrieval behavior is unchanged.

Exercise: add a result for “python env” that is unrelated to environment setup. Explain why the no-result rate improves while search quality does not, and design an evaluation that catches the failure.

NeuraPath's Data Analytics with Generative AI course connects event analysis with practical product diagnosis. A useful search report identifies the unmet intent and proposes a measurable improvement.

Continue learning

This article is part of the Customer and product analytics sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Attribution windows: why marketing reports disagree.
• Continue with Design a product analytics event naming standard.