Quick summary
Summarize this blog with AI
A machine learning system design interview asks you to make explicit choices under constraints. Use a concrete product and a simple baseline before proposing a model. This worked design is a fictional nearby-restaurant recommendation service; its volumes and latency numbers are assumptions for discussion, not measured performance claims.
Clarify the product and constraints
The service recommends currently available restaurants for discovery. Assume one million daily active users and ten recommendation requests per active user per day. The goal is a useful restaurant choice, with latency, availability, inappropriate results, and stale opening information as guardrails. A click is an observable intermediate event, not proof of a successful visit.
Clarify geography, freshness needs, inventory, personalization, permitted location use, and what outcome labels can actually be collected. Use consented, minimized location information appropriate to the feature. Decide how users without personalization or location permission receive useful results.
Estimate capacity and label assumptions
Ten million requests per day is approximately 115.7 requests per second on average. An assumed tenfold peak gives approximately 1,157 requests per second. This does not include retries, background work, replication, or capacity headroom.
daily_users = 1_000_000
requests_per_user = 10
requests_per_day = daily_users * requests_per_user
average_rps = requests_per_day / (24 * 60 * 60)
assumed_peak_rps = average_rps * 10
print(f"daily requests: {requests_per_day:,}")
print(f"average: {average_rps:.1f} requests/second")
print(f"assumed peak: {assumed_peak_rps:.1f} requests/second")
Suppose the target end-to-end p95 is 150 milliseconds. A provisional engineering budget might allocate 10 milliseconds to request handling, 40 to retrieval, 60 to ranking, and 40 to other work and headroom. These allocations are design targets. Adding measured component p95 values does not generally give the end-to-end p95; measure the complete request path under realistic load.
Start with a usable baseline
Retrieve restaurants within an allowed area, filter known unavailable results, and rank by a transparent combination of distance and a suitable quality or popularity signal. Define fallback behavior when freshness or feature services fail. A baseline lets you validate serving, logging, and evaluation before introducing model complexity. Google’s Rules of Machine Learning similarly emphasizes simple initial models and reliable infrastructure.
Describe the request and data paths
| Component | Responsibility | Failure behavior |
|---|---|---|
| Catalog ingestion | Validate IDs, location, availability and update times | Quarantine invalid records and expose freshness |
| Candidate retrieval | Return a bounded geographically relevant set | Use a safe cached candidate set or show an explicit empty result |
| Feature lookup | Provide permitted, time-consistent features | Use documented defaults with missingness indicators |
| Ranker | Score candidates within a time budget | Fall back to the baseline on timeout or invalid output |
| Policy and presentation | Apply availability, diversity and product constraints | Do not display known-invalid candidates merely to fill the list |
| Event collection | Record exposure, version, outcome and permitted context | Track loss and delay; do not treat missing events as negative outcomes |
Version model artifacts, feature definitions, and relevant catalog snapshots. Keep training and serving transformations consistent. Bound candidate count and request work so a pathological query cannot trigger unbounded scoring.
Build labels and evaluation without future information
Define a label that matches an observable outcome and its window. Impression data is needed to interpret clicks; unshown restaurants do not supply ordinary negative click labels. Account for the baseline policy’s exposure bias when interpreting offline results.
Split by time to emulate deployment and prevent future outcomes or future restaurant attributes from entering earlier examples. Evaluate retrieval coverage as well as ranking, because a ranker cannot recover a good restaurant that retrieval omitted. Examine new users, new restaurants, low-activity areas, and permission-limited users separately.
Choose a model only after identifying what it must improve. A compact supervised ranker may be sufficient; a large neural model needs evidence that its incremental value justifies training, serving, and operational cost. Offline ranking improvement is a screening result, not proof of online product benefit.
Plan rollout and operations
Validate the artifact and feature contract, compare a new model in shadow traffic where appropriate, and use a bounded rollout with a baseline fallback. Define rollback criteria before launch. Monitor request failures, latency distribution, feature freshness, missingness, candidate coverage, and the chosen product outcomes.
Investigate distribution shift and delayed labels without automatically retraining on corrupted data. Record ownership and the process for reviewing an incident. A cache can reduce latency but serve stale availability; a more complex ranker can improve offline relevance while exceeding the latency budget. Explain these tradeoffs with measurements you would collect.
Conclude the interview with the unresolved assumptions most likely to change the design. A complete answer connects product value, data quality, serving behavior, evaluation, and recovery rather than listing technologies.
