Writing
- Design an agent that is allowed to take an irreversible action
Two tool calls can look identical in a trace and differ completely in what happens if the agent was wrong. Reversibility is the property nothing in a tool schema records, approval decays into a rubber stamp under volume, and the ceiling that holds is the one written in code rather than in a prompt.
- Evals as a system, not a notebook
Most eval sets are assembled from examples somebody watched work, which makes them a record of what was already fine. And a release gate cannot catch the most likely cause of a regression, because that change happens on somebody else's deploy schedule.
- A semantic cache is a hash table that skipped the equality check
A dictionary hashes the key and then compares the full key before it returns anything. A semantic cache does the first half and calls it a lookup. The comparison was always missing, and until this month the only way to add it cost more than the cache saved.
- Semantic caching, and the day it serves the wrong answer
A cache that matches on meaning has a failure mode an exact cache cannot have. It can return a confidently wrong answer to a question nobody asked, and the hit rate metric you are optimising counts that as a success.
- Vector search internals, and why recall at k lies to you
Recall is measured against what brute force in the same embedding space would have returned, not against the answer the user wanted. It is a measure of agreement with your own index, and a filter is enough to break the graph it is measured on.
- Design a multi tenant rate limiter that is fair, not just safe
Rate limiters get built as capacity controls and then get used as fairness controls. The noisy neighbour in a real system is almost never an attacker, it is your largest and best paying customer, and a limit that treats those two the same is the design error.
- Design webhook delivery you can be held to
Sending an HTTP request is not the problem. The problem is the consumer that was down for six hours, wants the backlog, wants it in order, and is sharing your worker pool with everyone who stayed up.
- Idempotency, and why exactly once is a story you tell the client
Exactly once delivery does not exist, exactly once effects do, and the gap between those two sentences is the entire engineering problem. The assumption that fails in production is not the mechanism, it is the width of the window.
- Design a job system for work that takes minutes, not milliseconds
Queue defaults were chosen for tasks that finish in under a second. Model calls do not, and when a long job meets a short visibility timeout the redelivery is not a retry, it is a second worker doing the same work at the same time.
- Tool calling, and why the model is the easy part
Most tool call failures are interface design failures wearing a reasoning failure's clothes. The error string you return is a prompt, the schema is a contract, and the retry you allow is a side effect you have already decided to perform twice.
- Design a real time voice agent, and where the latency actually goes
The model call is the part everyone optimises and it is rarely the largest term in the budget. Most of the delay a caller feels is spent deciding that they have finished speaking, and that decision is a product choice wearing a threshold's clothes.
- Guardrails that fired on the wrong people
A user said "you are the bomb" and the safety layer stopped the conversation. The library passed our tests because our tests were written by engineers, and real users do not talk like engineers.
- Bloom filters, and the cost of being wrong cheaply
The interesting tradeoff is not memory against accuracy. It is who downstream pays for a false positive, and whether the rate you designed for survives the set outgrowing the filter.
- The needle in a haystack test, and what long context did not fix
A green needle chart means a model can find a sentence it has already seen the words of. Real corpora contradict themselves, real questions share no vocabulary with the answer, and both of those are missing from the test.
- Consistent hashing, and the three things the ring diagram never tells you
The ring solves rebalancing. It does not solve hot shards, its load balance is weaker than the picture suggests, and the virtual node count nobody revisits is a real operational decision.
- Fanout on write versus fanout on read, and where the boundary actually sits
The interview answer picks one strategy for the whole system. Production picks per account, and the thing that sets the boundary is the follower count distribution, not the request rate.
- RAG, its types, and when retrieval is the wrong tool
Most teams debug a retrieval problem as if it were a model problem. Retrieval grounds an answer, it does not give a model knowledge, and there is a class of question no retrieval strategy will ever answer.
- Design a URL shortener that survives a hot key
The standard system design answer optimises the write path, which is the easy half. What breaks a shortener in production is the distribution of reads, and one link is enough to do it.