Occasional writing on things I find worth documenting.
The naive approach injects all tool definitions into context on every request. That's why it's expensive. Here's what we did instead.
Read ↗Three years, four full rebuilds, one very chaotic hackathon deployment, and what I actually learned from building something that a lot of people used and I still chose to walk away from.
Read →9.5x higher throughput. 54x lower P99 latency. Zero runtime allocations. Here's exactly how.
Read ↗j/k to navigate · enter to open · esc to deselect