mikegzhou.com
q = (0, 0)

new york

Mike Zhou

 

move to query · click to prefill
↓

01 about

I'm an engineer working on inference infrastructure at Reflection in New York.

That's the part of the stack that takes a trained model and serves it: batching, KV caches, GPU utilization, latency. More here eventually.

02 roughly what that looks like

continuous batching. each color is a request; bright cell = prefill. util —

03 elsewhere