01 about
I'm an engineer working on inference infrastructure at Reflection in New York.
That's the part of the stack that takes a trained model and serves it: batching, KV caches, GPU utilization, latency. More here eventually.
new york
I'm an engineer working on inference infrastructure at Reflection in New York.
That's the part of the stack that takes a trained model and serves it: batching, KV caches, GPU utilization, latency. More here eventually.