jemalloc: The Infrastructure You Forgot Existed Until Meta Broke It
Meta just publicly admitted they buried jemalloc under technical debt and are trying to fix it. Here's why this actually matters.
#systems
2 transmissions tagged #systems
Your 4090 Is Probably Memory-Bound
On a 24 GB card, single-GPU LLM inference is usually constrained by memory traffic and KV cache growth long before raw math throughput becomes the limit.