
Prefill/decode disaggregation has the elegance of a clean architecture diagram. Prompt processing is compute-heavy. Token generation is memory-bandwidth-heavy. Put them on separate GPU pools, scale each phase independently, and stop them…
View original source — Hacker Noon ↗



