Abstract: In this paper, we propose scalable on-package memory expansion architectures to address the growing memory demands of large-scale AI inference workloads. To achieve high bandwidth and low ...