Case Study: Memory Expansion for Text Summarization
At Flash Memory Summit 2023 in Santa Clara, California, Lightelligence demonstrated the benefits of memory expansion using CXL to improve workload efficiency for Large Language Model OPT-66B, an AI model used for text summarization. Using two Photowave™ PCIe cards to connect an AMD Genoa server and Samsung Memory Expansion Module, CXL memory expansion over optics demonstrated a 2.4x throughput advantage compared to SSD/NVMe disk offloading with performance remaining constant.