SambaNova Systems

SambaNova Systems is an American AI hardware and software company that designs the Reconfigurable Dataflow Unit (RDU), a family of domain-specific accelerators built around a dataflow architecture rather than the GPU-style SIMT design used by NVIDIA. Its fourth-generation chip, the SN40L, is aimed at training and serving large language models at trillion-parameter scale with high memory bandwidth and low inference latency ^[1][2][3].

History

SambaNova Systems was founded in 2017 in Palo Alto, California, by Stanford professor Kunle Olukotun, along with Rodrigo Liang and Christopher Ré, to commercialize reconfigurable dataflow computing for AI workloads. The company raised over a billion dollars in venture funding, making it one of the most heavily funded AI-chip startups, and developed a full stack spanning silicon, software, and systems ^[1][2].

The RDU line evolved through several generations — the SN10, SN30, and finally the SN40L, introduced in September 2023 as a fourth-generation RDU built for both training and inference of large language models ^[1][2][4]. In 2025, reports emerged that Intel was exploring an acquisition of the startup, reflecting the broader industry scramble for AI accelerator alternatives to NVIDIA ^[4].

Key Concepts

The SN40L is fabricated on TSMC's 5-nanometer process as a 2.5D Chip-on-Wafer-on-Substrate (CoWoS) package containing two accelerator dies and High Bandwidth Memory (HBM) ^[2][3]. Each socket delivers 638 BF16 TFLOPS of peak compute using 1,040 Pattern Compute Units (PCUs) paired with 1,040 Pattern Memory Units (PMUs), and a three-tier memory system — on-chip SRAM, on-package HBM, and off-package DDR — that collectively provides over 1.5 TB of high-capacity memory ^[1][2][3].

Instead of executing a fixed instruction stream, the RDU compiles the neural network into a dataflow graph that is mapped onto reconfigurable compute and memory tiles at compile time, allowing the hardware to be reconfigured for each model. This design keeps data moving through on-chip memory and reduces the energy and latency of moving weights from off-chip DRAM, which is the dominant bottleneck ("memory wall") for inference ^[1][2]. SambaNova claims the SN40L can hold multiple large models in memory simultaneously and switch between them in microseconds, which it positions as an advantage for agentic AI workloads that orchestrate many models ^[3].

Applications

SambaNova sells SambaRack, a datacenter system of 16 SN40L RDUs, together with the SambaNova Suite software stack; the chips are also offered as a cloud service (SambaCloud) hosting open models such as Llama, DeepSeek, and Qwen ^[1][4]. Because RDU systems can run models entirely on-premises, the company markets them to enterprises and governments that require data privacy, and its SambaManaged offering is positioned as a turnkey sovereign-AI platform that can stand up a national inference cloud in about 90 days with air-cooled, 10 kW racks ^[1][6]. The SN40L is designed to serve models up to 5 trillion parameters on a single system node with 256K+ token sequence lengths, and its memory-centric design is highlighted for multimodal and agentic workloads where GPU systems lose efficiency ^[2][3][5].

>See Also

• NVIDIA
• Groq
• Cerebras Systems
• Tensor Processing Unit
• AI Data Centres in Malaysia
• Sovereign AI