NVIDIA Nemotron

NVIDIA Nemotron is a family of open-weight large language models developed by NVIDIA Corporation, designed to deliver leading efficiency and accuracy for building specialised AI agents. Unlike NVIDIA's primary business of designing the GPUs that power AI training and inference, the Nemotron family represents NVIDIA's direct entry into model development — offering open weights, training data, and post-training recipes under permissive licensing that permits unrestricted commercial use.^[1][2]

History and Evolution

NVIDIA began releasing Nemotron models as part of its broader strategy to demonstrate and accelerate AI adoption on its hardware platforms. Early Nemotron models were based on standard Transformer architectures and served primarily as reference implementations and research contributions.

The Nemotron 3 generation marked a significant architectural shift, adopting a hybrid Mamba-2 and Mixture-of-Experts (MoE) architecture with selective Attention layers. This design combined the efficiency of state-space models (Mamba) with the capacity and routing efficiency of MoE, while retaining attention mechanisms where they provide the most value.^[3]

On 11 August 2026, NVIDIA released Nemotron 3.5 Lightning 30B, the first model in the Nemotron 3.5 series and currently the smallest member of the Nemotron 3 family. The release coincided with NVIDIA's first open-source AI model launch under CEO Jensen Huang's expanded model strategy.^[1][4]

Nemotron 3.5 Lightning

Nemotron 3.5 Lightning is a 30-billion-parameter Mixture-of-Experts model with only 3 billion active parameters, meaning that for any given token, only a fraction of the model's total capacity is activated. This design provides the capacity of a larger dense model at the compute cost of a much smaller one.^[1][3]

Key Specifications

| Attribute | Value | |---|---| | Total parameters | 30 billion (31.6B) | | Active parameters | 3 billion (3.6B) | | Context length | Up to 1 million tokens | | Architecture | Mamba-2 + MoE + Attention hybrid | | Quantisation | NVFP4 and BF16 checkpoints | | License | OpenMDW 1.1 |

The model supports English and coding languages, with additional support for Spanish, French, German, Italian, and Japanese. It is explicitly designed for long-running autonomous agents, sub-agent workhorse deployments, and agentic workflows — the high-volume execution layer where frontier reasoning models would be too expensive or slow for every step.^[3]

Performance

According to independent benchmarks by Artificial Analysis, Nemotron 3.5 Lightning achieves performance comparable to OpenAI's gpt-oss-120b at approximately one-quarter of the total parameters. NVIDIA reports up to 4× faster token generation and 30% faster time-to-completion compared to open models in its class. Its Time per Intelligence Index Task is approximately 0.5 minutes based on output speeds achieved with NVFP4 quantised weights.^[1]

Design Philosophy: Agentic Execution

NVIDIA positions Nemotron 3.5 Lightning as the execution-layer model for always-on AI agents. In agentic workflows, a system typically uses a frontier reasoning model for complex planning and decision-making, but delegates high-volume, repetitive execution tasks — tool calls, result validation, sub-agent delegation — to a faster, cheaper model. Nemotron 3.5 Lightning is purpose-built for this execution role.^[3]

To support this model-routing paradigm, NVIDIA simultaneously released NeMo Switchyard, an open-source library for model routing that allows developers to dynamically direct tasks to the appropriate model based on complexity, cost, and latency requirements.^[2]

Local and Edge Deployment

A defining feature of the Nemotron 3.5 Lightning release is its emphasis on local and edge deployment. NVIDIA collaborated with vLLM, Ollama, llama.cpp, and LM Studio to provide optimised local deployment, offering both NVFP4 and GGUF model formats. Unsloth provided day-one support with optimised and quantised models for efficient local deployment via Unsloth Studio.^[2]

The model runs locally on NVIDIA RTX PCs, NVIDIA DGX Spark, and OEM GB10 systems, as well as NVIDIA Jetson edge platforms, scaling up to RTX PRO workstations, DGX Station, and GB300 deskside systems. This positions Nemotron as a key enabler of the local AI and privacy-preserving AI movements, where models run on user-owned hardware rather than cloud APIs.^[2]

>See Also

• NVIDIA
• Mixture of Experts
• Mamba SSM
• Agentic AI
• Open Weights
• Quantisation
• Speculative Decoding

References