NVIDIA Nemotron 3.5 Lightning fact-checked: 30B/3B, 1M context, RTX and Mac kept separate

Updated October 9, 2026; original August 12–15 launch research preserved. NVIDIA released Nemotron 3.5 Lightning on August 11. NVIDIA describes it as a 30B Mixture-of-Experts model with 3B active parameters, combining Mamba-2, MoE and selected attention layers. The official release includes BF16 and NVFP4 checkpoints, up to a one-million-token context window, speculative-decoding options, and training data and recipes under OpenMDW 1.1. ¹²³

The important part is what Lightning is not: it is not positioned as NVIDIA’s most capable general reasoning model. It is an execution model for high-volume agent loops: tool calls, coding steps, validation, retrieval work and sub-agent tasks where latency and throughput can matter more than squeezing out the last point of general intelligence.

That trade-off is visible in independent results. Artificial Analysis scores Nemotron 3.5 Lightning at 24 on its Intelligence Index, below Qwen3.6 35B A3B at 32 and Muse Glimmer at 35. Yet its pre-release test of the final NVFP4 weights on DeepInfra reached nearly 670 output tokens per second. ⁴⁵⁶

Verified specifications

Item	Verified status on Aug. 12, 2026	Caveat
Release	Aug. 11, 2026	official
Parameters	NVIDIA: 30B total / 3B active	AA reports 31.6B / 3.6B
Architecture	hybrid MoE: Mamba-2 + MoE + selected attention	official
Maximum context	up to 1M tokens	serving products can expose less
Modalities	text in / text out	no native image input
License	OpenMDW 1.1	check the license for your use
Optimized checkpoint	NVFP4	NVIDIA deployment path
Reference checkpoint	BF16	customization/post-training focus
Official consumer GPU listed	GeForce RTX 5090	not a promise for every laptop GPU
Apple Silicon	community GGUF and MLX paths	not official NVIDIA Mac support
OpenRouter standard route	262K, $0.05/M input, $0.20/M output	provider-specific
Fireworks serverless	262K, $0.05/$0.01/$0.20 input/cache/output	provider-specific

Why do parameter counts differ?

NVIDIA consistently uses 30B total / 3B active and names the checkpoints 30B-A3B. Artificial Analysis lists 31.6B total / 3.6B active, while Fireworks reports 32.9B parameters. ¹⁵⁷

Those figures may reflect rounding or different counting conventions, particularly in a sparse hybrid architecture. The reviewed NVIDIA material does not explicitly reconcile the numbers, so the safest wording is to preserve the official 30B/3B name while noting the more exact third-party metadata instead of silently mixing them.

Architecture: sparse execution rather than brute-force size

Lightning interleaves Mamba-2 and MoE components with selected attention layers. Only a fraction of the total model participates in each token’s computation. NVIDIA also ships Multi-Token Prediction and separate DSpark and DFlash speculative-decoding components. ¹⁸⁹

That makes the design unusually aligned with agent systems. Many agent steps are repetitive and bounded: parse a tool result, change a file, inspect logs, validate JSON, execute a command, summarize retrieved evidence, or delegate a small task. NVIDIA’s model-routing story is to reserve heavier models for planning and use Lightning for the execution layer. NeMo Switchyard is the corresponding routing library. ²¹⁰

Benchmark results: strong agent numbers, with an important methodology warning

NVIDIA’s NVFP4 release table reports:

Benchmark	NVFP4
MMLU Pro	81.62
GPQA Diamond	75.57
HLE, text-only/no tools	10.47
SciCode	31.38
SWE-bench Verified	52.80
SWE-bench Multilingual	36.47
Terminal-Bench 2.1	23.46
PinchBench	83.43
BrowseComp	36.81
GDPval-AA-V2	865
IFBench loose	72.88
AA-LCR	49.19

Source: ¹.

These numbers are useful for comparing NVIDIA’s BF16 and NVFP4 releases because they were produced under a common harness. They should not be pasted next to another vendor’s self-reported scores as if every prompt, tool, inference setting and grader were identical. NVIDIA explicitly says its values may differ from vendor-reported numbers and publishes evaluation recipes for reproduction. ¹

Independent view: intelligence is good, speed is the headline

Artificial Analysis currently places Lightning at an Intelligence Index of 24. Its launch analysis says Qwen3.6 35B A3B scores 32 and Muse Glimmer scores 35. ⁴⁵

The differentiator is serving speed. Artificial Analysis measured almost 670 output tokens/s on a pre-release DeepInfra endpoint using the final NVFP4 weights. Its provider page later showed 662.9 t/s for DeepInfra, 522.8 t/s for Fireworks and 256.6 t/s for Nebius at the time of retrieval. ⁴⁶

Those are cloud endpoint results, not an RTX 5090 or Mac benchmark. Throughput varies with batching, request shape, hardware and provider load. The correct conclusion is therefore narrower: Lightning can be served extremely quickly in optimized NVFP4 infrastructure.

Is it better than Qwen3.6 or Muse Glimmer?

For broad intelligence, the current Artificial Analysis evidence says no: both comparison models rank higher on the composite Intelligence Index. ⁴

For an agent that performs hundreds of cheap execution steps, however, Lightning can be the better engineering choice. Total task cost depends on more than an intelligence score: generation speed, tokens consumed, failure/retry rate, provider price, caching and whether the model is good enough for the specific step all matter.

A useful architecture is therefore:

1. a stronger planner for difficult decomposition and ambiguous decisions;
2. Nemotron 3.5 Lightning for high-volume execution;
3. escalation back to the planner when confidence checks fail.

That is a testable system design rather than a blanket claim that one model is universally better.

One-million-token context: capability versus product limit

The official checkpoint supports up to one million tokens. ¹

But OpenRouter’s standard paid route currently lists 262K, and Fireworks also lists 262K. OpenRouter separately shows a free NVIDIA route with 1M metadata. ¹¹¹²⁷

This distinction matters for buyers and local users. Always separate:

• the model’s architectural maximum;
• the provider’s configured maximum;
• the context size that is practical for your latency and memory budget.

Publishing “1M context” without that distinction is technically incomplete.

NVIDIA hardware: listed support versus tested recipe

The official NVFP4 model card names NVIDIA Blackwell, Hopper and Ampere deployment paths, but the hardware matrix qualifies them. DGX Spark / GB10 uses W4A16 through Marlin, not a native FP4 tensor-core path; H100/H200 also use W4A16. GB200 has a native FP4 path. The GeForce RTX 5090 is named, but NVIDIA’s matrix says its recipe/backend and validated context must be confirmed. Listing a GPU is not identical to publishing a reproducible validated full-context benchmark for that card.

NVIDIA shows DGX Spark and H100 as example single-GPU deployments. It advertises up to 1M context, but the installed serving recipe and available memory determine the actual limit. NVFP4 weights are not an Apple Metal/MLX-ready checkpoint; treat the Mac ports separately rather than reusing CUDA performance and quantization numbers.

Mac: community GGUF/MLX is real, but overhead is real too

NVIDIA does not publish an official Apple-Silicon serving recipe for the NVFP4 checkpoint. Local Mac users should use a separately converted GGUF or MLX weight set. For example, the ggml-org conversion documents the following llama.cpp serving path:

brew install llama.cpp
llama serve -hf ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF:Q4_K_M

The community 8-bit MLX build is another option; its listed package is approximately 33.6GB. The GGUF Q4_K_M listing is approximately 25.48GB. These numbers refer to files, not GPU memory usage, peak RAM or sustained interactive inference.

Mac memory decision: 24GB is below the 25.48GB Q4 file before any overhead; 32GB leaves very little useful headroom; 48–64GB is more practical for Q4/Q5 with short-to-medium context, and 64GB+ for the 8-bit conversion. Start with bounded context, record actual memory pressure in Activity Monitor, and evaluate tool calling and text output using the same tasks as your baseline. The model’s 1M-context figure is not a safe local default.

Do not present an RTX/NVFP4 vendor throughput chart as an Apple-Silicon benchmark, and do not assume speculative-decoding features work in third-party GGUF/MLX packages simply because they exist in the NVIDIA release. Community quantizers may remove or alter optional prediction heads and runtime-specific functionality. No M-series throughput claims were measured for this update.

NVIDIA NVFP4 is not interchangeable with a 4-bit GGUF

NVIDIA’s first-party quantization recipe describes shrinking its 66GB BF16 checkpoint to a roughly 22GB NVFP4 checkpoint. That’s a 3:1 file-size reduction, not evidence that its 3B active-parameter count turns the full model into a 3B checkpoint. Most of the total model is quantized, selected components retain higher precision, and metadata adds overhead. NVIDIA names lightning_w4a16_nvfp4_4o6 as its published recipe.

Why the published checkpoint uses post-training quantization plus distillation

NVIDIA applies post-training quantization (PTQ) to produce an NVFP4 student checkpoint, followed by quantization-aware distillation (QAD) with a frozen BF16 teacher. The published four_over_six recipe retains W4A16 Mamba linears, combining different quantization choices for specific components. This seeks to recover accuracy after size reduction; it does not validate any third-party Q4_K_M conversion against NVIDIA’s published GPU benchmarks.

Deployment	Published weight/package size	What it does not guarantee
NVIDIA BF16	66GB in NVIDIA’s quantization guide	A proven peak-memory footprint at 1M context
NVIDIA NVFP4	approximately 22GB after NVIDIA-specific PTQ/QAD	Native Apple Metal kernels or equivalent Mac performance
Community GGUF Q4_K_M	approximately 25.48GB in the linked conversion	Identical quantization or NVIDIA RTX throughput
Community MLX 8-bit	approximately 33.6GB package listed by the community	Independently confirmed Mac inference speed or 1M local context

A Mac test anyone can reproduce: First serve the Q4_K_M conversion with a 4,096-token context window, increasing it only after verifying memory pressure. Record the exact Hugging Face revision, downloaded model digest, llama.cpp release, unified-memory capacity and start arguments. Use five small, synthetic code/JSON/tool-calling tasks, three attempts each. Log accepted outputs, tool failures, runtime-reported tokens per second, wall-clock latency, peak memory and swap. Because no local M-series benchmark run was executed for this review, cloud-side speed must never be repackaged as a Mac result.

NVIDIA’s official recipe hub warns that its open-data-only training reproductions may not match published model benchmarks, which included additional proprietary data. This is a reproducibility qualifier, not proof that the downloadable Instruct weights themselves have degraded quality.

API pricing and three cost scenarios

OpenRouter’s standard route currently lists $0.05 per million input tokens and $0.20 per million output tokens. ¹¹

Fireworks lists $0.05 input, $0.01 cached input and $0.20 output per million tokens. ⁷

Using Fireworks’ published rates and ignoring taxes or other platform charges:

Workload	Input	Output	Estimated model cost
light	2M	0.5M	$0.20
medium	25M	5M	$2.25
heavy	250M	50M	$22.50
heavy, with 80% of input cached	250M	50M	$14.50

The cached heavy example saves about 35.6% versus treating all input as uncached. That 80% cache-hit assumption is illustrative, not a forecast.

Cost per token is also not cost per completed task. Artificial Analysis found Lightning relatively verbose in its benchmark suite, so a deployment should track tokens per successful task and retry rate, not only the price card. ⁵

Which checkpoint should you use?

Use NVFP4 when the goal is optimized NVIDIA inference on supported hardware. ¹

Use BF16 when you need reference weights for customization, post-training or generating your own quantization. NVIDIA explicitly directs deployment users toward NVFP4. ¹³

Use GGUF for llama.cpp, LM Studio, Ollama and broad local-runtime compatibility. The currently available builds are community conversions. ¹⁴¹⁵¹⁶

Use MLX for an Apple-Silicon-native workflow, with the explicit understanding that the conversions are new and should be validated against your tool-calling and context requirements. ¹⁷¹⁸

Deployment paths at a glance

The following overview separates official NVIDIA paths, Apple Silicon paths and cloud routes.

Where Lightning makes the most sense

The model is best suited to:

• agent tool execution;
• coding and terminal loops;
• structured extraction and transformation;
• RAG post-processing;
• log and file analysis;
• sub-agents;
• private local assistants;
• high-volume tasks where a frontier model would be unnecessary.

For difficult planning, broad reasoning or tasks where a wrong intermediate action is expensive, the current evidence supports using a stronger planner and routing bounded execution steps to Lightning.

A Mac evaluation that does not confuse vendor and community builds

For an Apple Silicon comparison, choose a specific GGUF or MLX revision and record its digest, quantization, runtime version, context limit and memory use. The official NVIDIA NVFP4 model card describes NVIDIA-specific validated hardware and kernels, not a promise of native Mac NVFP4 acceleration. Community GGUF and MLX conversions have their own performance characteristics.

Run the same short tool-using task three times and report the fraction of successful completions, latency and memory pressure from Activity Monitor. Confirm tool-call JSON against a schema rather than scoring fluent prose as correctness. A 1-million-token model maximum is not evidence that Ollama, MLX and every Mac configuration can serve that context. NVIDIA’s model card is the source for the 30B/3B and OpenMDW-1.1 claims; the separate Artificial Analysis evaluation reports independent speed tests on cloud hardware, not measured Apple Silicon results.

Practical role

Source separation: Hardware compatibility and 1M context come from NVIDIA’s model card. GGUF/MLX runtime availability and file sizes come from community conversions. Prices and throughput in the August launch snapshot refer to third-party cloud providers; recheck the selected provider before buying capacity.

Nemotron 3.5 Lightning is best understood as a fast open-weight worker model, not as a new general-intelligence champion.

Its strongest independent signal is the combination of a 24 Artificial Analysis Intelligence Index with exceptionally high measured output speed on an optimized NVFP4 endpoint. Its official release is unusually complete (shipping BF16 and NVFP4 weights, training recipes, speculative decoding modules, a 1M maximum context, and a permissive model license), though deployment details still matter. ¹³⁴

For Apple Silicon, the practical situation is clear but incomplete: GGUF and MLX paths exist, but credible per-Mac benchmarks are still missing. Until those measurements exist, any evaluation promising a specific M-series tokens/s number or effortless 1M local context would be getting ahead of the evidence.

The predecessor, Nemotron 3 Ultra, is covered in the Ollama Cloud guide.

Source register and limitations

Model specifications originate from the vendor model card; provider prices, external benchmarks and community builds are attributed to their respective authors. This is not an independently repeated Mac performance evaluation.

Footnotes

1. huggingface.co: source 1 ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸

2. developer.nvidia.com: source 6 ↩ ↩²

3. github.com: source 10 ↩ ↩²

4. artificialanalysis.ai: source 15 ↩ ↩² ↩³ ↩⁴ ↩⁵

5. artificialanalysis.ai: source 16 ↩ ↩² ↩³ ↩⁴

6. artificialanalysis.ai: source 17 ↩ ↩²

7. fireworks.ai: source 23 ↩ ↩² ↩³

8. huggingface.co: source 4 ↩

9. huggingface.co: source 5 ↩

10. developer.nvidia.com: source 9 ↩

11. openrouter.ai: source 21 ↩ ↩²

12. openrouter.ai: source 22 ↩

13. huggingface.co: source 2 ↩

14. huggingface.co: source 24 ↩

15. huggingface.co: source 25 ↩

16. huggingface.co: source 26 ↩

17. huggingface.co: source 27 ↩

18. huggingface.co: source 28 ↩