NVIDIA Nemotron 3.5 Lightning fact-checked: 30B/3B, 1M context, RTX and Mac kept separate

Research snapshot: August 12, 2026 · editorial review: August 15, 2026. NVIDIA released Nemotron 3.5 Lightning on August 11. NVIDIA describes it as a 30B Mixture-of-Experts model with 3B active parameters, combining Mamba-2, MoE and selected attention layers. The official release includes BF16 and NVFP4 checkpoints, up to a one-million-token context window, speculative-decoding options, and training data and recipes under OpenMDW 1.1. [S01][S06][S10]

The important part is what Lightning is not: it is not positioned as NVIDIA’s most capable general reasoning model. It is an execution model for high-volume agent loops—tool calls, coding steps, validation, retrieval work and sub-agent tasks where latency and throughput can matter more than squeezing out the last point of general intelligence.

That trade-off is visible in independent results. Artificial Analysis scores Nemotron 3.5 Lightning at 24 on its Intelligence Index, below Qwen3.6 35B A3B at 32 and Muse Glimmer at 35. Yet its pre-release test of the final NVFP4 weights on DeepInfra reached nearly 670 output tokens per second. [S15][S16][S17]

Verified specifications

Item	Verified status on Aug. 12, 2026	Caveat
Release	Aug. 11, 2026	official
Parameters	NVIDIA: 30B total / 3B active	AA reports 31.6B / 3.6B
Architecture	hybrid MoE: Mamba-2 + MoE + selected attention	official
Maximum context	up to 1M tokens	serving products can expose less
Modalities	text in / text out	no native image input
License	OpenMDW 1.1	check the license for your use
Optimized checkpoint	NVFP4	NVIDIA deployment path
Reference checkpoint	BF16	customization/post-training focus
Official consumer GPU listed	GeForce RTX 5090	not a promise for every laptop GPU
Apple Silicon	community GGUF and MLX paths	not official NVIDIA Mac support
OpenRouter standard route	262K, $0.05/M input, $0.20/M output	provider-specific
Fireworks serverless	262K, $0.05/$0.01/$0.20 input/cache/output	provider-specific

Why do parameter counts differ?

NVIDIA consistently uses 30B total / 3B active and names the checkpoints 30B-A3B. Artificial Analysis lists 31.6B total / 3.6B active, while Fireworks reports 32.9B parameters. [S01][S16][S23]

Those figures may reflect rounding or different counting conventions, particularly in a sparse hybrid architecture. The reviewed NVIDIA material does not explicitly reconcile the numbers, so the safest wording is to preserve the official 30B/3B name while noting the more exact third-party metadata instead of silently mixing them.

Architecture: sparse execution rather than brute-force size

Lightning interleaves Mamba-2 and MoE components with selected attention layers. Only a fraction of the total model participates in each token’s computation. NVIDIA also ships Multi-Token Prediction and separate DSpark and DFlash speculative-decoding components. [S01][S04][S05]

That makes the design unusually aligned with agent systems. Many agent steps are repetitive and bounded: parse a tool result, change a file, inspect logs, validate JSON, execute a command, summarize retrieved evidence, or delegate a small task. NVIDIA’s model-routing story is to reserve heavier models for planning and use Lightning for the execution layer. NeMo Switchyard is the corresponding routing library. [S06][S09]

Benchmark results: strong agent numbers, with an important methodology warning

NVIDIA’s NVFP4 release table reports:

Benchmark	NVFP4
MMLU Pro	81.62
GPQA Diamond	75.57
HLE, text-only/no tools	10.47
SciCode	31.38
SWE-bench Verified	52.80
SWE-bench Multilingual	36.47
Terminal-Bench 2.1	23.46
PinchBench	83.43
BrowseComp	36.81
GDPval-AA-V2	865
IFBench loose	72.88
AA-LCR	49.19

Source: [S01].

These numbers are useful for comparing NVIDIA’s BF16 and NVFP4 releases because they were produced under a common harness. They should not be pasted next to another vendor’s self-reported scores as if every prompt, tool, inference setting and grader were identical. NVIDIA explicitly says its values may differ from vendor-reported numbers and publishes evaluation recipes for reproduction. [S01]

Independent view: intelligence is good, speed is the headline

Artificial Analysis currently places Lightning at an Intelligence Index of 24. Its launch analysis says Qwen3.6 35B A3B scores 32 and Muse Glimmer scores 35. [S15][S16]

The differentiator is serving speed. Artificial Analysis measured almost 670 output tokens/s on a pre-release DeepInfra endpoint using the final NVFP4 weights. Its provider page later showed 662.9 t/s for DeepInfra, 522.8 t/s for Fireworks and 256.6 t/s for Nebius at the time of retrieval. [S15][S17]

Those are cloud endpoint results, not an RTX 5090 or Mac benchmark. Throughput varies with batching, request shape, hardware and provider load. The correct conclusion is therefore narrower: Lightning can be served extremely quickly in optimized NVFP4 infrastructure.

Is it better than Qwen3.6 or Muse Glimmer?

For broad intelligence, the current Artificial Analysis evidence says no: both comparison models rank higher on the composite Intelligence Index. [S15]

For an agent that performs hundreds of cheap execution steps, however, Lightning can be the better engineering choice. Total task cost depends on more than an intelligence score: generation speed, tokens consumed, failure/retry rate, provider price, caching and whether the model is good enough for the specific step all matter.

A useful architecture is therefore:

1. a stronger planner for difficult decomposition and ambiguous decisions;
2. Nemotron 3.5 Lightning for high-volume execution;
3. escalation back to the planner when confidence checks fail.

That is a testable system design rather than a blanket claim that one model is universally better.

One-million-token context: capability versus product limit

The official checkpoint supports up to one million tokens. [S01]

But OpenRouter’s standard paid route currently lists 262K, and Fireworks also lists 262K. OpenRouter separately shows a free NVIDIA route with 1M metadata. [S21][S22][S23]

This distinction matters for buyers and local users. Always separate:

• the model’s architectural maximum;
• the provider’s configured maximum;
• the context size that is practical for your latency and memory budget.

Publishing “1M context” without that distinction is technically incomplete.

Official local deployment on NVIDIA hardware

The NVFP4 model card lists Blackwell hardware including DGX Spark/GB10, GB200 and GeForce RTX 5090, plus H100/H200 and an Ampere W4A16 path. NVIDIA gives DGX Spark and H100 as concrete single-GPU deployment examples. [S01]

The BF16 card gives H100/A100 80GB single-GPU examples and notes that its single-H100 recipe uses 256K context even though the model maximum is 1M. [S02]

So “runs on a single GPU” should not be rewritten as “runs on a typical laptop GPU.” The official evidence is considerably more specific.

Running Nemotron 3.5 Lightning on Apple Silicon

There is no NVIDIA-official Apple Silicon deployment path in the reviewed model cards. There are already multiple community paths, however.

llama.cpp / GGUF

The ggml-org conversion documents:

brew install llama.cpp
llama serve -hf ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF:Q4_K_M

Source: [S24].

Bartowski’s quantization page lists Q4_K_M at 25.48 GB, Q5_K_M at 26.96 GB and Q8_0 at 35.00 GB. [S25]

MLX

The mlx-community 8-bit conversion is listed at 33.6 GB and documents an OpenAI-compatible MLX-LM server:

uv tool install mlx-lm
mlx_lm.server --model "mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-8bit"

Source: [S27].

A second MLX/oMLX conversion is already available from Vontra. [S28]

The important caveat is maturity. A downloadable conversion proves that the model can be loaded through the runtime; it does not prove perfect support for every long-context, function-calling, reasoning-template or speculative-decoding feature. As of the research date, there is not yet a sufficiently mature, reproducible Apple-Silicon benchmark matrix to quote trustworthy tokens/s across M-series machines.

Memory reality

A 25.48 GB GGUF file is not a 25.48 GB peak-memory guarantee. Runtime state and context add memory pressure. Based on file size alone:

• 24 GB unified memory is below the Q4_K_M weight-file size;
• 32 GB leaves limited headroom;
• 48–64 GB is a much more plausible target for Q4/Q5 plus practical context;
• 64 GB or more is a safer starting point for the 33.6 GB 8-bit MLX conversion.

Those are engineering estimates, not measured performance claims. A proper Mac test should publish peak memory, prompt processing speed, generation speed, context length and power draw.

API pricing and three cost scenarios

OpenRouter’s standard route currently lists $0.05 per million input tokens and $0.20 per million output tokens. [S21]

Fireworks lists $0.05 input, $0.01 cached input and $0.20 output per million tokens. [S23]

Using Fireworks’ published rates and ignoring taxes or other platform charges:

Workload	Input	Output	Estimated model cost
light	2M	0.5M	$0.20
medium	25M	5M	$2.25
heavy	250M	50M	$22.50
heavy, with 80% of input cached	250M	50M	$14.50

The cached heavy example saves about 35.6% versus treating all input as uncached. That 80% cache-hit assumption is illustrative, not a forecast.

Cost per token is also not cost per completed task. Artificial Analysis found Lightning relatively verbose in its benchmark suite, so a deployment should track tokens per successful task and retry rate, not only the price card. [S16]

Which checkpoint should you use?

Use NVFP4 when the goal is optimized NVIDIA inference on supported hardware. [S01]

Use BF16 when you need reference weights for customization, post-training or generating your own quantization. NVIDIA explicitly directs deployment users toward NVFP4. [S02]

Use GGUF for llama.cpp, LM Studio, Ollama and broad local-runtime compatibility. The currently available builds are community conversions. [S24][S25][S26]

Use MLX for an Apple-Silicon-native workflow, with the explicit understanding that the conversions are new and should be validated against your tool-calling and context requirements. [S27][S28]

Deployment paths at a glance

The following overview separates official NVIDIA paths, Apple Silicon paths and cloud routes.

Where Lightning makes the most sense

The model is best suited to:

• agent tool execution;
• coding and terminal loops;
• structured extraction and transformation;
• RAG post-processing;
• log and file analysis;
• sub-agents;
• private local assistants;
• high-volume tasks where a frontier model would be unnecessary.

For difficult planning, broad reasoning or tasks where a wrong intermediate action is expensive, the current evidence supports using a stronger planner and routing bounded execution steps to Lightning.

Practical role

Nemotron 3.5 Lightning is best understood as a fast open-weight worker model, not as a new general-intelligence champion.

Its strongest independent signal is the combination of a 24 Artificial Analysis Intelligence Index with exceptionally high measured output speed on an optimized NVFP4 endpoint. Its official release is unusually complete—BF16 and NVFP4 weights, training recipes, speculative decoding modules, 1M maximum context and a permissive model license—but deployment details still matter. [S01][S10][S15]

For Apple Silicon, the practical situation is clear but incomplete: GGUF and MLX paths exist, but credible per-Mac benchmarks are still missing. Until those measurements exist, any article promising a specific M-series tokens/s number or effortless 1M local context would be getting ahead of the evidence.