NVIDIA Expands Open Models for Local AI Agents on RTX and DGX Hardware
Developers can now run capable AI agents on their own machines, keeping code, files and credentials on the device while still getting responsive assistance. A wave of new open-weight models and tools from NVIDIA and its partners, released through August 2026, brings coding agents, video generation and always-on assistants to local hardware including GeForce RTX GPUs, RTX PRO GPUs, DGX Spark, DGX Station and Jetson. The common thread is local inference: work that used to depend on the cloud can run on a single system, without counting against token limits.
What can these local models actually do?
The releases cover coding, agent workflows, world modeling and generative media. Several are sized to fit a single GPU, which makes private, self-hosted AI practical for individual developers and small teams.
Frontier-quality coding on a single GPU
Qwen3.8-27B is a 27-billion-parameter open model built as the local companion to Qwen3.8-Max, sized for one GPU and tuned for coding workflows that use local files, tools and project context. Optimized with multi-token prediction, it reaches 131 tokens per second on a single GeForce RTX 5090. NVIDIA measured this on an RTX 5090 with an Intel Core Ultra 9 285K, 64GB of RAM and Windows 11, using a Q4_K_M checkpoint on llama.cpp. Day-zero support is available on RTX GPUs, RTX PRO GPUs, DGX Spark, DGX Station and Jetson, with developers able to start through llama.cpp, Ollama, Unsloth and LM Studio Bionic.
Always-on agents built for tool use
Meta released Muse Glimmer, a 30-billion-parameter dense open-weight model with a context window above 120K, purpose-built for coding and local agentic AI. It delivers over 200 tokens per second on an RTX 5090 and is small enough to run on a single consumer GPU. Its design targets custom agents, private data processing across local files and messages, credential handling that keeps inference on the device, multistep tool calls with error recovery, and long-running workflows that can resume interrupted sessions with context intact. Developers can run it with vLLM or llama.cpp, and fine-tune it locally with NVIDIA NeMo Automodel using private data.
NVIDIA also expanded its Nemotron 3 family with Nemotron 3.5 Lightning, a customizable open 30B mixture-of-experts model for always-on agents. It delivers up to 4x faster token generation and 30% faster time to completion compared with open models in its class, and because it ships as open weights, developers can fine-tune it on their own examples to match a preferred writing style or a specialty.
A portable personal agent
Perplexity introduced Portable Computer, a local agent app optimized for DGX Spark that runs private workflows which do not count toward token limits. It connects to apps including Google Drive, Gmail, Slack and GitHub, and lets users switch between local models for everyday tasks and cloud models for complex work. The local experience uses one-click inference setup and a post-trained Qwen 3.8 27B model, with a fine-tuned Nemotron 3.5 Lightning variant and support for GeForce RTX and RTX PRO GPUs, Windows and DGX Station in progress.
Which open models cover media and world modeling?
- Cosmos 3 Edge: a 4-billion-parameter open world model for robotics, autonomous vehicles and vision AI, at a quarter the size of Cosmos 3 Nano, running on device on DGX Spark and Jetson.
- MiniMax-H3: a 33-billion-parameter open-weight model that generates video and natively synchronized stereo audio from text, images, video, audio or a mix, accessible through ComfyUI.
- Poolside AI Laguna S 2.1: a 118-billion-parameter open-weight agentic coding model for hours-long tasks, with an NVFP4 checkpoint that runs on a single DGX Spark.
- DeepSeek-V4-Flash: a 284-billion-parameter mixture-of-experts model with 13 billion active parameters and a 1 million-token context window, runnable locally on a DGX Station via community GGUF versions.
- Thinking Machines Lab Inkling-Small: a 276-billion-parameter open-weight multimodal model that activates 12 billion parameters per token, reasons across text, images and audio, and runs on a single DGX Station or two DGX Spark systems.
- Alibaba Wan-Animate-2: a 14-billion-parameter open-weight model that transfers motion and facial expressions from a driving video onto a static character image, with day-zero ComfyUI support.
- LTX-2.5: an open video generation model adding multishot support for continuous multi-cut sequences and an upgraded diffusion video decoder, delivering up to 20% faster performance and 40% memory savings on an RTX 6000 PRO GPU.
Unsloth also launched Unsloth Desktop, described as the first fully open-source desktop app that both trains and runs AI models locally, combining inference, image and video diffusion, fine-tuning, agent integrations, web research and code execution.
How do developers run larger models locally?
Some of the larger open models, such as GLM 5.2 and DeepSeek V4 Flash, need multiple GPUs or DGX Spark systems working together. The NVIDIA Sync app, available for Windows and macOS, clusters multiple DGX Spark systems for more memory, inference performance and training throughput. It automatically detects connected systems and provides private remote access through Tailscale. Its Cluster Assistant configures two or more DGX Spark systems as a high-speed cluster once they are connected through ConnectX-7 ports, routing workloads across nodes and monitoring system health. Arriving later in August, DGX Spark gains a native ARM64 Linux build of Google Chrome installable in one click, and a new NVIDIA Sync Resource Monitor for real-time and historical views of CPU and GPU usage across a single system or a full cluster.
FAQ
How fast does Qwen3.8-27B run on an RTX 5090?
Optimized with multi-token prediction, Qwen3.8-27B reaches 131 tokens per second on a single GeForce RTX 5090. NVIDIA measured this using a Q4_K_M checkpoint on llama.cpp with an Intel Core Ultra 9 285K, 64GB of RAM and Windows 11.
What is Meta Muse Glimmer built for?
Muse Glimmer is a 30-billion-parameter dense open-weight model with a context window above 120K, purpose-built for coding and local agentic AI. It runs on a single consumer GPU, delivers over 200 tokens per second on an RTX 5090, and supports custom agents, private data processing, credential handling on the device, multistep tool calls and long-running workflows.
Can multiple DGX Spark systems be combined to run larger models?
Yes. The NVIDIA Sync app and its Cluster Assistant configure two or more DGX Spark systems as a high-speed cluster once they are connected through ConnectX-7 ports, providing the memory, inference performance and training throughput needed for larger models such as GLM 5.2 and DeepSeek V4 Flash.
Related coverage
This article summarizes reporting from blogs.nvidia.com.