Table of Contents
Nous Research has added one-click local model setup to Hermes Agent, giving users a simpler way to run open AI models directly on their own machines instead of routing every agent session through a cloud provider.
The change removes much of the setup work Hermes users usually handle with local inference, including choosing an engine, downloading the right build, tuning quantization and deciding how much of a model should fit into GPU memory.
Hermes can now download and manage a llama.cpp runtime, select a hardware-appropriate build and handle memory configuration automatically. Once a model is downloaded, the company says local chats require no account, API key or network access.
"The new easy setup feature is presented automatically upon downloading the app, or you can access it within the Providers section in Settings."
Local Models Move Into The Hermes App
The local-model workflow is designed to make on-device AI feel closer to selecting a normal model provider. Users can open Settings → Providers → Local Models, install the runtime, choose a model from the catalog, download it, and click "Use."
Hermes then starts and stops the local server with the app, while still allowing users to switch back to a cloud model from the same model picker.
Local inference gives those agent workflows a different operating model. Instead of sending every prompt, file summary or recurring task through a remote API, users can keep more activity on their own hardware. That gives privacy-conscious users and developers a more direct path to running persistent agents without ongoing cloud inference costs.
NVIDIA Pushes Easier Local AI At IFA 2026
NVIDIA’s IFA 2026 local AI announcement mentioned Hermes as part of a broader push to make local agents easier to install, faster to run and more practical on consumer and workstation hardware.
"Once it is running, Hermes works the way it does anywhere else. It uses tools, maintains context across tasks, remembers information between sessions and creates reusable skills over time, allowing the agent to become more capable with continued use. Running the model locally on a GPU keeps performance fast while keeping data on the system." - NVIDIA
NVIDIA says the Windows setup for Hermes can automatically detect an NVIDIA GPU, choose a suitable model and configuration, and run through integrated llama.cpp with NVIDIA inference optimizations already in place. Linux support for that NVIDIA-focused one-click path is coming, according to the company.
Gerardo Delgado
Windows and Linux systems can use NVIDIA GPUs or CPU fallback, macOS support targets Apple Silicon through Metal, and Vulkan builds serve AMD GPUs. Hermes marks model fit by memory status, showing whether a model can run entirely in GPU memory, spill into system RAM at slower speed or exceed the machine’s capacity.
Hardware requirements depend on model size. Nous says GPUs with 8 GB or more of memory can run smaller catalog models comfortably, while 16 GB or more is recommended for 27B–35B class models at higher quality. The managed runtime is also optional: users who already run a llama-server or another OpenAI-compatible local endpoint can point Hermes to their own setup.
Why It Matters For Agent Users
The change lowers friction. Local AI has appealed to users for privacy, latency and cost reasons, but it has often required them to understand backend selection, model formats, CUDA or Metal support, memory limits and context settings before doing anything useful. Hermes now absorbs more of that complexity inside the app.
The update also fits a broader shift in AI software. As agents become more persistent and tool-connected, users (and more notably, businesses) are weighing what data models can access, how much control they have over the runtime, and whether repeated background work depends on external APIs.
The new Hermes workflow now makes local agents more accessible to users who want private, always-available automation running on machines they control.
