Flint Local Coding Model: How to Run It on Your Mac or GPU
The Flint local coding model runs on your own machine, so your code never leaves your computer and there is no per-token bill. Flint is Recursive’s open coding model with about 2.5B active parameters, which makes it fast on a good laptop. In the video below I run it locally and use it for real coding. This guide shows every way to install it, how much hardware you need and how to plug it into a coding agent.
Key takeaways
Easiest route: Ollama, with a 23 GB 4-bit build.
On a Mac, Recursive’s rec runtime needs an M4 Pro or M5 with at least 48 GB of memory.
On NVIDIA, use the NVFP4 GGUF with llama-server, or vLLM’s Flint Docker image.
Hook it into OpenCode, OpenHands, Pi or Codex through an OpenAI-compatible API.
Pick your setup
Recursive’s setup guide splits this by hardware. Here is the short version:
Your hardware
Best route
Download size
Mac with M4 Pro / M5 Pro / M5 Max, 48 GB+
rec (Recursive’s runtime) or Ollama
About 23–30 GB
NVIDIA GPU with 48 GB+ memory, or DGX/RTX Spark
llama-server with the NVFP4 GGUF
About 22.9 GB
Single Blackwell data-centre GPU
vLLM Docker image
About 22.9 GB (NVFP4) or 80 GB (BF16)
One B300 or H200
Full BF16 with SGLang
About 80 GB
If you have less memory than this, Flint won’t run well locally. Use the free OpenCode tier or OpenRouter instead.
Option 1: Ollama (easiest)
Ollama is the quickest way to start. Recursive’s docs use this command:
ollama run flint:40b-a2b-qat-int4
The Ollama tags page lists a 23 GB build, an 80 GB mlx-bf16 build and a 26 GB q4_1 build, each with 256K context. Ollama can also start a coding agent for you with ollama launch opencode, and the same works for pi, codex and vscode.
Want private local AI working inside your business? Inside the AI Profit Boardroom you get the Agent OS — Hermes, OpenClaw, OpenCode, Claude and Codex in one place — plus step-by-step Hermes training and four live coaching calls a week.
rec is Recursive’s own runtime for Apple silicon. It needs an M4 Pro, M5 Pro or M5 Max with at least 48 GB of unified memory and about 30 GB of free disk. Recursive says it gets 1.4–2.6x the throughput of llama.cpp on an M4 Pro with 48 GB.
rec launch opencode
rec launch pi
rec gives you an OpenAI-compatible API at http://127.0.0.1:11500/v1 and an Anthropic-compatible /v1/messages endpoint, so tools built for either API can use it.
For NVIDIA cards, download flint-nvfp4.gguf (about 22.9 GB). Recursive recommends at least 48 GB of GPU memory, or DGX Spark unified memory. It has been tested on B300, RTX PRO 6000 Blackwell, RTX Spark and DGX Spark. The model card has the exact llama-server command. Once it runs, your local API sits at port 8080.
Option 4: vLLM in Docker
The vLLM blog has a dedicated image, vllm/vllm-openai:flint, for CUDA 13.0 on x86_64 and arm64. Use these flags for tool calls and reasoning:
For the NVFP4 build, also add --hf-config-path RecursiveSI/flint --tokenizer RecursiveSI/flint. It runs on a single Blackwell GPU, and vLLM reports about 500 tokens a second for one request on a GB200 at 100K context.
Start it: openhands serve --mount-cwd, then open localhost:3000.
Go to Settings > LLM > Advanced and set the provider to None.
With Ollama: model openai/flint:40b-a2b-qat-int4, base URL http://127.0.0.1:11434/v1, key ollama.
With llama.cpp: model openai/flint-local, base URL on port 8080 /v1.
If OpenHands runs in Docker, use host.docker.internal in place of 127.0.0.1.
Give it at least 32K of context. For hard problems, allow 32,768+ output tokens; Recursive’s OpenCode guide uses 65,536.
Settings that work
Use temperature 1.0, top-p 0.95 and top-k 20. Reasoning is always on, so short answers can still take a moment while it thinks. Plan for about 256K context: the model card lists 262,144, and the blog’s 512K figure is a stretch limit. For the background on the model itself, read my Flint coding model guide.
It is slow or crashes: you are probably short on memory. Close other apps, use the 4-bit build, or lower the context length.
Tool calls fail in vLLM: check you added the glm47 tool-call and reasoning parsers listed above.
The NVFP4 build won’t load in vLLM: add the hf-config-path and tokenizer flags pointing at RecursiveSI/flint.
OpenHands in Docker can’t reach the model: swap 127.0.0.1 for host.docker.internal.
Answers cut off on hard tasks: raise the output budget to 32,768 tokens or more.
Local or cloud: which should you pick?
Run Flint locally if your code is private, you code all day, or you are tired of usage limits. Your only cost is the hardware you already own and the power it uses. Use the cloud route if you only code now and then or don’t have 48 GB of memory. OpenCode’s free tier offers Flint-40B with rate limits, and OpenRouter lists a free version too. Many people start in the cloud, then move local once they know they like it. Whichever you choose, the same OpenAI-compatible settings work, so switching later only means changing the base URL and model name in your coding agent.
My verdict on the Flint local coding model
If you have a Mac with 48 GB or a 48 GB GPU, Flint is one of the fastest good coding models you can run at home. Start with Ollama, move to rec or vLLM if you want more speed, and connect it to OpenCode or OpenHands. For private, no-bill coding help, the Flint local coding model is well worth setting up.
About Julian Goldie
I’m Julian Goldie — founder of Goldie Agency and the AI Profit Boardroom. I test new AI agents almost every day for 432K+ YouTube subscribers and run Hermes, OpenClaw, OpenCode, Claude and Codex side by side in my own Agent OS. Claim your free strategy session or join the AI Profit Boardroom.
FAQ: flint local coding model
How much memory do I need to run Flint locally?
Recursive recommends at least 48 GB, either Mac unified memory or GPU memory, for the 4-bit builds of about 23 GB.
What is the easiest way to run Flint locally?
Ollama: run ollama run flint:40b-a2b-qat-int4, then ollama launch opencode to start coding with it.
Does Flint run on a Mac?
Yes. On an M4 Pro, M5 Pro or M5 Max with 48 GB+, use Recursive’s rec runtime or Ollama. Recursive reports over 125 tokens a second on an M4 Pro MacBook Pro.
Which coding agents work with local Flint?
OpenCode, OpenHands, Pi, Codex and VS Code, through its OpenAI-compatible API. rec also offers an Anthropic-compatible endpoint.
What context should I use with Flint locally?
Plan for about 256K tokens. The model card lists 262,144, and Recursive recommends staying within 256K.
Ready to put Hermes to work in your business? Join the AI Profit Boardroom for the Agent OS, setup training and daily updates.
