What hardware actually runs local AI
Brian Niceley · July 2026
The wrong way to size a local-AI machine is to begin with a GPU ranking. Begin with the work: one person drafting from a few documents is a different system from five people searching a shared matter archive while an agent runs in the background.
I size in this order: workflow, quality floor, working-set size, acceptable wait time, simultaneous users, and only then hardware. That order prevents an expensive machine from becoming a very fast answer to the wrong question.
The five inputs that matter
- 01The job. Drafting, document retrieval, transcription, and agent actions put different pressure on the machine. Bring examples from last week.
- 02The quality floor. A smaller model may be quick and adequate for a narrow workflow; a harder reasoning job may need a larger local model or an honest cloud exception.
- 03Memory. The model, context, document index, and supporting services all need room. Advertised model size is not the entire working set.
- 04Latency and concurrency. One patient user and several people expecting interactive responses are different capacity plans.
- 05Operations. Noise, power, backups, repairability, operating system, and who supports the box matter after the benchmark is over.
Three honest budget lanes
Use what you own
Best for proving one workflow before buying anything. Expect compromises in model size or speed. If the useful result appears, that measured workload becomes the dedicated-machine specification.
Dedicated box · roughly $2,000–$4,000
A common range for one or a few focused workflows with enough memory and storage to avoid living at the edge. The exact choice depends on the model and whether several people use it at once.
Larger local system · roughly $4,000–$8,000
More memory, larger models, longer context, or concurrent users. Beyond this range I stop using a generic lane and quote the actual shared-server or specialist requirement.
Those are planning ranges, not a shopping promise. Current manufacturer pricing spans the same broad territory — from Apple’s Mac Studio configurations to Dell’s AI workstation catalog. Prices move; the quote records the actual parts at cost.
The worksheet I use before a quote
- What exact task should work on day one?
- What data may never leave the machine?
- What is the smallest acceptable answer quality?
- How long may one response take?
- How many people or background jobs run at once?
- What hardware already exists?
- Who owns backup, patching, and physical security?
A good intake sometimes ends with “use the computer you already have.” Sometimes it ends with “this job still belongs in the cloud.” Hardware comes after those answers, because owning the wrong machine is not sovereignty.