---
title: Run an LLM locally with LM Studio
date: 2026-06-13
tags: ai, llm, lm-studio, local
summary: No cloud, no per-token cost, nothing leaves your machine. How to get a local model running in about ten minutes.
---

Running an AI model on your own machine is easier than it sounds. No account, no per-token fees, and your code never leaves your laptop.

There are several tools for this: Ollama, llama.cpp, Jan and LM Studio. I use **LM Studio**. It is a normal desktop app with a built-in model search and chat window, so there is nothing to build and no terminal needed.
I tried Ollama first but ended up with LM Studio because it's better integrated and IT doesn't block it :-)

## Why LM Studio

- It is one installer. There is no separate "GPU version".
- Under the hood it runs [llama.cpp](https://github.com/ggml-org/llama.cpp), so you get the same engine with a UI on top. On Apple Silicon it can also use MLX.
- Model search and download are built in. No hunting around Hugging Face.
- It can run a local server that speaks the OpenAI API. That is how you connect it to your IDE or your own app, which the next two posts are about.

## 1. Install

Download it from `lmstudio.ai`.

- **Windows:** run the `.exe` installer.
- **macOS:** open the `.dmg` and drag the app to Applications. Apple Silicon works out of the box via Metal.
- **Linux:** download the `.AppImage`, make it executable and run it. No sudo needed.

## 2. GPU acceleration

With an Nvidia card and up-to-date drivers, LM Studio finds the GPU on its own. On a Mac, Metal is used. To check, open a chat and look at the inference settings. If the GPU is working, you will see VRAM usage listed.

On Linux with an AMD card, make sure your ROCm drivers are current.

## 3. Download a model

1. Open the **Search** tab.
2. Type a model name, for example `Llama`, `Mistral`, `Phi` or `Qwen`.
3. Pick a `.gguf` file ending in **`Q4_K_M`**. That is a good balance between size and quality. Avoid the smallest versions unless you really have to, they make things up a lot.
4. Download it, then select it from the model dropdown at the top.

## Which model fits my hardware?

The model has to fit into your VRAM. If it doesn't, the rest spills into normal RAM, which still works but is much slower. As a rough guide:

| Your hardware | What to try |
|---|---|
| 4 GB VRAM, 8 to 16 GB RAM | Small models, around 1 to 3B parameters |
| 8 GB VRAM, 16 GB RAM | 7B models. This is the sweet spot |
| 16 GB VRAM, 32 GB RAM | 13 to 14B models |
| 24 GB VRAM or more | 27 to 34B models |

A site like [willitrun.ai](https://willitrun.ai) lets you enter your GPU and RAM and tells you if a model will fit, so you don't download 40 GB just to watch it crawl.

## Is it as good as the cloud?

No. A local 7B model is not a big hosted model. On complex reasoning or long context you will notice. That is just what consumer hardware can do today.

But the gap is closing, and the tooling you learn now will carry over when better local hardware arrives.

## It is still AI

Local or not, always review what it writes. It will answer questions that have no answer, invent APIs that don't exist, and miss context it never got. Treat it like a junior developer who has read a lot but has not shipped much: useful, but it needs checking.

Next: [connecting it to your IDE](/p/2026-08-07-local-llm-github-copilot/).