The Weekend Build: Getting Your First Local Model Running

Enough theory. This is the post where we actually do it.

I’ll walk you through the build the way I’d explain it to a friend over coffee: the shape of the thing, the key decisions, and the moments people get stuck. I’m deliberately not going to drown you in every click, because the apps change their menus often enough that a click-by-click guide goes stale fast.

And there’s a better reason to skip the exhaustive clicks, which I hinted at last post and want to make the spine of this one: your best step-by-step guide is a chatbot. Throughout this build, the move is to keep a frontier AI open in another window and let it walk you through your exact situation. In my testing Claude was the most accurate, and its mid-tier Sonnet model was plenty: fast and reliable enough that you needn’t spend premium quota on it. Treat what follows as the map; treat the chatbot as the guide who knows your specific roads.

Set aside a couple of hours on a weekend. Make a coffee. This is genuinely enjoyable.

Step zero: check your kitchen, then tell the guide

Before anything, find out how much fast memory your computer has, because that decides everything.

On a Windows PC, open Task Manager, go to the Performance tab, click your GPU, and read “dedicated GPU memory.” That’s your VRAM. On a recent Mac, it’s simpler and more generous: thanks to the unified memory we covered last post, nearly all your system memory is usable, so the number to note is just your total RAM.

Now do the single most useful thing in this whole guide: open your chatbot and tell it exactly what you found. “I have a Windows PC with an Nvidia card and 12 gigabytes of VRAM” or “I have a Mac mini with 32 gigabytes of memory.” Ask it to recommend a local model and the right size for your machine, and to walk you through running it. Everything below will now make sense as context for that conversation.

Step one: install the app store

Download and install LM Studio. It’s free and installs like any normal program. Nothing clever or risky.

This is the app store for AI models from earlier in the series. Open it and you’ll see a clean interface with a search box and a chat window. If you’ve ever installed and used a desktop app, you can navigate it. For the precise install clicks on your operating system, follow LM Studio’s own current guide, or just ask your chatbot, screenshot in hand if you get stuck.

Step two: pick and download a model

Here’s where last post pays off. You know your memory. Now you choose a model that fits inside it. Or rather, you let your chatbot tell you which one, and you go find it in LM Studio.

Each model comes in several sizes labelled in what looks like gibberish: tags like “Q4,” “Q5,” “Q8.” Think of these as compression settings, like saving a photo at different quality levels. A higher number means better quality but a bigger file needing more memory; a lower number squeezes it smaller to fit modest hardware, at a slight cost to sharpness. For most people starting out, “Q4” is the sweet spot. Fits comfortably, quality drop barely noticeable.

If you want a concrete starting point: ask your chatbot for a well-regarded model in the 7-to-14 size range (the number roughly tracks capability), grab the Q4 version, and watch LM Studio’s little memory indicator confirm it’ll fit. The app warns you if a model is too big, which removes most of the guesswork. Download it. A few minutes, depending on your connection, since these files are large.

Step three: load it and chat

Once downloaded, you load the model: telling your computer to clear the counter and lay this recipe out on it. There’s a loading screen with various settings, and this is the one moment where the options look genuinely intimidating.

My honest advice: for your first run, leave almost everything on its defaults. The one setting worth knowing is “context length”: how much the model holds in mind at once, like how long a conversation it can follow before forgetting the start. Start modest; turn it up later once you know your machine handles the model comfortably. And if a setting puzzles you, paste it to your chatbot and ask what it does for your hardware. That’s precisely the kind of question it answers well.

Then you just chat. A window appears, you type, and your own computer answers. The first time the words start appearing (generated entirely on a machine in your own home, no internet, no meter running) is a genuinely satisfying moment. I grinned like an idiot.

When it doesn’t work the first time

It might not be perfectly smooth, and I won’t pretend otherwise. The two common stumbles:

The model is too big and your computer chokes. Everything crawls or it refuses to load. The fix is simple: grab a smaller version, a lower Q number or a smaller model. This is the most common beginner mistake and the easiest to fix.

The answers come slowly. If it’s bearable, you might accept it, as I do with my patient model. If not, drop to something smaller. Speed and size pull against each other, and finding your balance is part of the fun.

Neither is a disaster. It’s the normal rhythm of learning what your machine likes. And this is exactly when the chatbot guide earns its keep: describe the symptom, your hardware, and the model you tried, and it’ll usually tell you the precise fix. I solved nearly every snag this way, faster than I’d have found it in any forum.

What you’ve actually accomplished

If you followed along, you now have a working AI running entirely on your own hardware. No subscription, no bill, no usage limit, nobody watching your questions. It’s yours.

A year ago this was a programmer’s project. You just did it on a weekend, with a frontier AI as your patient, accurate co-pilot the whole way.

In the final post: what to actually do with this, where local AI shines, where you’ll still want the frontier tools, and why the hours you just spent will look like a smart investment when the cheap subscriptions get expensive.