Living With Local AI: What It's Great At, What It Isn't, and Why Now

So you’ve got a model running on your own machine. Now what?

This final post is the honest debrief. After living with a local setup for a while alongside my frontier subscriptions, here’s where it genuinely earns its place, where it doesn’t, and why I think the weekend you spent on it was time well spent.

Where local AI genuinely shines

Anything private. This is the big one. When the AI runs on your own computer, your questions never leave the building. For anything sensitive (personal matters, early-stage ideas, financial details, a difficult email you’re working through) there’s a real peace of mind in knowing it’s not travelling to someone else’s warehouse. The model is a closed room, and you hold the only key.

Everyday drafting and thinking. Summarising a long document, rewording an email, brainstorming, talking through a decision, getting unstuck on a first draft. For this bread-and-butter work, a good local model is more than capable. This is most of what most people use AI for, and you can now do it for free, all day, with no meter running.

Working offline. On a plane, in a cabin, on dodgy hotel wifi, your AI keeps working because it was never relying on the internet in the first place. There’s something quietly liberating about that.

No anxiety about limits. Because nothing is metered, I use my local model more freely than the paid ones. I don’t think twice about asking it something trivial. That freedom changes how you work in a way that’s hard to appreciate until you have it.

Where it falls short

I promised honesty, so here it is. Local AI is not a complete replacement, and anyone telling you otherwise is selling something.

The hardest problems still go to the frontier. When I need the absolute best reasoning (a genuinely thorny analysis, something where I want the smartest possible help) I still reach for the frontier tools. Their warehouse-sized models are more capable than anything I can fit at home, and for the hard stuff that gap is real.

Local models know less about right now. The frontier tools can search the web and pull in current information. A basic local setup can’t, out of the box. For anything that depends on recent events, the cloud tools win.

It takes a little care and feeding. Not much, but some. You manage your own models, occasionally update things, fiddle when something misbehaves. It’s a car you own, with the small maintenance that implies, rather than a taxi where someone else handles everything.

So I run both, and I’ve stopped thinking of it as a competition. Local for the private, the everyday, and the offline. Frontier for the hardest problems and anything that needs the live internet. Each covers the other’s weak spots.

Why now, specifically

Back in the first post, I made an argument I want to return to. The frontier AI we’re all paying for is almost certainly priced below what it costs to run. The providers are in a land-grab, and land-grabs end. When they do, the comfortable monthly subscriptions get less comfortable.

I’m not predicting doom. The frontier tools will still be worth paying for. But depending entirely on something you don’t understand and don’t control is a weak position, and the time to build an alternative is while you’re not under pressure to.

Right now, learning local AI is a relaxed weekend project. You can take your time, make mistakes, enjoy the tinkering. If you wait until the day the prices jump and you suddenly need an alternative, you’ll be learning all of this in a hurry and a bad mood. Skills built calmly are worth far more than skills built in a panic.

There’s also a simpler reason: it’s genuinely satisfying. There’s a particular pleasure in pointing at a computer in your spare room and saying “that’s my AI, it answers to me, and it costs me nothing.” In an era where so much of our technology is rented from a handful of enormous companies, owning a real, capable piece of it feels good. It feels like the difference between renting your whole life and owning a few things outright.

What I’d tell a friend

If you take one thing from this series, let it be this: you have more options than the monthly subscription screen suggests.

The frontier tools are brilliant and I’m keeping mine. But there is a whole world of capable AI that runs on hardware you might already own, costs nothing to use, keeps your business to itself, and can’t be repriced out from under you. A year ago it was a hobby. Today it’s a weekend project that a curious, non-technical person can genuinely pull off.

The launch special is still running. The smart move is to learn to cook while the eating is cheap.

Thanks for reading along. If you build something over a weekend, I’d love to hear how it went.

That’s the series. Six posts, one ordinary gaming PC, a frontier AI as my patient setup guide, and an assistant that finally answers to me.