Why the Future of Local LLMs Doesn’t Depend on Being the Best
I’m going to do something very different from my usual blog posts, which typically fall into one of three categories:
- A walkthrough of how to configure something
- An account of how I tracked down and fixed a problem
- An introduction to a product I have just started working with
This blog post will have no deployment to follow along with, no repository to clone, and no portal screenshots because what I wanted to try was writing an opinion post where you’ll see more of my thinking as if we were interacting in person and had an open discussion about a topic.
Let me provide a bit of background because this opinion comes from a year of working with local models and following the testing that many other professionals have published. Anyone who has followed this blog over the past year will know that I bought an HP ZBook Ultra G1a with an AMD Ryzen AI Max+ Pro 395 and 128GB of LPDDR5X specifically so I could load large open-weight / open-source models into unified memory, and I have repeatedly written about what I have done with that machine, including but not limited to, Vibe Coding a Local Browser Agent with GitHub Copilot and connecting Browser MCP to models running in LM Studio. I have also experimented with OpenAI’s gpt-oss 120B, several sizes of Alibaba’s Qwen line, Google’s Gemini, other popular models, and pointed at real work rather than the benchmark prompts.
There were two primary reasons why I looked at local models. First, I was curious about whether the experience people were getting from the latest AI models could be reproduced on hardware that was realistically within reach of a home user. Second, I liked the idea of keeping everything local. Sending private information to a hosted service is always a judgement call, while running the same workload on a machine sitting under my desk largely removes that question altogether. Not surprisingly, the privacy argument didn’t stay front and center for long because the more time I spent with these models, the more I found myself paying attention to what they could actually do.
Before going any further, I’d like to state that I believe the newest models from Anthropic, OpenAI, or whichever new or existing company takes the lead in the future are going to continue to set the bar. The best local models likely will not match them in LLM performance metrics but what I’d like to point out is that the gap between “best available” and “good enough to be genuinely useful” is rapidly shrinking.
Where a local model typically comes up short is a long chain of dependent steps such as an instruction buried three paragraphs into a prompt, or a task where the right answer requires holding several constraints in mind simultaneously. Those failures are real and I have experienced all of them (e.g. I tried using Qwen 3.6 35B to draft a blog post for me and it would loop for hours and never finish). What keeps impressing me is how many other tasks these models can handle that do not get enough attention. When I want a document summarized, structure pulled out of messy text, a first pass at something I intend to edit anyway, or an answer about a file I would prefer not to upload, a decent local model handles it respectably. It is not an equal result by any stretch when compared to what Anthropic and OpenAI offer, but it is a sufficient one.
That word, sufficient, is the point where I keep landing back on the late Clayton Christensen. The Innovator’s Dilemma, published in 1997, is one of my favourite books, and I think a lot of people reduce it to a much simpler argument than the one Christensen was actually making. Most summaries reduce it to the idea that cheaper products eventually beat more expensive ones and I feel that’s not really the argument. My understanding of Christensen’s argument is that performance keeps improving whether customers need it or not and at some point a product crosses the line from “not good enough” to “good enough” for a large part of the market. After that, buying decisions start to include other considerations such as cost, convenience, and ownership. Being the best technical product is still useful, but it is no longer the only feature customers care about.
The example from the book that has always stuck with me is steel. Minimills melting scrap in electric arc furnaces entered at the very bottom of the market with rebar, the reinforcing bar buried inside concrete where nobody sees it and the only attribute anyone cared about was the price. Rebar carried a gross margin of roughly seven percent for the big integrated mills, so when the minimills came for it there was no fight at all, the incumbents simply walked away and watched their own average margins improve as a result. That decision appeared to be rational for the math on paper every single time they made it. The minimills then worked their way up into bars and rods, then angle iron, then structural beams, and eventually into sheet steel, and at each stage the integrated producers gave up the low end and climbed toward higher-margin product. Every step up looked like good business on a spreadsheet right until there was nothing left above them to climb to.
Why that history matters in the context of this discussion is that I think we spend too much time talking about what AI models can do at their limits and not enough time talking about what most people actually use them for.
When I think about how someone who is not a developer might use AI at home, the list is surprisingly ordinary. Summarizing a document before a meeting. Finding an inconsistency in a spreadsheet. Drafting an email. Asking questions about a PDF without reading forty pages. Automating a few repetitive tasks around the house. None of these are the sort of benchmark-defining problems that make headlines, and none of them require frontier-level reasoning.
What they do require is a model that is accurate enough to be useful, cheap enough to use constantly, and capable of working with information that never leaves the machine it’s running on. That combination matters because local models already perform many of these tasks reasonably well today.
This is the part of Christensen’s argument that keeps coming back to me. The question is not whether local models can match the best available models. The question is whether they have crossed the threshold from “not good enough” to “good enough” for a meaningful percentage of everyday work. Once that happens, other factors start to matter. Cost matters. Privacy matters. Ownership matters. Convenience matters.
The flat section in the diagram above is meant to represent exactly that point. The capability required for many everyday tasks is not increasing anywhere near as quickly as model capability itself. If local models continue improving while the requirements of those tasks remain largely unchanged, the gap between “best” and “useful” eventually becomes more important than the gap between “best” and “second best.”
The reason I think this is worth paying attention to is that it isn’t just the models that are improving. Three different developments are advancing at the same time.
The first is model quality. Open-weight releases are arriving so quickly that it is becoming difficult to keep up. OpenAI released gpt-oss under a permissive licence, and the pace of open-weight releases has only accelerated since then. Alibaba continues to push out new Qwen generations. Every few months there seems to be another model that is noticeably better than the one before it.
The second is hardware. The reason I bought the HP ZBook Ultra G1a was because unified memory made it possible to run models locally that I would have assumed required a workstation-class NVIDIA GPU with 128 GB of memory. The alternative was a stack of Mac Studios, which remains one of the most capable approaches for local inference, but I was not interested in spending more than $20,000 to experiment with local models. A couple of years ago I would not have expected to load a model this large into a machine that still feels like a laptop. Looking back, the hardware improved faster than I expected.
The third is tooling. Running local models used to feel like a hobby project. Today I can install LM Studio or Ollama in a few minutes and be productive almost immediately. MCP is making it easier for models to interact with tools and external data without every developer inventing their own solution. The model matters, but the surrounding ecosystem matters just as much. A powerful model that cannot access anything useful is still just a conversation. Connect it to files, scripts, and applications and it becomes a tool.
Having made the case above, I want to state that the comparison is not exactly the same because in Christensen’s model, the disruptor is usually a new entrant selling into a segment the incumbent does not care enough to defend. A small company starts at the low end of the market and gradually works its way up while the established players focus on more profitable opportunities. OpenAI released gpt-oss themselves, Alibaba publishes an open Qwen line under a permissive licence while keeping its strongest Max models behind an API. Rather than an outsider attacking the incumbents, some of the frontier labs are participating in both sides of the market at the same time.
One possibility is that the frontier labs understand this risk better than anyone else. Christensen’s books are not obscure. They have been taught in business schools and discussed in boardrooms for decades. If the risk is that open models eventually become good enough for a large percentage of workloads, then you would rather be the company that disrupted itself than the company that got disrupted by someone else.
Viewed through that lens, publishing open models starts to look less like a contradiction and more like a hedge. The frontier labs may be protecting a future in which open models become good enough for a substantial portion of the market. If that happens, the more interesting question becomes where the profits go when model capability becomes easier to obtain. The Innovator’s Solution, the 2003 book Christensen wrote with Michael Raynor, addresses that question more directly than the first book. It introduces what they called the Law of Conservation of Attractive Profits, the observation that when one layer of a technology stack becomes modular and commoditized, profits do not disappear. They move to whichever adjacent layer remains scarce and difficult.
If that pattern applies here, then the scarce resource may not be the model itself. It may be the hardware that can run large models efficiently and the software that turns raw inference into something people can actually use.
Looking back, I think I had already made that bet myself. I did not buy the HP ZBook Ultra G1a because there was a specific model I wanted to run. Models are replaced every few months. I bought it because I wanted the hardware to run whatever models became interesting next. The investment was in the capability, not any particular model.
There are two ways I could be wrong about all of this, and both are plausible enough that I am not treating any of it as settled. The first is that the definition of “good enough” may not stand still. Much of this post assumes that the tasks people care about tomorrow will look broadly similar to the tasks they care about today. Summarizing documents, drafting emails, answering questions, and automating small pieces of work. But the frontier labs are not standing still either. Every improvement in model capability creates opportunities for entirely new ways of working. If the next few years are defined by agents, autonomous workflows, or capabilities that are difficult to imagine today, the bar for what counts as “good enough” may keep rising. Local models may eventually catch up, but the crossover point could remain further away than I think because the target itself keeps moving.
The second is pricing. Hosted inference will eventually become less expensive so while local models may win on privacy, ownership, and control, they may no longer have a compelling economic advantage. Buying a workstation to run models locally is a bet that those benefits, together with the long-term cost savings of self-hosted inference, outweigh the convenience of simply paying for an API. I am not sure that calculation will always work in favour of the machine on the desk.
The comparison that comes to mind is streaming. Netflix succeeded because it made watching content legally easier than pirating it. For a long time the tradeoff was obvious. As prices rose and content became fragmented across multiple services, some people started revisiting alternatives they had previously abandoned. AI could follow a similar pattern. If hosted inference stays inexpensive and convenient, local models may remain a niche interest. If the economics change, or if privacy and ownership become more important, running a model on hardware you control starts to look much more attractive.
Another framework I keep coming back to is Everett Rogers’ diffusion of innovations curve. Most people have probably seen some version of it through Simon Sinek’s talks, where he highlights the gap between early adopters and the broader market. The details of the framework are less important than the underlying idea. New technologies usually start with enthusiasts who are willing to tolerate rough edges, work around limitations, and spend time learning something new. Eventually a point is reached where the product becomes simple enough, useful enough, and accessible enough that people who are not enthusiasts start using it too.
Sinek often points to a threshold of roughly fifteen to eighteen percent adoption where a technology starts moving beyond enthusiasts and into the mainstream. Whether that specific percentage matters is open to debate, but the broader idea resonates with me. There is often a point where a product stops feeling like something only hobbyists care about and starts looking like a practical tool for everyone else.
Local models feel like they are somewhere along that curve today. Running them is far easier than it used to be, which is one of the reasons I think they are worth paying attention to. At the same time, most people are not spending their evenings comparing models, experimenting with quantizations, or deciding how much memory they need in a workstation. They just want something that works. What I cannot tell is whether we are still in the early adopter phase or already beginning the transition to something broader. The difference is obvious in hindsight and remarkably difficult to see while you are living through it.
What I do not know is where the crossover point is. Maybe it is five years away. Maybe it is much closer. Malcolm Gladwell’s tipping point is probably the right mental image for it. A slow accumulation of changes that looks insignificant while it is happening and obvious once it has happened. Model quality improves. Hardware improves. Tooling improves. Each change on its own feels incremental until one day the combined effect is enough to change how people work. Frontier models will stay ahead, and the people doing the hardest work will keep reaching for them, but I think the ceiling stops being the thing that decides what most people run.
What I wish I could do is sit down with the founders and technical leaders at these organizations and ask them where they think this ends up. Not the public answer, but the honest one. Do they see local models eventually becoming good enough for a large percentage of workloads, or do they see a future where the frontier keeps pulling away? That is the part I find most interesting, because I suspect they have a much clearer view of the road ahead than the rest of us do.
I’ve been thinking about this for quite a while now, but I could be completely wrong. If you think I am, I’d genuinely like to hear why. If you’ve seen this pattern play out in another industry, or you think I’m overlooking something obvious, send me a note. I’m far more interested in finding the holes in this argument than convincing myself it’s right.
The following are list is for those who are interested in the books I referenced in this blog post:
