Local LLMs on Phil Wheat

Client Software Stack — The Road Not Taken

Fri, 17 Jul 2026 12:01:54 -0500

I’ve reviewed a lot of packages to find what worked best. Since finding out what didn’t work and why is as important as seeing what does work, here’s a list of some of the tools I tried — and why I don’t use them today. Some of them are really good; they just don’t meet my needs, and some just didn’t click at all. I’ll go through each of them and you can see if they might work for you even if they didn’t work for me.
(For contrast, if you missed what I use that did make the cut from these evaluations, you can find that list at Client Software Stack — What I Run

Client Software Stack — What I Run

Fri, 10 Jul 2026 13:55:45 -0500

We’ve covered the hardware, the OS options, and the Servers. Now to get to where the rubber meets the road - the client software. This is the stuff you actually see day to day as you’re working with your own processes and data, so in effect it is the most important part of the system. It is easy to just log into a site or download an app that is pre configured to run against someone’s hosted service or SaaS offering, but to run against your own resources, takes a bit more care. I’ll go through what I use on a regular basis for my work, and why those particular tools have made it to the top spots in my workflows. These aren’t the only tools, I use, but between them they handle probably 90% of all the LLM based work I do. And hopefully this will either help you find tools that work for you, or at least give you a better idea of what you should be looking for when you choose your client application(s).

The Server Software Stack — What I Run Locally

Mon, 29 Jun 2026 08:31:41 -0500

If you’ve been digging into local LLMs, you’ve probably noticed something: the server landscape is enormous. Everyone has their favorite, things move fast, and the documentation often assumes you already know which path to take. I’ve been running servers on metal — no Docker, no Proxmox — and I want to share what I’ve actually tried, what works, and what I’ve settled on.

You can build an AI lab from the things you find at home.

Fri, 26 Jun 2026 08:31:16 -0500

Well, at least at my home… and with a few strategic purchases. (Hat tip to Bill Sutton - Do It Yourself - Bill Sutton) I’ve worked with lots of various hardware (designed a bit myself), and that’s where I’ll start. Hardware has always inspired some religious wars, and some valid criticism. It’s always tradeoffs and balance. Cost/Features. Capabilities/Battery/Heat. Reliability/Flexibility. All that was in mind as I set up my systems.

The escape from Token Tyranny - Local LLMs.

Wed, 24 Jun 2026 15:08:39 -0500

I’ve been doing neural networks for more than 30 years. I’ve pushed custom classification models into production devices, poked and prodded every approach I could find to turn data into information — and maybe knowledge. So, when LLMs arrived, I had two questions: is attention really all you need? And can I build a stable baseline to test that — one that isn’t going to change under my feet every week?