Run a 104GB Local LLM on a 48GB Mac for Private, Fixed-Cost Automation

The quiet case for owning inference

You do not need a data center to keep your notes, drafts, and internal workflows private. You need a better margin decision. A local model on a Mac can be the durable trick: no per-token billing, no outside prompt trail, and a machine you already own. The catch is that local inference is not a faster version of the cloud. It is a slower, narrower, more predictable version. If you accept that, it can become the calm backbone of a one-person company.

The first-principles question is not whether the model is impressive. It is whether the job is worth the hours. If the task is private, batchable, and prompt-light, local inference can save you from renting intelligence for every small decision. If the task needs fast answers, long context, tool use, or polished structured output, the cloud may still be the better tool. The margin is not in the model. It is in choosing the right job for the machine you already have.

That is why the setup should feel less like a growth hack and more like a boundary. You are not trying to replace every API call. You are trying to own a small, private layer of automation that does not leak your work, does not surprise you with usage fees, and does not require you to become a machine-learning engineer. The goal is a quiet endpoint you can trust for the jobs that fit it.

The local inference margin checklist

Run this checklist before you spend time on the setup. If the answer to any item is no, the local path is probably not the right margin for that job.

1. Use local only when the job is private, batchable, and prompt-light. If the work is public, urgent, or context-heavy, leave it in the cloud.
2. Require Apple Silicon, macOS 14+, and about 110 GB of free disk, with a 512 GB Mac as the realistic minimum.
3. The model is 104 GB on disk at 4-bit and streams weights from SSD.
4. The auto memory cap is 33 GB, and a 128 GB Mac gets the same plan as a 48 GB Mac.
5. An 8,000-token prompt takes about a minute on a 48 GB Mac before the first token appears.
6. Avoid tools, images, JSON-schema output, and logprobs; the API subset returns a clear 400 for them.
7. Expose Ollama or OpenAI chat APIs from a single Swift binary with no Python.
8. Measure first-token time and tokens per second before automating. If the numbers do not fit the workflow, do not automate yet.

Keep the margin after setup

Once the endpoint is running, treat it like a small internal service, not a magic assistant. Give it one job at a time. Summarize a batch of notes. Draft a private memo. Extract a list from a document. Classify an inbox. Rewrite a paragraph in your own voice. The best local automations are boring, repeatable, and easy to inspect. If you have to chase a hallucination, debug a schema, or wait for a tool call, the job may be too large for the local path.

Measure before you automate. Run the same prompt a few times and note how long the first token takes and how fast the answer streams. If the wait is acceptable for a batch job, keep it local. If it makes you check your phone, move the job elsewhere. The point is not to prove that a Mac can run a large model. The point is to build a private, fixed-cost layer that saves you hours without stealing your attention.

Finally, keep the boundary visible. A local LLM is not a reason to stop using better tools. It is a reason to choose more deliberately. When the work is private and the pace is slow, own the inference. When the work is public, fast, or complex, rent the capability. That is the margin: fewer moving parts, fewer leaks, fewer surprises, and a machine that does the quiet work while you do the work that only you can do.

Do not let the setup become a hobby. The machine should serve the business, not the other way around. If a local automation saves you a small amount of time and keeps your work private, it has earned its place. If it creates a second system to babysit, it has not.