Most small businesses don't need a private AI model. If you can't send certain information to an outside company at all, you need one. That usually means patient records, client files, or anything you've promised in writing to keep confidential. Even then, the cheaper first move is usually a business-tier cloud plan where the AI company signs an agreement to protect that data. A model on your own hardware is the right answer for a narrower group than the marketing suggests, and it comes with a job attached: You become the one keeping it safe.
A local model is an AI model that runs on a computer you own, so nothing you type leaves the building. With a cloud model, the work happens on the AI company's servers, and your words travel there and back. Most of the decision comes down to which of those two your data is allowed to do.
Do you need a private AI model?
Start with what you'd actually put in. In the four levels of AI use, privacy barely matters at Level 1, where you're asking a chat window general questions. It starts to matter at Level 2, when you hand the model your own material, and it matters a great deal at Levels 3 and 4, where the model reads your systems or acts inside them. A retail shop asking for help wording a spring price letter has nothing to protect. A therapy practice pasting in a client's session notes has a federal law to answer to.
So the useful question is whether anyone has a legal or contractual right to decide where that data goes, and the answer splits most businesses cleanly. A retail shop's sales numbers, a contractor's bid template, and a lodge's booking policies are yours to share with any vendor you trust. Patient information belongs to a regime with rules. Client information in a law office comes with a professional duty. Those two cases are where most small offices run into the question, and they're worth taking one at a time.
One question splits into three options, an ordinary plan, a business plan with a signed agreement, or a local model
Do you hold data you can't send out?
No
an ordinary plan
Yes, with an agreement
a business plan with a signed agreement
Yes, with no agreement possible
a local model
Large companies ask where the data goes and what the contract says before anyone compares answer quality. A small office faces the same two questions with no legal department, so nobody is going to ask them for you.
This is plain-English background, not legal advice. For a decision about your practice, your lawyer, your compliance consultant, or your state bar's ethics line is the right call.
Who has a legal reason to keep data in?
Two groups, mostly. Medical and dental offices answer to HIPAA, which requires a signed business associate agreement before a vendor handles patient information. Law offices answer to confidentiality rules, and accounting firms in professional services carry similar duties. The plan-by-plan details, and what the bar opinions say about running AI in-house, are in whether ChatGPT is HIPAA compliant.
Everyone else, the retail shop, the contractor, and the lodge, can usually share their own business data with a vendor they trust. Their privacy question is a reasonable one about a vendor's terms, and a business plan answers it.
The middle option owners miss
The private-AI conversation usually skips a middle option. If you run a dental office, you don't have to choose between pasting patient notes into a free chat app and building a server in the supply closet. The business and enterprise tiers of the major AI products, signed under a BAA or a confidentiality agreement with the settings turned on, cover what a regulated small office usually needs, and the security work stays with a company that has a team for it.
When a local model makes sense
Three situations, and an office usually knows right away whether it's in one. The vendor won't sign an agreement with a business your size. Your clients, your bar, or a contract say the data never leaves the building, full stop. Or the job is narrow and repetitive, like pulling dates out of intake forms all day, and a mid-size model running on your own machine handles it well enough that a subscription stops making sense.
If you're in one of those, the next two questions are practical. What a local LLM is and how it works covers where the model runs and how it answers with no internet connection. Sizing a local AI model covers the memory, the bits, and the context window, which decide whether the machine you have is enough. The models that make local AI possible are the open-weight ones, and frontier vs. open-source AI explains what you give up and keep by choosing one.
If none of the three fits, the business plan is the answer, and the money you'd have spent on hardware is better spent on an afternoon setting the plan up properly.
Data that already went out is a separate problem. If someone at the front desk pasted a patient's chart into a personal chat account last spring, no new plan reaches back for it, so the first step may be finding out how often that's already happened.