Resources · Choosing the right AI
Is open-source AI secure enough for a regulated industry?
Updated 2026-11-17
Regulators regulate the data, not the model
Nothing in HIPAA, the FTC Safeguards Rule, or the Massachusetts data security regulation names an AI model as allowed or banned. Those rules were written about systems that hold sensitive information, and they ask the same handful of questions about every one of them: who can get in, what gets recorded, what is encrypted, and which outside companies handle the data.
So “is this model compliant” is the wrong question, for an open model and for a paid cloud one. A model is one part. The system around it, and the paperwork that describes that system, is what passes or fails. That is good news for open-source AI, because the system is the part you control.
If you are still deciding whether open models are trustworthy in general, start with is open-source AI safe for your business. This guide picks up where that one stops: what changes when a regulator, an auditor, or a client security questionnaire is in the picture.
What the rules actually ask for
Three rule sets cover most regulated small businesses we talk to in Massachusetts. Read side by side, they overlap more than they differ.
HIPAA Security Rule, for health information. The technical safeguards require that systems holding electronic patient information allow access only to the people and programs granted it, with a unique ID for each user. They require mechanisms that record and examine activity in those systems. They require protection against improper alteration or destruction, a way to verify that a person is who they claim to be, and protection for data moving over a network. Before any of that, the rule requires a risk analysis: a thorough written look at what could go wrong.
FTC Safeguards Rule, for financial information. This one reaches further than most owners expect. It covers many businesses that are not banks, tax preparers and mortgage brokers among them. It requires a named person responsible for the security program, a risk assessment, encryption of customer information at rest and in transit, multi-factor authentication for anyone accessing an information system, logging of what authorized users do, and a written incident response plan. Firms that hold information on fewer than five thousand consumers are excused from a few of the written pieces, the incident response plan among them. The rest still applies.
201 CMR 17.00, for personal information about Massachusetts residents. This applies to nearly every business in the state that runs payroll or keeps a card on file. It requires a written information security program, secure user logins, encryption of personal information crossing public networks or sitting on laptops, monitoring for unauthorized use, and reasonably current firewall protection and security patches.
All three also have a clause about outside companies. HIPAA wants satisfactory assurances from a business associate before it handles patient information. The Safeguards Rule and 201 CMR 17.00 both tell you to pick service providers capable of protecting the data and to require that protection by contract.
Law firms have a parallel duty of confidentiality that works the same way in practice. We cover that separately in private LLMs for law firms.
Where an open model on your own hardware is the stronger answer
The vendor question goes away for the AI step. Every cloud AI tool is an outside company receiving your data, which makes it a service provider under these rules. You have to vet it, sign the right agreement, and keep doing both as its terms change. An open model on a machine in your office sends the prompt to nobody. No outside party receives or stores anything, so there is no vendor to vet for that step.
You can prove it. Block outbound traffic from the machine at the firewall and keep the log. “The data does not leave” stops being a promise in a vendor’s policy and becomes a fact you can show an auditor on a screen.
The model does not change underneath you. Cloud models are updated and retired on the vendor’s schedule. A model file on your drive is the same file next year unless you replace it. If you tested the system on your own cases before putting it to work, those tests still describe what is running.
Retention is yours. Chat history, uploaded documents, and logs live on one machine, kept as long as your policy says and wiped when it says.
One honest limit on all of this. Anyone you hire to maintain that machine who can reach the data on it is a service provider under the same rules. That includes us. The vendor question does not vanish. It shrinks to a party you chose, under a contract you signed.
Where it is weaker, and what you have to build
There is no report to hand over. A large cloud vendor can give your auditor an independent security report and a signed agreement. With your own system, you produce the evidence yourself.
You are the security team for the stack. The model is data. The software that serves it is ordinary software with ordinary vulnerabilities, and it needs patching on a schedule, like anything else on your network.
Logins are often missing out of the box. Many of the free tools for running a model start with no login at all, or one shared password. That fails the unique-user requirement on day one. Named accounts have to be put in front of it.
Logging is not automatic. These rules want a record of who used the system and when. That has to be turned on, kept for a set period, and reviewed by someone. The logs hold regulated data themselves, so they get the same protection.
The strongest models are still in the cloud. An open model on a single machine handles drafting, summarizing, sorting, and answering questions from your documents well. For the hardest reasoning work, the top cloud models are ahead. Our frontier vs private on-prem AI guide covers that trade.
None of these is exotic. It is the same work a careful office already does for its file server. It is still real work, and somebody has to own it.
Know where the model file came from
A model you download is software from outside your company, and the supply chain deserves five minutes of care.
Hugging Face, where most open models are published, documents the main risk plainly: the older pickle format can run arbitrary code when a file is loaded. The safetensors format was built to store model weights safely instead. OWASP lists supply chain among its top ten risks for applications built on language models, for this reason.
The rule that covers it: take the file from the publisher’s own repository, use a data-only format, record the file’s hash and the download date, and keep that record with your security paperwork. CISA and its partner agencies published joint guidance on deploying and operating AI systems built by someone else, which is exactly what a downloaded open model is. It is worth handing to whoever sets up the machine.
What about models from Chinese labs?
Several of the strongest open models come from Chinese labs, Qwen from Alibaba and DeepSeek among them. In a regulated business somebody will ask about that, so have the answer ready.
Using a company’s app or hosted service sends your data to that company’s servers. Running the downloaded model on your own isolated machine does not. A model file is a table of numbers. It cannot open a network connection. The serving software does the networking, you chose that software, and with outbound traffic blocked there is nowhere for anything to go.
What remains is two things. The first is the answers themselves, which can carry blind spots, so test the model on your own work before trusting it. The second is paper: a client contract or an internal policy may restrict where your software comes from, whatever the technical facts are. If one does, pick an open model from an American or European publisher. There are good ones.
The risks that come with any AI
Some risks have nothing to do with open versus closed, and an honest review covers them too.
Instructions hidden in documents. OWASP ranks prompt injection first on its list. Text inside a file or an email can tell the model to do something you did not ask for. It matters most when the AI can take actions and matters far less when all it does is draft.
Too much authority. OWASP calls this excessive agency. An AI that can send, pay, delete, or file by itself is a different risk than one that prepares work for a person. In a regulated office, start with the second kind. Our guide to what AI agents are explains where that line sits.
Wrong answers stated confidently. Every model does this sometimes. The control is a person who reviews the output and signs their name to it, written into the policy.
What an auditor will want to see
Have these on paper before anyone asks. Most of it is a few pages.
- A risk assessment that names the AI system. What data it touches, what could go wrong, and what you did about each item.
- A model inventory. Which model, which version, where it was downloaded from, the file hash, and the date.
- Proof of isolation. The firewall rule that blocks outbound traffic from the machine, and how updates get in.
- Named user accounts. No shared logins, and multi-factor authentication where your rule requires it.
- Activity logs. Who used it and when, how long the logs are kept, and who reviews them.
- Encryption. On the drive, and on the network between the staff computers and the machine.
- A patch record. For the operating system and the serving software.
- Retention and disposal. What is stored, for how long, and how the drive is wiped at the end.
- The human review rule. What the AI may draft, and what a person must approve.
- An incident plan. What happens if the machine is stolen or broken into.
- Contracts. With anyone outside the business who can reach the data.
The NIST AI Risk Management Framework is a free, voluntary structure for this kind of review, and it is a reasonable outline to follow if you have no other. The staff rules belong in your written AI policy. Our small business AI policy template is the starting document, and the AI policy generator guide walks through the data clauses for health, client, and financial information.
Open source on someone else’s servers is a different question
“Open source” describes the license on the model. It says nothing about where your data goes. Plenty of companies rent access to open models over the internet, and using one of them puts you right back in the vendor question: an outside company is receiving your data, and the rules above apply to it in full.
The privacy benefit comes from where the model runs. If a salesperson tells you a product is safe because it is built on open source, ask where the computer is.
When it is the wrong call
- Nobody will own the upkeep. A private AI system that goes unpatched and unwatched is worse than a well-run cloud tool under a signed agreement.
- Your data can go to a vendor under the right agreement, and you need the strongest model. The business tiers of the big cloud tools exist for this. See is your business data safe with ChatGPT.
- The regulated data is not part of the job. Marketing copy and public research do not need a private machine.
- The volume is tiny. A few questions a week does not justify hardware.
One more plain statement. We are engineers. We build the system and the documentation that goes with it. Whether that satisfies your regulator is a call for your compliance officer or counsel, and the list above is what they will ask for.
Where to start
If the data cannot leave the building, the private route is a real option. We priced it in how much it costs to self-host an LLM and covered the machine in what on-prem AI hardware to buy. For a practice-specific view, see AI for medical and dental offices.
Our private AI page covers what we install and support. If you are not sure you need it, the assessment is the fixed-price step that answers that, and sometimes the answer is that a cloud tool under the right agreement is enough.
Questions people ask
Is open-source AI HIPAA compliant?
Do I need a business associate agreement to run an open model on my own server?
Is it safe to use a model from a Chinese lab, like Qwen or DeepSeek, in a regulated business?
Can a downloaded model file carry malware?
Does Massachusetts 201 CMR 17.00 allow AI to touch personal information?
Sources
- 45 CFR 164.312, HIPAA Security Rule technical safeguards (Cornell Legal Information Institute)
- 45 CFR 164.308, HIPAA Security Rule administrative safeguards (Cornell Legal Information Institute)
- 45 CFR 160.103, HIPAA definitions including business associate (Cornell Legal Information Institute)
- 16 CFR 314.4, FTC Safeguards Rule elements (Cornell Legal Information Institute)
- 16 CFR 314.2, FTC Safeguards Rule definitions and examples of financial institutions (Cornell Legal Information Institute)
- 16 CFR 314.6, FTC Safeguards Rule exceptions (Cornell Legal Information Institute)
- 201 CMR 17.03, Duty to Protect and Standards for Protecting Personal Information (Cornell Legal Information Institute)
- 201 CMR 17.04, Computer System Security Requirements (Cornell Legal Information Institute)
- CISA, Joint Guidance on Deploying AI Systems Securely
- Hugging Face, Pickle Scanning
- Hugging Face, Safetensors
- OWASP Top 10 for LLM Applications 2025
- NIST AI Risk Management Framework
Want this figured out for your business?
The assessment tells you where AI is worth it for you, fixed price, and the fee comes off the build.
Get an assessment →