HIPAA and AI: How Healthcare Organizations Can Protect Patient Data

Artificial intelligence is quickly becoming part of everyday healthcare operations. Clinical teams use AI-assisted tools to draft notes and summarize records. Administrative staff use them to prepare correspondence, analyze documents, and streamline routine tasks. Organizations are also exploring AI for coding, scheduling, patient communication, fraud detection, and workforce training.

But convenience can create risk—especially when employees enter patient information into an AI tool before anyone has reviewed the vendor, approved the workflow, or established clear rules.

HIPAA does not prohibit the use of artificial intelligence. It does, however, require covered entities and business associates to protect protected health information, or PHI, whenever that information is created, received, maintained, or transmitted.

The practical question is therefore not simply, “Is this an AI tool?”

It is: “What information does the tool receive, what happens to that information, and have we implemented the safeguards required to protect it?”

Does HIPAA Apply to Artificial Intelligence?

HIPAA is technology-neutral. The Privacy and Security Rules apply to protected health information regardless of whether an organization handles it through an electronic health record, cloud platform, mobile device, transcription service, or AI application.

The HIPAA Security Rule requires covered entities and business associates to use appropriate administrative, physical, and technical safeguards to protect the confidentiality, integrity, and availability of electronic PHI.

An AI system may fall within that environment when it:

• Receives clinical notes, claims, recordings, images, or patient messages
• Creates summaries or recommendations based on identifiable patient data
• Stores prompts, uploaded files, transcripts, or generated responses
• Connects to an electronic health record or another system containing PHI
• Uses patient information to analyze operations or train a model
• Sends information to other vendors, subprocessors, or external services

The fact that an AI product is new does not create an exception to established HIPAA responsibilities.

A Current Regulatory Note

HHS has proposed significant changes intended to strengthen the HIPAA Security Rule, including more specific requirements for risk analysis, asset inventories, incident response, encryption, multifactor authentication, and security testing.

At the time of publication, these changes remain proposed, and HHS states that the current Security Rule remains in effect.

Healthcare organizations should follow the current rule while monitoring the proposal and preparing for stronger cybersecurity expectations.

What Counts as PHI When Employees Use AI?

PHI includes identifiable information related to an individual’s health, healthcare, or payment for healthcare when it is held or transmitted by a covered entity or business associate.

In an AI workflow, PHI may appear in more places than employees realize. It can exist in:

• Prompts typed into a chatbot
• Uploaded medical records or spreadsheets
• Audio submitted to an AI transcription service
• Images, lab results, or billing documents
• Patient messages copied into a writing assistant
• AI-generated summaries containing patient details
• Logs, saved conversations, or exported reports
• Metadata associated with an uploaded file

Removing a patient’s name may not be enough. Dates, geographic details, medical record numbers, rare diagnoses, and combinations of facts can still identify an individual.

Under HHS guidance, health information is considered de-identified under HIPAA only when it satisfies either the Safe Harbor method or the Expert Determination method. Organizations should review the full HHS de-identification guidance before treating data as no longer protected.

Six Common HIPAA Risks Created by AI Tools

1. Employees Use Unapproved Public AI Tools

One of the most immediate risks is “shadow AI”—employees using applications that have not been reviewed or approved by their organization.

A well-meaning employee might paste a patient email into a public chatbot and ask it to draft a response. Another might upload a clinical note to create a summary. If the service is not authorized to receive PHI, that action could expose patient information outside the organization’s approved environment.

A policy that simply tells employees to “use AI responsibly” is not specific enough. Employees need to know which tools are approved, what information may be entered, and who to contact before trying a new use case.

2. Vendor Data Practices Are Unclear

AI services may retain prompts, use submitted information to improve their models, transfer data to subprocessors, or store information in multiple locations.

Before allowing an AI vendor to handle PHI, an organization should understand:

• What information the vendor collects
• Where and for how long it is retained
• Whether customer data is used to train or improve models
• Which subprocessors can access the information
• How information is encrypted and separated from other customers’ data
• Whether administrators can control access and delete stored information
• How the vendor detects and reports security incidents

A privacy statement or “HIPAA-ready” marketing claim is not a substitute for reviewing the service’s actual contract, security controls, and configuration.

3. A Required Business Associate Agreement Is Missing

An AI vendor that creates, receives, maintains, or transmits PHI on behalf of a covered entity or business associate will generally be acting as a business associate. A written Business Associate Agreement, or BAA, may therefore be required before PHI is shared with the vendor.

A proper BAA establishes permitted uses and disclosures, required safeguards, incident-reporting responsibilities, subcontractor obligations, and procedures for returning or destroying PHI. HHS provides an overview of the required provisions in its Business Associate Agreement guidance.

However, signing a BAA does not automatically make an AI implementation compliant. The healthcare organization must still evaluate the vendor, configure the service appropriately, control access, train users, and manage the risks identified through its own analysis.

For a deeper look at vendor responsibilities, read Evolve’s guide to HIPAA requirements for contractors and vendors.

4. The AI Tool Receives More Information Than It Needs

The HIPAA Privacy Rule generally requires organizations to make reasonable efforts to limit certain uses, disclosures, and requests for PHI to the minimum necessary to accomplish the intended purpose. There are exceptions, including certain treatment-related disclosures, so the rule must be applied to the specific situation.

Even when the minimum necessary standard does not technically apply, limiting an AI system’s data access is a sound security practice.

An appointment-reminder tool, for example, should not receive a patient’s complete medical record if it only needs a name, telephone number, appointment date, and location.

Organizations should consult the HHS minimum necessary guidance and configure each workflow around the smallest appropriate dataset.

5. AI-Generated Content Is Inaccurate or Exposed

Generative AI can produce incomplete, misleading, or incorrect information. It may also repeat sensitive details from the material it was given.

Human review is especially important when AI output will affect:

• Clinical documentation
• Treatment or diagnostic decisions
• Medical coding and billing
• Patient instructions
• Coverage determinations
• Legal or compliance communications

Organizations should define who is responsible for reviewing AI output and how errors are reported and corrected. AI should support accountable human decision-making—not obscure it.

6. The Organization Cannot Reconstruct What Happened

Without appropriate logging and monitoring, an organization may not know who used an AI tool, what records were accessed, what information was submitted, or where the resulting content was sent.

Relevant controls may include:

• Unique user accounts
• Role-based access
• Multifactor authentication
• Activity and access logs
• Alerts for unusual behavior
• Retention and deletion controls
• Periodic access reviews
• Documented incident-response procedures

These controls help organizations detect inappropriate activity, investigate suspected disclosures, and demonstrate that their safeguards are operating as intended.

How to Use AI More Safely in a HIPAA-Regulated Environment

Healthcare organizations can take advantage of AI without treating every experiment as an uncontrolled compliance risk. A structured process makes the difference.

1. Create an Inventory of AI Tools and Use Cases

Start by identifying where AI is already being used. Include official applications, embedded AI features in existing software, free web tools, browser extensions, transcription services, and employee-created accounts.

For each use case, document:

• The business purpose
• The responsible department
• The type of data involved
• Whether the system receives PHI
• Which people and vendors can access the information
• Where information is stored
• How long it is retained

Update the inventory whenever a tool, integration, model, or data flow changes.

2. Conduct and Document a Risk Analysis

The HIPAA Security Rule requires an accurate and thorough assessment of risks and vulnerabilities to electronic PHI. HHS describes risk analysis as a foundational step in selecting appropriate safeguards.

An AI-focused risk analysis should consider:

• Unauthorized disclosure through prompts or uploads
• Excessive user or system access
• Vendor and subprocessor risks
• Data retention and model-training practices
• Weak authentication or missing audit logs
• Inaccurate or manipulated output
• Connections to external systems
• Incident detection and response
• Changes introduced by model or product updates

Review the HHS guidance on HIPAA risk analysis when building this process.

Organizations can also use the voluntary NIST AI Risk Management Framework to supplement their HIPAA compliance program with broader AI governance practices.

3. Review Vendors Before Sharing PHI

Vendor review should involve compliance, privacy, information security, legal, clinical, and operational stakeholders as appropriate.

Ask prospective AI vendors:

• Will you sign a BAA covering this specific service?
• Do you use customer prompts, files, or outputs to train models?
• What data do you retain, and can we control the retention period?
• Which subcontractors or subprocessors can access our information?
• Where is our data stored and processed?
• How is data encrypted in transit and at rest?
• Does the platform support role-based access and multifactor authentication?
• What audit logs and administrative reports are available?
• How quickly will you notify us of a security incident?
• What happens to our data when the agreement ends?
• How are customers notified when the product, model, or data practices change?

The answers should be documented rather than left in a sales presentation or informal email.

4. Configure the Tool Around the Approved Use

An acceptable contract cannot compensate for unsafe configuration.

Limit access by role, disable unnecessary integrations, establish retention settings, require strong authentication, and restrict data exports where possible.

Organizations should also test whether user conversations appear in shared histories or whether sensitive output can be accessed by other departments.

5. Establish a Written AI Acceptable-Use Policy

A practical policy should explain:

• Which AI tools are approved
• Whether approved tools may receive PHI
• Which uses require additional review
• What information employees must never enter
• How users should verify generated content
• Whether AI output may be added to a medical record
• How to report an accidental disclosure or suspicious result
• What disciplinary or corrective process applies to violations
• Who can approve new tools and use cases

Use examples drawn from actual employee workflows. Concrete scenarios are easier to follow than broad principles.

6. Train Employees on AI-Specific Privacy Risks

Technology controls matter, but employees still decide what to type, upload, copy, approve, and send.

HIPAA training for an AI-enabled workforce should teach employees to:

• Recognize PHI in prompts, files, recordings, images, and output
• Use only organization-approved AI tools
• Follow the organization’s rules for handling PHI
• Avoid assuming that removing a name fully de-identifies a record
• Limit information to what is appropriate for the task
• Review AI output before relying on or distributing it
• Report mistakes immediately
• Ask for approval when a use case is unclear

The HIPAA Privacy Rule requires workforce training on applicable privacy policies and procedures, while the Security Rule calls for a security-awareness and training program. Training should be updated when new technologies or practices materially change how employees handle PHI.

7. Monitor Use and Respond to Incidents

Review access logs, permissions, vendor reports, and reported concerns. If PHI may have been accessed, used, or disclosed impermissibly, immediately activate the organization’s incident-response and breach-assessment procedures.

Not every security incident is a breach. However, under the HIPAA Breach Notification Rule, an impermissible use or disclosure of PHI is presumed to be a breach unless the covered entity or business associate demonstrates—through a documented risk assessment—that there is a low probability the PHI was compromised.

That assessment must consider at least four factors:

1. The nature and extent of the PHI involved, including the types of identifiers and likelihood of re-identification
2. The unauthorized person who used the PHI or received the disclosure
3. Whether the PHI was actually acquired or viewed
4. The extent to which the risk to the PHI was mitigated

If the organization cannot demonstrate a low probability of compromise, notification to affected individuals, HHS, and—in some cases—the media may be required.

Employees should know that reporting a mistake quickly is safer than attempting to hide it. Early reporting helps the organization preserve evidence, limit exposure, conduct the required assessment, and meet any applicable notification deadlines.

8. Reassess AI Systems Over Time

AI products change quickly. A vendor may introduce a new model, alter its retention practices, add an integration, or begin using a new subprocessor.

Reassess the system when:

• A new use case is introduced
• The tool gains access to additional data
• A model or major feature changes
• A new vendor or subprocessor becomes involved
• A security incident occurs
• Regulations or organizational policies change

Approval should not be treated as permanent.

A Practical Example: Unsafe Use Versus Controlled Use

Unsafe Scenario

A staff member copies a patient’s message and recent treatment notes into a free public chatbot to draft a response. The organization has not approved the tool, reviewed its data practices, or entered into a BAA with the provider.

The employee may have disclosed PHI to an unauthorized service, and the organization may have little visibility into where the information was stored or how it will be used.

More Controlled Scenario

The organization approves a defined patient-communication workflow using a properly evaluated service. A BAA is in place when required, access is limited by role, retention is configured, activity is logged, and employees are trained to submit only the appropriate information. A qualified team member reviews every response before it is sent.

No AI workflow is risk-free, but the second scenario creates accountability, visibility, and safeguards around the use of patient information.

AI Governance Is a Workforce Issue

Healthcare organizations often approach AI as an information-technology project. In practice, AI governance also depends on the people using the technology.

Employees need more than a list of prohibited tools. They need to understand why patient information must be protected, how approved workflows differ from public applications, and what to do when they make a mistake.

That makes workforce training an essential part of responsible AI adoption. Policies establish the rules; training helps employees apply them during real work.

Build a Stronger HIPAA Training Program

AI is changing how healthcare teams work, but the foundation of HIPAA compliance remains familiar: understand where PHI goes, limit access, evaluate vendors, document decisions, implement appropriate safeguards, and train the workforce.

Evolve e-Learning Solutions provides flexible HIPAA Privacy and Security training for covered entities, business associates, healthcare professionals, administrative teams, and support staff.

Courses can be delivered through Evolve’s learning platform or integrated into an existing LMS in SCORM-compatible formats. Organizations can assign role-appropriate training, track completion, and maintain records that support their compliance program.

Contact Evolve e-Learning to request a course preview or discuss a HIPAA training program for your team.

Frequently Asked Questions About HIPAA and AI

Does HIPAA Prohibit Healthcare Organizations From Using AI?

No. HIPAA does not ban artificial intelligence. Covered entities and business associates must ensure that any use of AI involving PHI complies with applicable privacy, security, and breach-notification requirements.

Can Employees Enter PHI Into a Public Generative AI Tool?

Employees should not enter PHI into an unapproved AI tool. A tool should receive PHI only when the organization has authorized the workflow, evaluated the vendor, established any required BAA, implemented appropriate safeguards, and trained the users.

What Makes an AI Tool HIPAA Compliant?

There is no single product feature that makes every use of an AI tool compliant. Compliance depends on the vendor relationship, contract, system configuration, security controls, authorized purpose, data practices, workforce behavior, and ongoing risk management.

Organizations should be cautious about relying solely on a vendor’s “HIPAA-compliant” marketing claim.

Does an AI Vendor Need a Business Associate Agreement?

An AI vendor will generally need a BAA when it creates, receives, maintains, or transmits PHI on behalf of a covered entity or another business associate. The determination depends on the service and relationship, so organizations should obtain appropriate legal or compliance guidance.

Can De-Identified Patient Data Be Used With AI?

Information that has been properly de-identified under the HIPAA Safe Harbor or Expert Determination method is no longer PHI under the Privacy Rule.

Simply deleting names is not necessarily sufficient. Organizations should also consider contractual, ethical, security, and re-identification risks.

Is a BAA Enough to Make an AI Workflow Compliant?

No. A BAA is an important contractual safeguard, but it does not replace risk analysis, access controls, appropriate configuration, workforce training, monitoring, or compliance with the Privacy and Security Rules.

Is Every AI Security Incident a HIPAA Breach?

No. A security incident is not automatically a breach. However, an impermissible use or disclosure of PHI is generally presumed to be a breach unless a documented assessment demonstrates a low probability that the PHI was compromised or an applicable exception applies.

How Often Should Employees Receive HIPAA and AI Training?

HIPAA requires training appropriate to workforce roles and calls for periodic security updates. Training should also occur when policies, systems, or job responsibilities materially change.

Many organizations provide annual refreshers as a practical way to reinforce expectations, address new risks, and document ongoing awareness.

This article provides general educational information and is not legal advice. Organizations should consult qualified privacy, security, and legal professionals about their particular circumstances.