De-Identifying PHI for Machine Learning: A Practical Guide

In July 2026, researchers from Imperial College London, the Technical University of Munich, and the Hasso Plattner Institute published findings in Nature that undercut a core assumption behind most healthcare AI programs. Using membership inference attacks against seven real-world clinical datasets — medical images, ECGs, and electronic health records — the team trained roughly 200 model versions per dataset and measured, patient by patient, how accurately an attacker could guess whether someone's record was used in training. For patients with rare diseases, unusual presentations, or membership in underrepresented groups, the attack succeeded with near-perfect accuracy, even though the datasets had already been de-identified (Medical Daily). As Imperial's Professor Stefanos Zafeiriou put it, "standard privacy evaluations can overlook serious risks to individual patients, particularly those from underrepresented groups, and these risks may grow as models become more powerful" (Medical Daily).

This isn't an isolated finding. A separate 2026 study found that AI models could predict six demographic attributes — including biological sex, neighborhood, and insurance type — from data that had already met HIPAA's Safe Harbor standard, increasing re-identification risk 37-fold (Censinet). Removing the 18 HIPAA identifiers was never designed to withstand a machine learning model looking for statistical patterns across an entire dataset — it was designed to stop a human from recognizing an obvious name or address.

Safe Harbor vs. Expert Determination: what each one actually buys you

HIPAA gives you two paths to legally de-identified data, and they trade off very differently for AI training pipelines.

Safe Harbor (45 CFR 164.514(b)(2)) requires removing or generalizing 18 specific identifiers — names, geographic data smaller than a state, all dates except year, phone and fax numbers, SSNs, and more — combined with no "actual knowledge" that the remaining data could still identify someone (Oklahoma Bar Journal). It's prescriptive and doesn't require a statistician, which makes it fast — but it's also inflexible. Dates get reduced to year only, ages over 89 get grouped into a single "90+" category, and ZIP codes with populations under 20,000 get zeroed out entirely (Nirmitee).

Expert Determination requires a qualified statistician or scientist to formally certify that the risk of re-identification is "very small." It's more work upfront, but it's far more flexible — you can often retain precise dates, finer geography, or other detail that AI models genuinely need to learn meaningful clinical patterns, as long as the statistical risk assessment supports it (Datavant). For teams training models where temporal sequencing or regional variation matters clinically, Expert Determination is usually the better fit — Safe Harbor's blunt generalization rules can strip out exactly the signal the model needs.

Either path gets your data out of HIPAA's scope once properly executed. Handling PHI under Business Associate Agreements without de-identifying it, by contrast, keeps the full Security Rule and Privacy Rule in force throughout the AI pipeline (Limina).

Why de-identification alone isn't enough for AI training

De-identification was built around a threat model where a human reviewer might recognize a patient. Machine learning changes that threat model in three ways regulators and researchers are still catching up to:

• Membership inference attacks determine whether a specific person's data was used to train a model at all — the exact vulnerability the Imperial College/TUM/HPI team demonstrated at near-perfect accuracy for outlier patients (Medical Daily).

• Model inversion and training-data extraction attacks attempt to reconstruct individual training examples directly from a trained model's outputs or parameters — an attack class already demonstrated successfully against large language models (PMC).

• Latent biometric signal leakage shows up specifically in medical imaging: one study found re-identification rates up to 94% in retinal imaging using foundation models, because facial contours and other identity-adjacent signals survive in the image data even after standard identifiers are stripped from the file metadata (Nature Digital Medicine).

The pattern across all three: risk scales with how distinctive a patient's data is, not with whether their name was removed. Patients with rare diseases, unusual symptom combinations, or membership in underrepresented demographic groups face materially higher exposure than the "average" patient in a dataset — which is also, uncomfortably, often the population researchers most want to study (Medical Daily).

A practical framework for de-identifying PHI before training

1. Classify your use case before choosing a method. If your model needs coarse population-level patterns, Safe Harbor's generalization is probably sufficient. If it needs temporal precision or fine-grained geography to learn real clinical signal, budget for Expert Determination instead.

2. Automate identifier detection in unstructured text. The majority of what healthcare AI systems process is unstructured clinical notes, not clean structured fields — and PHI hides in narrative text in ways rule-based systems miss. NLP-based de-identification models, including ones jointly developed by Philips and AWS for HL7 messages, are specifically built to catch identifiers embedded in free text that field-level scrubbing would never find (AWS).

3. Use surrogation, not deletion, where relationships matter. Replacing PHI with realistic synthetic pseudonyms — rather than deleting it outright — preserves the temporal and relational structure a model needs (visit sequences, referral chains, medication timelines) while still removing the identifying value. Consistent surrogation across a batch keeps these relationships intact (Microsoft Azure Health Data Services).

4. Add differential privacy as a second layer, not a replacement. Differential privacy introduces mathematically bounded noise during training itself, which — unlike static de-identification — degrades predictably and can't be undone by combining the data with outside sources later. As HPI's Professor Georg Kaissis explained, "differential privacy introduces small modifications into the training data that do not affect the model's calculations but make [membership inference attacks] significantly more difficult" (Medical Daily). It also satisfies formal guarantees that traditional de-identification schemes simply don't provide (Nature Machine Intelligence).

5. Consider synthetic data for the highest-risk cohorts. For rare-disease or small-subgroup training data — exactly the populations most exposed under membership inference — training on validated synthetic records instead of real de-identified ones adds a genuine extra layer of separation between the model and any real patient (Nature Digital Medicine).

6. Document everything. Both Safe Harbor and Expert Determination require documentation of the method used, and Expert Determination specifically requires a written statistical justification. Regulators, IRBs, and plaintiffs' counsel will all ask for this paper trail if a model's provenance is ever questioned (Morgan Lewis).

What's changing on the regulatory side

In January 2025, HHS's Office for Civil Rights proposed the first update to the HIPAA Security Rule in over a decade, and it explicitly addresses AI training data: covered entities would need to inventory AI software that handles ePHI and treat AI training datasets as subject to Security Rule controls, not just production clinical systems. Finalization is targeted for 2026 (Physician AI Handbook). Separately, 2025 guidance clarified that using PHI to train AI models may not automatically qualify as treatment, payment, or healthcare operations under existing HIPAA — which means individual patient authorization could be required in cases many teams currently treat as covered by existing consent (Physician AI Handbook).

HIPAA de-identification also isn't the end of the compliance analysis. State consumer-health-data laws, human-subjects research rules, IRB approval or waiver requirements, and contractual data-use restrictions can all still constrain how de-identified data gets used for AI training, validation, or eventual commercialization — even after it's technically left HIPAA's scope (Morgan Lewis).

A quick checklist before your next training run

• Have you documented which de-identification method — Safe Harbor or Expert Determination — applies to this dataset, and why?

• Does your de-identification pipeline catch PHI embedded in unstructured clinical notes, not just structured fields?

• Have you assessed re-identification risk specifically for outlier patients (rare diseases, underrepresented groups), not just average-case risk across the dataset?

• Are you layering differential privacy or synthetic data on top of static de-identification for your highest-risk cohorts?

• Have you checked whether state consumer-health-data laws or IRB requirements apply on top of HIPAA for this specific use case?

The bottom line

De-identification is necessary, but the 2026 research is clear that it's no longer sufficient on its own — especially for the rare-disease and underrepresented patients whose data often matters most for clinical AI. The teams getting this right are treating de-identification as the first layer of a stack, not the whole solution: automated detection that actually catches PHI in free text, surrogation that preserves useful structure, differential privacy or synthetic data for the highest-risk records, and documentation that can survive a regulator's or plaintiff's questions.

C² Data Privacy Platform discovers PHI across structured and unstructured healthcare data — including the clinical notes most de-identification tools miss — and applies consistent, relationship-preserving masking before that data ever reaches a training pipeline. Book a demo to see it run against your own schema.

Sources: Medical Daily — "Patients Can Be Identified from Medical AI Training Data with Near-Perfect Accuracy", Censinet — AI in Data De-Identification: Ethical Issues, Oklahoma Bar Journal — Processing Health Records With AI Under HIPAA, Nirmitee — De-Identifying Healthcare Data for AI Training, Datavant — Where AI Meets Health Data Privacy, Limina — How to Build a HIPAA-Compliant AI Pipeline, PMC — Addressing contemporary threats in anonymised healthcare data, Nature Digital Medicine — Cautious optimism on foundation models in medical imaging, AWS — Philips and AWS Automate PHI de-identification with Machine Learning, Microsoft Azure Health Data Services — De-identification service overview, Nature Machine Intelligence — Reconciling privacy and accuracy in AI for medical imaging, Morgan Lewis — When AI Informs Diagnosis: Privacy, Consent, and Liability Considerations, Physician AI Handbook — Privacy, HIPAA, and Patient Data Security, C² Data Technology.