Using local LLMs to find sensitive data before masking

The risk with realistic data

Production data is useful because it has the patterns teams need. It has real shapes, edge cases, missing fields, strange notes, and relationships between records.

But using production data directly creates risk. Names, IDs, emails, phone numbers, addresses, account numbers, and free-text notes can leak into lower environments, demos, dashboards, or training jobs.

The pattern: local LLM as a locator

The useful pattern is not asking the model to generate the final dataset. Instead, use a local LLM as a sensitive-data locator inside the trusted environment.

The model reads chunks of files and returns structured findings: what looks sensitive, what type it is, and where it appears. Then a normal application does the replacement with fake-but-consistent data.

That separation matters. The LLM helps with understanding. The application keeps control.

How the flow works

A chunker reads files or exports and sends small sections to the local model. The model returns JSON findings with exact positions. A masking application replaces those values with realistic fake values.

A cache like Redis stores the mapping. If "Jane Lee" becomes "Maria Santos" in one file, the same original value gets the same replacement in the next file. The same rule applies to IDs, emails, phone numbers, account numbers, and other fields.

The audit layer stores what rule ran, when it ran, and how many values were replaced. It does not need to store raw sensitive values.

Where this applies

For healthcare, this can cover Primary Care Encounters, Emergency Room Encounters, Symptom-Driven Encounters, Conditions, Allergies, Medications, Vaccinations, Observations and Vitals, Labs, Procedures, and CarePlans.

The same architecture also works across many other domains:

Why this improves team experience

Engineering, QA, analytics, support, and AI teams can work with useful data without waiting for manual cleanup or asking for risky access. The dataset still looks realistic. Records still connect to each other. But the sensitive values are replaced.

This gives non-production teams useful structure while security teams keep a stronger boundary around real data.

It also supports incident investigation. When a production error depends on a specific data shape, the same pattern can be recreated in a new ephemeral environment with sanitized data. Engineers can reproduce the issue from representative structure and share the environment without exposing real patient, customer, or account data.

The bigger point

Local LLMs do not need to replace traditional data tooling. They can become a layer in the pipeline, especially where unstructured text makes rule-based detection difficult.

For sensitive data workflows, the combination is strong: local LLMs for understanding, deterministic systems for control.