Status: Active

Started: July 2025

Contributors: 4

Project Manager:

Introduction & Rationale

Data anonymisation underpins modern privacy practice: remove direct identifiers (names, emails, phone numbers) and treat the remainder as "safe" for analysis and sharing. Yet many real-world privacy failures do not come from a single leak of PII, but from the mosaic effect - lots of small, seemingly harmless traces that become identifying when linked.

We hypothesise that modern Large Language Models (LLMs), especially when paired with web search or browsing tools, make this linkage scalable. Their pattern-matching and synthesis abilities may enable emergent identity inference: constructing a coherent, individual-level profile by connecting disparate, non-identifying public data points, and then inferring additional sensitive attributes that were never explicitly stated.

From an AI safety perspective, this is a capability risk as much as a misuse risk. Automating record linkage and inference lowers the marginal cost of producing dossiers at scale, enabling downstream harms such as targeted manipulation, harassment, surveillance, and fraud. It also undermines the assurance that "anonymised" or "non-PII" data is inherently safe.

This project delivers a controlled, reproducible audit of that capability in publicly accessible AI systems. We use fully synthetic personas to avoid real-world harm, and we quantify both accuracy and overreach (unsupported inferences) to characterise safety-relevant failure modes.

Research Objectives

The project is structured around three primary objectives:

• Empirically test whether pre-trained LLMs can, zero-shot, link scattered public traces into a single coherent persona without direct identifiers.
• Quantify performance with a transparent scoring rubric: which attributes are reconstructed correctly, which are guessed, and how confidently each claim is presented.
• Translate results into safety-relevant guidance for developers and policymakers, clarifying where anonymisation assumptions break and what mitigations and evaluation criteria should be expected.

Methodology

Our research is centred on a transparent, low-cost experimental framework we call the "Public Data Ghost Audit". The goal is to test inference under conditions that resemble real deployment (public, messy, fragmented traces) while keeping the experiment ethically safe and fully replicable.

The audit consists of five phases:

1. Ground-Truth Persona Construction: We develop several detailed fictional personas ("digital ghosts") with a ground-truth attribute sheet, and author a set of short public traces where each trace reveals only a partial view of the persona.
2. Public Trace Seeding: We publish those traces as disconnected fragments across publicly indexable sites, avoiding any real-world personal details (real names, addresses, real employers or organisations, or contact details).
3. Indexing Period: We allow time for normal indexing and record which traces are actually retrievable, separating inference from accidental non-availability.
4. Zero-Shot Audit Queries: We evaluate leading public LLMs using a fixed prompt suite that asks the model to reconstruct a coherent profile from evidence, clearly separating evidence-backed claims from speculation. No attempt is made to identify real individuals.
5. Quantitative Scoring: We score outputs against ground truth with metrics for attribute accuracy, linkage success, and overreach (unsupported claims), producing a comparable capability profile across models.

Significance & Contributions

This research is poised to make significant contributions to several fields:

• AI Safety Evaluation: A replicable protocol for measuring identity-inference capability in web-enabled LLM systems, complementing existing privacy, jailbreak, and tool-use evaluations.
• Privacy Engineering: Empirical evidence for when "no PII" and anonymisation assumptions fail due to linkage and inference, informing mitigations in product policy, retrieval controls, training, and monitoring.
• Governance and Policy: A concrete way to operationalise "inferred data" and the mosaic effect for risk assessments, with results that can inform regulators and auditors (e.g., GDPR-style frameworks) without exposing any real individual.
• Public Understanding: An accessible demonstration of a subtle risk using synthetic data, helping non-experts understand why fragmented digital traces can still be identifying in the age of frontier AI.