Glonce

OpenAI's Privacy Filter collapses on non-Latin PII, study finds

researchThe Glonce editorsarXiv🔥72Glonce rating72Impact65Novelty65Relevance95Surprise50Players75Credibility80

Researchers ran the first independent, systematic evaluation of OpenAI's Privacy Filter (OPF), a 1.5B-parameter bidirectional PII detector, testing it zero-shot across 42 synthetic benchmarks covering 22 languages and 5 domains. They compared it against Presidio, XLM-RoBERTa, and GPT-4o.

On PII-annotated benchmarks, OPF scored F1=0.855 on AI4Privacy and 0.464 on SPY medical, ahead of Presidio (0.431 and 0.273 on the same two benchmarks) and XLM-RoBERTa (0.269 and 0.111). But on multilingual named-entity recognition, the picture flips: XLM-RoBERTa beat OPF on all 13 Indic and non-Latin languages tested. GPT-4o came out ahead on medical, legal, and financial PII, averaging F1=0.643 on the SPY benchmarks and 0.527 on Gretel, while OPF led on structured synthetic PII (0.71 average) and on customer-support text (0.60).

The sharpest weakness showed up when PII was embedded in narrative prose rather than structured fields: OPF's F1 on NER benchmarks ranged from 0.04 to 0.57, and its scores collapsed outright on non-Latin scripts, falling to 0.04 on Arabic and 0.03 on Cyrillic. Broken down by PII type, OPF was strongest on structurally regular categories such as email (0.78) and phone numbers (0.76), and weakest on culturally variable ones such as person names (0.40) and addresses (0.49). In customer-support and medical/legal settings, OPF also showed a recall bias: precision ran 0.31 to 0.54 while recall ran 0.70 to 0.85, meaning it caught most PII but flagged many false positives. Global precision across all domains spanned 0.31 to 0.86.

Key facts

  • First independent evaluation of OpenAI's Privacy Filter (OPF), a 1.5B-parameter PII detector, across 42 benchmarks spanning 22 languages and 5 domains.
  • Zero-shot OPF scores F1=0.855 on AI4Privacy and 0.464 on SPY medical, ahead of Presidio (0.431, 0.273) and XLM-RoBERTa (0.269, 0.111) on those benchmarks.
  • XLM-RoBERTa beats OPF on all 13 Indic and non-Latin languages tested in multilingual named-entity recognition.
  • OPF's accuracy collapses on narrative prose in non-Latin scripts: F1 falls to 0.04 on Arabic and 0.03 on Cyrillic.
  • GPT-4o leads on medical, legal, and financial PII (SPY average 0.643, Gretel 0.527); OPF leads on structured synthetic PII (0.71 average) and customer support (0.60).

Why it matters

OpenAI's Privacy Filter is meant to strip personal data before it reaches a model or a downstream system. This is the first outside test of how well it actually does that, and it shows the tool's real-world reliability is far more uneven than a single headline accuracy number would suggest, especially once text stops being neatly structured and stops being in English or another Latin-script language.

Who it affects

Anyone building automated PII redaction or anonymization on top of OPF, particularly for products serving non-English or non-Latin-script users (Arabic, Cyrillic, Indic languages) or for regulated domains like medical, legal, and financial text, where the study found OPF weakest and GPT-4o stronger.

How to use it

The results point to matching the tool to the task rather than treating OPF as a universal filter: it performs well on structured, English-language PII such as emails and phone numbers, and on customer-support text, but the study suggests GPT-4o for medical, legal, or financial narrative PII, and XLM-RoBERTa for non-Latin and Indic-language content.

How solid is it

The evaluation is systematic and broad, 42 synthetic benchmarks across 22 languages and 5 domains, run zero-shot against three named baselines. The abstract does not give the authors' names, institutional affiliations, a publication date, or a description of how the 42 benchmarks were built; only three of them (AI4Privacy, SPY, Gretel) are named individually.

Risks and caveats

In customer-support and medical/legal settings OPF is recall-biased, catching 70 to 85 percent of true PII but with precision as low as 0.31 to 0.54, meaning a substantial share of flagged content is false positives. More seriously, its near-total collapse on non-Latin scripts (Arabic 0.04, Cyrillic 0.03) means deployments that assume uniform PII protection across languages would be leaving real personal data unredacted in narrative text.