Benchmarks

We measure instead of claiming. Every number on this page comes from a logged measurement run on the same German gold corpus, and every number carries its measurement date. If we haven't measured it, it isn't here.

Last updated: 2026-08-16 · Diese Seite auf Deutsch →

Free PII engines compared (standalone)

Four freely available detection systems and Saklam Bridge, each standalone on the same corpus with the same metric. We deliberately don't name commercial vendors here — we only compare what anyone can reproduce for free.

* Follow-up measurement with identical corpus and identical metric. Presidio and Kiji each measured in their standard setup — both products allow custom recognizers / regex rules (at which point you own the pattern maintenance yourself).
Intermediate state of the same series: Bridge v0.2.12 scores 467/489 = 95.5 % on the same corpus (2026-07-22). The gain up to v0.2.17 comes from a deterministic additional layer for reference and case numbers, organization names and title chains — no new model, no tuning on the corpus.

In fairness: what the others do well

• GLiNER2-PII delivered the field's best address result (98 % vs. Saklam's 94 %) and was strong on employee and customer IDs.
• The OpenAI filter was strong on person names and secrets — in the first run it beat our then-current version on person names. Those findings went into v0.2.10 the same day (person names at 93 % afterwards).
• Kiji (Apache 2.0) was flawless on email addresses (53/53) and strong on tax IDs (90 %) and IBANs (88 %) — its gaps stem from a label schema that doesn't know German domain identifiers (case numbers, courts, client numbers, registers).
• The shared gap of the free engines is German structural reality: organizations, courts, case numbers, tax IDs, grouped IBANs.
• Saklam has gaps too — 23 of the 489 values (as of 2026-07-17) were deliberately not tuned away on the corpus: context-free short IDs belong in a customer's custom configuration, and rare foreign name forms are a model limit we keep working on. No benchmark overfitting. As of 2026-08-16 there are 7 — all of them person names without a context anchor: bare surnames, user handles, rare foreign name forms.

Ongoing series — latest measurements

We continuously measure newly released models as the NER component inside the Saklam pipeline: same corpus, same regex layer, identical thresholds. Numbers within this series are comparable to each other — but not to the standalone table above: there, e.g. GLiNER2 runs without our pipeline (71.2 %), here as a component inside it (95.1 %). That's why every row carries its measurement date, and why we never mix numbers from different measurement states in one comparison row.

• SauerkrautLM-LFM2.5-GLiNER (Aug 9): just below the reference, with a genuine strength — the model is format-sensitive on structured IDs (customer, employee and client numbers) and catches formats other candidates in the series miss.
• GLiNER2-PII (Aug 7): close to the reference inside the pipeline; a few additional gaps on person names.
• gliner-pii-edge (Aug 7): very small and roughly 7× faster — but with systematic gaps on person names. Speed is cheap, recall isn't.

Methodology

Why the corpus isn't public

The obvious question: "Show us the corpus." The honest answer: a gold corpus full of realistic German PII patterns — names, IBANs, case numbers, employee IDs inside real-looking documents — would itself be exactly the kind of artifact this engine protects against. Publishing it would also let every engine (ours included) optimize against the test set — at which point it measures nothing. Instead: the documents follow the pattern of the examples below (newly constructed, not copied from the corpus), and the live demos at the bottom let anyone measure with their own text.

Sehr geehrte Frau Kollegin Dr. Wenkstern, in der Sache unseres Mandanten
Halil Demirbaş (Az. 4 O 2231/26) übersenden wir den Vergleichsvorschlag.
Die Zahlung über 4.850 € geht auf das Konto DE21 3704 0044 0532 0130 87.
Rückfragen gern an h.demirbas@web.de.

Sehr geehrte Frau Kollegin Dr. [PER_9c41f2ab], in der Sache unseres Mandanten
[PER_5e17d803] (Az. [CAS_7d21e0a3]) übersenden wir den Vergleichsvorschlag.
Die Zahlung über 4.850 € geht auf das Konto [IBA_2f90c6d4].
Rückfragen gern an [EMA_b34a9e01].

Mitarbeiterin Annika Brendel hat zum 31.10. gekündigt. Bitte formuliere
ein wohlwollendes Zwischenzeugnis, Abteilung Vertrieb Süd,
Vorgesetzter: Timo Kaltenbach.

Mitarbeiterin [PER_41d8e07f] hat zum 31.10. gekündigt. Bitte formuliere
ein wohlwollendes Zwischenzeugnis, Abteilung Vertrieb Süd,
Vorgesetzter: [PER_c290ab54].

Rechnung an Brandt & Cie. Steuerberatung, z. Hd. Frau Melisa Yavuz,
USt-IdNr. DE314592807 — bitte prüfe die Positionen auf Plausibilität.

Rechnung an [ORG_88f1c3d2], z. Hd. Frau [PER_67ab04e9],
USt-IdNr. [VAT_d10c58fa] — bitte prüfe die Positionen auf Plausibilität.

All examples newly constructed in the style of the corpus; all names, IBANs and identifiers are invented, placeholder IDs illustrative. Note what does not get masked: amounts, deadlines, the actual matter — that's the payload the model needs to work with. What gets masked is whatever makes the request attributable to a person or company.

Measure it yourself

You don't have to take our word for it. The same detection runs in our live demos right in your browser — paste your own text and inspect the result line by line:

Or right inside the chat window: Saklam browser extension for ChatGPT & Co. →