Summary
A preprint on arXiv measures eleven open-weights prompt injection guards on two corpora of legal contracts. The authors report that existing guards struggle, with document-level false positive rates as high as 92.39%. Two guards fall below chance on paired AUC. The paper notes that no false positive rate had been published for these guards on legal contracts, and the snippet cuts off during its comparison with a five-regex baseline.
AI-assisted summary based on the listed source.
What happened
Specialized prompt injection guards are deployed in front of document analysis systems, but no false positive rate has been published for any of them on legal contracts. We measure eleven open-weights guards on two corpora of legal contracts and find that existing guards struggle: document-level false positive...
Why it matters
Teams that place these guards in front of contract review or document analysis systems may be relying on tools whose false positive rates on legal text were not previously published. The results suggest that guard performance on legal contracts should be tested directly before deployment. Watch for the full paper's baseline comparison and for any follow-up evaluations that test these findings.
Signal Intelligence
Signal Strength 95%
Technical label SOURCE-BACKED
Public Interest 18
Category SECURITY
Reader Depth TECHNICAL
Signal Strength reflects source quality, relevance, freshness and evidence. Public Interest helps organize discovery; it is not proof of truth.
Public Interest components
Recognizable Entity Score 0
Practical Impact Score 8
Novelty Interest Score 48
Consequence Score 30
Curiosity Score 0
Shareability Score 18