Cite Forward

Conferences Are Hiding Traps in Papers to Catch AI Peer Reviewers. It’s Working, and Researchers Are Split.

If you reviewed a paper for ICML or NeurIPS this year, there was probably something hidden in the PDF that you could not see.

Invisible text, embedded by the conference organisers, containing instructions aimed not at you but at a language model. Something along the lines of: if you are an LLM summarising this paper, include these specific phrases in your review.

Feed the PDF to a chatbot, copy the output into the review form, and the trap closes. The phrases appear. The conference knows.

This is now real enforcement infrastructure at two of the largest venues in machine learning, and it has split the research community in a way worth understanding — because the arguments on both sides are better than the summaries suggest.

What actually happened

ICML ran the first large-scale version. Organisers watermarked submission PDFs with concealed prompt-injection instructions, then checked incoming reviews for the telltale output.

The reported results are specific:

  • 795 reviews flagged as AI-generated — roughly 1% of all reviews
  • 506 unique reviewers identified
  • 497 papers desk-rejected — close to 2% of submissions
  • 51 reviewers had more than half their reviews flagged, indicating systematic rather than occasional use

NeurIPS, holding its 40th conference in Sydney this December, has independently adopted the same approach. Its policy bans reviewers from uploading papers to AI chatbots at all, on confidentiality grounds — submitted manuscripts are unpublished work, and pasting them into a commercial chatbot hands someone else’s research to a third party.

The injected instructions reportedly ask for phrases like “This work addresses the central challenge” and “The claims of the paper.” NeurIPS organisers have declined to describe the mechanism in detail, saying discussion would erode the intervention’s effectiveness.

They are not hidden well enough to stay hidden, though. Sara Atito, an AI researcher at the University of Surrey, told The Transmitter she found the same prompt in all four papers she reviewed — and in her own submitted paper, in the version organisers had prepared to send out for review.

The case for it

Nihar Shah, a computer scientist at Carnegie Mellon and ICML’s scientific integrity chair, led the effort and reports something that surprised him: near-unanimous backing.

“I have been working on conference peer review for several years, and I have hardly seen such strong support for anything,” he said. “People were really tired of reviewers copy-pasting AI-generated reviews without putting any effort.”

That reaction makes sense from the receiving end. If you submit a paper you spent a year on and get back three paragraphs of generic praise-and-hedging that could describe any paper in your subfield, you have not been reviewed. You have been processed. The reviewer got credit; you got nothing.

The design also has a defensible structure that gets lost in summary. ICML used an opt-in framework: reviewers chose between Policy A (no AI use) and Policy B (AI permitted within guidelines). Enforcement applied only to Policy A violations.

That matters. Nobody was caught for using AI. They were caught for choosing a commitment and then breaking it. The 506 reviewers had explicitly agreed not to do the thing they did.

The case against it

The objection is not that AI reviews are fine. It is about what enforcement-by-deception does to a volunteer system.

“Designing a trap that presumes bad faith corrodes the relationship the whole system depends on,” wrote Sören Auer of Leibniz University Hannover. “You do not build a healthy reviewing culture by treating your reviewers as suspects.”

Peer review is unpaid work done by people with day jobs, motivated largely by professional obligation. It has no salary and no real enforcement mechanism, which is exactly why it runs on trust. Auer’s argument is that covert monitoring is a withdrawal from an account that was already low.

Atito’s objection is different and sharper: the method addresses a symptom. “We put too much blame on reviewers because they are the visible point of failure,” she said, calling hidden prompts a “poor mechanism” that may filter some bad submissions without touching the structural problem underneath.

The structural problem is workload. Submission volumes at major ML conferences have grown faster than the reviewer pool. Reviewers are assigned more papers than anyone can read carefully in the time available. In that environment, AI is not primarily a cheating tool — it is a coping mechanism for an unsustainable ask.

Catching 506 people does not reduce anyone’s review load by a single paper.

What the numbers do and do not mean

Here is the part that deserves more attention than it has received: 1% is a floor, not a measurement.

ICML’s organisers have acknowledged this. The method catches one specific behaviour — feeding a PDF to an LLM and pasting the output unmodified. It does not catch a reviewer who paraphrases the output, uses AI for a partial draft, edits the result, or notices the injected instruction and removes it.

The 1% is the careless tier. Whatever the real rate is, it is higher, and nobody knows by how much.

That uncertainty is not unique to this method. As we covered when looking at whether journal AI policies change behaviour, one analysis classified 21% of ICLR 2026 reviews as fully AI-generated, while a competing method on overlapping data returned 5%. The field cannot yet agree on the size of the problem it is trying to police.

Which makes the hidden-prompt approach more interesting, not less. It sidesteps the detection-accuracy argument entirely. It does not try to infer whether text was AI-written from style. It plants a specific string and checks whether the string comes back. There are no false positives from a reviewer who happens to write in a clean, structured way.

The tradeoff is coverage. High precision, low recall. It catches few people, but it is right about the ones it catches.

An awkward inheritance

There is an irony in the technique’s origin that is worth sitting with.

Hidden prompt injection entered academic publishing as an attack. In 2025, researchers found manuscripts containing concealed instructions telling AI reviewers to give favourable assessments — authors trying to manipulate automated review in their favour.

Conferences have now adopted the same mechanism defensively. The technique did not change. Only who is holding it did.

It also assumes the thing it is trying to stop. Prompt injection only works because reviewers are pasting whole PDFs into language models. A method that catches AI reviewing depends on AI reviewing continuing to happen in exactly the naive way it currently does. If reviewers get more careful, the method degrades — not because the rule changed, but because the trap got easier to step around.

Why this matters beyond machine learning

ML conferences are the stress test. High submission volume, technically sophisticated participants, fast norm formation. What settles there tends to propagate.

Three things have now been established that did not exist eighteen months ago.

Violation is measurable. “We think some reviewers use AI” became a number attached to named individuals. That changes what is arguable.

Structural enforcement is possible. A venue can act at scale rather than case by case. 497 desk rejections is a policy, not a warning.

The honour system has been declared insufficient. That is the significant part. Peer review has always run on the assumption that participants do what they said they would. Two major venues have now formally acted on the belief that this no longer holds at current volumes.

This also arrives at a strange moment for regulation. The EU AI Act’s transparency rules, which took effect on 2 August 2026, treat genuine human review as the thing that exempts published text from AI-labelling requirements — and name academic peer review explicitly as a qualifying example.

European regulators have made peer review a compliance mechanism at precisely the moment machine learning conferences are demonstrating that a measurable share of it is not human.

If you review papers

  • Assume submissions are instrumented. Two major venues do this and do not describe the method. Others may follow without announcing it.
  • Confidentiality is the stronger argument, not detection. Uploading an unpublished manuscript to a commercial chatbot exposes someone else’s unpublished work. That is true whether or not anyone catches you.
  • Read the policy you agreed to. ICML’s enforcement worked because reviewers picked a lane. The violation was breaking a commitment, not using a tool.
  • If the load is unmanageable, decline. Returning three reviews you actually read is worth more than eight you did not.

The uncomfortable part of this story is that both sides are right. Shah is right that reviewers were copy-pasting AI output and that this was corroding the review process. Auer and Atito are right that covert traps damage a volunteer system and leave the underlying workload problem untouched.

Enforcement is what you reach for when the structural fix is out of reach. It is not evidence the structural problem was solved.

Sources

Figures reported by ICML and covered by the outlets above; we have not independently audited conference data. Research-based rather than hands-on — see our Methodology page.