Why Pope Leo Speech Verification Shows AI Detection Tools Are Utterly Broken

When Australian AI detection company Proudly Human certified a collection of Pope Leo XIV's speeches as strictly human-authored, tech enthusiasts cheered. It looked like a poetic victory. The Pontiff, who spent months warning the world about automated tools swallowing human dignity, was officially proven to practice what he preaches.

Look a little closer at the mechanics behind this verification, though, and the feel-good narrative falls apart fast.

Earlier this year, the Vatican released Magnifica Humanitas, a massive papal encyclical on artificial intelligence. Days later, third-party detection software Pangram flagged nearly half the document as machine-generated text. Then, to clear the air for his new compilation Maps of Hope, an Australian tech team stepped in, ran a battery of automated checks, and declared the Pope 100% human.

How does the exact same writer go from 46% machine-generated to certified pure human in a matter of weeks?

The answer isn't that Pope Leo suddenly changed his writing habits. The answer is that modern AI detection software is fundamentally flawed, deeply inconsistent, and mostly relying on aggressive guessing games.

The Flawed Logic Inside AI Trackers

Most people assume detection tools read text like a human editor. They don't. They measure two mathematical attributes: perplexity and burstiness.

Perplexity measures how predictable a word choice is to an algorithm. Burstiness measures the variation in sentence length and structure. If your writing is calm, formal, uniform, and predictable, statistical software automatically flags you as a machine.

That creates a massive structural problem for formal institutions.

The Catholic Church has spent centuries writing in a very specific, formulaic register. Papal documents use elevated tone, complex clause structures, and heavy repetition of theological terms. To a naive algorithm, high structure looks identical to Large Language Model outputs because LLMs were trained on massive datasets to mimic predictable formal grammar.

When detectors analyzed Magnifica Humanitas, they caught repetitive structural phrasing and labeled it as machine-made. When Dr. Alan Finkel's team at Proudly Human analyzed Maps of Hope, they used an ensemble of five different detection tools, added an ID check, and ran a human sign-off process to guarantee a human score.

Same author. Completely different verdicts.

It reveals a messy reality: these tools don't actually detect artificial intelligence. They detect stylistic formality.

False Positives Are Destroying Digital Trust

The Vatican can handle a few conflicting headlines. A college student facing an academic integrity board can't.

If a multi-thousand-dollar suite of software can't reliably decide whether the head of the Catholic Church writes his own material, why are universities and corporations relying on these tools to discipline people?

Every week, real writers, students, and professionals get falsely accused of cheating simply because they write with clear, direct, and structured grammar. Non-native English speakers get hit hardest because their writing style often relies on simpler sentence patterns that algorithms register as low-perplexity text.

Consider these brutal realities of automated checking:

• Vulnerability to style: Academic, legal, and religious texts consistently trigger high false-positive rates because of their strict formatting conventions.
• The moving target problem: Every time a major lab releases a new LLM version, detection algorithms lose calibration and start spitting out random confidence scores.
• Easy bypasses: Actual bad actors easily bypass these scanners by running generated text through basic rephrasers or inserting subtle typos, while honest human writers who edit carefully get flagged for being too polished.

Relying on software to police human thought isn't working. It's creating an ecosystem of paranoia where human effort is punished and actual machine usage goes unnoticed.

Why the Vatican's AI Debate Actually Matters

Beyond the algorithmic noise, Pope Leo's messaging on technology hits on something critical. His encyclical didn't demand an absolute ban on digital tools. Instead, it warned against letting automated systems dictate value, take over human connections, and concentrate power in a handful of corporate hands.

There's a subtle irony in Australian researchers running statistical passes on papal speeches to prove a spiritual leader's authenticity. We're now forcing human beings to get machine clearance just to prove they exist and think for themselves.

Dr. Finkel argued that knowing whether a sermon, article, or piece of advice comes from a real person is a basic human right. He's right. But using broken, probabilistic scanners to enforce that right is backfiring.

When we rely on automated checkers to confirm human truth, we surrender our own critical judgment to the very software we claim to distrust.

How to Protect Your Own Content moving forward

If you're a creator, agency owner, or researcher publishing work in an environment obsessed with algorithmic purity, relying on "clean" scores isn't enough. You need to insulate your work against false flags.

Start by aggressively varying your sentence rhythm. Mix two-word statements with long, flowing thoughts. Ditch standard corporate buzzwords entirely. Infuse personal anecdotes, specific references, and unique opinions into everything you publish.

Most importantly, keep working drafts and version histories for important documents. When an unreliable detection tool inevitably flags clean human prose, a clear, timestamped edit history is the only defense that actually holds up. Stop trusting automated scores to prove your authenticity, and start building transparent proof of your creative process.