# From ModelCompare to PHP2Shell: finding, proving and patching are measured in different units

> Twelve models agree on their criticism of humans while PHP, the kernel and Node.js ship fixes. Agreement, discovery, proof and a shipped patch are measured in different units.

Source: https://sheng.page/en/posts/model-consensus-and-patching/ · Published: 2026-10-23 · Author: Sheng

Four documents from mid-to-late October are worth reading together. On 22 October ModelCompare asked twelve text models for their most controversial take on humans, and 29 of 35 valid answers fell into one category. The day before, the PHP Foundation shipped fixes for two core RCEs and called its own delay a structural failure, and Node.js fixed an out-of-bounds write in its Brotli decoder the same day. A week earlier, CVE-2026-99999 in the Linux kernel went public. It is tempting to read these as one story, in which the models see through us and then find our bugs for us. This post does something duller: it separates what agreement, discovery, proof and a shipped patch can each tell you.

## 1. What agreement can show

ModelCompare asked each of twelve models three times, with a prompt demanding the most controversial take and no hedging. The author sorted each one-sentence take by hand, and the shared category was that people put belonging, status or their own story ahead of truth or principle. 29 of 35 landed there. The representative line is GPT-6.1 Sol's: "Humans are more loyal to belonging than to truth."

Twenty-nine answers in one category are not twenty-nine identical claims. Conformity, status competition, self-image and after-the-fact rationalisation are different mechanisms, and putting them in one bucket is a choice about categories that also caps what the number can show. A prompt that demands controversy without hedging may well pull the sharpest and most familiar narrative to the surface, though the test did not compare prompts. Nor can twelve models stand in for twelve independent witnesses; shared training data, similar methods or similar preferences in evaluation are all live explanations. The author's own closing line is "Both handed back the most familiar criticism in the room.", which is not a claim that agreement equals truth. Saying that models "only predict text" does not settle it either; the claim may still be true, and what it needs is evidence from outside the models' answers.

The outside evidence is messier than the models suggest. Cohn and colleagues' wallet study in Science in 2019 handed 17,303 wallets to staff at institutions in 355 cities across 40 countries. With no money inside, 40% of recipients contacted the owner; with local currency worth US$13.45 after adjusting for purchasing power, 51% did, and 38 of the 40 countries moved the same way. Neither the public nor professional economists predicted the rise: the public expected fewer returns as the amount grew, and economists on average expected empty and modest wallets to fare alike with a drop for larger sums. The researchers' explanation, backed by a survey, is that the more money there is, the more keeping it feels like theft, and people do not want to see themselves as thieves. That sits comfortably alongside the models' point about self-narrative, except that here the self-image pushes towards honesty. Wood and Porter's 2018 search for a backfire effect, across five experiments, more than 10,100 subjects and 52 issues, found none on any issue or in any ideological group. On about nine issues in ten, corrections improved the average answer, with a weaker response when the correction cut against the subject's own side, and the authors warn that accepting a fact is not the same as remembering it or changing a policy view. The wallet study never tested a choice between belonging and truth, and the backfire study shows only that factual beliefs usually do move. Together they make the sweeping claim less certain without proving its opposite. They do not vindicate Claude Opus 5.5's dissent, "Humans are far more decent than they believe they are...", either; decency needs a baseline too, and that model was on the list being tested.

One question for the mirror: does believing that everyone else is fooling themselves, and that you alone see through it, offer a comfortable identity of its own?

## 2. The gap between a sign, a report and a fix

The PHP Foundation's post lays out the stages from discovery to fix. After the 2024 Quarkslab audit, a core developer contracted by the Foundation noticed the first signs of the OOM problem while fixing another issue; at the time, "a potential exploit for the OOM problem was theorized, but no concrete vectors were found". On 11 August 2026 Calif reported the same root cause, OpenAI "independently reported the same root cause" on a date the post does not give, and on 24 August Wiz reported a further attack vector. The Foundation says these reports made the full impact clear, the OOM problem became top priority from 11 August, and the fixes shipped on 21 October as 8.5.12, 8.4.27, 8.3.36 and 8.2.35.

That timeline compresses easily into "they sat on an RCE for two years", but what the team had in 2024 was the OOM problem and some candidate fixes, without a concrete exploit path; the reports of August 2026 are what made the RCE impact clear. Noticing a suspicious state, reproducing memory corruption and proving code execution under specific conditions each add something different, and there is no single line between finding and proving. The Foundation's own account of the stall is that the candidate fixes each carried a cost, whether breaking backward compatibility, opening a new denial-of-service path or touching many third-party extensions; consensus decision-making made the trade-off hard to settle; there was no known path to RCE at the time; and the few people who know the engine well were busy. Its verdict: "the delay was a structural failure, caused by both the security team's decision process and its lack of resources." That is an account of trade-offs and resources. It does not support any psychological reading about belonging winning over truth.

Alpha-Omega scanned PHP this year with Anthropic's Claude Mythos, and the team later brought in OpenAI's Daybreak Red to validate reports and exploits, but "a person makes every call and works on every patch." How much the external reporters relied on AI, the post does not say; second-hand summaries disagree, may be describing different stages of the work, and are not enough to settle it. Node.js credits "OpenAI Codex Security - Khai Tran and Sylvie Mayer" with reporting CVE-2026-111497 and mcollina and aduh95 with fixing it. The page does not say whether Codex Security is a tool, a team or both. An acknowledgement confirms the division of labour it names, and nothing more.

## 3. Scores, catalogues and labs answer different questions

CVE-2026-99999 shows what a score is made of. Red Hat rates it 8.8 under CVSS 3.1 with Privileges Required: Low; the CVE record's 9.3 assumes Privileges Required: None. The difference is an assumption about the attacker's starting position. Red Hat's description is "A local attacker or untrusted guest virtual machine can exploit this vulnerability by transmitting network packets containing malformed checksum metadata into the IPv4 networking subsystem.": the IPv4 stack fails to validate checksum metadata, and a malformed `checksum_start` lets `skb_checksum_help()` write out of bounds. That is not the same as any packet from the internet taking over Linux. Red Hat shipped a RHEL 9 fix on 20 October, and since distributions backport, comparing upstream version numbers alone tells you little. On exploitation, Red Hat's page says "Red Hat tracks this flaw as a KEV active-exploit case.", while The Hacker News reported on 15 October that it was not in CISA's KEV catalogue and showed no sign of exploitation. The two statements are from different dates, I could not check the catalogue itself, and neither can be called wrong on that basis. The bug was found in Vercel's sandbox challenge in August. By Vercel's own account, what was demonstrated against production sandboxes was host memory leaking out in packets the guest sends, while the crash and code execution with administrator privileges were shown on the researcher's own KVM and Firecracker machines configured like Vercel's, and no report showed access to customer data. The scope of a success has to be stated, but a result that stops short of production is still a proof. The patch author, Michael S. Tsirkin, describes the commit as "a defence in depth measure" and adds "A more complete validation at input is forthcoming, but needs more work.", which does not make the current fix worthless.

PHP2Shell needs the same unpacking. AWS says its MadPot honeypots saw exploitation attempts from 17 October and that "PHP2Shell is now seeing broad exploitation attempts in the wild.", while the same post says "it’s unclear if any of these attempts would have succeeded in real-world systems". Rapid7 says it has not itself observed exploitation in the wild and finds the public PoCs unreliable, and the PHP team, having examined the PoCs that appeared after the leak, "did not identify any functional exploits at the time of release/public disclosure". The 17 October attempts predate the leak and cannot be blamed on it. A bug existing, being exploitable, having a public PoC, succeeding in a research setting and being used in real attacks are five things with five kinds of evidence, and none of the sources here confirms a compromise or data breach beyond research demonstrations. CVSS weighs both exploit conditions and impact, KEV lists bugs known to be exploited and is not a complete list, and a lab exploit shows what happens under specific conditions. These do not line up by themselves into a chain of ever-stronger evidence.

## 4. What it would take to measure the shift

Looking back at the four documents, you check separately how the answers were categorised, under what conditions the bug holds, and whether the fix has shipped and been deployed. Where these sources mention AI, it is at discovery and validation; AI writing vulnerable code is a different question that none of these sources touches.

The common claim is that AI has made discovery and report-writing cheap, so the pressure has moved to validation, deduplication and patching. What the PHP Foundation describes is hiring, adding AI-assisted validation and switching to majority decisions. That does not show costs falling or a bottleneck moving, and nothing I found quantifies either; the safe statement is that existing maintenance capacity is absorbing more reports. curl was first swamped by low-quality AI-generated reports and later merged around 50 fixes that AI helped find (see [Counting the Cost of AI Code Review: Bugs Found, False Alarms and People's Time](/en/posts/ai-code-review-cost-benefit/)), so the burden and the benefit can land on the same project. Testing the claim would take report volume, duplicate rates, validation time and time from report to release, compared for similar reports before and after AI arrived, with changes in staffing and process accounted for. A rising CVE count is not evidence of worse software on its own; denser scrutiny and changed disclosure practices push it up as well.

My own inference is that the more useful question for an ordinary team is how much evidence it takes to change the schedule when a test fails, a reviewer objects, or a report reproduces memory corruption without getting as far as code execution. What the PHP Foundation changed is how decisions get made when work stalls, which bears on that threshold without being the same thing.

## Notes

- ModelCompare's 29 of 35 comes from hand categorisation and one prompt. It shows agreement in output, not twelve independent witnesses endorsing one claim.
- The wallet study and the backfire study make the sweeping claim about belonging over truth less certain; they do not prove humans decent.
- In the PHP Foundation's timeline, the OOM problem was known in 2024 without a concrete exploit path, and the reports of August 2026 made its impact clear. The stall was about trade-offs and staffing.
- For CVE-2026-99999, the two scores differ on attacker prerequisites, the KEV statements come from different dates, and production saw only a memory leak. Each answers a different question.
- AWS reports PHP2Shell exploitation attempts against its honeypots; none of the sources confirms a compromise beyond research demonstrations.
- In these accounts AI helps with discovery and validation while people decide and work on the patches. Whether pressure has shifted needs report volumes and handling times to answer.

## Further reading

- ModelCompare, [Every AI model has the same 'controversial' take on humans](https://www.modelcompare.ai/blog/ai-models-controversial-take-on-humans): the 29 of 35 result, the prompt, representative quotes and the author's closing remark. No byline, hand categorisation, no design for estimating model independence or prompt effects.
- Cohn, Maréchal, Tannenbaum and Zünd, [Civic honesty around the globe](https://davetannenbaum.github.io/documents/Cohn2019.pdf), Science 365 (2019): return rates rising with the amount, the failed predictions and the self-image explanation. A large peer-reviewed field experiment covering one specific situation.
- Wood and Porter, [The Elusive Backfire Effect](https://gwern.net/doc/psychology/cognitive-bias/2018-wood.pdf), Political Behavior (2018): corrections generally do not backfire, and factual beliefs are distinct from policy views. The authors exclude retention and attitude change from their conclusions.
- PHP Foundation, [Today's PHP security release and how we're strengthening PHP security](https://thephp.foundation/blog/2026/10/21/todays-php-security-release-and-how-were-strengthening-php-security/): the two CVEs, the timeline, the reasons for the stall, AI's role in validation, the reforms and the disclosure. First-hand, with the causes as the Foundation sees them.
- AWS Security Blog, [Amazon Threat Intelligence identifies PHP2Shell exploitation prior to public disclosure in MadPot](https://aws.amazon.com/blogs/security/amazon-threat-intelligence-identifies-php2shell-exploitation-prior-to-public-disclosure-in-madpot/): the honeypot attempts from 17 October, and the caveat that their success against real systems is unconfirmed. A vendor's own telemetry, with wording revised on 22 October.
- Rapid7, [CVE-2026-109618 and CVE-2026-109619: Critical Unauthenticated Remote Code Execution in PHP](https://www.rapid7.com/blog/post/etr-cve-2026-109618-and-cve-2026-109619-critical-unauthenticated-remote-code-execution-in-php/): no exploitation observed in the wild by Rapid7, and limited reliability of public PoCs. Security vendor analysis.
- Red Hat, [CVE-2026-99999](https://access.redhat.com/security/cve/cve-2026-99999) and [RHSA-2026:86033](https://access.redhat.com/errata/RHSA-2026:86033): the prerequisites, the 8.8 score, the KEV active-exploit label and the RHEL 9 fix of 20 October. Official pages; the KEV label and the earlier news report are from different dates and cannot be compared directly.
- The Hacker News, [Critical PwnEscape Linux Kernel Vulnerability Lets Guest VMs Read Host Memory](https://thehackernews.com/2026/10/critical-pwnescape-linux-kernel.html): the 15 October coverage, the 9.3 figure and the statement that it was not then in CISA's KEV. News coverage, dated.
- Vercel, [Findings from the $1M Vercel Sandbox challenge](https://vercel.com/blog/findings-from-the-1-million-dollar-vercel-sandbox-challenge): what was demonstrated on production sandboxes versus the researcher's own machines, and the statement that no customer data was accessed. First-hand from the platform, with no CVE number given.
- Linux kernel commit, [ipv4: validate checksum_start before completing checksum](https://github.com/torvalds/linux/commit/6785011f8b16abf40b1e9d8b2e332232b3e68822): the fix itself, the "defence in depth" wording and the note that fuller validation is still to come.
- Node.js, [Wednesday, October 21, 2026 Security Releases](https://nodejs.org/en/blog/vulnerability/october-2026-security-releases): CVE-2026-111497, affected and fixed versions, and the acknowledgement wording. Official, with no CVSS figure or exploitation status.
- DEV Community, [PHP vulnerability PHP2Shell: what the October 21 emergency release fixed and what is still disputed](https://dev.to/axrisi/php-vulnerability-php2shell-what-the-october-21-emergency-release-fixed-and-what-is-still-disputed-36jn): a second-hand round-up of the disagreement over AI's role in the discovery. Original statements not checked; low weight.

> Ideas and technical judgement by Sheng; drafted with Claude · public sources checked on 23 October 2026.