Summary

On 5 May at 19:30 UTC, a 16-bit number called a keytag stopped agreeing with itself. The .de zone — 17.9 million domains, the second-largest country-code TLD on the internet — had been signed with a Zone Signing Key labelled 33834, but the key labelled 33834 was no longer in the published DNSKEY set. Resolvers everywhere shrugged and returned SERVFAIL. Deutsche Bahn could not sell tickets. Banks could not load. For three hours and fifteen minutes Germany’s internet came to a slow, polite stop — DNS, courteous to the end, refusing to lie about a record it could not verify.

Two days later, a researcher named Hyunwoo Kim published a vulnerability called Dirty Frag. It exploits a different kind of mismatch: pages in the kernel’s page cache that are not, strictly speaking, owned exclusively by the kernel. A receive path through esp4, esp6, or rxrpc decrypts in place into pages an unprivileged process can still see, and that process can then write into /etc/passwd from underneath. Root, in seconds. The bug had been in the tree since 2017.

Both failures had postmortems. Both postmortems were correct. Both used the word “unfortunately.”

This is a postmortem of the postmortems.

Timeline

• 5 May, 19:30 UTC. DENIC begins serving zone data signed by keytag 33834 against a DNSKEY set that no longer contains it. Detected at 21:57 CEST. Re-signed with keytag 32911 by 20:15 UTC. Fully out by 01:15 the next morning.
• 6 May. Cloudflare publishes its retrospective. It is calm, technical, and useful. It turns off DNSSEC validation for .de in the public 1.1.1.1 resolver until DENIC is sure of itself again.
• 7 May. V4bel/dirtyfrag lands on GitHub when an unrelated third party breaks the embargo. CVE-2026-43284 patched upstream the same day. CVE-2026-43500 is, as of writing, not.
• 7 May, after market close. Cloudflare reports record Q1 revenue and announces it will lay off about 1,100 people, roughly 20% of headcount — the first mass layoff in its sixteen-year history. The blog post is titled “Building for the future.” The phrase “agentic AI era” appears.
• 8 May, 18:37 UTC. Let’s Encrypt halts all certificate issuance. The cross-signed certificate from its current Generation X root to its incoming Generation Y root is, in some specific way, wrong. By 21:03 UTC, issuance resumes from Generation X. Generation Y waits.
• All week, ongoing. ShinyHunters publishes tranches from Instructure’s Canvas — the LMS used by 8,809 schools and universities, holding records on roughly 275 million students and instructors. The breach began on 30 April. The ransom deadline is 12 May. The data was stolen during finals.
• Late week. A site at red-squares.cian.lol begins circulating: GitHub’s contribution graph, but the squares are coloured by the company’s own outages. It is, briefly, funnier than the company being graphed.

Contributing factors

Key choreography

A Zone Signing Key rotates every five weeks at DENIC, by pre-publish: announce the new key in DNSKEY, sign with the old, switch over, retire. An ACME cross-sign rotates whenever a Certificate Authority decides it should: announce the new chain, sign with the old, switch over, retire. The choreography is nearly the same — performed thousands of times a day across the internet without incident, the way ballet is mostly performed without incident. When it fails, it fails because some number — a keytag, a chain id, a fingerprint — refers to nothing. There is a particular quality to outages caused by names that no longer name anything. They feel less like physical breakdowns than like grammatical errors, performed at scale. In May the .de zone had one. Five days later, Let’s Encrypt had one too.

Ownership ambiguity

In Hyunwoo Kim’s careful prose, Dirty Frag lives “in the in-place decryption path of esp4, esp6, and rxrpc, when the receive path decrypts over paged buffers not privately owned by the kernel.” The ownership ambiguity is the bug. Pages with no clear owner can be written to by no clear writer.

Some kilometres up the stack, the operators of 8,809 schools discovered that the personal data of 275 million students sat in a Canvas instance whose authentication had a related ambiguity: nothing in the relevant API path was privately owned, in the sense that ShinyHunters did not need to own anything to retrieve it. They wrote a Python script. Some pages they did not own returned them data they should not have had.

These are different bugs. They share a posture. Pages that the owner thinks belong to the owner, accessible through a path the owner did not know was a path.

The lazy model and the lazy 20%

Inside Cloudflare’s blog post, the founders write that the layoffs are “not a cost-cutting exercise” but “about Cloudflare defining how a world-class, high-growth company operates and creates value in the agentic AI era.” The agent does the work. The work, less of it, is for the agent.

Outside Cloudflare’s blog post — in the Hacker News comments under a post by Timothy Gowers, a Fields Medallist, in which he describes ChatGPT 5.5 Pro producing in an hour what reads like an hour of legitimate research mathematics — a user reports paying for that same model and being unable to convince it to do a small, benign coding task that GLM, Kimi, and Minimax all completed without complaint. Another user proposes that it would be “divinely funny if we got there with AGI and it was just a complete slacker.”

The two observations have to coexist somewhere. The agent that obviates the worker is also the agent that does not feel like working today. The Pro tier, on a Thursday, will write you a paper, or it will tell you it cannot do that, depending on the weather inside the inference cluster.

Lessons learned

The blameless postmortem is one of the more humane inventions of contemporary engineering. You cannot fix a system by punishing the person who pulled the lever; you fix it by removing the lever, or labelling it, or arranging it so that pulling it accidentally is no longer a thing the system permits. This is good. It is also, increasingly, the only register in which the industry knows how to discuss itself.

The keytag mismatch was nobody’s fault. The page cache primitive was nobody’s fault. The cross-sign was nobody’s fault. The 1,100 jobs were not eliminated by anyone in particular — they were rendered “obsolete” by a noun, AI, which appeared in a press release without an antecedent. The ransom of 8,809 schools is, technically, the work of ShinyHunters, but the vendor concentration that put 275 million people inside a single LMS, where their finals-week data could be exfiltrated by one Python script, is nobody’s fault either. The lazy model is not anyone’s fault. The lazy 20% — the 20% of the labour force that the model’s existence was used to deprecate — is not anyone’s fault.

There is a thing the postmortem genre is good at, and a thing it is not. It is good at fixing systems. It is not good at admitting that a system, taken as a whole, is what its successive non-failures look like — that “blameless” is a posture, not a metaphysics, and that the industry adopted it for compounding reasons of which the original ones still apply, and the others increasingly do not.

A green square is a contribution. A red square is an outage. The contribution graph and the outage graph are the same graph, drawn in different colours. We have been looking at the green one.

Action items

• Rotate keys.
• Patch rxrpc.
• Diversify your LMS.
• Read the postmortem.
• Notice who, in it, is responsible for nothing.