How the Technology Industry Is Erasing Its Own History
In May 2024, the Pew Research Center published a finding that should unsettle anyone who treats the web as a permanent record — 38 percent of webpages that existed in 2013 had become inaccessible within a decade, and a quarter of all pages created between 2013 and 2023 were already gone by October 2023. The technology industry produced much of that content, and it is also the content’s most efficient destroyer, deleting newsrooms after acquisitions, retiring blogs during platform migrations, and shutting down entire services that held decades of material. Announcements replicated across multiple independent endpoints, through channels such as this resource and the archives that ingest them, survive at measurably higher rates than documents living on a single corporate domain. What follows is an examination of the measured rate of digital decay, the corporate mechanics that accelerate it, the costs it imposes on research and law, and the engineering principles that actually keep a public record alive.
The Decay Rate Has Been Measured, and It Is Steep
The Pew study is the largest recent attempt to quantify what researchers call link rot. The team sampled nearly one million webpages from the Common Crawl repository, roughly ninety thousand per year from 2013 through 2023, and then checked whether each page still resolved. Beyond the headline figures, the details are worse than most engineers assume. Even pages captured in 2021 showed a failure rate of about one in five after only two years. Sixteen percent of dead pages sat on domains that still worked, meaning someone deliberately removed or restructured them; only nine percent died because an entire domain went dark. Deletion, not bankruptcy, is the dominant cause of loss.
The rot propagates into the reference layer of the web. Pew found that 54 percent of Wikipedia articles contain at least one dead link in their references section, and 23 percent of news webpages carry at least one broken link. Legal scholarship documented the same failure earlier: research led by Jonathan Zittrain at Harvard found that roughly half of the URLs cited in United States Supreme Court opinions no longer point to the material the justices relied on. When the citations underpinning encyclopedias, journalism, and constitutional law decay at these rates, the technology industry’s own documentary record — release notes, version announcements, security advisories, founding-era blog posts — decays faster still, because nobody is legally obligated to preserve it.
Why Companies Delete Their Own Records
Digital decay in the corporate world is rarely accidental. It follows from four routine business events, each of which treats historical content as a liability or an inconvenience rather than an asset.
Acquisitions are the most destructive. When a startup is absorbed, its domain typically redirects to the acquirer within months, and everything beneath it — the engineering blog, the changelog, the announcement archive — vanishes unless an archivist happened to crawl it first. Platform migrations run a close second: a new content management system rarely imports a decade of old posts, and URL schemes change without redirects because redirects require someone to care. Rebrands sever whatever survived the migration. And deliberate shutdowns erase content at a scale the other three cannot match. Yahoo deleted GeoCities in 2009, removing tens of millions of pages of early web culture in a single decision. Myspace admitted in 2019 that a botched server migration had destroyed roughly fifty million user-uploaded songs, twelve years of independent music history. Google has retired hundreds of products, and with several of them went forums, hosted code repositories, and documentation that other projects had linked to for years.
The pattern is consistent enough to state as a rule: any document whose only copy sits on infrastructure controlled by a single company has an expected lifespan bounded by that company’s next reorganization.
The Costs Land on People Who Never Made the Decision
The losses are not abstract. Historians of computing now report that reconstructing events from the 2000s is harder than reconstructing events from the 1980s, because the earlier era left paper — manuals, trade magazines, printed press kits — while the later era left URLs that no longer resolve. Security researchers hit the same wall when a vendor’s original advisory disappears and the only remaining description of a vulnerability is second-hand. Journalists citing a company’s own announcement of a product’s capabilities find, years later during litigation or accountability reporting, that the primary source is gone and the company disputes the paraphrase. Academic authors watch peer reviewers reject citations that returned valid pages at submission time.
There is also a newer cost that did not exist five years ago. Retrieval systems behind AI assistants answer questions about companies using whatever documents remain indexed and archived. A firm that has erased its own record has, in effect, delegated its history to third parties — competitors’ blog posts, forum threads, and whatever fragments survive in web archives. The version of events that persists is the version the company had the least control over.
Redundancy Is the Only Strategy That Works
Preservation research from the Internet Archive, the LOCKSS program at Stanford, and the Perma.cc project at Harvard converges on a single principle: content survives in proportion to the number of independently operated copies that exist. Formats and metadata matter, but replication across organizations with different failure modes matters more. Applied to a technology company’s public record, that principle translates into a short set of concrete practices:
- Syndicate announcements at publication time, so that wire networks, aggregators, and news databases hold copies on infrastructure the company does not control and cannot later delete in a redesign.
- Trigger archive captures deliberately, submitting each significant page to the Wayback Machine and similar services on the day it goes live rather than hoping a crawler arrives before the next migration.
- Preserve URL contracts, treating published permalinks as API commitments and mapping every restructuring with permanent redirects instead of letting old paths die.
- Write acquisition and shutdown clauses for content, making the survival of documentation and announcement archives an explicit line item in migration plans, the way databases and DNS already are.
- Keep dated, immutable exports offline, because an internal archive in a standard format is the cheapest insurance against the day every external copy proves incomplete.
None of these practices is technically difficult. What they require is the recognition that a public record is infrastructure, with the same need for redundancy as anything else a company refuses to run on a single server.
An industry that measures uptime in fractions of a percent tolerates a documentary loss rate of nearly forty percent per decade, mostly through its own routine decisions. The web does not forget by accident; it forgets because deletion is the default and preservation is nobody’s job — and the companies that assign that job now will be the only ones whose history is still theirs to tell.