A Prophet Of The Web

While doing the research for a future talk, I came across an obscure but impressively prophetic report entitled Accessibility and Integrity of Networked Information Collections that Cliff Lynch wrote for the federal Office of Technology Assessment in 1993, 32 years ago. I say "obscure" because it doesn't appear in Lynch's pre-1997 bibliography.

To give you some idea of the context in which it was written, unless you are over 70, it was more than half your life ago when in November 1989 Tim Berners-Lee's browser first accessed a page from his Web server. It was only about the same time that the first commercial, as opposed to research, Internet Service Providers started with the ARPANET being decommissioned the next year. Two years later, in December of 1991, the Stanford Linear Accelerator Center put up the first US Web page. In 1992 Tim Berners-Lee codified and extended the HTTP protocol he had earlier implemented. It would be another two years before Netscape became the first browser to support HTTPS. It would be two years after that before the ITEF approved HTTP/1.0 in RFC 1945. As you can see, Lynch was writing among the birth-pangs of the Web.

Although Lynch was insufficiently pessimistic, he got a lot of things exactly right. Below the fold I provide four out of many examples.

"Owning" e-books

The basic aspiration of the LOCKSS Program when we started a quarter century ago was to enable libraries to continue their historical mission of collecting, preserving, and providing readers with access to academic journals. In the paper world libraries which subscribed to a journal owned a copy; in the digital world they could only rent access to the publisher's copy. This allowed the oligoply academic publishers to increase their rent extraction from research and education budgets.

LOCKSS provided a cheap way for libraries to collect, preserve and provide access to their own copy of journals. The competing e-journal preservation systems accepted the idea of rental; they provided an alternate place from which access could be rented if it were denied by the publisher.

Similarly, libraries that purchased a paper book owned a copy that they could loan to readers. The transition to e-books meant that they were only able to rent access to the publisher's copy, and over time the terms of this rental grew more and more onerous.

Below the fold I look into a recent effort to mitigate this problem.

Video Game Preservation

Source

I have written fairly often about the problems of preserving video games, most recently last year in in Video Game History. It was based upon Phil Salvador's Survey of the Video Game Reissue Market in the United States. Salvador's main focus was on classic console games but he noted a specific problem with more recent releases:

The largest major platform shutdown in recent memory is the closure of the digital stores for the Nintendo 3DS and Wii U platforms. Nintendo shut down the 3DS and Wii U eShops on March 27, 2023, resulting in the removal of 2,413 digital titles. Although many of these are likely available on other platforms, Video Games Chronicle estimates that over 1,000 of those games were exclusive to those platforms’ digital stores and are no longer available in any form, including first-party Nintendo titles like Dr. Luigi, Dillon’s Rolling Western, Mario & Donkey Kong: Minis on the Move, and Pokémon Rumble U. The closures also affected around 500 historical games reissued by Nintendo through their Virtual Console storefronts, over 300 of which are believed not available on any other platform or service.

Below the fold I discuss recent developments in this area.

Video Game History

Arguably, video games have become the most important entertainment medium. 2022 US video game revenue topped $85B, compared with global movie industry revenue of $76.7B. Game history is thus an essential resource for scholars of culture, but the industry's copyright paranoia means they have little access to it.

Salvador Table 1

In 87% Missing: the Disappearance of Classic Video Games Kelsey Lewin of the Video Game History Foundation describes their recent study in cooperation with the Software Preservation Network, published by Phil Salvador as Survey of the Video Game Reissue Market in the United States. The report's abstract doesn't mince words:

Only 13 percent of classic video games published in the United States are currently in release (n = 1500, ±2.5%, 95% CI). These low numbers are consistent across platform ecosystems and time periods. Troublingly, the reissue rate drops below 3 percent for games released prior to 1985—the foundational era of video games—indicating that the interests of the marketplace may not align with the needs of video game researchers. Our experiences gathering data for this study suggest that these problems will intensify over time due to a low diversity of reissue sources and the long-term volatility of digital game storefronts.

Below the fold I discuss some of the details.

Crypto: My Part In Its Downfall

I was asked to talk about cryptocurrencies to the 49^th Asilomar Microcomputer Workshop. I decided that my talk would take the form of a chronology, so I based the title on a book by the late, great comic Spike Milligan. It became Crypto: My Part In Its Downfall¹.

Below the fold is the text, with links to the sources.

Zittrain On Internet Rot

I spent two decades working on the problem of preserving digital documents, especially those published on the Web, in the LOCKSS Program. So I'm in agreement with the overall argument of Jonathan Zittrain's The Internet Is Rotting, that digital information is evanescent and mutable, and that libraries are no longer fulfilling their mission to be society's memory institutions. He writes:

People tend to overlook the decay of the modern web, when in fact these numbers are extraordinary—they represent a comprehensive breakdown in the chain of custody for facts. Libraries exist, and they still have books in them, but they aren’t stewarding a huge percentage of the information that people are linking to, including within formal, legal documents. No one is. The flexibility of the web—the very feature that makes it work, that had it eclipse CompuServe and other centrally organized networks—diffuses responsibility for this core societal function.

And concludes:

Society can’t understand itself if it can’t be honest with itself, and it can’t be honest with itself if it can only live in the present moment. It’s long overdue to affirm and enact the policies and technologies that will let us see where we’ve been, including and especially where we’ve erred, so we might have a coherent sense of where we are and where we want to go.

In our first paper about LOCKSS, Vicky Reich and I wrote:

Librarians have a well-founded confidence in their ability to provide their readers with access to material published on paper, even if it is centuries old. Preservation is a by-product of the need to scatter copies around to provide access. Librarians have an equally well-founded skepticism about their ability to do the same for material published in electronic form. Preservation is totally at the whim of the publisher.

A subscription to a paper journal provides the library with an archival copy of the content. Subscribing to a Web journal rents access to the publisher’s copy. The publisher may promise "perpetual access", but there is no business model to support the promise. Recent events have demonstrated that major journals may vanish from the Web at a few months notice.

Although I agree with Zittrain's big picture, I have some problems with his details. Below the fold I explain the issues I have with them.

Venture Capital Isn't Working

I was an early employee at three VC-funded startups from the 80s and 90s. All of them IPO-ed and two (Sun Microsystems and Nvidia) made it into the list of the top 100 US companies by market capitalization. So I'm in a good position to appreciate Jeffrey Funk's must-read The Crisis of Venture Capital: Fixing America’s Broken Start-Up System. Funk starts:

Despite all the attention and investment that Silicon Valley’s recent start-ups have received, they have done little but lose money: Uber, Lyft, WeWork, Pinterest, and Snapchat have consistently failed to turn profits, with Uber’s cumulative losses exceeding $25 billion. Perhaps even more notorious are bankrupt and discredited start-ups such as Theranos, Luckin Coffee, and Wirecard, which were plagued with management failures, technical problems, or even outright fraud that auditors failed to notice.

What’s going on? There is no immediately obvious reason why this generation of start-ups should be so financially disastrous. After all, Amazon incurred losses for many years, but eventually grew to become one of the most profitable companies in the world, even as Enron and WorldCom were mired in accounting scandals. So why can’t today’s start-ups also succeed? Are they exceptions, or part of a larger, more systemic problem?

Below the fold, some reflections on Funk's insightful analysis of the "larger, more systemic problem".

RISC vs. CISC

The architectural debate between Complex Instruction Set Computers (CISC) and Reduced Instruction Set Conputers (RISC) really took off in the 1980s:

In particular, two projects at Stanford University and the University of California, Berkeley are most associated with the popularization of this concept. Stanford's MIPS would go on to be commercialized as the successful MIPS architecture, while Berkeley's RISC gave its name to the entire concept and was commercialized as the SPARC.

For the last decade or more the debate has seemed frozen, with the CISC x86 architecture dominating the server and desktop markets, while the RISC ARM architecture dominated the mobile market. But two recent developments are shaking things up. Below the fold, some discussion.

Liability In The Software Supply Chain

Atlantic Council Report On Software Supply Chains was already rather long when I got to the last of the report's recommendations that I wanted to discuss, the one entitled Bring Lawyers, Guns and Money. It proposes imposing liability on actors in the software supply chain, and I wrote:

The fact that software vendors use licensing to disclaim liability for the functioning of their products is at the root of the lack of security in systems. These proposals are plausible but I believe they would either be ineffective or, more likely, actively harmful. There is so much to write about them that they deserve an entire post to themselves.

Below the fold is the post they deserve.

Informational Capitalism

In The Law of Informational Capitalism, Prof. Amy Kapczynski of the Yale Law School reviews two books, Shoshana Zuboff’s The Age of Surveillance Capitalism and Julie Cohen’s Between Truth and Power: The Legal Constructions of Informational Capitalism to document the legal structures on which the FAANGs and other "big tech" companies depend for their power.

Below the fold, some commentary on her fascinating article.

The Death Of Corporate Research Labs

In American innovation through the ages, Jamie Powell wrote:

who hasn’t finished a non-fiction book and thought “Gee, that could have been half the length and just as informative. If that.”

Yet every now and then you read something that provokes the exact opposite feeling. Where all you can do after reading a tweet, or an article, is type the subject into Google and hope there’s more material out there waiting to be read.

So it was with Alphaville this Tuesday afternoon reading a research paper from last year entitled The changing structure of American innovation: Some cautionary remarks for economic growth by Arora, Belenzon, Patacconi and Suh (h/t to KPMG’s Ben Southwood, who highlighted it on Twitter).

The exhaustive work of the Duke University and UEA academics traces the roots of American academia through the golden age of corporate-driven research, which roughly encompasses the postwar period up to Ronald Reagan’s presidency, before its steady decline up to the present day.

Arora et al argue that a cause of the decline in productivity is that:

The past three decades have been marked by a growing division of labor between universities focusing on research and large corporations focusing on development. Knowledge produced by universities is not often in a form that can be readily digested and turned into new goods and services. Small firms and university technology transfer offices cannot fully substitute for corporate research, which had integrated multiple disciplines at the scale required to solve significant technical problems.

As someone with many friends who worked at the legendary corporate research labs of the past, including Bell Labs and Xerox PARC, and who myself worked at Sun Microsystems' research lab, this is personal. Below the fold I add my 2c-worth to Arora et al's extraordinarily interesting article.

Carl Malamud Wins (Mostly)

In Supreme Court rules Georgia can’t put the law behind a paywall Timothy B. Lee writes:

A narrowly divided US Supreme Court on Monday upheld the right to freely share the official law code of Georgia. The state claimed to own the copyright for the Official Code of Georgia Annotated and sued a nonprofit called Public.Resource.Org for publishing it online. Monday's ruling is not only a victory for the open-government group, it's an important precedent that will help secure the right to publish other legally significant public documents.

"Officials empowered to speak with the force of law cannot be the authors of—and therefore cannot copyright—the works they create in the course of their official duties," wrote Chief Justice John Roberts in an opinion that was joined by four other justices on the nine-member court.

Below the fold, commentary on various reports of the decision, and more.

Never Let A Crisis Go To Waste

On March 13^th, an Elsevier press release entitled Elsevier gives full access to its content on its COVID-19 Information Center for PubMed Central and other public health databases to accelerate fight against coronavirus announced:

From today, Elsevier, a global leader in research publishing and information analytics specializing in science and health, is making all its research and data content on its COVID-19 Information Center available to PubMed Central, the archive of biomedical and lifescience at the US. National Institutes of Health’s National Library of Medicine, and other publicly funded repositories globally, such as the WHO COVID database, for as long as needed while the public health emergency is ongoing. This additional access allows researchers to use artificial intelligence to keep up with the rapidly growing body of literature and identify trends as countries around the world address this global health crisis.

Elsevier and the other oligopoly academic publishers have reacted similarly in earlier virus outbreaks. Prof. John Willinsky pounced on this admission that these companies normal restrictive access policies based on copyright ownership slow the progress of science, and thus violate the US Constitution's intellectual property clause:

That Congress shall have Power...To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries.

Below the fold I provide some details of his proposal.

Falling Research Productivity Revisited

Last year, in Falling Research Productivity, I commented on Are Ideas Getting Harder to Find? by Nicholas Bloom et al. Now, The Economist's current issue has a Free Exchange column entitled How to get more innovation bang for the research buck that takes off from the same paper:

In a paper by Nicholas Bloom, Charles Jones and Michael Webb of Stanford University, and John Van Reenen of the Massachusetts Institute of Technology (MIT), the authors note that even as discovery has disappointed, real investment in new ideas has grown by more than 4% per year since the 1930s. Digging into particular targets of research—to increase computer processing power, crop yields and life expectancy—they find that in each case maintaining the pace of innovation takes ever more money and people.

Follow me below the fold for some commentary on a number of the other papers they cite.

Regulating Social Media: Part 1

It has become obvious that self-regulated social media are a threat to pretty much every country's national security. This is intended to be the start of a series looking at the range of suggestions as to how, at least in the United States, it might be done, including (I hope) at least these:

• How to Regulate (and Not Regulate) Social Media by Jack Balkin
• Bipartisan legislation would force Big Tech to allow interoperability with small competitors by Cory Doctorow
• The Good And The Bad Of The ACCESS Act To Force Open APIs On Big Social Media by Mike Masnick
• Testimony by Maciej Cegłowski to the Senate Committee on Banking, Housing, and Urban Affairs for their hearing on Privacy Rights and Data Collection in a Digital Economy
• A Framework for Regulating Competition on the Internet by Ben Thompson.
• A Better Internet Is Waiting for Us by Annalee Newitz

Below the fold, I start with the first of them.

Emulation as a Service

I've written before about the valuable work of the Software Preservation Network (SPN). Now they have released their EaaSI Sandbox, in which you can explore the capabilities of "Emulation as a Service" (EaaS), a topic I discussed in my report Emulation and Virtualization as Preservation Strategies. Below the fold I try EaaSi for the first time.

Carl Malamud's Text Mining Project

For many years now it has been obvious that humans can no longer effectively process the enormous volume of academic publishing. The entire system is overloaded, and its signal-to-noise ratio is degrading. Journals are no longer effective gatekeepers, indeed many are simply fraudulent. Peer review is incapable of preventing fraud, gross errors, false authorship, and duplicative papers; reviewers cannot be expected to have read all the relevant literature.

On the other hand, there is now much research showing that computers can be effective at processing this flood of information. Below the fold I look at a couple of recent developments.

Ten Hot Topics

The topic of scholarly communication has received short shrift here for the last few years. There has been too much to say about other topics, and developments such as Plan S have been exhaustively discussed elsewhere. But I do want to call attention to an extremely valuable review by Jon Tennant and a host of co-authors entitled Ten Hot Topics around Scholarly Publishing.

The authors pose the ten topics as questions, which allows for a scientific experiment. My hypothesis is that all these questions, while strictly not headlines, will nevertheless obey Betteridge's Law of Headlines, in that the answer will be "No". Below the fold, I try to falsify my hypothesis.

It's The Enforcement, Stupid!

Kim Stanley Robinson is a remarkable author. In 1990 he concluded his Wild Shore triptych of novels describing alternate futures for California with Pacific Edge:

Pacific Edge (1990) can be compared to Ernest Callenbach's Ecotopia, and also to Ursula K. Le Guin's The Dispossessed. This book's Californian future is set in the El Modena neighborhood of Orange in 2065. It depicts a realistic utopia as it describes a possible transformation process from our present status, to a more ecologically-focused future.

Why am I writing about this now, nearly three decades later? Follow me below the fold for an explanation.

Selective Amnesia

Last year's series of posts and PNC keynote entitled The Amnesiac Civilization were about the threats to our cultural heritage from inadequate funding of Web archives, and the resulting important content that is never preserved. But content that Web archives do collect and preserve is also under a threat that can be described as selective amnesia. David Bixenspan's When the Internet Archive Forgets makes the important, but often overlooked, point that the Internet Archive isn't an elephant:

On the internet, there are certain institutions we have come to rely on daily to keep truth from becoming nebulous or elastic. Not necessarily in the way that something stupid like Verrit aspired to, but at least in confirming that you aren’t losing your mind, that an old post or article you remember reading did, in fact, actually exist. It can be as fleeting as using Google Cache to grab a quickly deleted tweet, but it can also be as involved as doing a deep dive of a now-dead site’s archive via the Wayback Machine. But what happens when an archive becomes less reliable, and arguably has legitimate reasons to bow to pressure and remove controversial archived material?
...
Over the last few years, there has been a change in how the Wayback Machine is viewed, one inspired by the general political mood. What had long been a useful tool when you came across broken links online is now, more than ever before, seen as an arbiter of the truth and a bulwark against erasing history.

Below the fold, some commentary on the vulnerability of Web history to censorship.