Compression vs. Preservation

An archive is in a hardware refresh cycle and they have asked me to comment on concerns arising because their favored storage hardware uses data compression, which may not be possible to disable even if doing so were a good idea. This is an issue I wrote about two years ago in Threats to stored data.

Because similar concerns keep re-appearing in discussions of digital preservation, I decided this time to discuss it in the same way as Cloud for Preservation, writing a post with a general discussion of the issues without referring to a specific institution. Below the fold, the details.

Betteridge's Law Violation

Erez Zadok points me to Wasim Ahmed Bhat's Is a Data-Capacity Gap Inevitable in Big Data Storage? in IEEE Computer. It is a violation of Betteridge's Law of Headlines because the answer isn't no. But what, exactly, is this gap? Follow me below the fold.

Longer talk at MSST2018

I was invited to give both a longer and a shorter talk at the 34^th International Conference on Massive Storage Systems and Technology at Santa Clara University. Below the fold is the text with links to the sources of the longer talk, which was updated from and entitled The Medium-Term Prospects for Long-Term Storage Systems.

Kai Li's FAST Keynote

Kai Li's keynote at the FAST 2013 conference was entitled Disruptive Innovation: Data Domain Experience. Data Domain was the pioneer of deduplication for backups. I was one of the people Sutter Hill asked to look at Data Domain when they were considering a B-round investment in 2003. I was very impressed, not just with their technology, but more with the way it was packaged as an appliance so that it was very easy to sell. The elevator pitch was "It is a box. You plug it into your network. Backups work better."

I loved Kai's talk. Not just because I had a small investment in the B round, so he made me money, but more because just about everything he said matched experiences I had at Sun or nVIDIA. Below the fold I discuss some of the details.