The Future of the Past - Challenges in Archival Storage

Overview

Explore the challenges and advancements in long-term digital data preservation in this USENIX ATC '20 conference talk. Delve into the evolving techniques for archival storage, including DNA and glass-based technologies, and their implications for preserving information for future generations. Examine the physical and semantic obstacles in maintaining digital data integrity over extended periods, ranging from decades to millennia. Gain insights into the importance of protecting not just bits, but also the ability to interpret data in the future. Learn about media reliability, data integrity, and the economics of archival storage from Professor Ethan L. Miller, an expert in storage systems research. Discover ongoing archival storage projects and the complexities of managing and searching vast amounts of preserved information. Consider the potential threats to our society's knowledge continuity and be inspired to contribute to solving these critical data preservation challenges.

Syllabus

Intro
Why archival storage?
Why do we care about archival storage
Preserving information over the millennia
Media reliability
Data integrity
Preserving meaning over time
Media: performance
Preserving the bits: media
Media: tape
Media: disk
Media: glass (Project Silica) Microsoft-led project to write data in glas • Performance
Media: comparison
Media: futures
Preserving the bits: long-term retention Several key questions about media? - Do bits last forever after they're written? - How likely are bits to "rot"? How much maintenance do they require? Some media are relatively "hands off"
So what's the best option?
The economics of archival storage
What happens when storage stops getting cheaper over time?
Consider cost over time
Reliability: "blast radius"
Preserving the bits: integrity & security
Archival storage projects
Preserving the meaning of data
Managing information: search
Summary: challenges in preserving data for the long- term
Conclusions
Acknowledgments