A Superfacility Model for Data-Intensive Science

Overview

Explore a comprehensive conference talk from JuliaCon 2017 on the Superfacility Model for Data-Intensive Science. Delve into the intersection of big data and scientific research, covering topics such as the Internet of Things, automated search and meta-data analysis, and high-performance computing facilities. Learn about the challenges of filtering and de-noising data, mathematical hurdles in energy science, and the application of machine learning in scientific contexts. Examine the growing disparity between available data and computational capabilities, and understand the Department of Energy's exascale Computing Project. Investigate the costs associated with computation and data movement within machines, and explore programming models for exascale computations. Gain insights into specific examples like whole-mantle seismic modeling and genome analysis, and discover the importance of high-level languages like Julia in tackling these complex computational challenges. The talk concludes with discussions on computer architecture specialization and high-performance computing policies, providing a comprehensive overview of the current landscape and future directions in data-intensive scientific research.

Syllabus

Welcome!.
"Big Data" and science.
Science and Internet of Things.
Many science challenges are at the boundary of theory and experiment.
Roadmap of the talk.
Science and search facilities.
Automated search and meta-data analysis.
Past and future high-performance facilities.
Filtering and de-noising data.
Math challenges in energy science data.
Machine learning for science.
Amount of available data grows faster than our computational capabilities.
DOE ECP, Department of Energy exascale Computing Project.
Computation and cost of energy used to perform it.
The most costly thing inside a machine is moving data around.
Data vs. simulations: The irregularity spectrum.
Programming models for exascale computations.
Example: whole-mantle seismic model.
Example: analysis of genome.
Problems with distributed hash tables.
Optimizing algorithm for matrix multiplication.
7 Giants of Data and 7 Dwarfs of Simulation.
Systems for data analysis.
Why high-level languages like Julia are key.
Specialization of computer architectures.
High-Performance Computing Policies.
Acknowledgements.