Completed
Problem that we encounter with I/O
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Celeste.jl - Petascale Computing in Julia
Automatically move to the next video in the Classroom when playback concludes
- 1 Welcome! Prabhat's part of the talk
- 2 Supercomputers at National Energy Research Supercomputing Center (NERSC)
- 3 Top 10 data analytics problems (link to blog post)
- 4 Celeste is at top of the list for many reasons
- 5 Cori Phase 2, Cray XC-40, the supercomputer that we use
- 6 NERSC big data stack
- 7 Performance and productivity tradeoff?
- 8 Celeste team accomplishment
- 9 The Celest.jl collaboration
- 10 Project goals. Jeff Regier's part of the talk
- 11 Examples of astronomical images that we want to analyze
- 12 Challenge of analyzing faint stars and galaxies
- 13 Our approach using Bayesian inference
- 14 The Celest.jl graphical model
- 15 Scientific color priors
- 16 The likelihood: p(images|catalog)
- 17 Tractable and intractable quantities in Bayesian inference
- 18 Problem with integral over too high dimensional space
- 19 Variational inference
- 20 Julia makes implementation of complicated functions possible
- 21 Results for small data
- 22 Validation of our methods against well-researched sky region Stripe 82
- 23 Numerical optimization scheme and making it parallel
- 24 Cori Phase 2 supercomputer, some technical information
- 25 Performance results of Celest.jl
- 26 Solving large-scale I/O problem. Keno Fischer part of the talk
- 27 Problem that we encounter with I/O
- 28 We take down the network with data traffic that we created
- 29 Standard high-performance guidelines Julia code
- 30 Majority of hot code in Celest.jl compute Hessian matrix of the objective function
- 31 Analyzing for loops in the code
- 32 Question: what is ILP (Instruction Level Parallelism)
- 33 How much ILP is required?
- 34 Putting a large chunk of Julia and assemble code in one big scalar inner loop and vectoring it
- 35 Improving Julia and LLVM to allow the aforementioned vectorization
- 36 Improvements from the previous point are slowly moving into Julia proper
- 37 Using StaticArrays.jl
- 38 Optimization of memory layout to avoid vector shuffling
- 39 Using multiple dispatch to utilize Hessian matrix structure
- 40 Optimization takeaways
- 41 Conclusions
- 42 Closing information
