Completed
Motivation Application Perspective Genome analysis by Our scientist Some stages of data pipeline begin to support bigdata platform
Class Central Classrooms beta
YouTube videos curated by Class Central.
Classroom Contents
Chameleon: Expanding Open-Source Ambari for HPC
Automatically move to the next video in the Classroom when playback concludes
- 1 Chameleon Expanding Open-Source Ambari for HPC
- 2 Motivation (Trend Perspective) HPC and Bigdata is converging
- 3 Motivation Application Perspective Genome analysis by Our scientist Some stages of data pipeline begin to support bigdata platform
- 4 Ambari Overview Apache Ambari is a 100% open source platform for provisioning managing and monitoring Hadoop clusters
- 5 Extension Points for custom service development Ambari view is a plugin that provides a way to connect custom functions to the web UI Ambari stack defines a set of everything needed to define service…
- 6 lustrefs Management Service Lustre Kernel Installation function(LustrekernelUpdater)
- 7 Account Management Service Hadoop does not support strong authentication by default • Hadoop supports Kerberes for that, but, causes performance
- 8 Hadoop-on-Lustre architecture Comparison between HDFS and Lustre
- 9 Related works for Hadoop-on-Lustre Xyrates • MapReduce Job shows theoretical performance gains on an appropriately designed Lustre based HIPC cluster with Infinband network Seagate's lustrels plugin
- 10 2 Hadoop-on-Lustre execution environment Hadoop-on-Lustre execution environment • Works for diskless cluster backed by Lustre • Uses secure container configuration for multi-tenancy
- 11 Motivation Dynamic metrics management is required
- 12 YARN Application Monitoring Service Time-series data monitoring
- 13 3 TimeScaleDB Open-Source time-series database optimized for fast
- 14 3 Data management structure Alter Table
- 15 HPC Resources Monitoring Provides HPC monitoring information through web UI
- 16 Summary HPC and big data convergence makes the distinction between data analytics and computational science's ecosystem disappear. Chameleon is a Bigdata platform operation management system consider…
