Big Data Processing on Small Computers

Overview

Explore the intersection of big data and small computers in this 21-minute Java conference talk. Delve into topics such as MapReduce on Hadoop, data centers, RAID arrays, distributed hashmaps, and memory models. Learn about reliable computation techniques, fork-join parallelism, task splitting, and distributed key-value stores. Gain insights into programming models for big data processing, including parallel Java implementations, HDFS, and append-only data structures. Discover how to leverage small computers for handling large-scale data challenges and optimize your approach to big data processing.

Syllabus

Intro
MapReduce on Hadoop
The Market
Data Centers
RAID Array
Distributed Hashmap
Memory Model
Reliable Computation
Fork Join
Fork Joint
Task Split
Task Death
Moron
REST API
Parallel Java
HDFS
Distributed KBstore
Distributed forkjoin
Programming model
Appendonly