Overview
Explore deep learning and natural language processing with Apache Spark in this Scala Days conference talk. Dive into the implementation of a Spark-ready Long Short-Term Memory (LSTM) neural network, a complex deep learning model used for challenging NLP tasks like automatic summarization, machine translation, and question answering. Learn about machine learning fundamentals, the intricacies of deep learning, and the challenges of scalability. Discover how to leverage Spark Notebook and Spark Streaming for interactive, real-time visualizations. Follow along as the speaker demonstrates practical examples, including word prediction, auto-generated Shakespeare text, and sentiment classification. Gain insights into data parallelization, back propagation, and the integration of deep learning frameworks like DL4J with Spark. Understand the structure of recurrent neural networks and their applications in NLP. Get hands-on experience with importing models, creating data structures, and performing training and prediction tasks within the JVM environment.
Syllabus
Introduction
Machine Learning
What is Deep Learning
About Scary Mind
Why is it not easy
What our stack looks like
Back Propagation
Data Parallelization
NLP
Recurrent Neural Net
Examples
Word Tyk
Autogenerated Shakespeare
Sentiment Classification
Thank you
Spark Notebook
Cluster
Imports
Google News
DL4J
Logistic Regression
Creating the model
Importing the model
Data structure
Review numbers
Data
Train
Prediction
The JVM
Taught by
Scala Days Conferences