Explore a conference talk detailing the development of a highly scalable Machine Learning platform for Machine Translation using Apache Mesos. Discover how to combine microservices architecture with Big Data technologies, including Kafka, HBase, and Hadoop, all running within a Mesos environment. Learn about the challenges faced and solutions implemented in creating this innovative platform. Gain insights into containerized microservices architecture based on Mesos, Docker, and Zookeeper. Understand the evolution of the architecture, lessons learned, and future improvements planned for this cutting-edge Machine Learning platform. Witness a demonstration of the platform's capabilities and discover how it enhances Machine Translation quality through adaptive techniques and Neural MT in the cloud.
How We Built a Highly Scalable Machine Learning Platform Using Apache Mesos
Overview
Syllabus
Intro
Machine Translation quality improvements
Adaptive Machine Translation idea
Update Example - EngFra
Update Example: Translation Model Adaptation
Neural MT in the Cloud
How can we do this?
What we had before?
A new platform
Architecture evolution
Lessons Learned
Cost efficient
Security
Platform high availability
Resource allocation
Releases are not as easy as expected
Investigations become more complex
Independent microservices
Periodically reevaluate assumptions
Future improvements
Demo Time!
Taught by
Linux Foundation
