Releasing the MAMMOTH - A Framework for Large-Scale Modular Multilingual NLP Models

Explore the challenges and advancements in large-scale multilingual natural language processing (NLP) models in this 53-minute conference talk by Jörg Tiedemann from the Finnish Center for Artificial Intelligence. Delve into the MAMMOTH framework, designed to address the complexities of supporting multiple languages and improving runtime efficiency in neural language models. Examine the benefits of modular architectures that balance task-specific components with parameter sharing, enabling effective cross-lingual transfer learning while maintaining language-specific modules for efficient inference. Learn about the various approaches to implementing multilingual NLP systems and the importance of systematic comparison. Discover the efforts to optimize scalability in multinode training on large HPC clusters like LUMI. Gain insights into the current stage of research, including initial results, hyper-parameter tuning, optimization of modular architectures, scalability benchmarks, and the ultimate goal of training a large-scale multilingual translation model using massively parallel datasets.