Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

YouTube

Mixture-of-Supernets - Improving Weight-Sharing Supernet Training with Architecture-Routed MoE

AutoML Seminars via YouTube

Overview

Learn about enhancing supernet architecture training through a 35-minute AutoML seminar that introduces the Mixture-of-Supernets formulation. Explore how Mixture-of-Experts (MoE) concepts generate flexible weights for subnetworks, leading to improved Neural Architecture Search (NAS) efficiency and high-quality architectures. Discover the practical applications in constructing efficient BERT and Machine Translation models while meeting user-defined constraints. Join speaker Ganesh Jawahar as he presents this ACL 2024 research that demonstrates significant improvements in retraining time and overall NAS effectiveness.

Syllabus

Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed MoE

Taught by

AutoML Seminars

Reviews

Start your review of Mixture-of-Supernets - Improving Weight-Sharing Supernet Training with Architecture-Routed MoE

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.

Someone learning on their laptop while sitting on the floor.