Explore advanced concepts in machine learning testing and explainability in this comprehensive lecture from the Full Stack Deep Learning Spring 2021 series. Dive into software testing best practices, continuous integration and delivery, and various types of ML system tests including infrastructure, training, functionality, and evaluation. Learn about shadow tests, A/B tests, labeling tests, and expectation tests, as well as challenges in operationalizing ML tests. Gain insights into explainable and interpretable AI, including techniques for using interpretable model families, distilling complex models, understanding feature contributions, and analyzing training data point impacts. Critically examine the need for explainability and consider important caveats in the field of explainable and interpretable AI.
ML Testing & Explainability - Full Stack Deep Learning - Spring 2021
Overview
Syllabus
- What's Wrong With Black-Box Predictions
- Types of Software Tests
- Software Testing Best Practices
- Sofware Testing In Production
- Continuous Integration and Continuous Delivery
- Testing Machine Learning Systems
- Infrastructure Tests
- Training Tests
- Functionality Tests
- Evaluation Tests
- Shadow Tests
- A/B Tests
- Labeling Tests
- Expectation Tests
- Challenges and Solutions Operationalizing ML Tests
- Overview of Explainable and Interpretable AI
- Use An Interpretable Family of Models
- Distill A Complex To An Interpretable One
- Understand The Contribution of Features To The Prediction
- Understand The Contribution of Training Data Points To The Prediction
- Do You Need "Explainability"?
- Caveats For Explainable and Interpretable AI
Taught by
The Full Stack
