Monitoring Machine Learning Concepts

Overview

Learn about the challenges of monitoring machine learning models in production, including data and concept drift, and methods to address model degradation.

To ensure the long-term success of machine learning models, it's essential to understand how to effectively monitor them in production. As machine learning becomes more widely adopted in the business world, post-deployment data science is emerging as an important field. This course covers all the essential concepts related to monitoring machine learning systems in production, in order to maintain business value, reduce the risk of failure, and increase visibility.

Syllabus

What is ML Monitoring

The first chapter will explain why businesses need to monitor your machine learning models in production. You will learn about the ideal monitoring workflow and the steps involved, as well as some of the challenges that monitoring systems can face in production.

Theoretical Concepts of monitoring

In Chapter 2, you'll discover the fundamental importance of performance monitoring in a reliable monitoring system. We'll explore the common challenges faced in real-world production environments, such as the availability of ground truth. By the end of the chapter, you'll know how to handle situations when ground truth data is delayed or absent , using performance estimation algorithms.

Covariate Shift and Concept Drift Detection

Now that you know the basics of covariate shift and concept drift in production, let''s dive a little bit deeper. At the end of this chapter, you will know the different ways to detect and handle them in real-world scenarios.