Customer Analytics and A/B Testing in Python

Overview

Learn how to use Python to create, run, and analyze A/B tests to make proactive business decisions.

The most successful companies today are the ones that know their customers so well that they can anticipate their needs.
Customer analytics and in particular A/B Testing are crucial parts of
leveraging quantitative know-how to help make business decisions that generate value. This course
covers the ins and outs of how to use Python to
analyze customer behavior and business trends as well as how to create, run,
and analyze A/B tests to make proactive, data-driven business decisions.

Syllabus

Key Performance Indicators: Measuring Business Success

This chapter provides a brief introduction to the content that will be covered throughout the course before transitioning into a discussion of Key Performance Indicators or KPIs. You'll learn how to identify and define meaningful KPIs through a combination of critical thinking and leveraging Python tools. These techniques are all presented in a highly practical and generalizable way. Ultimately these topics serve as the core foundation for the A/B testing discussion that follows.

Exploring and Visualizing Customer Behavior

This chapter teaches you how to visualize, manipulate, and explore KPIs as they change over time. Through a variety of examples, you'll learn how to work with datetime objects to calculate metrics per unit time. Then we move to the techniques for how to graph different segments of data, and apply various smoothing functions to reveal hidden trends. Finally we walk through a complete example of how to pinpoint issues through exploratory data analysis of customer data. Throughout this chapter various functions are introduced and explained in a highly generalizable way.

The Design and Application of A/B Testing

In this chapter you will dive fully into A/B testing. You will learn the mathematics and knowledge needed to design and successfully plan an A/B test from determining an experimental unit to finding how large a sample size is needed. Accompanying this will be an introduction to the functions and code needed to calculate the various quantities associated with a statistical test of this type.

Analyzing A/B Testing Results

After running an A/B test, you must analyze the data and then effectively communicate the results. This chapter begins by interleaving the theory of statistical significance and confidence intervals with the tools you need to calculate them yourself from the data. Next we discuss how to effectively visualize and communicate these results. This chapter is the culmination of all the knowledge built over the entire course.