Self-Driving Cars with Duckietown

Overview

Robotics and AI are all around us and promise to revolutionize our daily lives. Autonomous vehicles have a huge potential to impact society in the near future, for example, by making owning private vehicles unnecessary!

Have you ever wondered how autonomous cars actually work?

With this course, you will start from a box of parts and finish with a scaled self-driving car that drives autonomously in your living room. In the process, you will use state-of-the-art approaches, the latest software tools, and real hardware in an engaging hands-on learning experience.

Self-driving cars with Duckietown is a practical introduction to vehicle autonomy. It explores real-world solutions to the theoretical challenges of autonomy, including their translation into algorithms and their deployment in simulation as well as on hardware.

Using modern software architectures built with Python, Robot Operating System (ROS), and Docker, you will appreciate the complementary strengths of classical architectures and modern machine learning-based approaches. The scope of this introductory course is to go from zero to having a self-driving car safely driving in a Duckietown.

This course is presented by Professors and Scientists who are passionate about robotics and accessible education. It uses the Duckietown robotic ecosystem, an open-source platform created at the MIT Computer Science and Artificial Intelligence Laboratory and now used by over 150 universities worldwide.

We support a track for learners to deploy their solutions in a simulation environment, and an additional option for learners that want to engage in the challenging but rewarding, tangible, hands-on learning experience of making the theory come to life in the real world. The hardware track is streamlined through an all-inclusive low-cost Jetson Nano-powered Duckiebot kit, inclusive of city track, available here.

This course is made possible thanks to the support of the Swiss Federal Institute of Technology in Zurich (ETH Zurich), in collaboration with the University of Montreal (Prof. Liam Paull), the Duckietown Foundation, and the Toyota Technological Institute at Chicago (Prof. Matthew Walter).

Syllabus

Module 0: Welcome to the course

Welcome to the course, by Prof. Emilio Frazzoli
You will familiarize yourself with the logistics and navigation interface of the course resources
You will start a learning journey in the world of robot autonomy with Duckietown

Module 1: Introduction to self-driving cars

The potentials and challenges
Levels of autonomy
The vision for autonomous vehicles (AVs)
Activities: You will set up your learning environment, and your Duckiebot, and make your first challenge submission

Module 2: Towards autonomy

Making a robot
Sensorimotor architectures
Stateful architectures
Logical and physical architectures
Application: You will create a reactive "Braitenberg" agent to avoid duckies and see how your agent compares to other submissions

Module 3: Modeling and Control

Introduction to control systems
Representations and models
PID control
Application: You will design an odometry function and PID controller to command your Duckiebot's angular velocity

Module 4: Robot Vision

Introduction to projective geometry
Camera modeling and calibration
Image processing
Application: You will develop image processing techniques necessary for visual lane servoing - controlling your Duckiebot to drive within markings

Module 5: Object Detection

Introduction to neural networks
Convolutional neural networks
One and two-stage object detection
Application: You will train a convolutional neural network (CNN) to detect duckies and integrate your model with ROS to run onboard your Duckiebot and avoid duckies

Module 6: State Estimation and Localization

Bayes filtering framework
Parameterized methods (Kalman filter)
Sampling-based methods (particle and histogram filter)
Application: You will build a state estimation algorithm combining the dynamics and sensor data of your Duckiebot in order to predict its pose as it travels through the world

Module 7: Planning I

Formalization of the planning problem
Application: You will create a collision checker to determine if your Duckiebot is crashing into an obstacle

Module 8: Planning II

Graphs
Graph search algorithms
Application: You will tackle a variety of path-planning challenges and leverage all the skills you've built thus far to navigate your Duckiebot in a variety of simulated environments

Module 9: Learning by Reinforcement

Markov decision processes
Value functions
Policy gradients
Domain randomization
Application: You will explore the capabilities and limitations of reinforcement learning models when applied to real-world robotics tasks such as lane following