Fundamentals of Reinforcement Learning

University of Alberta and Alberta Machine Intelligence Institute via Coursera

Go to class Write review

Details

Go to class

Provider

Coursera
Pricing

Free Online Course (Audit)
Languages

English
Certificate

Paid Certificate Available
Duration & workload

15 hours 9 minutes
Sessions

On-Demand
Level

Intermediate
Subtitles

Arabic, French, Portuguese, Italian, German, Russian, English, Spanish, Thai, Indonesian, Kazakh, Hindi, Swedish, Korean, Greek, Chinese, Ukrainian, Japanese, Polish, Dutch, Turkish, Hungarian, Bengali, Pashto, Urdu, Azerbaijani, Farsi

A Best Course of All Time

Found in

Part of

Reinforcement Learning

3.5

Overview

Reinforcement Learning is a subfield of Machine Learning, but is also a general purpose formalism for automated decision-making and AI. This course introduces you to statistical learning techniques where an agent explicitly takes actions and interacts with the world. Understanding the importance and challenges of learning agents that make decisions is of vital importance today, with more and more companies interested in interactive agents and intelligent decision-making. This course introduces you to the fundamentals of Reinforcement Learning. When you finish this course, you will: - Formalize problems as Markov Decision Processes - Understand basic exploration methods and the exploration/exploitation tradeoff - Understand value functions, as a general-purpose tool for optimal decision-making - Know how to implement dynamic programming as an efficient solution approach to an industrial control problem This course teaches you the key concepts of Reinforcement Learning, underlying classic and modern algorithms in RL. After completing this course, you will be able to start using RL for real problems, where you have or can specify the MDP. This is the first course of the Reinforcement Learning Specialization.

Syllabus

Welcome to the Course!

Welcome to: Fundamentals of Reinforcement Learning, the first course in a four-part specialization on Reinforcement Learning brought to you by the University of Alberta, Onlea, and Coursera. In this pre-course module, you'll be introduced to your instructors, get a flavour of what the course has in store for you, and be given an in-depth roadmap to help make your journey through this specialization as smooth as possible.

An Introduction to Sequential Decision-Making

For the first week of this course, you will learn how to understand the exploration-exploitation trade-off in sequential decision-making, implement incremental algorithms for estimating action-values, and compare the strengths and weaknesses to different algorithms for exploration. For this week’s graded assessment, you will implement and test an epsilon-greedy agent.

Markov Decision Processes

When you’re presented with a problem in industry, the first and most important step is to translate that problem into a Markov Decision Process (MDP). The quality of your solution depends heavily on how well you do this translation. This week, you will learn the definition of MDPs, you will understand goal-directed behavior and how this can be obtained from maximizing scalar rewards, and you will also understand the difference between episodic and continuing tasks. For this week’s graded assessment, you will create three example tasks of your own that fit into the MDP framework.

Value Functions & Bellman Equations

Once the problem is formulated as an MDP, finding the optimal policy is more efficient when using value functions. This week, you will learn the definition of policies and value functions, as well as Bellman equations, which is the key technology that all of our algorithms will use.

Dynamic Programming

This week, you will learn how to compute value functions and optimal policies, assuming you have the MDP model. You will implement dynamic programming to compute value functions and optimal policies and understand the utility of dynamic programming for industrial applications and problems. Further, you will learn about Generalized Policy Iteration as a common template for constructing algorithms that maximize reward. For this week’s graded assessment, you will implement an efficient dynamic programming agent in a simulated industrial control problem.

Taught by

Martha White and Adam White

Reviews

4.9 rating, based on 24 Class Central reviews

4.8 rating at Coursera based on 2782 ratings

Start your review of Fundamentals of Reinforcement Learning

Anonymous

This course has very good outline and appropriate level for RL beginners. The presentation and description in the lectures is simple but very accurate. I can totally follow it without reading the textbook. The workload is small. I finished the entir…

This course has very good outline and appropriate level for RL beginners. The presentation and description in the lectures is simple but very accurate. I can totally follow it without reading the textbook. The workload is small. I finished the entire course in a week. The coding assignment is well organized and insightful as well. However, the quiz is sometimes confusing without enough details from the lecture. But I think it will be fine if you have more time than me and can read the materials from the textbook they told you. The math is harder for beginners than most other ML introductory course, which is unavoidable because that’s the most important part in reinforcement learning. Better to start with some background in probability and stochastic process.
Anonymous

The course is overall very good, and it actually introduces you to Reinforcement Learning from scratch. Lectures are very clear, quizzes are challenging and the course relies on a text book, provided when you enroll. The only weak point, but not a serious issue, is that most of the lectures do not add content to what is in the book. Since studying the book is in fact mandatory, they could have used the lectures to better explain some concepts, assuming people read the book. Sometimes they do, but not so often.
Luiz Cunha

Fantastic Course. That's the RL MOOC I have been waiting for so long. No surprise it is from Students of RL guru R. Sutton at Uni of Alberta. Very clearly and simply explained. Exercise and Test difficulty spot on. Wouldn't change a iota from this Course. Can't wait to do the rest of this RL specialization
Stewart Adamson

This is a great course on Reinforcement Learning (RL) and I thoroughly recommend it. This is the first course in the four course Reinforcement Learning specialization from the Alberta Machine Intelligence Institute (AMII) at University of Alberta. T…

This is a great course on Reinforcement Learning (RL) and I thoroughly recommend it. This is the first course in the four course Reinforcement Learning specialization from the Alberta Machine Intelligence Institute (AMII) at University of Alberta. The course introduces the key concepts and goals of RL and follows the standard text on the subject, (Sutton & Barto 2018), very closely. AMII is the "home" of Rich Sutton and Andy Barto the authors of Reinforcement Learning an Introduction which is used throughout the specialization. It is available as a free PDF as part of the course material and each week of the course starts with a reading exercise from the book covering the algorithms to be covered in that week's videos, quizzes and assignments. Sutton & Barto 2018 is also used by Stanford and DeepMind in their RL courses. In the final week of the course, you get to implement a Dynamic Programming algorithm in a Jupyter notebook in Python as a programming assignment. You can check out the syllabus on Coursera.org for details of this course and the other courses in the specialization.
Anonymous

First of all this course is based on an excellent book, "Reinforcement Learning, An Introduction - 2nd edition" by Sutton and Barto. The text is a clearly written with graphs and illustrations. I especially like the bibliographical and historical re…

First of all this course is based on an excellent book, "Reinforcement Learning, An Introduction - 2nd edition" by Sutton and Barto. The text is a clearly written with graphs and illustrations. I especially like the bibliographical and historical remarks to get context since I'm a contextual learner.

Ok, so I'm not a huge fan of multiple selection questions (the ones with the square boxes). But it is what it is.

The Jupyter notebooks for weeks 1 and 4 give you a feel of research quality results with just simple coding required. For an introductory course it's a good balance, but I suspect the programming workload to increase progressively in later courses.

My recommendation is to take this course even if machine learning isn't your speciality. You're sure to find plenty of applications for the tools you'll learn here.
Prateek Chandra Jha

This course comes straight from the capital of reinforcement learning and that is Alberta Machine Intelligence Institute which is co-headed by Rich Sutton himself. You can be sure of the fact that this course is free of all fat and nutritious. The c…

This course comes straight from the capital of reinforcement learning and that is Alberta Machine Intelligence Institute which is co-headed by Rich Sutton himself. You can be sure of the fact that this course is free of all fat and nutritious. The course gives you the fundamental understanding of Markov Decision Processes and approximate Dynamic Programming required to build expert RL systems in course 2 and 3 of the specialization. I recommend everyone who has ever thought of breaking into RL come and take this course. It's hands-on with explanation of all key conceptual paradigms in Python and gives you that much needed head start you've been looking for so long after you heard about AlphaZero, AlphaGo and other recent advancements in AI.

Regards & Thanks

Prateek.
Anonymous

I enjoyed taking this course and feel that it has expanded my tool-kit. The course was well constructed with reading assignments from the book followed by relatively short videos and assignments. I spent around 4-5 hours a week and most of the time was on the reading assignment. I enjoyed the programming assignments though it would have been nice if they had provided lesser scaffolding and asked us to write more code. The quizzes were ok and some of the questions were rather ambiguous and could have been improved with better wording. Overall I had a great learning experience and look forward to completing the series.
Matteo Hee

This course formed an exceptional introduction to RL as a field while only requiring a rudimentary background in Python. I used this course to grow my understanding of Python (which is a new programming language for me) as well as introduce some of the problems of Reinforcement Learning.

The "guest lecturers" in each module provided an interesting breadth of perspective on the material, while the primary instructors were clear, articulate, and engaging.
Anonymous

Fundamentals of Reinforcement Learning is one of the best Online Courses I did on Coursera. I like that the course is based on a text book (Reinforcement Learning by Sutton), so you can really dig into the theory. Also the exercises are very helpful and ambitious which I like. I haven't found much advanced online courses which are so well explained like this one.
Andrei Petrovskii

The course covers basics of reinforcement learning. It presents main ideas of the topic. The course instructors are very clear. Understanding of the course does not require neither advanced skills in mathematics nor in programming. However the presented concepts are useful, if you are beginner and know nothing about reinforcement learning
Anonymous

This is a great course which require a genuine commitment. The teachers have made a great effort to make you understand the Bellman equation in details! The quizzes and the coding exercises have an appropriate level of difficulty. You really have to take your pencil and your paper before answering.
Anonymous

Really good course. It's really interesting and explains really well the basic concepts of the reinforcement learning. The mix between reading the text book and the videos gives the ability to understand very well, and the programming assignments let's you put to test the things you've learned.
Anonymous

I really like the course. It's simple and easy to understand. I managed to finish the entire course in a day. The programming assignments are well thought out and are a good mix of easy and challenging, while also giving a good understanding of the underlying concepts.
Anonymous

It is a reallui good course. It is basically an introduction course to RL but it has good reference (that you have to read) and video lectures which explain the reference book with some examples. The resources, such as notebooks, are well done and challenging enough.
Anonymous

This course provided great value for me, the content and explanations are of good quality. Quizzes and programming exsersies are challenging enough to help you grasp nessesary concepts and get hands on experience. Look forward to the next course in the spesialisation.
Anonymous

It gets the point across and the examples used are decent. But it's not a very engaging course. The assignments are not very difficult and walk you through the problem well. I found myself skipping through the lectures a bit, but learnt the basic ideas of RL fine.
Anonymous

This course enforces one to become strong with the fundamentals of RL and implementing it in code just adds icing on the cake by giving confidence. I would recommend one to take this course and move ahead in this field.
Anonymous

Best place to start reinforcement learning if you are new to it. Challenging assignments based concepts. Quizzes to test your fundamentals. Everything is organised.
Much more importantly good discussion forum.
Kim Falk

I found the course very interesting. The videos very good an informative. I like the fact that the videos are describing theory (with real world examples) and not trying to teach Python while doing it.
Anonymous

This course is from the number 1 origin of reinforcement learning.
I think this course is a basic course and passing it alone without the other courses in the Specialization is not enough.