Reinforcement Learning: Machine Learning Meets Control Theory

Welcome back. So I'm really excited to do this  lecture on reinforcement learning. I've been wanting to do this for a long time.

Those of  you who know me know that I love control theory and machine learning and reinforcement learning is  kind of at this sweet spot between these two super important fields. Okay, so reinforcement learning  is essentially a branch of machine learning that deals with how to learn control strategies to  interact with a complex environment. And one of the ways I think about this, the way I'm going  to define this, is that reinforcement learning is a framework for learning how to interact  with the environment from experience.

This is a very biologically inspired idea. . .

this is  what animals do. So through trial and error, through experience, through positive and negative  rewards and feedback, they learn how to interact with their environment. OK good.

So before I jump  in I want to show some motivating videos. I really like this one where reinforcement learning is  used to learn how to walk in this artificial environment. And there's a lot of papers like this  where people use reinforcement learning as kind of an optimization framework to learn how to control  a complex system, in this case a bipedal walker, often in a simulated environment.

And this just  looks really cool and it's a difficult control problem. This is a really hard non-linear control  problem. Now the goal would be to take what you learn here and start to port that over into  the real world to make better robots and better actual physical agents that can interact with  the world alongside us, to learn how to learn like humans and animals do.

So another video  I love. . .

this is my dog Mordecai and my wife has trained him. . .

this is a treat on his nose. . . 

to hold the treat on his nose until she says ok, after which he can then grab the treat and eat it.  This is not an easy trick to learn and this again this goes to show you anytime you, anybody who's  trained an animal, a dog or any other animal, has done some type of reinforcement learning or  reinforcement training.