1

When studying Reinforcement learning, and exactly when it comes to Model-Free RL, there are two methods we use generally:

  • TD learning
  • Monte Carlo

When is each one of them used over the other? In other words, how do we figure out what method is best for our problem?

Ilyes Yamoun
  • 37
  • 1
  • 6
  • Related https://stackoverflow.com/questions/22723830/when-to-use-a-certain-reinforcement-learning-algorithm – Simon Apr 30 '19 at 07:44

1 Answers1

2

Sections 6.1 and 6.2 of Sutton & Barto give a very nice intuitive understanding of the difference between Monte Carlo and TD learning.

Having said that, there's of course the obvious incompatibility of MC methods with non-episodic tasks. In that case, you will always need some kind of bootstrapping.

Kris
  • 22,079
  • 3
  • 30
  • 35