5

I am trying to make a PPO model using the stable-baselines3 library. I want to use a policy network with an LSTM layer in it. However, I can't find such a possibility on the library's website although it exists on the previous version of stable-baselines here https://stable-baselines.readthedocs.io/en/master/modules/policies.html#stable_baselines.common.policies.MlpLstmPolicy.

Does this possibility exist in stable-baselines3 (not stable-baselines)? if not, is there any other possibility I can do this? Thanx.

mac179
  • 1,540
  • 1
  • 14
  • 24

2 Answers2

3

From the migration doc.

https://stable-baselines3.readthedocs.io/en/master/guide/migration.html

Breaking Changes¶

LSTM policies (MlpLstmPolicy, CnnLstmPolicy) are not supported for the time being (see PR #53 for a recurrent PPO implementation)

bnye
  • 41
  • 5
3

Currently this functionality does not exist on stable-baselines3.

However, on their contributions repo (stable-baselines3-contrib) they have an experimental version of PPO with LSTM policy. I have not tried it myself, but according to this pull request it works.

You can find it on the feat/ppo-lstm branch, which may get merged onto master soon.

mac_or_cheese
  • 31
  • 1