Stack Overflow
Stack Overflow

Questions tagged [rllib]

Ray RLlib is an open-source Python library for Reinforcement Learning. Use with applicable framework tags, such as TensorFlow or PyTorch.

105 questions
12
votes
3 answers

Change Logdir of Ray RLlib Training instead of ~/ray_results

I'm using Ray & RLlib to train RL agents on an Ubuntu system. Tensorboard is used to monitor the training progress by pointing it to ~/ray_results where all the log files for all runs are stored. Ray Tune is not being used. For example, on starting…
Nyxynyx
  • 61,411
  • 155
  • 482
  • 830
5
votes
1 answer

cannot import name 'type_spec_registry' from 'tensorflow.python.framework'

I am working on my Master's project with MARL, and i decided to use RLLib. I created a conda environment for my project and installed RLLib: pip install "ray[rllib]" tensorflow Later on, i tried to run one experiment: rllib train --algo DQN --env…
Borja
  • 51
  • 1
  • 2
5
votes
3 answers

Ray on slurm - Problems with initialization

I write this post because since I use slurm, I have not been able to use ray correctly. Whenever I use the commands : ray.init trainer = A3CTrainer(env = “my_env”) (I have registered my env on tune) , the program crashes with the following message…
Pierre houdouin
  • 51
  • 1
  • 3
5
votes
2 answers

Number of time steps in one iteration of RLlib training

I am new to reinforcement learning and I am working on the RL of a custom environment in OpenAI gym with RLlib. When I create a custom environment, do I need to specify the number of episodes in the __init__() method? ALso, when I train the agent…
user3443033
  • 737
  • 2
  • 6
  • 21
5
votes
1 answer

RLLib - Tensorflow - InvalidArgumentError: Received a label value of N which is outside the valid range of [0, N)

I'm using RLLib's PPOTrainer with a custom environment, I execute trainer.train() two times, the first one completes successfully, but when I execute it for the second time it crashed with an error:…
Devid Farinelli
  • 7,514
  • 9
  • 42
  • 73
3
votes
0 answers

RLlib: Multiple training phases with different configurations

I want to do some complicated training using RLlib and I'm not sure how. I have an environment for two agents, and I want to train the first agent while I'm forcing the policy of the second agent to be a hard-coded policy that I write. I want to run…
Ram Rachum
  • 84,019
  • 84
  • 236
  • 374
3
votes
0 answers

RLlib: Using a CNN to train on a 2D grid environment

I’m a newbie to RLlib. I developed a few RLlib environments before and successfully used PPO to train agents on them. Now I developed a 2D grid environment where agents roam around and conquer territory (basically mark some cells as belonging to…
Ram Rachum
  • 84,019
  • 84
  • 236
  • 374
3
votes
0 answers

How really make action masking in Ray (rllib)?

1) It's unclear how to make action masking just more complex in rllib than we can find in examples. This mask works good from example action_mask_model.py with class TorchActionMaskModel(TorchModelV2, nn.Module) self.observation_space = Dict({ …
sirjay
  • 1,767
  • 3
  • 32
  • 52
3
votes
1 answer

Multiple-Actions in one step, Reinforcement learning

I am trying to write a custom openAI Gym environment in which the agent takes 2-actions in each step, one of which is a discrete action and the other is continuous one. I am using Ray RLLib and using SAC algorithm as it supports both discrete and…
JoCode
  • 31
  • 2
3
votes
1 answer

Ray RLllib: Export policy for external use

I have a PPO policy based model that I train with RLLib using the Ray Tune API on some standard gym environments (with no fancy preprocessing). I have model checkpoints saved which I can load from and restore for further training. Now, I want to…
deepmindz
  • 598
  • 1
  • 6
  • 14
3
votes
1 answer

What does "num_envs_per_worker" in rllib do?

For the life of me I don't get what "num_envs_per_worker" does. If the limiting factor is policy evaluation why would we need to create multiple environments? Wouldn't we need to create multiple policies? ELI5 please? The docs say: Vectorization…
Andriy Drozdyuk
  • 58,435
  • 50
  • 171
  • 272
2
votes
0 answers

How to select a policy update rule for PPO when using action masking in Ray RLLib?

I am training a PPO agent in a custom environment using the Ray RLLib library. As my action space has a lot of illegal actions, I have defined a custom model as suggested by the Ray documentation, to mask these actions out. The model is defined as…
Jakob Sejten
  • 31
  • 2
2
votes
1 answer

Get the neural network weights out of a Tensorflow `Graph`

I'm using RLlib to train a reinforcement learning policy (PPO algorithm). I want to see the weights in the neural network underlying the policy. After digging through RLlib's PPO object, I found the TensorFlow Graph object. I thought that I would…
Ram Rachum
  • 84,019
  • 84
  • 236
  • 374
2
votes
1 answer

Ray monitoring fails when binding to empty address

I'm learning to use RLlib. I've been running it in my debugger on an example script, and it works, but for some reason I get an error message about the monitoring service failing. This is the traceback: File…
Ram Rachum
  • 84,019
  • 84
  • 236
  • 374
2
votes
1 answer

RAM Usage Keeps Going Up While Training an RL Network Using RLLib and TensorFlow

I have been using older versions of Ray and TensorFlow, but recently transitioned to the following most up-to-date versions on a Linux Ubuntu 20.04 setup. ray==2.0.0 tensorflow==2.10.0 cuDNN==8.1 CUDA==11.2 While training a single-agent network, I…
troymyname00
  • 670
  • 1
  • 14
  • 32
1
2 3 4 5 6 7