Cannot define custom metrics in ray

Question

I'm using a framework called FLOW RL. It enables me to use rllib and ray for my RL algorithm. I have been trying to plot non learning data on tensorboard. Following ray documentation ( link ), I have tried to add custom metrics. Therefore, I need to use the info dict, which is accessed by on_episode_step(info). An "episode" element is supposed to be present in this dictionary. That lets me access to my custom scalars. However, every time I try to access to the episode element, I get an error because it does not exist in the info dict. Is this normal?

File "examples/rllib/newGreenWaveGrid2.py", line 295, in on_episode_start episode = info["episode"] KeyError: 'episode'

def on_episode_step(info):
    episode = info["episode"]
    whatever = abs(episode.last_observation_for()[2])
    episode.user_data["whatever"].append(whatever)



if __name__ == '__main__':
    alg_run, gym_name, config = setup_exps()
    ray.init(num_cpus=N_CPUS + 1, redirect_output=False)
    trials = run_experiments({
        flow_params['exp_tag']: {
            'run': alg_run,
            'env': gym_name,
            'config': {
                **config,
                'callbacks': {
                    "on_episode_start": on_episode_start,
                    "on_episode_step": on_episode_step,
                    "on_episode_end": on_episode_end,

                }
            },
            'checkpoint_freq': 20,
            'max_failures': 999,
            'stop': {
                'training_iteration': 200,
            },
        },


    })

Cannot define custom metrics in ray

0 Answers0