0

I am a newbie in reinforcement learning working on a college project. The project is related to optimizing x86 hardware power. I am running proprietary software in Linux distribution (16.04). The goal is to use reinforcement learning and optimize the power of the System (keeping the performance degradation of the software as minimum as possible). The proprietary software is a cellular network.

As we already know, the primary functional blocks of Reinforcement learning are Agent and Environment. The basic idea is to use the cellular network running on x86 hardware as the environment for RL. This environment interacts with the agent implementing RL using state, actions, and reward.

From reading different materials, I could understand that I need to make my software as a custom environment from where I can retrieve the state features. The state features are the application layer KPIs like latency, throughput. Action space may include instructions to Linux to change the power (I can use some predefined set of power options). I did not decide about the reward function.

I read this post and decided that I should use OpenAI gym to create my custom environment.

My doubt is that using OpenAI gym for creating custom environments (for these type of setup) is correct. Am I going in the right direction (or) is there any alternative/best tools to create a custom environment. any tutorial or direction to create this custom environment is appreciated.

NewToCoding
  • 199
  • 1
  • 2
  • 15
  • If you want to use RL, gym is the most used interface for learning algorithms. For how to create your env, read this answer https://stackoverflow.com/questions/52727233/registering-a-custom-environment-in-openai-gym/52739974#52739974 – Simon Jun 01 '19 at 10:28
  • @Simon But is it the correct approach to create a custom environment. I mean, for the use case I explained above, OpenAI gym is the correct approach to create a custom environment. – NewToCoding Jun 01 '19 at 10:31
  • If you want to solve it with RL in python using already implemented algorithms: yes. You can also make your own environment without following gym structure, but honestly it doesn't make sense, since gym structure is very minimal (see the example I linked). And if you don't follow it, you most likely won't be able to use algorithms from libraries you'll find on git (baseline and rll-lab are the most famous). – Simon Jun 01 '19 at 10:56
  • @Simon can you please let me know what is Baseline and rll-lab? – NewToCoding Jun 01 '19 at 22:23
  • They are repositories with already implemented algorithms for RL, and can be easily interfaced with gym. You can find them immediately searching on Google. – Simon Jun 01 '19 at 22:45
  • @Simon if I use OpenAI gym for the custom environment in RL, do I have to model the behavior of the cellular network in the custom environment or just create an interface such that the custom environment code (such as the example you linked in the above comment) could interact with the cellular network? – NewToCoding Jun 02 '19 at 18:08
  • The second. You just need an interface with the 4 functions of the example (_seed, step, reset, get_obs). – Simon Jun 02 '19 at 18:42

0 Answers0