OpenAI Gym experiments

My implementations of normalized advantage functions (NAF) for continuous actions spaces and dueling network architecture (DUEL) for discrete action spaces.

Example results with NAF:

Example results with DUEL:

Prerequisites

You will need:

Python 2.7
OpenAI Gym
Keras
Numpy
Scikit-Learn (if using imagination rollouts)

In Ubuntu that would be:

sudo apt-get install python-numpy python-sklearn
pip install --user gym keras

If you want to run Mujoco environments, you also need to acquire trial key and install the binaries. Then you can install Mujoco support for OpenAI Gym:

pip install --user gym[mujoco]

Running the code

There are three main starting points:

python duel.py <envid> - run DUEL against environment with discrete action space,
python naf.py <envid> - run NAF against environment with continuous action space,
python nag_ir.py <envid> - run NAF with imagination rollouts.

You can override default hyperparameters with command-line options, use -h to see them or check out the code.

Some other utility scipts:

python test.py <envid> - test script to run random actions against the environment,
python naf_search.sh - example how to run crude hyperparameter search for NAF,
python duel_search.sh - example how to run crude hyperparameter search for DUEL.