Tuned DoubleDQN with prioritized experience replay

[muupan]

Merge #301 first.

This PR improves train_dqn_ale.py:

• use tuned DoubleDQN as better default settings (used in the Double DQN paper)
• add --prioritized to enable prioritized experience replay
• remove --activation and --use-sdl

[muupan]

Below are preliminary results with --max-episode-len 4500. Results with --max-episode-len 27000 are not ready yet. Prioritized experience replay helps, and lr/4 stabilizes training. Results of Seaquest looks weird, though.

[muupan]

Now the results with --max-episode-len 27000 are ready (except DQN). BeamRider's scores are affected by 4500 vs 27000.

Commands:

• DQN (before this PR, only 10M steps): examples/ale/train_dqn_ale.py --env {env_id}
• Tuned DoubleDQN + prioritized replay lr/4 (after this PR): examples/ale/train_dqn_ale.py --eval-interval 1000000 --prioritized --lr 6.25e-5 --env {env_id}
• Tuned DoubleDQN + prioritized replay (after this PR): examples/ale/train_dqn_ale.py --eval-interval 1000000 --prioritized --env {env_id}
• Tuned DoubleDQN (after this PR): examples/ale/train_dqn_ale.py --eval-interval 1000000 --env {env_id}

Each configuration was ran with three different random seeds. The figures below show the average evaluation scores with confidence bounds are shown.

[muupan]

I'll merge it because it's reviewed and approved by @prabhatnagarajan

[prabhatnagarajan]

LGTM.