A2C - #149
Merged
Merged
Conversation
Contributor
Author
|
CartPole-v0 score
CartPole-v1 score
Pendulum-v0 score |
Contributor
Author
|
Pong-v0
|
Contributor
Author
|
result of breakout
|
Member
|
This is a very important enhancement, thank you! Please give the developer team some time to discuss the interface of parallel synchronous training before reviewing it. |
muupan
reviewed
Oct 12, 2017
| dist_entropy = F.mean(pout.entropy) | ||
| elif isinstance(pout, distribution.GaussianDistribution): | ||
| log_prob = pout.log_prob(actions, keepdims=True) | ||
| dist_entropy = F.mean(F.sum(log_prob, axis=-1)) |
Member
There was a problem hiding this comment.
Why not using GaussianDistribution.entropy?
Contributor
Author
There was a problem hiding this comment.
I'd been misunderstanding meaning of differential entropy.
Modified to use pout.entropy(GaussianDistribution.entropy) in case of continuous action space.
Contributor
Author
|
@muupan |
Member
|
That is better, thank you. > This PR should split interface of parallel synchronous training and a2c? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add A2C proposed by following thesis.
https://arxiv.org/abs/1708.05144
https://blog.openai.com/baselines-acktr-a2c/
A2C is a synchronous, deterministic variant of Asynchronous Advantage Actor Critic (A3C).