Skip to content

[rllib] Allow envs to be auto-registered; add on_train_result callback with curriculum example #3451

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 13 commits into from
Dec 4, 2018
Merged

[rllib] Allow envs to be auto-registered; add on_train_result callback with curriculum example #3451

merged 13 commits into from
Dec 4, 2018

Conversation

@ericl
Copy link
Contributor

@ericl ericl commented Dec 1, 2018

No description provided.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 2, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9691/
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 2, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9692/
Test FAILed.

richardliaw
richardliaw reviewed Dec 2, 2018
View reviewed changes
python/ray/rllib/agents/agent.py
if self.config["callbacks"].get("on_train_step"):
self.config["callbacks"]["on_train_step"]({
"agent": self,
"result": result,
Copy link
Contributor

@richardliaw richardliaw Dec 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.copy()?

Copy link
Contributor

@richardliaw richardliaw Dec 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(or deepcopy)

Copy link
Contributor Author

@ericl ericl Dec 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the advantage here is the user can mutate it if they want.

python/ray/rllib/agents/agent.py
self.__setstate__(extra_data)


def _register_if_needed(env_object):
Copy link
Contributor

@richardliaw richardliaw Dec 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this happen earlier? i.e., before the config is pickled, somewhere in trial.py

Copy link
Contributor Author

@ericl ericl Dec 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it really matters, since the pickled config can't be resolved anyways. Basically it just adds a little more text there.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 2, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9701/
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 2, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9700/
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 2, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9702/
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 3, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9708/
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 3, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9709/
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 4, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9728/
Test FAILed.

richardliaw
richardliaw reviewed Dec 4, 2018
View reviewed changes
doc/source/rllib-training.rst

.. image:: custom_metric.png

Example: Curriculum Learning
Copy link
Contributor

@richardliaw richardliaw Dec 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can probably put together a page full of examples (in a later PR)

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 4, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9737/
Test FAILed.

@ericl ericl added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Dec 4, 2018
@ericl ericl merged commit ce355d1 into ray-project:master Dec 4, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@richardliaw richardliaw richardliaw approved these changes

Assignees

@richardliaw richardliaw

Labels

tests-ok The tagger certifies test failures are unrelated and assumes personal liability.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ericl @AmplabJenkins @richardliaw