GoalEnv for Robot-Supervisor scheme #103
Conversation
|
Thanks @KelvinYang0320 ! I'll review this PR in the following days. |
|
@tsampazk Thank you! |
That sounds nice so we can actually have a usage example to make it more clear. |
There was a problem hiding this comment.
Left some inline comments, several minor PEP8 comments that should had failed the automated tests (something we should look into) and two major comments about code/class hierarchy structure, that need further discussion.
As a general comment, i think we should also look into how the new GoalEnv classes interact with the emitter/receiver classes. Right now emitter/receiver classes seem to not have access to the GoalEnv stuff.
Yes, I am still trying to integrate GoalEnv with emitter/receiver. Thank you for reviewing the code! |
I integrate GoalEnv from gym/core.py with deepbots for Robot-Supervisor scheme.
GoalEnvalso inheriteEnv, but it imposes a required structure on theobservation_space.reset(self):self.observation_spacemust be a Goal-compatible observation space.It is required to contain at least three elements, namely
observation,desired_goal, andachieved_goal.Refer to robot_env.py in gym.
compute_reward(self, achieved_goal, desired_goal, info):Since in our panda example, we need to get the
observationto computeachieved_goal, and then we can compute the reward by usingachieved_goalanddesired_goal.Our original
get_reward(action)is actually not that suitable for robotics control task.So I think we can let users implement their own
step(), so I addraise NotImplementedErrorinstep()function ofRobotGoalSupervisor.I think my coding style or implementation is not good enough.
Any suggestions would be appreciated! 😄