Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

breakable-bottles observation space correction #93

Merged
merged 3 commits into from
Apr 15, 2024
Merged

breakable-bottles observation space correction #93

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
04575e1
Changes the observation space of breakable_bottles to state that the …
Apr 12, 2024
00f33f1
Added some text in the BreakableBottles class description explaining …
Apr 14, 2024
652e910
Attempted fix for pre-commit whitespace trim failure.
Apr 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions mo_gymnasium/envs/breakable_bottles/breakable_bottles.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,13 @@ class BreakableBottles(Env, EzPickle):
The observation space is a dictionary with 4 keys:
- location: the current location of the agent
- bottles_carrying: the number of bottles the agent is currently carrying (0, 1 or 2)
- bottles_delivered: the number of bottles the agent has delivered (0 or 1)
- bottles_delivered: the number of bottles the agent has delivered (0, 1 or 2)
- bottles_dropped: for each location, a boolean flag indicating if that location currently contains a bottle

Note that this observation space is different from that listed in the paper above. In the paper, bottles_delivered's possible values are listed as (0 or 1),
rather than (0, 1 or 2). This is because the paper did not take the terminal state, in which 2 bottles have been delivered, into account when calculating
the observation space. As such, the observation space of this implementation is larger than specified in the paper, having 360 possible states instead of 240.

## Reward Space
The reward space has 3 dimensions:
- time penalty: -1 for each time step
Expand Down Expand Up @@ -96,11 +100,11 @@ def __init__(
{
"location": Discrete(self.size),
"bottles_carrying": Discrete(3),
"bottles_delivered": Discrete(2),
"bottles_delivered": Discrete(3),
"bottles_dropped": MultiBinary(self.size - 2),
}
)
self.num_observations = 240
self.num_observations = 360

self.action_space = Discrete(3) # LEFT, RIGHT, PICKUP
self.num_actions = 3
Expand Down Expand Up @@ -220,7 +224,7 @@ def get_obs_idx(self, obs):
*[[bd > 0] for bd in obs["bottles_dropped"]],
]
)
return np.ravel_multi_index(multi_index, tuple([self.size, 3, 2, *([2] * (self.size - 2))]))
return np.ravel_multi_index(multi_index, tuple([self.size, 3, 3, *([2] * (self.size - 2))]))

def _get_obs(self):
return {
Expand Down
Loading