Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scene Selection in HITL App #1954

Open
ambervg opened this issue May 15, 2024 · 6 comments
Open

Scene Selection in HITL App #1954

ambervg opened this issue May 15, 2024 · 6 comments

Comments

@ambervg
Copy link

ambervg commented May 15, 2024

System information & package versions

  • OS: Ubuntu 22.04.1
  • Architecture: x86_64
  • habitat-sim version: 0.3.0
  • habitat-lab version: 0.3.0
  • habitat-hitl version: 0.3.0

❓ Questions and Help

Problem description:

For a simple habitat HITL app I want to use a scene that differs from the default one. I have tried to add a scene .glb file to the .yaml configuration file like this:

habitat:
  # https://github.com/facebookresearch/habitat-lab/blob/main/habitat-lab/habitat/config/default_structured_configs.py
  simulator:
    scene: "data/scene_datasets/habitat-test-scenes/skokloster-castle.glb"

However, when running the script, the scene is still the default one instead of the just defined skokloster-castle. It seems like the scene is being overwritten somewhere. Let’s take the minimal app as example. I am wondering which files have to be linked at what location(s) so that the scene being used is changed.

I have checked the passed configurations with print(omegaconf.OmegaConf.to_yaml(config)) in the main function and will link the results of that here: used_config.txt

I feel like this question should have a very straightforward answer, but I am stuck on it and would greatly appreciate it if anyone could point me in the right direction. So far, I have not been able to find the answer to this in the documentation, but maybe I am not looking in the right places.

My questions:

  1. For the minimal.py & minimal.yaml example, which files would I have to link at what location(s) so that the actual scene being used is changed?
  2. Where can the scene still be overwritten?
  3. Do I need more files than just the .glb and .navmesh for a scene to be loaded successfully?
@ojss
Copy link

ojss commented May 21, 2024

I have the same issue! perhaps, @0mdc @aclegg3 any insights you could share?

@0mdc
Copy link
Contributor

0mdc commented May 21, 2024

The HITL app sits on top of habitat-lab rather than just the sim, which works with episode sets instead of scenes. I'm also interested in adding support for loading specific scenes (via config or at any time via a command).

As a work-around, you can create an episode set (json.gz file) with only 1 episode which loads skokloster-castle.glb.

You can use an existing file as a template, e.g. data/versioned_data/hab3-episodes/train/social_rearrange.json.gz.
This one is downloaded using the python -m habitat_sim.utils.datasets_download --uids hab3-episodes command.

@ambervg
Copy link
Author

ambervg commented May 30, 2024

Tagging @rutadesai based on our discussion this morning.

@ambervg
Copy link
Author

ambervg commented Jun 10, 2024

I have been trying to get this work-around to work by using run_episode_generator.py to create an episode set, such as data/versioned_data/hab3-episodes/train/social_rearrange.json.gz. Let me start by describing the steps that I have taken.

Steps I have taken:

  1. Creating a .yaml file with configs for run_episode_generator.py. I will link one of my attempts here: config_ep_v10.zip. It is a .zip format because GitHub did not accept .yaml.
  2. I run the below command to create rearrange_ep_dataset_v10.json.gz. I will then copy this to my desired location.
python habitat-lab/habitat/datasets/rearrange/run_episode_generator.py --config habitat-lab/habitat/datasets/rearrange/configs/config_ep_v10.yaml --out habitat-lab/habitat/datasets/rearrange/configs/rearrange_ep_dataset_v10.json.gz --run --num-episodes 1
  1. Now, I go to hitl_rearrange.yaml and at habitat.dataset.data_path I link path/to/rearrange_ep_dataset_v10.json.gz.
  2. Ensure that hitl_rearrange.yaml is in fact being used by rearrange.py in this line.
  3. Finally running python path/to/rearrange.py. However, this results in an error for me: KeyError: 'any_targets|0'. I will attach a file with the full error message here: error_msg_keyerror.txt.

Flow:
If I understand it correctly, the flow is as follows:
image

My question:

  • Any tips on how to handle the KeyError: 'any_targets|0'? The error does not only happen for the episode sets that I create myself, but also when I use the config_ep.yaml files that are provided in the repo.

@rutadesai @0mdc

@0mdc
Copy link
Contributor

0mdc commented Jun 10, 2024

Hmm. I believe that the problem is due to the config activating incompatible pddl code.
hitl_rearrange.yaml is indeed referencing pop_play.yaml, which is probably breaking with this setup.

Let me see if I can come up with a bare-bone configuration that doesn't do all this unnecessary work.

@ambervg
Copy link
Author

ambervg commented Jun 18, 2024

@0mdc That would be super helpful. Thank you so much.

Based on recommendations from @rutadesai + @aclegg3 I tried changing something in the config_ep.yaml because the PDDL implementation is hardcoded to look for "any_targets" with an "s".

So from:

object_target_samplers:
  - name: 'any_target'

To:

object_target_samplers:
  - name: 'any_targets'

This changed the error from KeyError: 'any_targets|0' to KeyError: 'any_targets|1'.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants