[Development] Add convert glb to usd objects. Issac Lab repo installation required.

[danieltmeta]

Motivation and Context

This pull request (PR) introduces the capability to convert file types in Isaac Lab, specifically converting files into .usda format. These conversion utilities are essential for Isaac Sim physics simulations to process scene instance JSONs and object meshes from Habitat-Sim.

The PR includes the conversion of scene instances, encompassing rigid body objects. However, it does not currently support articulated objects or the stage. Future updates will address these limitations.

Additionally, a separate set of functions is required to convert robot URDF files into .usda format for use in Isaac Sim. This functionality will be implemented in a subsequent update.

How Has This Been Tested

Unit tests will be added to code function and the usda file output.

Types of changes

• [Docs change] Addition or changes to the documentation
• [Refactoring] Large changes to the code that improve its functionality or performance
• [Dependency Upgrade] Upgrades one or several dependencies in habitat
• [Bug Fix] (non-breaking change which fixes an issue)
• [Development] A pull request that add new features to the habitat-lab task and environment codebase. Development Pull Requests must be small (less that 500 lines of code change), have unit testing, very extensive documentation and examples. These are typically new tasks, environments, sensors, etc... The review process for these Pull Request is longer because these changes will be maintained by our core team of developers, so make sure your changes are easy to understand!
• [Experiment] A pull request that add new features to the habitat-baselines training codebase. Experiments Pull Requests can be any size, must have smoke/integration tests and be isolated from the rest of the code. Your code additions should not rely on other habitat-baselines code. This is to avoid dependencies between different parts of habitat-baselines. You must also include a README that will document how to use your new feature or trainer. You will be the maintainer of this code, if the code becomes stale or is not supported often enough, we will eventually remove it.

Checklist

• My code follows the code style of this project.
• I have updated the documentation if required.
• I have read the CONTRIBUTING document.
• I have completed my CLA (see CONTRIBUTING)
• I have added tests to cover my changes if required.

[eundersander]

Some good progress here! I left some comments.

My biggest request is to add a test file (the last comment in usd_conversion_scene.py).

Also, I see that pre-commit and python lint CI tests are failing (see below "Some checks are failing"). Try to get those passing. Have you installed pre-commit locally? See instructions.

Let me know when you've addressed my comments and I'll take another look!

[eundersander]

Let's add test/test_isaac_sim.py to habitat-lab in this PR. Look at our other test files in that folder for reference. Let's have a test that tries to convert a toy habitat scene instance (this is something you suggested a while back). The test can assume the source assets are present on the machine (or maybe check for them and skip the test if they're missing). The test should inspect the output usda file and check for expected contents.

[danieltmeta]

Migrating unit tests and example data from PR #2123

[danieltmeta]

Need to delete PR #2123 after unit tests are reviewed.

[eundersander]

I left some comments. A few top-level comments:

• Let's add nice CLIs for both of these files. This means using argument-parsing and offering a friendly command-line interface.
• You need to fill out the PR description. What's autopopulated there is a template for you to edit. This is an important part of documenting a code change.
• As I requested earlier, let's add a minimal test file to this PR.

[eundersander]

Good progress here!

Please flesh out your PR description a bit. You don't have to explain what Isaac Sim is, but give some background for why we need these conversion utils. Mention that we convert scene instances including rigid objects (but not any included articulated objects or the stage yet). Mention the URDF conversion for converting robot URDF files. For Type of change, specify [Development].

Let's move meshColored inside data/usd_conversion_data so all the isaac-related test data is in one subfolder.

It looks like maybe you're not using test/data/usd_conversion_data/102343992.scene_instance.json for testing, so you can remove it from this PR.

I suspect you don't need to add test/data/usd_conversion_data/objects_EXAMPLE2/1efdc3d37dfab1eb9f99117bb84c59003d684811.glb to this repo, because it's a render GLB and it won't be used/loaded during the conversion (or am I missing something?). In general, we want to be really careful and selective about adding binary files to a git repo, since they are usually large.

[eundersander]

Just bumping this comment. Let's not name HSSD in comments.

[eundersander]

I left a few more comments. Also, your test is failing in CI:

==================================== ERRORS ====================================
___________________ ERROR collecting test/test_isaac_sim.py ____________________
ImportError while importing test module '/home/runner/work/habitat-lab/habitat-lab/habitat-lab/test/test_isaac_sim.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../../miniconda3/envs/habitat/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_isaac_sim.py:7: in <module>
    from lxml import etree as ET
E   ModuleNotFoundError: No module named 'lxml'

I believe the fix here is to add lxml to habitat-lab/requirements.txt.

[danieltmeta]

Oh I see, you wanted the CLI to be consolidated into a single command with all of the functions you mentioned for urdf to usd.

[eundersander]

Your test is still failing, due to Isaac not installed on the CI machine.

Some thoughts:

• Can the CI machine install Isaac? Needs to be Ubuntu 22 and have an Nvidia GPU.
• If yes, how should we install it? Since Isaac is optional for Habitat-lab, let's not add it to requirements.txt. Instead, it can be a separate pip install in the CI script.
• If no, you should configure your test to skip if Isaac isn't installed (if it can't be imported). This means CI won't run your test, but your test is still valuable; it can be run locally by folks that want to understand these conversion scripts.

[danieltmeta]

Pausing work here. Trying to run test/test_isaac_sim.py for unit testing, but it requires Isaac Sim and Issac lab to be installed on github actions CI. The github action file related to this unit test is at .github/actions/install_isaac_deps/action.yml. Basically, ./isaac_modified.sh is copied into the IsaacLab repo, and then installed instead of isaaclab.sh. Isaaclab.sh is not run because there is a pathing issue with the global constant ISAACLAB_PATH and extract_isaacsim_path(). The below is the error message related to missing files that somehow were not cloned into the IsaacLab repo.

https://github.com/facebookresearch/habitat-lab/actions/runs/13173253221/job/36767237376

[facebook-github-bot]

Hi @danieltmeta!

Thank you for your pull request.

We require contributors to sign our Contributor License Agreement, and yours needs attention.

You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!