Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot load elasticity plugin as defined in xml #439

Open
sachinkundu opened this issue Dec 9, 2023 · 2 comments
Open

Cannot load elasticity plugin as defined in xml #439

sachinkundu opened this issue Dec 9, 2023 · 2 comments

Comments

@sachinkundu
Copy link

sachinkundu commented Dec 9, 2023 •
edited
Loading

I am trying to create a flexcomp object available in mujoco 3.

The XML definition is

<mujoco model="cloth">
    <statistic center=".4 0 .8" extent="1.3"/>

    <option wind="0 0 0" density="10" solver="CG" tolerance="1e-6">
    </option>

    <extension>
        <plugin plugin="mujoco.elasticity.shell"/>
    </extension>

    <worldbody>
        <camera name="top" pos="0.071 -0.018 2.756" xyaxes="0.019 -1.000 0.000 0.998 0.019 0.054"/>
        <camera name="fixed" mode="track" pos="-1.787 -0.875 1.327" xyaxes="0.442 -0.897 0.000 0.476 0.235 0.848"/>
        <geom name="floor" type="plane" size="0 0 .1"/>
        <light diffuse=".6 .6 .6" specular="0.2 0.2 0.2" pos="0 0 4" dir="0 0 -1"/>
        <body>
            <body name="object" pos="0 0 0.75">
                <flexcomp type="grid" count="9 19 1" spacing=".05 .05 .05" mass="10"
                          name="flag" radius="0.001">
                    <edge equality="true" damping="0.001"/>
                    <plugin plugin="mujoco.elasticity.shell">
                        <config key="poisson" value="0"/>
                        <config key="thickness" value="1e-2"/>
                        <!--Units are in Pa (SI)-->
                        <config key="young" value="3e6"/>
                    </plugin>
                </flexcomp>
                <site rgba="0 0 0 0" size="0.005" pos="0 0 -0.045" name="bottom_site"/>
                <site rgba="0 0 0 0" size="0.005" pos="0 0 0.03" name="top_site"/>
                <site rgba="0 0 0 0" size="0.005" pos="0.03 0.03 0" name="horizontal_radius_site"/>
            </body>
        </body>
    </worldbody>
</mujoco>

I can load this xml in mujoco simulate stand alone application. As you can see it uses engine plugin. When running simulate I have to put libelasticity.so in mujoco_plugin folder next to the application simulate.

I define a ClothObject

class ClothObject(MujocoXMLObject):
    """
    Cloth object
    """

    def __init__(self, path, name):
        super().__init__(path, name=name, obj_type="collision", duplicate_collision_geoms=False)

and create this object as part of my environment initialization and add the object to the environment's placement initializer.

    def _create_cloth(self):
        self.cloth = ClothObject(str((Path(self.asset_path) / "cloth.xml").resolve()), "cloth")

During the run, after MujocoEnv._load_model() is run, a valid looking xml is generated(with my flexcomp cloth definition included). However this xml is not good enough for loading the sim as it fails with error

ValueError: Error: unrecognized plugin 'mujoco.elasticity.shell

There is plugin/libelasticity.so in /miniconda3/envs/robosuite/lib/python3.8/site-packages/mujoco and _load_all_bundled_plugins() in __init__.py of mujoco loads the plugin without an error.

The same xml and running test_load_plugin in bindings_test.py in /miniconda3/envs/robosuite/lib/python3.8/site-packages/mujoco also runs fine.

This makes me think that somehow robosuite run is changing the library loading path but I cannot seem to figure out where.

How can I load the engine plugins when running robosuite with mujoco 3?
@sachinkundu
Copy link
Author

I have been debuggin a bit and surely there is something funky with loading an xml with plugins with robosuite.

Here is a simple script

import mujoco

MY_TEST_XML = r"""
<mujoco model="cloth">
    <statistic center=".4 0 .8" extent="1.3"/>

    <option wind="0 0 0" density="10" solver="CG" tolerance="1e-6">
    </option>

    <extension>
        <plugin plugin="mujoco.elasticity.shell"/>
    </extension>

    <worldbody>
        <camera name="top" pos="0.071 -0.018 2.756" xyaxes="0.019 -1.000 0.000 0.998 0.019 0.054"/>
        <camera name="fixed" mode="track" pos="-1.787 -0.875 1.327" xyaxes="0.442 -0.897 0.000 0.476 0.235 0.848"/>
        <geom name="floor" type="plane" size="0 0 .1"/>
        <light diffuse=".6 .6 .6" specular="0.2 0.2 0.2" pos="0 0 4" dir="0 0 -1"/>
        <body>
            <body name="object" pos="0 0 0.75">
                <flexcomp type="grid" count="9 19 1" spacing=".05 .05 .05" mass="10"
                          name="flag" radius="0.001">
                    <edge equality="true" damping="0.001"/>
                    <plugin plugin="mujoco.elasticity.shell">
                        <config key="poisson" value="0"/>
                        <config key="thickness" value="1e-2"/>
                        <!--Units are in Pa (SI)-->
                        <config key="young" value="3e6"/>
                    </plugin>
                </flexcomp>
                <site rgba="0 0 0 0" size="0.005" pos="0 0 -0.045" name="bottom_site"/>
                <site rgba="0 0 0 0" size="0.005" pos="0 0 0.03" name="top_site"/>
                <site rgba="0 0 0 0" size="0.005" pos="0.03 0.03 0" name="horizontal_radius_site"/>
            </body>
        </body>
    </worldbody>
</mujoco>
"""

model = mujoco.MjModel.from_xml_string(MY_TEST_XML)
data = mujoco.MjData(model)

which works.

I put some debug statements in self compiled mujoco binary(and replacing this in conda env for robosuite site-packages) and I can see that in void mjCModel::ResolvePlugin in user_model.cc active_plugins.size() == 1. However when running robosuite with the same xml(as part of full xml spec) active_plugins.size() == 0 which then errors out with ValueError: Error: unrecognized plugin 'mujoco.elasticity.shell'

Function void mjXReader::Extension(XMLElement* section) at Line 2548 in xml_native_reader.cc has a line ReadAttrTxt(elem, "plugin", plugin_name, /* required = */ true); which sets plugin state as true. This function is processed as part of mujoco run but does not get hit when running robosuite.

This means that somehow robosuite generates the xml in some other manner and hence bypasses the plugin loading logic of mujoco.

I am now lost as how to get robosuite working with plugins

@sachinkundu
Copy link
Author

Should have noticed this before but it seems the generated xml which gets sent to mujoco.MjModel.from_xml_string does not have

<extension>
        <plugin plugin="mujoco.elasticity.shell"/>
</extension>

which would explain why mujoco never loads the plugin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@sachinkundu