Generalize dartsim tests so they can run with other physics engines

[iche033]

The tests in dartsim/src directory could be generalized so we can use them for other engines like TPE.

[scpeters]

I think one approach to do this would be to create a component of ignition-physics called tests or feature-tests and add an ign physics command in that component that would load physics implementation plugins and test them somehow (can gtest be used from a library?). I've experimented with the ign plugin --info command in gazebosim/gz-plugin#32 as a proof of concept, since it uses ignition::plugin::Loader from a component library, and it seems to be working (aside from some SIP issues on macOS that can be worked around).

[scpeters]

I was thinking of syntax like the following to test a single plugin with a single test:

ign physics --feature-test TestName --plugin /path/to/libPhysicsPlugin.so

If we use gtest under the hood, we may want to be able to pass through gtest command-line arguments, so that we can get junit xml files out or select a subset of test cases to run.

[mjcarroll]

I think you could also write the tests in such a way that they load one plugin as part of the test fixture. and then execute the test.

This is similar to how it is done in ign-rendering to verify across engines: https://github.com/ignitionrobotics/ign-rendering/blob/ign-rendering4/test/test_config.h.in

[scpeters]

I think you could also write the tests in such a way that they load one plugin as part of the test fixture. and then execute the test.

This is similar to how it is done in ign-rendering to verify across engines: https://github.com/ignitionrobotics/ign-rendering/blob/ign-rendering4/test/test_config.h.in

That looks similar to how osrf/gazebo handles generalized physics tests. I believe it requires the list of engines to be tested is known at configure time. To simplify the workflow for 3rd-party physics implementation plugin developers, I don't want this set of ignition-physics tests to require any knowledge of which plugins it will be tested against at configure time of ign-physics.

[scpeters]

so maybe my comments go beyond the initial description of this issue, but I think it is important and will scale better for 3rd-party plugin developers

[mjcarroll]

Ah, in that case, I suppose you could make the library a command line argument to any gtest-based test.

I'm hesitant to add a ruby dependency to be able to execute physics unit tests.

[scpeters]

Ah, in that case, I suppose you could make the library a command line argument to any gtest-based test.

I'm hesitant to add a ruby dependency to be able to execute physics unit tests.

that's a fair point. If we can stuff all the test logic in a library, then perhaps we could wrap it in gtest executables for unit test purposes while still exposing it via ign physics. Best of both worlds?

[scpeters]

Ah, in that case, I suppose you could make the library a command line argument to any gtest-based test.
I'm hesitant to add a ruby dependency to be able to execute physics unit tests.

that's a fair point. If we can stuff all the test logic in a library, then perhaps we could wrap it in gtest executables for unit test purposes while still exposing it via ign physics. Best of both worlds?

after some discussion with @mjcarroll, we reached roughly the following:

• Create gtest executable(s) in the core component of ignition-physicsN that accept the path to a physics implementation plugin as a command-line argument.
• Use add_test in the dartsim and tpe components with the path to their plugins. I'm guessing the gtest executables will need to be exported as targets for this to work?
• Install these gtest executables to /usr/lib/ignition-physicsN/ so they can be used by ign physics test