Removing unit tests from ARMI's API

[john-science]

You can import all of ARMI's unit test code:

from armi.settings.tests.test_settingsIO import SettingsReaderTests
from armi.reactor.test.test_blocks import TestDetailedNDensUpdate

I think this is the wrong paradigm. I think a more usual, and modern approach is what the NumPy team did. Instead of importing the NumPy team's unit tests, they provided a set of useful testing tools at:

from numpy import testing

The NumPy team reaps benefits from their approach:

1. Their API isn't clogged up with their unit tests!
2. They can change, add, refactor, or rename unit tests without breaking tests in downstream projects.
3. They can curate a small set of high-quality testing tools.

Thoughts?

[john-science]

Here is a full list of ALL imports I can find (in any downstream project) of ARMI's unit tests:

import armi.materials.tests.test_materials._Material_Test
import armi.nuclearDataIO.cccc.tests.test_labels
import armi.nuclearDataIO.tests.test_xsCollections
import armi.nuclearDataIO.tests.test_xsLibraries
import armi.physics.neutronics.fissionProductModel.tests.test_lumpedFissionProduct
import armi.physics.neutronics.globalFlux.tests.test_globalFluxInterface
import armi.physics.neutronics.latticePhysics.tests.test_latticeInterface
import armi.reactor.converters.tests.test_axialExpansionChanger
import armi.reactor.tests.test_assemblies.buildTestAssemblies
import armi.reactor.tests.test_blocks.buildSimpleFuelBlock
import armi.reactor.tests.test_components.TestShapedComponent
import armi.reactor.tests.test_reactors.loadTestReactor
import armi.reactor.tests.test_reactors.reduceTestReactorRings
import armi.tests.ARMI_RUN_PATH
import armi.tests.ArmiTestHelper
import armi.tests.getEmptyHexReactor
import armi.tests.ISOAA_PATH
import armi.tests.mockRunLogs
import armi.tests.test_plugins.TestPlugin
import armi.tests.TEST_ROOT

Some of these are only used in one place in the world and I could just copy a little test code into a downstream repo with no problems.

A quick review shows that 95% of the imports of ARMI's testing features come from these few things, that could be moved to armi.testing:

import armi.reactor.tests.test_reactors.loadTestReactor
import armi.reactor.tests.test_reactors.reduceTestReactorRings
import armi.tests.ARMI_RUN_PATH
import armi.tests.ArmiTestHelper
import armi.tests.getEmptyHexReactor
import armi.tests.ISOAA_PATH
import armi.tests.mockRunLogs
import armi.tests.TEST_ROOT

[albeanth]

I think a more usual, and modern approach is what the NumPy team did. Instead of importing the NumPy team's unit tests, they provided a set of useful testing tools....

What do you think this would look like for ARMI? A quick glance looks like those numpy testing tools are basically assertions. The functionality used by plugins goes beyond assertions.

[drewj-tp]

We could provide fixtures or other testing-specific utilities in armi.testing. One example could be armi.testing.loadTestReactor(...)

[john-science]

This is the way.

Yeah, I think ARMI should provide some easy-to-use tools (that we mostly already do) that will help other people write unit tests for ARMI-based projects.

For instance:

# helpful testing tools
from armi.testing import loadTestReactor
from armi.testing import mockRunLogs
from armi.testing import loadTestFuelBlock

# paths to settings files for various test reactors
from armi.testing import TEST_REACTOR_SMALLEST
from armi.testing import TEST_REACTOR_C5G7
from armi.testing import TEST_REACTOR_GODIVA
from armi.testing import TEST_REACTOR_ZPPR

That's not a complete list, but just for the flavor of the thing.

[albeanth]

Ah, yeah this makes sense to me. Seems like a consideration for ARMI 1.0 planning?

[john-science]

I love that idea. Done!

[drewj-tp]

FWIW I've personally liked having the tests totally outside the API, something like

armi/
    physics/
    testing/
    reactor/
        blocks.py
        ...
    ...
tests/
    reactor/
        test_reactors.py