Skip to content

devel CreateComponent before sep 2010

Jump to bottom
Jeff Squyres edited this page Sep 2, 2014 · 3 revisions

Create a new MCA component (prior to September 2010)

This wiki page goes through the basics of creating a new component in an existing Open MPI Modular Component Architecture (MCA) framework. It is assumed that the reader already has at least some familiarity with the internals of Open MPI.

NOTE: This page is relevant to versions of Open MPI prior to September 2010 (e.g., version 1.4.x and prior). Newer versions of the Open MPI source base (e.g., the SVN trunk and possibly later versions of the 1.5.x series) are probably more relevant to the most recent version of this wiki page, which can be found here.

Related wiki pages

  • Note that OMPI's top-level "autogen.sh" is pivotal to the discovery, configuration, and building of components in the Open MPI source code tree. Be sure to see the autogen.sh wiki page for details about the role of autogen.sh.
  • How to create a new MCA framework.

Removing a component

As you'll see below, a component is solely contained within a single directory tree. As such, removing a component is accomplished via "rm -rf" of the component directory tree and re-running autogen.sh and configure.

Since it would be an abstraction violation for one component to refer to another (with the sole exception of "common" pseudo-components), it should be to "rm -rf" a component without changing any other part of the OMPI source code tree.

Creating the component

There are several different options when creating a new component; this document walks through the most common: a component that has some build-time requirements (i.e., it can't be built unless one or more support libraries are present, such as those for a specialized network type) that are checked via the top-level Open MPI configure script and some run-time requirements.

Note that every framework has different rules and interfaces that its components must adhere to. This document does not attempt to cover those rules; see the framework's main interface .h file for details.

  1. This document assumes that you are creating a component in the framework named "foo".
  2. Pick a component name. It must obey the following guidelines:
    1. Be valid as a C variable name (no spaces/punctuation other than "_", start with a letter, etc.)
    2. Be all lower case
    3. Be unique across all other components in this MCA framework (i.e., in the "foo" framework). It is permissible to have another component with the same name in a different framework (and is actually somewhat desirable, if all the components of the same name shared characteristics, such as support the same specialized network).
    4. Is at least somewhat descriptive (be friendly to your fellow developers!)
  3. Create a directory with the component name in /mca/foo/. For the purposes of this document, we'll assume that your component name is "bar" (i.e., /mca/foo/bar/).
  4. A file named "configure.params" must be present in the top-level component directory. More details on this file are listed below.

The entire source code of the component must go into the single directory tree /mca/foo/bar/. The only exception to this rule is "common" pseudo-components, described below.

The configure.params file

A file named "configure.params" must exist in the component top-level directory (in this example, /mca/foo/bar/configure.params). If this file does not exist, autogen.sh will assume that the directory does not contain a valid component and will skip it.

The configure.params file must contain valid Bourne shell script syntax (although it does not need to have its executable bit set). It is executed during autogen.sh to specify configure and build behaviors of the component.

configure.params may assign the following shell variables:

  • PARAM_CONFIG_FILES: A space-delimited list of files that need to be generated by OMPI's top-level configure. Specifically, these files are added to AC_CONFIG_FILES. For example, if you have a Makefile.am in your component directory (which is recommended; see below), you would list Makefile in PARAM_CONFIG_FILES. If your component has multiple directories and you have multiple Makefile.am's, you need to list all of them, relative to the top-level component directory. For example:
PARAM_CONFIG_FILES="Makefile src/Makefile src/a/Makefile src/b/Makefile"
  • PARAM_VERSION_FILE: This field is only used when a configure.stub file is used to configure the component (see below). If specified, it is the name of a file, relative to the component directory, that specifies the version of the component. The file must be the same format as the top-level OMPI VERSION file (see that file for a discussion of its format). The following #defines will automatically be created (where "foo" and "bar" are specific to the framework and component) in a file named --version.h in the component directory:
    • MCA_foo_bar_MAJOR_VERSION
    • MCA_foo_bar_MINOR_VERSION
    • MCA_foo_bar_RELEASE_VERSION
    • MCA_foo_bar_GREEK_VERSION
    • MCA_foo_bar_SVN_VERSION
    • MCA_foo_bar_VERSION
  • PARAM_CONFIG_PRIORITY: This value is only meaningful if the component's framework has defined the configure module to be STOP_AT_FIRST_PRIORITY; it is ignored otherwise. If the framework defined STOP_AT_FIRST_PRIORITY, all components should set a PARAM_CONFIG_PRIORITY value; failure to do so will result in that component's priority automatically being set to zero.

Configuring the component

A component has multiple choices about how to configure itself (which is different than the decision about how to build itself -- see below). Specifically, the top-level OMPI configure script must be informed as to whether a given component wants to build or not.

This decision is made by either having one of the following files in the component's directory (each of which is described in detail below), or having none of these files:

  1. configure.m4 '''(this is the preferred file to use)'''
  2. configure
  3. configure.in or configure.ac
  4. configure.stub ''(this method is very old and may not work anymore)''

If none of these files are present, the top-level OMPI configure script assumes that the component has "configured" itself successfully. That is, having no file is a shortcut for assuring that the component will always be built.

The contents of the various "configure*" files are described below.

Using a "configure" file

If a component has its own configure script, it will be invoked by the top-level OMPI configure script. The return status of the configure script determines what happens to that component:

  • If the component configure script returns 0, the top-level OMPI configure script assumes that the component wants to be built, and therefore the relevant "make" targets will traverse into the component directory during the build process.
  • Any other return status results in the top-level OMPI configure script ignoring that component during the build process.

Using a "configure.in" or "configure.ac" file

If the component has its own configure.in or configure.ac file, OMPI's autogen.sh will turn into a full configure script via the proper GNU Autoconf / Automake / Libtool bootstrap commands.

The component is then treated just like it had its own configure script (see above).

Using a "configure.stub" file

If the component has a configure.stub file, OMPI's autogen.sh will turn into a full configure script OMPI's top-level configure script determines whether the component wants to build via the same rules as components that have their own configure script (see above).

NOTE: This method has not been used in a long, long time (specifically, when we introduced the configure.m4 method, the configure.stub method fell into disuse). It probably still works, but may have bit rotted...?

The configure.stub file can contain a variety of fields that eventually get expanded into a full configure script. This method has not been used in so long that there are no examples in the current OMPI trunk, and I'm not really motivated to look up what they are. :-)

Once the configure script is generated, the component is then treated just like it had its own configure script (see above).

Using a "configure.m4" file

''Using a configure.m4 file is the preferred method for configuring files that are in the main Open MPI code base.''

If the component has a configure.m4 file, it is slurped into the top-level OMPI configure script by autogen.sh. configure.m4 can contain a few different hooks that are executed at strategic points during configure that determine both how the component is configured and whether it will be built or not.

  • It is required that the configure.m4 file define an m4 macro named MCA_<framework>_<component>_CONFIG (in this example, MCA_foo_bar_CONFIG). The macro can execute any of the normal Autoconf / Automake tests and determine if it can built itself. The macro is also passed two parameters: $1 and $2. $1 should be executed if the macro determines that the component wants to be built; $2 should be executed otherwise.
    • It is permissible to invoke AC_ARG_WITH and AC_ARG_ENABLE, which will add --with/--without and --enable/--disable command line options to Open MPI's top-level configure script. These can be useful mechanisms to pass in parameters to your component's configure macro. For example, the openib BTL component invokes AC_ARG_WITH to create the --with-openib command line parameter. If a parameter is supplied, it is assumed to be the path location where the OpenFabrics header files and libraries are located. NOTE: If you use AC_ARG_WITH or AC_ARG_ENABLE, please also use AC_HELP_STRING to make a proper help string for when users invoke "configure --help".
    • You can generally call any AC_ or AM_ built-in tests that you want to determine if your component can build (e.g., test for the presence of libraries and/or header files, etc.). The only notable exception is AM_CONDITIONAL: '''you must not invoke AM_CONDITIONAL in the MCA_<framework>_<component>_CONFIG macro! ''' (you will get Automake errors if you do). Instead, you can invoke AM_CONDITIONAL in the MCA_<framework>_<component>_POST_CONFIG macro, described below.
    • You may AC_DEFINE[UNQUOTED] any values that you wish, but should generally prefix them with upper-case versions of "_" (for example, "FOO_BAR_WANT_GADGETS"), since preprocessor macros have only a single namespace.
    • Similarly, you may AC_SUBST any values that you wish, but must also obey the prefix rule. For example, it is permissible to AC_SUBST([foo_bar_CPPFLAGS]). You can then use this value in your component's Makefile.am (for example -- assuming that Open MPI's top-level configure is generating your Makefile). This can be useful if you need to pass additional -I, -L, and/or -l flags to build your component. See various other Open MPI component Makefile.am's for examples of this technique, such as the AC_SUBST's at the bottom of source: trunk/ompi/mca/btl/openib/configure.m4 and their corresponding use in source: trunk/ompi/mca/btl/openib/Makefile.am.
    • It is highly suggested to be verbose in the output of configure. Use AC_MSG_CHECKING and AC_MSG_RESULT to indicate what you are checking so that a human can read the output and generally surmise what is happening, and how/why individual tests succeed or fail.
    • If you set any of the compiler/assembler/linker flags in your configure.m4 script, you must reset them back to their original values before returning. For example (but not limited to): CPPFLAGS, CFLAGS, CXXFLAGS, FFLAGS, FCFLAGS, LDFLAGS, LIBS.
    • If your component cannot compile on a given platform, do not call AC_MSG_ERROR. AC_MSG_ERROR will abort the whole top-level OMPI configure script. Instead, just execute $2, which will cause OMPI to ignore this component and proceed with the rest of the build.
    • However, it is strongly suggested that if a specific option is requested (e.g., via a --with or --enable option), and that option is not possible, then you should AC_MSG_WARN to explain the problem and then AC_MSG_ERROR to abort. Specifically: if a user specifically asks for something and you cannot deliver it, it's an abortable error. But default configurations that will not build are not errors; it is perfectly acceptable to skip a component in a build.
    • OMPI defines several helper m4 macros that may be useful, for example:
      • OMPI_VAR_SCOPE_PUSH / OMPI_VAR_SCOPE_POP: For listing all the temporary shell variables that you will use in your macro. The _POP macro will unset all of them (to be social/safe with the rest of the Bourne shell code in configure).
      • OMPI_CHECK_PACKAGE: Checks for header files and corresponding library files (e.g., do all the work for checking for the OpenFabrics header and library files). See the comments in source: trunk/config/ompi_check_package.m4 for details.
  • The component can optionally define an MCA_<framework>_<component>_POST_CONFIG macro if it needs to invoke AM_CONDITIONAL. See below for an example.

Here's a trivial configure.m4 example:

# This macro is executed to determine if the component wants to be
# included in the Open MPI build process.  It is only executed if the
# user did not exclude this component and/or framework via the
# --with-mca-no-build OMPI configure command line switch (this is why
# you can't put AM_CONDITIONALs in this macro: the whole macro is
# executed conditionally).

AC_DEFUN([MCA_foo_bar_CONFIG],[
     OMPI_VAR_SCOPE_PUSH([foo_bar_message])

     # We need the header file "gadgets.h" to build this component
     AC_CHECK_HEADERS([gadgets.h], 
                      # If we have gadgets.h, see if we have the
                      # optional gadgets/froobles.h file
                      foo_bar_message=happy
                      [AC_CHECK_HEADERS([gadgets/froobles.h],
                                        [foo_bar_have_froobles=1
                                         foo_bar_message="super happy"])
                       foo_bar_happy=1],
                      [foo_bar_message=unhappy
                       foo_bar_happy=0])

     # This is a contrived message/example just to use $foo_bar_message,
     # just so that we can OMPI_VAR_SCOPE_PUSH/POP.
     AC_MSG_CHECKING([for bar happiness])
     AC_MSG_RESULT([$foo_bar_message])

     # Execute $1 if everything was happy above; $2 otherwise.  You
     # can imagine other logic flows to execute $1 or $2; this is just
     # one example.
     AS_IF([test "$foo_bar_happy" = "1"], [$1], [$2])
     AC_VAR_SCOPE_POP
])

# This macro is *always* executed during configure, and it is
# therefore safe to execute AM_CONDITIONAL in it.  $1 will be "1" if
# the MCA_btl_openib_CONFIG macro was executed.  Hence, you likely
# want to ensure to test the value of $1 in the AM_CONDITIONAL test
# clause (good defensive programming).

AC_DEFUN([MCA_btl_openib_POST_CONFIG], [

     # If we need to change build behavior depending on whether we
     # have gadgets/froobles.h, we can AM_CONDITIONAL with the result
     # of the test from the main _CONFIG macro, above.
     AM_CONDITIONAL([MCA_foo_bar_HAVE_FROOBLES], 
                    [test $1 -eq 1 -a "x$foo_bar_have_froobles" = "x1"])
])

Source code of the component

Generally, the source code is entirely up to you. :-)

However, the source code that is specific to this component must be entirely within the component directory (with the one exception of "common" pseudo-components, described below). That being said, you can call any library function in your component's library layer and below. For example, if your component is in the OMPI layer, it can call any OMPI, ORTE, or OPAL library call. If your component is in the ORTE layer, then it can only call any ORTE or OPAL library call (not OMPI).

It is illegal for components to directly call functions in other components (except for common pseudo-components; see below). Do not link your component against any other component (except for common pseudo-components) or assume that you can call any other component's internal functions. You must use published framework interfaces to call functions in other components.

You can have whatever directory layout you want in your component; many OMPI components are a single, top-level directory, but some have a multi-level directory structure. The specific directory layout of your component is irrelevant; the main requirement is that the output library/DSO must be in the top-level directory (see "Building the component", below).

Building the component

If a component was configured successfully, it can build itself however it wants. The only requirement is that it supports standard Automake "make" targets, such as (but not limited to):

  • all
  • install
  • clean
  • distclean
  • dist
  • test

Specifically, OMPI's top-level build process will traverse into the component's directory with any of the standard Automake "make" targets. These targets are expected the work in the same way as Automake-generated Makefiles.

Most (all?) components in the Open MPI tree have Makefile.am's and add themselves to the list of files to be generated by OMPI's top-level configure script (via the PARAM_CONFIG_FILES value in configure.params).

Components are built in either "DSO" mode or static mode (never both simultaneously):

  • If the component is building in dynamic shared object (DSO) mode, meaning that it creates a standalone dynamic library file that is dynamically opened at run-time, it must create a DSO named "mca__.", where is whatever extension is relevant for that platform (e.g., "so", "dll", etc.).
  • If the component is building in static mode, meaning that it will be included in the upper-level project library, it must create a Libtool convenience library named "libmca__.la".

NOTE: It is possible that these output filenames may change in the future in order to simplify some of the logic in component Makefile.am's.

Currently, the only way to tell which way a component is to be built is via the AM_CONDITIONAL OMPI_BUILD_<framework>_<component>_DSO. If it is true, the component is to be built as a DSO. Otherwise, it needs to be built statically. '''This may be problematic for components that use their own configure scripts, or a configure.stub file. This may need to be fixed, if anyone cares! ''' (it's not a huge problem for the OMPI code base because all components use the configure.m4 method)

There are many good examples of component Makefile.am's in the OMPI source tree. Two such examples are:

  • source: trunk/ompi/mca/btl/tcp/Makefile.am: A "plain vanilla" Makefile.am that simply lists several header and source files and builds them into either a DSO or a Libtool convenience library.
  • source: trunk/ompi/mca/btl/openib/Makefile.am: A fairly complex Makefile.am, that has all the same elements as the TCP BTL makefile, but also includes features such as:
    • Flex-generated source code
    • AMCA parameter file installation
    • Run-time help message file installation
    • Header and source files for the component itself
    • Conditional compilation of several component source files
    • Dependency on external libraries, and therefore adds CPPFLAGS, LDFLAGS, and LIBS to the component build process

"Common" pseudo-components

It is not uncommon to need to share some common code between multiple different components in different frameworks. For example, OpenFabrics-based components in both the point-to-point and collective MPI frameworks may want to share queue pair management code. Rather than duplicating this code in each component, it may be better to put this code in a single OpenFabrics "common" component.

NOTE: There is currently a limitation that the "common" pseudo-framework can only exist in the Open MPI project (i.e., "ompi/"). The ORTE and OPAL trees do not have a "common" pseudo-component. If you need a "common" pseudo-component in ORTE or OPAL, please send mail to the OMPI developer's mailing list.

The common framework in the OMPI project (i.e., ompi/mca/common) is unlike all other frameworks. There are no rules about interfaces, there are no requirements for framework open and close functions, there is no ompi/mca/common/base directory, etc. Simply put, the "common" tree is a repository for code shared between components in any OMPI project framework.

That being said, pseudo-components in the common framework do abide by the same configure.params / configure.m4 / etc. rules described above. As such, for code to be compiled in the common framework, each pseudo-component must configure successfully (just like a real component).

Just like real components, pseudo-components can be built as DSOs or statically. When built statically, the code is slurped into the upper-level libmpi.la and is therefore available to any component that wishes to use that code. When built as a DSO, it can be convenient to simply link components against the DSO.

A lengthy comment in source: trunk/ompi/mca/common/sm/Makefile.am explains the rationale and how it works. This Makefile.am also provides a template for how to compile common pseudo-components.

For example, say that the "yow" component in the "foo" framework and the "yow" component in the "bar" framework need to share some code.

  1. A developer can create the ompi/mca/common/yow/ pseudo-component and put some code in there that is used by both foo::yow and bar::yow components.
  2. Both the foo::yow and bar::yow components can link against the yow pseudo-component by adding $(top_ompi_builddir)/ompi/mca/common/yow/libmca_common_yow.la to their LIBADD line in the DSO case. See source: trunk/ompi/mca/btl/sm/Makefile.am as an example.
Clone this wiki locally