Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix LDFLAGS definition and usage #199

Merged
merged 1 commit into from
Feb 28, 2024

Conversation

SeanBryan51
Copy link
Collaborator

@SeanBryan51 SeanBryan51 commented Jan 22, 2024

ld flags are currently specified incorrectly. For example:

  1. -lnetcdf is not required since it is a dependency of the netCDF Fortran library - only -lnetcdff is needed.
  2. ld flags are being split across two variables: LDFLAGS and LD when there should only be one (LD typically refers to the path to the ld executable). Having the flags split across two variables allows for the flags to be passed to the compiler in an inconsistent way.

See #198 for more details.

This change removes the LD variable and any hard-coded linker flags or include flags in the Makefile or build script. Linker flags and include flags are replaced with a query to pkg-config so that the correct flags are used. Using pkg-config also insulates the build system from changes to library versions and dependencies.

This PR should ideally be done after #197 is merged so that we can verify compiled object files have not been altered as a result of this change.

Compiled object files are identical between this branch and the main branch. However, the executables differ between the two branches due to changes in the compiler arguments at the linking step.

Regression tests using benchcab* (bitwise comparison of model output via nccmp) show that model output is bitwise identical between the current branch and the main branch for serial and MPI model runs.

*executables were built manually for the spatial case (see this related issue: CABLE-LSM/benchcab#244).

Fixes #198


📚 Documentation preview 📚: https://cable--199.org.readthedocs.build/en/199/

@SeanBryan51 SeanBryan51 linked an issue Jan 22, 2024 that may be closed by this pull request
@SeanBryan51 SeanBryan51 force-pushed the 198-ld-flags-used-in-offline-build-are-incorrect branch 2 times, most recently from a13c268 to 56954e3 Compare January 24, 2024 05:52
@SeanBryan51 SeanBryan51 marked this pull request as ready for review January 24, 2024 23:39
@SeanBryan51 SeanBryan51 force-pushed the 198-ld-flags-used-in-offline-build-are-incorrect branch 5 times, most recently from 1e6d325 to aaebf2a Compare January 25, 2024 06:26
@ccarouge
Copy link
Collaborator

@harshula Is pkg-config available anywhere? Do system administrators have to decide to set it up? I want to make sure this would work on other systems as well without too many modifications.

@harshula
Copy link

Hi, Good question! My understanding is that it is a default package on GNU/Linux distros.

@SeanBryan51
Copy link
Collaborator Author

SeanBryan51 commented Jan 31, 2024

This is more of a question around the CMake implementation (#200) but it is still relevant. If pkg-config is not a standard tool on most systems, should parts of the build which depend on pkg-config be guarded by an "if on Gadi" check? Currently, the CMake implementation needs pkg-config to link against the netcdf-fortran library. We could possibly look into linking against netcdf dependency using find_package directly instead of relying on pkg-config?

@harshula I see this was done for libaccessom2: ACCESS-NRI/libaccessom2@3311b58. Micael's CMake implementation for mom6 also uses find_package(NetCDF), except it avoids the complications with Gadi since it uses netcdf libraries installed via spack. We could try find_package(NetCDF) and fall back to using pkg-config if that fails?

@harshula
Copy link

With Spack, it's as simple as, depends_on("pkgconf", type="build"). Have you come across any systems that Cable is compiled on that does not have pkg-config? If you are worried by an unlikely scenario, try pkg-config and then fallback to something else.

@SeanBryan51
Copy link
Collaborator Author

It also looks like the find_package(NetCDF) approach will likely require maintaining a FindNetCDF.cmake file which is not ideal: https://stackoverflow.com/questions/41571163/write-cmake-find-netcdf-for-c-and-fortran-library. Using pkg-config seems like a better alternative to this.

Copy link
Collaborator

@ccarouge ccarouge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am approving but I need to check the portability of pkg_config. We have at least one active developer in Europe who does not use Gadi. And I have no idea what's available on their system.
Depending on what comes out of the enquiry, further adaptation might be required.

Should the prepend_path be guarded by a "if gadi" condition? Or do we only worry about that for the CMake version.

@SeanBryan51
Copy link
Collaborator Author

Should the prepend_path be guarded by a "if gadi" condition? Or do we only worry about that for the CMake version.

It is implied that anything under host_gadi is specific to Gadi.

`ld` flags are currently specified incorrectly. For example:
1. `-lnetcdf` is not required since it is a dependency of the netCDF
Fortran library - only `-lnetcdff` is needed.
2. `ld` flags are being split across two variables: `LDFLAGS` and `LD`
when there should only be one (`LD` typically refers to the path  to the
`ld` executable). Having the flags split across two variables allows for
the flags to be passed to the compiler in an inconsistent way.
See #198 for more details.

This change removes the `LD` variable and any hard-coded linker flags or
include flags in the Makefile or build script. Linker flags and include
flags are replaced with a query to `pkg-config` so that the build system
is insulated from changes to library versions and dependencies.

Fixes #198
@SeanBryan51 SeanBryan51 force-pushed the 198-ld-flags-used-in-offline-build-are-incorrect branch from af1f126 to 379a9cd Compare January 31, 2024 23:09
@harshula
Copy link

harshula commented Feb 1, 2024

On Rocky Linux:

[root@3c6c3cddd8f4 /]# cat /etc/os-release 
NAME="Rocky Linux"
VERSION="8.9 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.9"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.9 (Green Obsidian)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2029-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-8"
ROCKY_SUPPORT_PRODUCT_VERSION="8.9"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.9"

pkgconf is installed by the Development Tools group:

[root@3c6c3cddd8f4 /]# dnf group info "Development Tools"
Last metadata expiration check: 0:28:59 ago on Thu Feb  1 05:30:48 2024.

Group: Development Tools
 Description: A basic development environment.
 Mandatory Packages:
   autoconf
   automake
   binutils
   bison
   flex
   gcc
   gcc-c++
   gdb
   glibc-devel
   libtool
   make
   pkgconf
   pkgconf-m4
   pkgconf-pkg-config
   redhat-rpm-config
   rpm-build
   rpm-sign
   strace
 Default Packages:
   asciidoc
   byacc
   ctags
   diffstat
   elfutils-libelf-devel
   git
   intltool
   jna
   ltrace
   patchutils
   perl-Fedora-VSP
   perl-Sys-Syslog
   perl-generators
   pesign
   source-highlight
   systemtap
   valgrind
   valgrind-devel
 Optional Packages:
   cmake
   expect
   rpmdevtools
   rpmlint

On RHEL clones, it's safe to conclude that pkg-config will be installed by default alongside well-known Development Tools.

@ccarouge
Copy link
Collaborator

ccarouge commented Feb 2, 2024

It is implied that anything under host_gadi is specific to Gadi.

Aahh, the diff view in the review didn't show the host_gadi line.

@ccarouge
Copy link
Collaborator

ccarouge commented Feb 2, 2024

Since I hadn't seen the almost entirety of the build script is within a host_gadi section, this seems to make my concerns about pkg_config irrelevant. On another machine, people will have to write their own host_x anyway and then changing pkg_config for something else in that host_x section is fine. Or are there planned changes that affect that conclusion?

@SeanBryan51
Copy link
Collaborator Author

Since I hadn't seen the almost entirety of the build script is within a host_gadi section, this seems to make my concerns about pkg_config irrelevant. On another machine, people will have to write their own host_x anyway and then changing pkg_config for something else in that host_x section is fine. Or are there planned changes that affect that conclusion?

@ccarouge it will be relevant for CMake as pkg-config is a required dependency, in CMakeLists.txt:

find_package(PkgConfig REQUIRED)

@SeanBryan51 SeanBryan51 merged commit 4be9a61 into main Feb 28, 2024
3 checks passed
@SeanBryan51 SeanBryan51 deleted the 198-ld-flags-used-in-offline-build-are-incorrect branch February 28, 2024 02:07
SeanBryan51 added a commit that referenced this pull request Mar 1, 2024
In preparation for building CABLE with [spack](https://spack.io/),
@harshula, @ccarouge and I have found areas where the current build
system for CABLE offline can be improved (see #188 for more details).
This change adds a [CMake](https://cmake.org/) based build system to
address these issues:

- #190
Although this problem can be solved in the Makefile by specifying the
relative paths to each source file, it requires updating all the
Makefile Fortran module dependency rules which is a pain. CMake
automatically handles Fortran module dependencies out of the box so
doing this with CMake is much easier.
- #191 
When invoking `cmake`, custom options, compilers and compiler flags can
be configurable by setting the appropriate variables via the [`-D`
flag](https://cmake.org/cmake/help/latest/manual/cmake.1.html#cmdoption-cmake-D),
for example: `cmake -DCMAKE_Fortran_COMPILER="mpif90"
-DCMAKE_Fortran_FLAGS="-O2 -fp-model precise" -S . -B build`. These can
be used for doing MPI only builds and enabling specific debug flags.
- #192 
As mentioned above, CMake automatically handles Fortran module
dependencies out of the box. Parallelised builds can be done by
specifying the `-j` flag to `cmake`.

The following pull requests should be merged before this pull request so
that we can demonstrate binary equivalence between the executables
compiled by both Makefile and CMake builds. This ensures no unwanted
changes are introduced by the transition.

- [x] #197
- [x] #199

Edit: the above PR's have been merged - the serial and MPI binaries
built by CMake (in release mode) are equivalent to the binaries built by
the Makefile based build system in the main branch.

Fixes #188, #190, #191, #192

<!-- readthedocs-preview cable start -->
----
📚 Documentation preview 📚:
https://cable--200.org.readthedocs.build/en/200/

<!-- readthedocs-preview cable end -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ld flags used in offline build are incorrect
3 participants