-
Notifications
You must be signed in to change notification settings - Fork 239
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'docu-for-openmp-and-log' into 'master'
[doc] Added documentation for OpenMP and debug output See merge request ogs/ogs!5100
- Loading branch information
Showing
3 changed files
with
126 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
+++ | ||
date = "2024-09-09T13:52:17+02:00" | ||
title = "OpenMP parallelization" | ||
author = "Christoph Lehmann" | ||
weight = 9 | ||
|
||
|
||
[menu] | ||
[menu.userguide] | ||
parent = "basics" | ||
+++ | ||
|
||
OpenGeoSys supports shared memory parallelization (multi threading) to some | ||
extent: by default OpenGeoSys is built with OpenMP support and OpenMP-enabled | ||
linear solvers will automatically make use of it. | ||
|
||
The number of threads a linear solver uses can be controlled with the | ||
environment variable | ||
[`OMP_NUM_THREADS`](https://www.openmp.org/spec-html/5.0/openmpse50.html). | ||
|
||
Note: | ||
|
||
* Some linear solvers and some preconditioners from the Eigen library run with a | ||
single thread only, see | ||
[here](https://eigen.tuxfamily.org/dox/TopicMultiThreading.html). | ||
* When using distributed memory parallelization (MPI) it might not make sense to | ||
use OpenMP at the same time: the different threads might compete for the same | ||
CPU resources. | ||
* There is an interference between the Intel MKL and OpenMP parallelization: If | ||
you use OGS together with the former, you might break the latter. In that case | ||
you might want to change the | ||
[`OGS_EIGEN_PARALLEL_BACKEND`](/docs/devguide/advanced/configuration-options/). | ||
|
||
In addition, the assembly of some processes in OGS can be run | ||
OpenMP-parallelized when setting the environment variable `OGS_ASM_THREADS` to | ||
the number of threads to be used. At the moment (2024-09-09) that's supported by | ||
the `HydroMechanics`, `TH2M`, `ThermoRichardsMechanics`, and | ||
`ThermoRichardsFlow` processes. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
+++ | ||
date = "2024-09-10T10:57:31+02:00" | ||
title = "Debugging Options" | ||
author = "Christoph Lehmann" | ||
weight = 1 | ||
+++ | ||
|
||
If you encounter an error or unexpected behaviour during an OGS simulation there | ||
are a couple of things you can do to investigate the error. These are, e.g... | ||
|
||
## Increasing the Log Level | ||
|
||
Running OGS with the [command line option](/docs/userguide/basics/cli-arguments/) | ||
`-l debug` will make OGS print a lot of debug output, which might help locating | ||
the cause of the error. | ||
|
||
## Writing local and/or global Vectors and Matrices | ||
|
||
When the environment variables `OGS_LOCAL_MAT_OUT_PREFIX="<some prefix>"` and | ||
`OGS_LOCAL_MAT_OUT_ELEMENTS="0 1 5 16 123"` (or `"*"` to select all elements) | ||
are set, OGS will write the local matrices and vectors it has assembled to | ||
`<some prefix>ogs_local_matrix.log`. That file can be checked, if the assembled | ||
Jacobian is singular, among others. `<some prefix>` can contain slashes, making | ||
output to an arbitrary directory possible. | ||
|
||
Likewise, output of global matrices can be requested via | ||
`OGS_GLOBAL_MAT_OUT_PREFIX="<some prefix>"`. That will make OGS write a series | ||
of files, one for each global matrix/vector for each non-linear iteration. | ||
|
||
Local matrix output is available for all processes in OGS, as of now | ||
(2024-09-10) global matrix output is available only for the `LARGE_DEFORMATION`, | ||
`SMALL_DEFORMATION`, `ThermalTwoPhaseFlowWithPP`, `ThermoRichardsMechanics` and | ||
`TH2M` processes, and only for serial (non-PETSc) runs. | ||
|
||
## Generating more output | ||
|
||
When the [`<time_loop/output/variables>`](https://doxygen.opengeosys.org/d2/d5b/ogs_file_param__prj__time_loop__output__variables) | ||
list in the OGS project file is empty, OGS will write all available data to its | ||
output files. | ||
|
||
Furthermore, you can set | ||
[`<time_loop/output/output_iteration_results>`](https://doxygen.opengeosys.org/dc/d50/ogs_file_param__prj__time_loop__output__output_iteration_results.html) | ||
to true. Then OGS will generate output files after each non-linear iteration, | ||
which might help to find the cause of a non-linear solver divergence. | ||
This option is available for VTK output only (as of Sep. 2024). | ||
|
||
When interpreting secondary variables it makes sense to enable | ||
[`<time_loop/output/output_extrapolation_residuals>`](https://doxygen.opengeosys.org/d1/d30/ogs_file_param__prj__time_loop__output__output_extrapolation_residuals.html). | ||
The generated extrapolation residuals provide a measure of the extrapolation | ||
errors of secondary variables. | ||
|
||
## Enabling Floating Point Exceptions | ||
|
||
If you are running OGS on Linux you can pass it the [command line option](/docs/userguide/basics/cli-arguments/) | ||
`--enable-fpe` while running OGS in a debugger. That command line option will | ||
raise a signal (i.e., make OGS crash) if there is a divide by zero or other | ||
illegal floating point operation. The debugger will notice that situation and | ||
stop at the respective source location. That, of course, requires an OGS build | ||
with debug information (this is different from debug output above) enabled. See | ||
[here](/docs/devguide/getting-started/build-configuration/#available-cmake-presets) | ||
for more information. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters