The directory examples into the root of the project contains some simple examples to try OFF. These examples are designed to learn OFF quickly and easily. The examples are categorized into 2 classes, namely one-dimensional and two-dimensional. The followings instructions drive you to the examples execution.
The first 1D example concerns the solution of the Riemann Problem of Sod, that is a classical 1D shock tube test. In order to complete the SOD example, IBM, OFF and POG must be properly compiled.
IBM and POG can be compiled only with serial support, whereas OFF has support for serial, OpenMP, MPI and OpenMP/MPI execution. POG can be compiled with or without the Tecplot Inc. proprietary IO library for saving Tecplot binary outputs.
Let us start cleaning the project root directory:
make cleanall
Then compile codes:
make IBM COMPILER=intel DEBUG=no OPENMP=no MPI=no NULj=yes NULk=yes
make POG COMPILER=intel DEBUG=no OPENMP=no MPI=no NULj=yes NULk=yes TECIO=yes
make clean
make codes COMPILER=intel DEBUG=no OPENMP=no MPI=no NULj=yes NULk=yes
make clean
make OFF COMPILER=intel DEBUG=no OPENMP=no MPI=no NULj=yes NULk=yes
make clean
make OFF COMPILER=intel DEBUG=no OPENMP=yes MPI=no NULj=yes NULk=yes
make clean
make OFF COMPILER=intel DEBUG=no OPENMP=no MPI=yes NULj=yes NULk=yes
make clean
make OFF COMPILER=intel DEBUG=no OPENMP=yes MPI=yes NULj=yes NULk=yes
To run the Sod's shock tube test navigate to its working directory:
cd examples/1D/shock_tube/SOD/
There are 3 sub-directories:
- ibm
- off
- pog
Into each of the above directories there is a bash script, run[IBM,OFF,POG].sh_ that help you to complete the example. Moreover, there are the necessary option files to run each code. The bash scripts are designed to be executed on a workstation where you can directly execute programs without dealing with a resource manager system that schedules the jobs, such as Torque. For the execution of this example on a system using a jobs scheduler see the subsection How-to facing with a jobs scheduler like Torque.
The 3 subdirectories ibm, off and pog indicate the 3 steps that must be completed in sequence.
Firstly, navigate to ibm directory and building the initial and boundary conditions and the mesh file (generating the input file for OFF):
cd ibm
Then you have 2 options:
- clean ibm directory:
run_IBM.sh -clean - run IBM for generating OFF input files:
run_IBM.sh -out
The above commands will produce the correct input for OFF. The produced files are placed into output directory inside ibm one.
Now, navigate to off directory
cd ../off
Again, you have 3 options:
- clean off directory:
run_OFF.sh -clean - run OFF in serial or OpenMP mode:
run_OFF.sh -no_mpi - run OFF in MPI mode:
run_OFF.sh -mpi
The above commands will perform the simulation. The produced files are placed into output directory inside the off one.
Finally, postprocess the result obtained. Navigate to pog directory:
cd ../pog
You have 4 options:
- clean pog directory:
run_POG.sh -clean - produce Gnuplot postprocessed files:
run_POG.sh -gnu - produce Tecplot postprocessed files:
run_POG.sh -tec - produce VTK postprocessed files:
run_POG.sh -vtk
The above commands will post-process the simulation results. The produced files are placed into output directory inside pog one. It is worth noting that the obtained results can be compared with the OFF reference ones contained into output-ref directory. Moreover, into the file exact_sod-100-t02-block-split.dat the exact solution of Sod's problem are reported for validation purpose.
The first 2D examples concern with 2D Riemann problems. The initial conditions are the same as the ones reported by Kurganov and Tadmor. In order to complete the SOD example, IBM, OFF and POG must be properly compiled.
IBM and POG can be compiled only with serial support, whereas OFF has support for serial, OpenMP, MPI and OpenMP/MPI execution. POG can be compiled with or without the Tecplot Inc. proprietary IO library for saving Tecplot binary outputs.
Let us start cleaning the project root directory:
make cleanall
Then compile codes:
make IBM COMPILER=intel DEBUG=no OPENMP=no MPI=no NULk=yes
make POG COMPILER=intel DEBUG=no OPENMP=no MPI=no NULk=yes TECIO=yes
make clean
make codes COMPILER=intel DEBUG=no OPENMP=no MPI=no NULk=yes
make clean
make OFF COMPILER=intel DEBUG=no OPENMP=no MPI=no NULk=yes
make clean
make OFF COMPILER=intel DEBUG=no OPENMP=yes MPI=no NULk=yes
make clean
make OFF COMPILER=intel DEBUG=no OPENMP=no MPI=yes NULk=yes
make clean
make OFF COMPILER=intel DEBUG=no OPENMP=yes MPI=yes NULk=yes
To run a 2D Riemann Problem example navigate to its working directory:
cd examples/2D/two_dimensional_riemann_problems/kt-c??/
Note: substitute ?? with one of 03,04,05,06,12,17
For each kt-?? there are 3 sub-directories:
- ibm
- off
- pog
Into each of the above directories there is a bash script, run[IBM,OFF,POG].sh_ that help you to complete the example. Moreover, there are the necessary option files to run each code. The bash scripts are designed to be executed on a workstation where you can directly execute programs without dealing with a resource manager system that schedules the jobs, such as Torque. For the execution of this example on a system using a jobs scheduler see the subsection How-to facing with a jobs scheduler like Torque.
The 3 subdirectories ibm, off and pog indicate the 3 steps that must be completed in sequence.
Firstly, navigate to ibm directory and building the initial and boundary conditions and the mesh file (generating the input file for OFF):
cd ibm
Then you have 2 options:
- clean ibm directory:
run_IBM.sh -clean - run IBM for generating OFF input files:
run_IBM.sh -out
The above commands will produce the correct input for OFF. The produced files are placed into output directory inside ibm one.
Now, navigate to off directory
cd ../off
Again, you have 3 options:
- clean off directory:
run_OFF.sh -clean - run OFF in serial or OpenMP mode:
run_OFF.sh -no_mpi - run OFF in MPI mode:
run_OFF.sh -mpi
The above commands will perform the simulation. The produced files are placed into output directory inside the off one.
Finally, postprocess the result obtained. Navigate to pog directory:
cd ../pog
You have 4 options:
- clean pog directory:
run_POG.sh -clean - produce Gnuplot postprocessed files:
run_POG.sh -gnu - produce Tecplot postprocessed files:
run_POG.sh -tec - produce VTK postprocessed files:
run_POG.sh -vtk
The above commands will post-process the simulation results. The produced files are placed into output directory inside pog one. It is worth noting that the obtained results can be compared with the OFF reference ones contained into output-ref directory.
It is worth noting that all the 2D Riemann problems are set to finish at 20 time steps in order to quickly complete the examples. However, the solutions obtained have no physical meaning because the final time of integration is too small. If you want to complete one of the 2D Riemann problems reaching the same physical time of the solution of Kurganov and Tadmor you must edit one input file into ibm directory of the kt-?? example chosen:
cd ibm
cd input
Into this directory there is the option file solver.dat: the fourth line should contain the maximum number of iterations performed, Nmax, just set it to 0 in order to finish the simulation at the physical time set below, Tmax, instead of using Nmax.
In the High Performance Computing (HPC) context it is usual to have a resources manager program that schedules the jobs in order to optimize the computing resources. Portable Batch System (PBS) is one of the most used scheduler considering all of its forks, e.g. Torque being based on the OpenPBS fork of the original PBS project. In this framework the bash script for running OFF on parallel MPI-enabled architecture cannot be used. Indeed, a job-submission script, suitable for the actual scheduler used, must be created. For the non expert users of Torque-like scheduler an example of job-submission script is here reported. Assuming the input files for OFF have already created, navigate to off directory and save the following job-submission example file to runjob.pbs:
#!/bin/bash
#PBS -N OFF
#PBS -q debug_eth
#PBS -l nodes=1:ppn=2
#PBS -l walltime=00:30:00
#PBS -j oe
#PBS -V
# in many PBS-like frameworks it is suggested to explicity declare/navigate the current working directory
cd $PBS_O_WORKDIR
# load here the modules eventually needed
module add ...
# set the necessary enviroments variables
export ...
# OFF stuffs
rm -rf output lockfile input # cleaning working directory
ln -fs ../ibm/output input # link to IBM output
mkdir -p output # creating outout directory
cd input ; ln -fs procmap-mpi.dat procmap.dat ; cd ../ # linking procmap.dat to map with MPI
# running OFF; note that the number of MPI process is left generic (mpi_procs) and you must specify other mpirun-options eventually nedeed (other_options)
mpirun -np mpi_procs [other_options] ./OFF off_options.datOnce the runjob.pbs file has been properly edited accordingly to your scheduler just submit it (maybe something like qsub runjob.pbs).
The final POG post-processing phase can be completed as detailed in the above sections.