Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

code to link SmartRedis tools #4045

Draft
wants to merge 22 commits into
base: master
Choose a base branch
from
Draft

code to link SmartRedis tools #4045

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
56880aa
code to link SmartRedis tools
jedwards4b Jul 8, 2021
e567cf2
add a SmartSim test to xatm component
jedwards4b Jul 16, 2021
ebcec81
add smartsim scripts
jedwards4b Jul 16, 2021
cdc4fb1
add a readme
jedwards4b Jul 19, 2021
2fdc56d
remove ifdefs from xatm
jedwards4b Jul 19, 2021
7aa8d79
remove commented section
jedwards4b Jul 19, 2021
a51afac
remove unused utils
jedwards4b Jul 19, 2021
a9ccf7d
use module for SmartRedis
jedwards4b Jul 20, 2021
f8562ae
add rpath to smartsim
jedwards4b Jul 21, 2021
845bfb8
add sleep to case run for smartsim db to start first
jedwards4b Jul 21, 2021
2004215
add gptl ifdefs to casper
jedwards4b Jul 22, 2021
7b94f9a
remove this sleep
jedwards4b Jul 22, 2021
b1823ff
add to README
jedwards4b Jul 22, 2021
51d4676
add documentation
jedwards4b Jul 22, 2021
347313a
add launch wrapper to test scheduler for smartsim test
jedwards4b Jul 23, 2021
85d4146
smartsim to cheyenne gnu
jedwards4b Aug 2, 2021
192c418
smartsim setup
jedwards4b Aug 2, 2021
f5739d0
now working on cheyenne
jedwards4b Aug 3, 2021
d5b122f
fix error when testmods not defined
jedwards4b Aug 16, 2021
f7181f9
rebase to cime master
jedwards4b Sep 10, 2021
98fd5a9
need to explicitly forward environment
jedwards4b Nov 11, 2021
febc3b3
updates to smartredis interface
jedwards4b Nov 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions config/cesm/machines/config_batch.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
batch_redirect: Whether a redirect character is needed to submit jobs.
batch_directive: The string that prepends a batch directive for the batch system.
jobid_pattern: A perl regular expression used to filter out the returned job id from a
queue submission.
queue submission.

===============================================================
batch_system
Expand Down Expand Up @@ -243,7 +243,12 @@

<directives queue="casper" compiler="intel">
<directive default="/bin/bash" > -S {{ shell }} </directive>
<directive> -l select={{ num_nodes }}:ncpus={{ max_tasks_per_node }}:mpiprocs={{ tasks_per_node }}:ompthreads={{ thread_count }}:mem=700GB:ngpus={{ ngpus_per_node }} </directive>
<directive> -l select={{ num_nodes }}:ncpus={{ max_tasks_per_node }}:mpiprocs={{ tasks_per_node }}:ompthreads={{ thread_count }} </directive>
</directives>

<directives queue="casper" compiler="gnu">
<directive default="/bin/bash" > -S {{ shell }} </directive>
<directive> -l select={{ num_nodes }}:ncpus={{ max_tasks_per_node }}:mpiprocs={{ tasks_per_node }}:ompthreads={{ thread_count }} </directive>
</directives>

<!-- Unknown queues use the batch directives for the regular queue -->
Expand Down Expand Up @@ -423,7 +428,7 @@
</batch_system>

<batch_system MACH="frontera" type="slurm" >
<!-- note this form of batch_submit causes some tests to fail on the system
<!-- note this form of batch_submit causes some tests to fail on the system
because env variables are not forwarded -->
<batch_submit>ssh login1 cd $CASEROOT ; sbatch</batch_submit>
<submit_args>
Expand Down Expand Up @@ -468,8 +473,8 @@
</submit_args>
<queues>
<!-- Want case.st_archiver to run on first available queue, but case.run to use sib2.9
We achieve this setting both queues as the default queue but limiting it
to jobs that use 8 tasks or fewer; other jobs will land in sib2.9 -->
We achieve this setting both queues as the default queue but limiting it
to jobs that use 8 tasks or fewer; other jobs will land in sib2.9 -->
<queue nodemin="1" nodemax="45">sib2.9</queue>
<queue default="true" jobmin="1" jobmax="8">sib2.9,sky2.4</queue>
</queues>
Expand All @@ -482,8 +487,8 @@
</submit_args>
<queues>
<!-- Want case.st_archiver to run on first available queue, but case.run to use sky2.4
We achieve this setting both queues as the default queue but limiting it
to jobs that use 8 tasks or fewer; other jobs will land in sky2.4 -->
We achieve this setting both queues as the default queue but limiting it
to jobs that use 8 tasks or fewer; other jobs will land in sky2.4 -->
<queue nodemin="1" nodemax="13">sky2.4</queue>
<queue default="true" jobmin="1" jobmax="8">sib2.9,sky2.4</queue>
</queues>
Expand Down Expand Up @@ -721,7 +726,7 @@
<queues>
<queue walltimemax="02:00:00" nodemin="1" nodemax="4" default="true">regular</queue>
</queues>
</batch_system>
</batch_system>

<batch_system MACH="zeus" type="lsf">
<batch_env>-env</batch_env>
Expand Down
18 changes: 16 additions & 2 deletions config/cesm/machines/config_compilers.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -826,6 +826,10 @@ using a fortran linker.
<NETCDF_PATH> $ENV{NETCDF}</NETCDF_PATH>
<PIO_FILESYSTEM_HINTS>gpfs</PIO_FILESYSTEM_HINTS>
<PNETCDF_PATH>$ENV{PNETCDF}</PNETCDF_PATH>
<CPPDEFS>
<!-- these flags enable nano timers -->
<append MODEL="gptl"> -DHAVE_NANOTIME -DBIT64 -DHAVE_VPRINTF -DHAVE_BACKTRACE -DHAVE_SLASHPROC -DHAVE_COMM_F2C -DHAVE_TIMES -DHAVE_GETTIMEOFDAY </append>
</CPPDEFS>
</compiler>

<compiler MACH="casper" COMPILER="pgi">
Expand Down Expand Up @@ -902,11 +906,15 @@ using a fortran linker.

<compiler MACH="casper" COMPILER="intel">
<CFLAGS>
<append> -qopt-report -xCORE_AVX2 -no-fma</append>
<append> -qopt-report -xCORE_AVX2 -no-fma </append>
</CFLAGS>
<FFLAGS>
<append> -qopt-report -xCORE_AVX2 -no-fma</append>
</FFLAGS>
<LDFLAGS>
<!-- the path to gcc should match that used in building the SmartRedis lib -->
<append>-Wl,-rpath /glade/u/apps/dav/opt/gnu/9.1.0/lib64 </append>
</LDFLAGS>
<CMAKE_OPTS>
<append DEBUG="TRUE"> -DPIO_ENABLE_LOGGING=ON </append>
</CMAKE_OPTS>
Expand All @@ -933,7 +941,7 @@ using a fortran linker.
<!-- -fallow-argument-mismatch is needed for building with the mpich library with gfortran10;
-fallow-invalid-boz is needed for some components that have non-standard uses of BOZ literal constants with gfortran10.
See https://github.com/ESMCI/cime/issues/3846 for details. -->
<append> -fallow-argument-mismatch -fallow-invalid-boz </append>
<!-- <append> -fallow-argument-mismatch -fallow-invalid-boz </append> -->
</FFLAGS>
<SLIBS>
<append> -ldl </append>
Expand All @@ -954,8 +962,14 @@ using a fortran linker.
<PFUNIT_PATH MPILIB="mpt" compile_threaded="TRUE">$ENV{CESMDATAROOT}/tools/pFUnit/pFUnit3.2.8_cheyenne_Intel17.0.1_MPI_openMP</PFUNIT_PATH>
<!-- SET to FALSE for intel 17 and 18 TRUE otherwise -->
<HAS_F2008_CONTIGUOUS>TRUE</HAS_F2008_CONTIGUOUS>
<LDFLAGS>
<!-- the path to gcc should match that used in building the SmartRedis lib -->
<append MPILIB="openmpi"> -std=c11 -gxx-name=/glade/u/apps/ch/opt/gnu/9.1.0/bin/g++ -Wl,-rpath,/glade/u/apps/ch/opt/gnu/9.1.0/lib64 -L/glade/u/apps/ch/opt/gnu/9.1.0/lib64 -lgcc </append>
</LDFLAGS>

</compiler>


<compiler MACH="cheyenne" COMPILER="pgi">
<SLIBS>
<append> -llapack -lblas </append>
Expand Down
Loading