Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fixes to products for when REPLAY IC's are used #2755

Merged
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
9e6206c
Update linters.yaml
EricSinsky-NOAA Jun 3, 2024
4e13fbe
Revert changes in linters.yaml
EricSinsky-NOAA Jun 4, 2024
1eb0c26
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 7, 2024
720276b
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 12, 2024
219adb2
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 20, 2024
37e2a6f
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 24, 2024
b7ba980
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 25, 2024
a3efe66
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 26, 2024
12e1ec1
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jul 8, 2024
557ac3f
Add changes related to offset
EricSinsky-NOAA Jul 8, 2024
6c2b7da
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Jul 10, 2024
15de909
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Jul 11, 2024
c1ddb4a
Replace NDATE with date
EricSinsky-NOAA Jul 11, 2024
c1ffe30
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Jul 16, 2024
0dba84e
Move where replay diag_table is defined
EricSinsky-NOAA Jul 16, 2024
fb440c6
Move dest_file outside if-statement
EricSinsky-NOAA Jul 16, 2024
5f393b1
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Jul 18, 2024
dcb8066
Add bugfix in exglobal_stage_ic
EricSinsky-NOAA Jul 18, 2024
2936472
Revert "Add bugfix in exglobal_stage_ic"
EricSinsky-NOAA Jul 24, 2024
8cf65f5
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Jul 24, 2024
77b00cf
Add fix for fv3 filename when fhr is decimal
EricSinsky-NOAA Jul 25, 2024
d1efec8
Address shell check error
EricSinsky-NOAA Jul 25, 2024
78c4b59
Add more fixes for filename when fhr is decimal
EricSinsky-NOAA Jul 29, 2024
5699dfc
Set default REPLAY_ICS to NO in predet/postdet
EricSinsky-NOAA Jul 29, 2024
ea4c3d5
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Jul 29, 2024
c2420d0
Address shell check errors
EricSinsky-NOAA Jul 29, 2024
2a477d9
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Jul 29, 2024
d7eb34a
Give extractvars ability to process replay cases
EricSinsky-NOAA Jul 30, 2024
c057a61
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Jul 30, 2024
f258475
Fix bug in config.resources for extractvars
EricSinsky-NOAA Jul 30, 2024
dcbfd59
Modify config.extractvars based on requirements
EricSinsky-NOAA Jul 30, 2024
d97dbf5
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Aug 1, 2024
75b0d5a
Condition based on REPLAY_ICS
EricSinsky-NOAA Aug 1, 2024
0e54133
Rearange if-block to address reviewer comment
EricSinsky-NOAA Aug 1, 2024
23a146b
Address shellcheck error
EricSinsky-NOAA Aug 1, 2024
44d6e59
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Aug 3, 2024
f3a0ad7
Make variable FV3_OUTPUT_FH_s local
EricSinsky-NOAA Aug 3, 2024
7a8f6ba
Avoid exporting arrays in bash
EricSinsky-NOAA Aug 3, 2024
c28c2a0
Fix potential bug
EricSinsky-NOAA Aug 5, 2024
bd5b258
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Aug 5, 2024
6476218
Organize netcdf comout directories for ocniceprod
EricSinsky-NOAA Aug 5, 2024
73a5101
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Aug 6, 2024
57107f4
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Aug 7, 2024
d5d1efb
Merge remote-tracking branch 'EMC/develop' into feature/offsetfixes
EricSinsky-NOAA Aug 8, 2024
57d6b11
Extract ice products with native grid instead
EricSinsky-NOAA Aug 8, 2024
d443bf9
Remove unused local variable
EricSinsky-NOAA Aug 8, 2024
aaab44a
Merge branch 'develop' into feature/offsetfixes
WalterKolczynski-NOAA Aug 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion parm/config/gefs/config.fcst
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,11 @@ export FSICS="0"

#---------------------------------------------------------------------
# Write more variables to output
export DIAG_TABLE="${PARMgfs}/ufs/fv3/diag_table"
if (( OFFSET_START_HOUR != 0 )); then
export DIAG_TABLE="${PARMgfs}/ufs/fv3/diag_table_replay"
else
export DIAG_TABLE="${PARMgfs}/ufs/fv3/diag_table"
fi
WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved

# Write gfs restart files to rerun fcst from any break point
export restart_interval=${restart_interval_gfs:-12}
Expand Down
2 changes: 1 addition & 1 deletion parm/config/gefs/config.resources
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ case ${step} in
export walltime_gefs="00:30:00"
export ntasks_gefs=1
export threads_per_task_gefs=1
export tasks_per_node_gefs="${ntasks}"
export tasks_per_node_gefs="${ntasks_gefs}"
EricSinsky-NOAA marked this conversation as resolved.
Show resolved Hide resolved
export walltime_gfs="${walltime_gefs}"
export ntasks_gfs="${ntasks_gefs}"
export threads_per_tasks_gfs="${threads_per_task_gefs}"
Expand Down
337 changes: 337 additions & 0 deletions parm/ufs/fv3/diag_table_replay

Large diffs are not rendered by default.

25 changes: 19 additions & 6 deletions ush/atmos_extractvars.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,16 @@ for outtype in "f2d" "f3d"; do
outdirpre="${subdata}/${outtype}"
[[ -d "${outdirpre}" ]] || mkdir -p "${outdirpre}"

nh=${FHMIN}
if [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
if [[ "${outtype}" == "f2d" ]]; then
nh=${OFFSET_START_HOUR}
elif [[ "${outtype}" == "f3d" ]]; then
nh=${FHOUT_GFS}
fi
else
nh=${FHMIN}
fi

while (( nh <= FHMAX_GFS )); do
fnh=$(printf "%3.3d" "${nh}")

Expand All @@ -45,11 +54,15 @@ for outtype in "f2d" "f3d"; do
outres="1p00"
fi

if (( nh <= FHMAX_HF_GFS )); then
outfreq=${FHOUT_HF_GFS}
else
outfreq=${FHOUT_GFS}
fi
if [[ "${outtype}" == "f2d" ]]; then
if (( nh < FHMAX_HF_GFS )); then
outfreq=${FHOUT_HF_GFS}
else
outfreq=${FHOUT_GFS}
fi
elif [[ "${outtype}" == "f3d" ]]; then
outfreq=${FHOUT_GFS}
fi

com_var="COMIN_ATMOS_GRIB_${outres}"
infile1="${!com_var}/${RUN}.t${cyc}z.pgrb2.${outres}.f${fnh}"
Expand Down
31 changes: 25 additions & 6 deletions ush/forecast_postdet.sh
Original file line number Diff line number Diff line change
Expand Up @@ -202,13 +202,23 @@ EOF

#============================================================================
if [[ "${QUILTING}" = ".true." ]] && [[ "${OUTPUT_GRID}" = "gaussian_grid" ]]; then
local FH2 FH3
local FH2 FH3 iicnt
iicnt=0
for fhr in ${FV3_OUTPUT_FH}; do
FH3=$(printf %03i "${fhr}")
FH2=$(printf %02i "${fhr}")
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.atmf${FH3}.nc" "atmf${FH3}.nc"
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.sfcf${FH3}.nc" "sfcf${FH3}.nc"
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.atm.logf${FH3}.txt" "log.atm.f${FH3}"
# When replaying, the time format outputted by model in filename is HH-MM-SS
# because first fhr is a decimal number
if [[ ${REPLAY_ICS:-NO} == "YES" ]] && (( fhr >= OFFSET_START_HOUR )); then
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.atmf${FH3}.nc" "atmf${FV3_OUTPUT_FH_hhmmss[${iicnt}]}.nc"
WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.sfcf${FH3}.nc" "sfcf${FV3_OUTPUT_FH_hhmmss[${iicnt}]}.nc"
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.atm.logf${FH3}.txt" "log.atm.f${FV3_OUTPUT_FH_hhmmss[${iicnt}]}"
iicnt=$(( iicnt + 1 ))
else
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.atmf${FH3}.nc" "atmf${FH3}.nc"
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.sfcf${FH3}.nc" "sfcf${FH3}.nc"
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.atm.logf${FH3}.txt" "log.atm.f${FH3}"
fi
if [[ "${WRITE_DOPOST}" == ".true." ]]; then
${NLN} "${COMOUT_ATMOS_MASTER}/${RUN}.t${cyc}z.master.grb2f${FH3}" "GFSPRS.GrbF${FH2}"
${NLN} "${COMOUT_ATMOS_MASTER}/${RUN}.t${cyc}z.sfluxgrbf${FH3}.grib2" "GFSFLX.GrbF${FH2}"
Expand Down Expand Up @@ -452,10 +462,19 @@ MOM6_postdet() {
(( midpoint = last_fhr + interval/2 ))

vdate=$(date --utc -d "${current_cycle:0:8} ${current_cycle:8:2} + ${fhr} hours" +%Y%m%d%H)
vdate_mid=$(date --utc -d "${current_cycle:0:8} ${current_cycle:8:2} + ${midpoint} hours" +%Y%m%d%H)
#If OFFSET_START_HOUR is greater than 0, OFFSET_START_HOUR should be added to the midpoint for first lead time
if (( OFFSET_START_HOUR > 0 )) && (( fhr == FHOUT_OCN ));then
vdate_mid=$(date --utc -d "${current_cycle:0:8} ${current_cycle:8:2} + $(( midpoint + OFFSET_START_HOUR )) hours" +%Y%m%d%H)
else
vdate_mid=$(date --utc -d "${current_cycle:0:8} ${current_cycle:8:2} + ${midpoint} hours" +%Y%m%d%H)
fi

# Native model output uses window midpoint in the filename, but we are mapping that to the end of the period for COM
source_file="ocn_${vdate_mid:0:4}_${vdate_mid:4:2}_${vdate_mid:6:2}_${vdate_mid:8:2}.nc"
if (( OFFSET_START_HOUR > 0 )) && (( fhr == FHOUT_OCN ));then
source_file="ocn_lead1_${vdate_mid:0:4}_${vdate_mid:4:2}_${vdate_mid:6:2}_${vdate_mid:8:2}.nc"
else
source_file="ocn_${vdate_mid:0:4}_${vdate_mid:4:2}_${vdate_mid:6:2}_${vdate_mid:8:2}.nc"
fi
dest_file="${RUN}.ocean.t${cyc}z.${interval}hr_avg.f${fhr3}.nc"
${NLN} "${COMOUT_OCEAN_HISTORY}/${dest_file}" "${DATA}/MOM6_OUTPUT/${source_file}"

Expand Down
30 changes: 30 additions & 0 deletions ush/forecast_predet.sh
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,30 @@ FV3_predet(){
fi

# Convert output settings into an explicit list for FV3
# Create an FV3 fhr list to be used in the namelist for when REPLAY_ICS is set to YES
# The FV3 fhr list for the namelist and the FV3 fhr list for the filenames
# are only different when REPLAY_ICS is set to YES
if [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
FV3_OUTPUT_FH_NML="$(echo "scale=5; ${OFFSET_START_HOUR}+(${DELTIM}/3600)" | bc -l)"
WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved
FV3_OUTPUT_FH_s=$(( OFFSET_START_HOUR * 3600 + DELTIM ))
WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved
local FHMIN_REPLAY=$(( FHMIN + FHOUT ))
local fhr=${FHMIN_REPLAY}
if (( FHOUT_HF > 0 && FHMAX_HF > 0 )); then
FV3_OUTPUT_FH_NML="${FV3_OUTPUT_FH_NML} $(seq -s ' ' "${FHMIN_REPLAY}" "${FHOUT_HF}" "${FHMAX_HF}")"
FV3_OUTPUT_FH_s="${FV3_OUTPUT_FH_s} $(seq -s ' ' "$(( FHMIN_REPLAY * 3600 ))" "$(( FHOUT_HF * 3600 ))" "$(( FHMAX_HF * 3600 ))")"
fhr=${FHMAX_HF}
fi
FV3_OUTPUT_FH_NML="${FV3_OUTPUT_FH_NML} $(seq -s ' ' "${fhr}" "${FHOUT}" "${FHMAX}")"
FV3_OUTPUT_FH_s="${FV3_OUTPUT_FH_s} $(seq -s ' ' "$(( fhr * 3600 ))" "$(( FHOUT * 3600 ))" "$(( FHMAX * 3600 ))")"
local hh mm ss s_total
for s_total in ${FV3_OUTPUT_FH_s}; do
# Convert seconds to HHH:MM:SS
(( ss = s_total, mm = ss / 60, ss %= 60, hh = mm / 60, mm %= 60 )) || true
FV3_OUTPUT_FH_hhmmss+=("$(printf "%03d-%02d-%02d" "${hh}" "${mm}" "${ss}")")
WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved
done
fi
WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved

# Create an FV3 fhr list to be used in the filenames
FV3_OUTPUT_FH=""
local fhr=${FHMIN}
if (( FHOUT_HF > 0 && FHMAX_HF > 0 )); then
Expand All @@ -153,6 +177,12 @@ FV3_predet(){
fi
FV3_OUTPUT_FH="${FV3_OUTPUT_FH} $(seq -s ' ' "${fhr}" "${FHOUT}" "${FHMAX}")"

# The FV3 fhr list for the namelist and the FV3 fhr list for the filenames
# are identical when REPLAY_ICS is set to NO
if [[ "${REPLAY_ICS:-NO}" == "NO" ]]; then
FV3_OUTPUT_FH_NML="${FV3_OUTPUT_FH}"
fi

# Other options
MEMBER=$(( 10#${ENSMEM:-"-1"} )) # -1: control, 0: ensemble mean, >0: ensemble member $MEMBER
PREFIX_ATMINC=${PREFIX_ATMINC:-""} # allow ensemble to use recentered increment
Expand Down
2 changes: 1 addition & 1 deletion ush/parsing_model_configure_FV3.sh
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ local JCHUNK3D=$((2*restile))
local KCHUNK3D=1
local IMO=${LONB_IMO}
local JMO=${LATB_JMO}
local OUTPUT_FH=${FV3_OUTPUT_FH}
local OUTPUT_FH=${FV3_OUTPUT_FH_NML}
local IAU_OFFSET=${IAU_OFFSET:-0}

# Ensure the template exists
Expand Down
12 changes: 12 additions & 0 deletions ush/parsing_namelists_FV3.sh
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,18 @@ local SDAY=${current_cycle:6:2}
local CHOUR=${current_cycle:8:2}
local MOM6_OUTPUT_DIR="./MOM6_OUTPUT"

if (( OFFSET_START_HOUR > 0 )); then
WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved
local current_cycle_p1
current_cycle_p1=$(date --utc -d "${current_cycle:0:8} ${current_cycle:8:2} + ${FHOUT_OCN} hours" +%Y%m%d%H)
local current_cycle_offset
current_cycle_offset=$(date --utc -d "${current_cycle:0:8} ${current_cycle:8:2} + ${OFFSET_START_HOUR} hours" +%Y%m%d%H)
local SYEAR1=${current_cycle_p1:0:4}
local SMONTH1=${current_cycle_p1:4:2}
local SDAY1=${current_cycle_p1:6:2}
local CHOUR1=${current_cycle_p1:8:2}
local CHOUR_offset=${current_cycle_offset:8:2}
fi

atparse < "${template}" >> "diag_table"


Expand Down
Loading