Updates ufs-weather-model, this updates RDHPCS to the newer spack-stack
allowing some temporary fixes to be reverted.
* removes upp submodule
* uses upp from the ufs-weather-model
* restores the build and link that were hacked during the Hera Rocky 8
transition to allow for UPP submodule
* Removes forecast directories in clean-up

Resolves NOAA-EMC#2617 
Resolves NOAA-EMC#2437

---------

Co-authored-by: Rahul Mahajan <[email protected]>

…OAA-EMC#2632)

This PR enables linking both global-nest fix files and non-nest ones at
the same time and users can run both nesting and non-nesting experiments
at the same time without worries about what fix files to be linked.

Resolves NOAA-EMC#2631

Adds a COM template to define a path to store obs processed for JEDI.
This will allow UFSDA to stop writing to COM_OBS, which should be
read-only as it belongs to obsproc in operations.

No functional change yet.

This PR addresses issue NOAA-EMC#2444. The following is accomplished:

- All `sleep` statements are replaced with `wait_for_file` for the
relevant scripts beneath `scripts` and `ush`;
- Indentation and shell-norms are updated where applicable. 

Note: The WAFS scripts are not updated as per @aerorahul direction.

Resolves NOAA-EMC#2444

---------

Co-authored-by: henrywinterbottom-wxdev <[email protected]>
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>

The PR allows the use of ICs from PSL's replay analysis. These replay
ICs will be used for GEFS reforecasting and SFS. Two main changes are
associated with these updates: (1) replay ICs being valid at 3Z, and (2)
the use of warm starts.

Resolves NOAA-EMC#1838

---------

Co-authored-by: Jessica Meixner <[email protected]>
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>
Co-authored-by: Rahul Mahajan <[email protected]>

This PR updates the `sorc/gdas.cd` and `sorc/gsi_utils` hashes. The
updated hashes bring in bug fixes, new UFS DA functionality, and a Gaea
build for gsi_utils.
 
Resolves NOAA-EMC#2640

…#2685)

This quick-fix PR is to update the Jenkins Pipeline's configurable
parameter list to include the **Hercules-EMC** node.
This allows Jenkins users to restart Jobs in the controller when no
updates have been made.

needed a comma

This PR removes the ocn_daily files that are produced by the ocean
component. These files can be recreated by averaging data that exists in
the 6 hour aveaged files if needed.

Fixes NOAA-EMC#2675
Fixes NOAA-EMC#2659 (by
removing them and making this obsolete)

…#2624)

Add a prepaeroobs job to prepare aerosol obs files for DA.

This job does quality control of the VIIRS aerosol raw observations and
convert them to ioda format.

Resolves NOAA-EMC#2623 
---------

Co-authored-by: ypwang19 <[email protected]>
Co-authored-by: TerrenceMcGuinness-NOAA <[email protected]>
Co-authored-by: Cory Martin <[email protected]>
Co-authored-by: David Huber <[email protected]>

This PR moves JCB into GDASapp. The PR
also bumps up the hash of GDASapp to what is in `feature/move_jcb`,
which at time of writing is develop plus the absorption of JCB into
GDASapp.
Note that I also took the changes from
NOAA-EMC#2641 to follow the
testing @RussTreadon-NOAA has done.

Optimize the gfswavepostpnt, gfswavepostbndpntbll, and gfswavepostbndpnt jobs

This is done by
1) reducing the number of calls to `sed`, `awk`, `grep`, and `cat` by
- performing operations on all files at once instead of looping over
each file
    - removing piped `cat` calls (e.g. `cat <file> | sed 'something'`)
    - combining `sed` and `grep` calls when possible
    - adding logic to `awk` calls instead of handling that logic in bash
2) minimizing as much as possible the amount of data on disk that has to
be read in (e.g. limiting sed to read only the line numbers it needs)
---------

Co-authored-by: Walter Kolczynski - NOAA <[email protected]>

Turns on high-frequency (hourly) output in the extended products test to
exercise that aspect of the code. This test only runs on WCOSS.

Also adds the hooks to optionally turn on the metplus jobs, but that is
deferred as they are not currently working correctly.

NCO has requested that each COM variable specify whether it is an input
or an output. This completes that process for the global-workflow
archive task.

Refs NOAA-EMC#2451 
---------

Co-authored-by: Walter Kolczynski - NOAA <[email protected]>
Co-authored-by: Rahul Mahajan <[email protected]>
Co-authored-by: David Huber <[email protected]>

This PR is a hotfix for an incorrectly named (e.g., non-existent) `COM/`
template.
Resolves NOAA-EMC#2696 
Refs NOAA-EMC#2451

…put files (NOAA-EMC#2688)

This PR addresses issue NOAA-EMC#1252. The following is accomplished:

- Prior to removing files, the existence of a file is checked prior to
attempting to remove; this is performed as noted
[here](NOAA-EMC#1252 (comment));
this PR only addresses the the `chgrp` issue.

Refs NOAA-EMC#1252

---------

Co-authored-by: David Huber <[email protected]>

Eliminates the post groups used for upp and products jobs so that each
task only processes one forecast hour. This is more efficient and
greatly simplifies downstream dependencies that depend on a specific
forecast hour.

Resolves NOAA-EMC#2666
Refs NOAA-EMC#2642

…#2687)

This fixes the online archiving portion of the `*arch` and `*earc00`
jobs, a prerequisite for running METplus.  This also reenables
METplus by default.

The approach previously taken created `FileHandler` dictionaries at
varying levels within the resulting yaml, which was not properly parsed
by `exglobal_archive.py`. This approach creates a single `FileHandler`
dictionary and is much less complicated overall.

Resolves NOAA-EMC#2673 NOAA-EMC#2647

Adds a table to HPC documentation stating the minimum support versions

Redefine resource variables based explicitly on RUN or CDUMP

Additionally, machine-specific resources are moved out of
config.resources and placed in respective config.resources.{machine}
files.

Resolves NOAA-EMC#177 NOAA-EMC#2672

NCO has requested that each COM variable specify whether it is an input
or an output. 
This completes that process for the global-workflow wave
model and products tasks.
Refs NOAA-EMC#2451

Updates the logic to be by run instead of for DO_JEDIOCNVAR to determine
how many layers and sets the exception value for MOM6 to be 1e-34 for
all scenarios. Note, we will no longer have zeros in the ocean grib
output and the DA will also run without issues.

Fixes NOAA-EMC#2615

This PR:
- removes stale temporary scratch run directories from `$DATAROOT/`
every 3 days.
- should help to scrub failed attempts.
- removes an unused variable `RUNDIR` defined in `config.base`

Activate the snow DA test on WCOSS.

…ut (NOAA-EMC#2692)

We are updating the Jenkins Pipeline with a try block around checkout to
capture errors for the user.
Also cleaned up Finalize and added section to clean out workspace on
success.

This PR:
- reverts NOAA-EMC#2681 in part 
- keeps some changes for `RUN`.
- is a hotfix
- should be merged ASAP after consensus w/ @guillaumevernieres
@CatherineThomas-NOAA @WalterKolczynski-NOAA

…OAA-EMC#2700)

This PR updates the `gdas.cd` hash to bring in new JCB conventions.
Resolves NOAA-EMC#2699

From NOAA-EMC#2654
This PR will move much of the staging code that take place in the python
initialization subroutines of the variational and ensemble DA jobs into
Jinja2-templated YAML files to be passed into the wxflow file handler.
Much of the staging has already been done this way, but this PR simply
expands that strategy.

The old Python routines that were doing this staging are now removed.
This is part of a broader refactoring of the pygfs tasking.

wxflow PR [NOAA-EMC#30](NOAA-EMC/wxflow#30) is a
companion to this PR.

Co-authored-by: danholdaway <[email protected]>
Co-authored-by: DavidNew-NOAA <[email protected]>
Co-authored-by: DavidNew-NOAA <[email protected]>
Co-authored-by: Dan Holdaway <[email protected]>

…OAA-EMC#2701)

This PR sets up the ability on Gaea for auto generation of a clean xml
file, i.e., an xml file that does not need any alterations before
running rocoto.
Refs NOAA-EMC#2572 
Refs NOAA-EMC#2664

Fixes an issue created from NOAA-EMC#2701 that added `CLUSTERS` to the
`gaea.yaml`.

This PR updates gempak jobs (gfs, gdas, and goes) from processing all
forecast hours at once to one forecast hour at a time. This will reduce
the job runtime to less than 5 min, so restart capability is not needed.

Resolves NOAA-EMC#1250 
Ref NOAA-EMC#2666 NOAA-EMC#2667  
---------

Co-authored-by: Walter.Kolczynski <[email protected]>

Defines `CLUSTERS` as an empty string for all hosts except Gaea and uses
the native `dict` `get` method to prevent grabbing an unset entry.

Updates UFS weather model hash to hash from 2024-06-24 which has orion
porting updates + a few namelist updates.

This PR:
- updates a few submodules (GSI, GSI-utils, GSI-monitor, UFS_utils,
GFS-utils) to include recent update to their modulefiles for
Orion+Rocky9 upgrade
- updates the modulefiles in global-workflow to load modules from
Orion+Rocky9 paths
- updates modulefiles for `gwsetup` and `gwci` as well.
- removes NCL and GEMPAK from Orion. NCL is not used and GEMPAK is not
installed.
- adds `parm/config.gfs/config.resources.ORION` to address GSI
performance degradation after Rocky 9 upgrade.

This PR:
- does not update the build for UPP. Standalone UPP is not available via
ufs-weather-model as of NOAA-EMC#2729
- will need a follow-up update for `prepobs` and `fit2obs` updated
locations when they are installed in `glopara` space on Orion.
 

# Type of change
- Maintenance (code refactor, clean-up, new CI test, etc.)

# Change characteristics
- Is this a breaking change (a change in existing functionality)? NO
- Does this change require a documentation update? NO

# How has this been tested?
This PR is not sufficient for Orion.
This PR must be tested on other platforms (Hera, WCOSS2) as this PR
updates submodules.

# Checklist
- [ ] Any dependent changes have been merged and published
- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] My changes generate no new warnings
- [ ] New and existing tests pass with my changes
- [ ] I have made corresponding changes to the documentation if
necessary

---------

Co-authored-by: Kate Friedman <[email protected]>

Add previous cycle's `fcst` as a dependency to `ocnanalprep`

This ensures that the availability of restart files to the latter. This addresses a
seldomly-encountered race condition where `ocnanalprep` fails due to the
lack of the files.

Archives the GDAS restartb dataset at a 6-hour offset from restarta

This allows cycled experiments to restart from the archives.
The tabbing for the master archive templates was also added to
improve readability.

Resolves NOAA-EMC#2722

…lass (NOAA-EMC#2635)

Adds contents of constructor and initialize methods to marine LETKF class

Partially addresses NOAA-EMC/GDASApp#1091
---------

Co-authored-by: Rahul Mahajan <[email protected]>
Co-authored-by: Cory Martin <[email protected]>

This PR has a few maintenance updates to the CI pipeline and adds a test
directory with Unit Tests
**Major Maintenance updates:**
- Added try blocks with appropriate messaging to GitHub PR of failure
for:
- - **scm** checkout
- - build fail (with error logs sent as gists)
- - create experiment fails with `stderr` sent to GitHub PR messaging
- Pre-stage FAILS from the above are now captured these fails allow
FINALIZE to update the label to FAIL (i.e. no more "hanging" CI state
labels in GitHub - see image below)
**Minor Maintenance updates:**
- Fix for STALLED cases reviled from PR 2700 (just needed a lambda
specifier)
- Fixed path to experiment directory in PR message (had dropped EXPDIR
in path)
- Needed `latin-1` decoder in reading log files for publishing
**Added python Unit Tests for CI functionality:**
- Installed **Rocoto** and **wxfow** in GitHub Runner for testing key CI
utility codes
- Cashed the install of Rocoto in the GitHub Runners to greatly reduce
stetup time for running the unit tests
- Unit Tests Python scripts added
- `test_rocostat.py`: rocoto_statcount() rocoto_summary()
rocoto_stalled()
  - `test_setup.py`: setup_expt() test_setup_xml()
  - `test_create_experment`: test_create_experiment()
  - - Runs all PR cases that do not have ICs in the GItHub Runner
- Reporting mechanism in the Actions tab for Python Unit Testing results
- Test case data for STALLED and RUNNING stored on S3 and pulled using
wget during runtime of tests

Update prepobs to v1.0.2 and fit2obs to v1.1.2

These versions now support Orion Rocky9. Updates are included
for new install locations on WCOSS2, Hera, Orion/Hercules, and Jet.

…OAA-EMC#2757)

This bugfix PR:
- fixes an issue where a user is unable to generate the XML for a GFS
forecast-only experiment with APP=S2SWA

Specifically, the changes are related to defining `aero_fcst_cdumps`.  
Following `setup_expt.py`, the user will have to set
`AERO_FCST_CDUMPS="gdas|gfs|both" depending on their use case in
`config.base`.

Removes git-lfs files from the `GDASApp` `jcb` submodule, allowing it to
be cloned on Hera.

This hotfix points to a non-authoritative branch of the GDASApp
(https://github.com/DavidHuber-NOAA/GDASApp/tree/hotfix/update_jcb) and
should be updated ASAP back to the authoritative repository.

…r Rocky 9 (NOAA-EMC#2774)

Fix metp* resources and check that they completed properly; add support for Orion Rocky 9

…2680)

# Description
This PR adds an optional task to the global-workflow to process a subset
of ocean, ice, wave and atmosphere products to be saved on WCOSS2 for
the GEFSv13 reforecast. This task is designed to process GEFS variables
so that specific reforecast product requirements are met. A new variable
in `config.base` called `DO_EXTRACTVARS` enables this task, which is
currently called `extractvars`. `DO_EXTRACTVARS` is set to `NO` by
default and is specifically a task designed to be executed for the
GEFSv13 reforecast.

Refs NOAA-EMC#1878 

# Type of change
- New feature (adds functionality)

# Change characteristics
- Is this a breaking change (a change in existing functionality)? NO
- Does this change require a documentation update? NO

# How has this been tested?
This has been cloned and tested on WCOSS2. This will need to be tested
on Hera and other platforms on which the reforecast may be running.

# Checklist
- [ ] Any dependent changes have been merged and published
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] New and existing tests pass with my changes
- [ ] I have made corresponding changes to the documentation if
necessary

---------

Co-authored-by: Rahul Mahajan <[email protected]>

…_SERVICE option (NOAA-EMC#2775)

Removes the namespace-based construction of EnKF member COM directories
in the enkf archive template.

Refactor the functionality of B-matrix generation from the GDASApp

Resolves NOAA-EMC#2743

This update is a combination of the gravity wave drag (GWD) versions
from the NOAA/GSL and NOAA/PSL

The main purpose of this PR is to remove the f000 from
atmosphere-related rocoto tasks when `REPLAY_ICS` is set to `YES`. In
cases where `REPLAY_ICS` is `YES` and `OFFSET_START_HOUR `is greater
than `0`, it becomes necessary to have the first lead hour set to
`OFFSET_START_HOUR ` for the atmosphere-related rocoto tasks. For
example, when `OFFSET_START_HOUR ` is set to `3`, then the minimum lead
time for the atmos_prod and atmos_ensstat rocoto tasks needs to be 3 and
the minimum lead time for the ocean_prod rocoto task needs to be 6
(assuming `FHOUT_OCN` is 6). This PR makes this rocoto workflow set up
possible by removing 0 from fhrs for atmosphere-related tasks in
`gefs_tasks.py` when replaying.

This PR also moves where f000 is being removed for the ocean_prod and
ice_prod tasks. The if-block that performs this f000 removal has been
moved from `tasks.py` to `gefs_tasks.py` and `gfs_tasks.py`.

---------

Co-authored-by: Rahul Mahajan <[email protected]>

…A-EMC#2727)

This overhauls resource-related variables to use a common set of
variables for each job. In the process, this also removed the use of
CDUMP in most cases.

Resolves NOAA-EMC#1299 NOAA-EMC#2693

Changes how METplus jobs run so that they run on the last GFS cycle for
a given `PDY`.

This is a departure from operations where the METplus jobs run on the
00Z cycle for the previous 3 cycles and 00Z (i.e. `${PDYm1}06` through
`${PDY}00`). With this PR, for gfs_cyc=4, METplus jobs will run on
`${PDY}18` for cycles 00-18. See
NOAA-EMC/EMC_verif-global#131 for more details.

# Description
Point to the latest hash of the gfs-utils repository, which contains the
bug fix to gaussian_sfcanl.

Resolves NOAA-EMC#2669.
Refs: NOAA-EMC/gfs-utils#73

This PR:
- replaces `check_netcdf.sh` checker for ocean post-processing with with
ocean output at the next forecast hour or finishing of the forecast job
for ocean prod
- removes no longer needed `ush/check_netcdf.sh`

…2762)

Made updates to the Jenkins Launching Script for robustness and less
ambiguous documentation:
- Clearer distinction between required user token for the remote api and
the systems token for launching
- Added pre-checks: `gh` is authenticating, named compliant token and
secret file exists
- More robust Jason based parser of the remote api response for checking
the state of the Node connection
- For `cron` use a 5 minute pause and recheck was added before
re-launching of the java agent
- Added concise header documentation of requirements and purpose 
---------

Co-authored-by: David Huber <[email protected]>
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>

Adds 2 hot fixes:

- METplus v9.1.3 has a bug in it that sometimes attempts to create
multiple copies of the same directory when running in parallel, causing
a Python error and downstream problems. This PR makes METplus run in
serial mode, preventing such issues.
- Corrects the name of the atmos_prod, ocean_prod, and ice_prod tasks in
workflow/rocoto/tasks.py (was accidentally changed to e.g. atmosprod)

Resets the default value of opt_diag to 2 corresponding to the land
surface upgrades in ufs-weather-model for HR4.

Resolves NOAA-EMC#2786

This PR:
- is a follow-up to a previous PR that aggressively pruned run
directories.
- removes run directories for the current cycle in the clean-up if the
cycle is successful. If the cycle is not successful, cleanup is not
called and all run directories for the cycle are safe from being purged.
- also updates the PR template to list/query for any updates to
submodules.

---------

Co-authored-by: David Huber <[email protected]>
Co-authored-by: David Huber <[email protected]>
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>

Quick bug fix for updating state labels in CI during finalize.
(did not reference GitHub CLI executable correctly in the pipeline
script)

Companion PR to GDASApp#1226

Addresses a few issues described in:

- GDASApp#1219
- GDASApp#1217
- GDASApp#1232

)

- This feature added an archive task for GEFS system. This task will run
in the service queue with a single processor.
The task executes with success.
- Only xml is generated.

Refs NOAA-EMC#832
Refs NOAA-EMC#2698

…A-EMC#2799)

Adds a check to the SSH command used to kill child PIDs of a defunct
driver instance on a different head node to prevent invalid kill
commands, preventing CI failures.

Resolves NOAA-EMC#2798

…A-EMC#2792)

This PR changes the G-W so that backgrounds for the atmospheric analyses
are staged as UFS cubed-sphere history files rather than FMS restarts.
The motivation being that FMS restarts include many data we don't need
for the analyses and thus take up too much disk space/bandwidth.

This is a companion PR to:
GDASApp#1236
JCB-GDAS#19

---------

Co-authored-by: danholdaway <[email protected]>
Co-authored-by: Dan Holdaway <[email protected]>
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>
Co-authored-by: RussTreadon-NOAA <[email protected]>
Co-authored-by: RussTreadon-NOAA <[email protected]>
Co-authored-by: Rahul Mahajan <[email protected]>
Co-authored-by: David Huber <[email protected]>

Adds the ability to run a forecast in segments instead of all at once.
To accomplish this, a new local `checkpnts` variable is introduced to
`config.base` to contain a comma-separated list of intermediate stopping
points for the forecast. This is combined with `FHMIN_GFS` and
`FHMAX_GFS` to create a comma-separated string `FCST_SEGMENTS` with all
the start/end points that is used by `config.fcst` and rocoto workflow.
Capability to parse these into python lists was added to wxflow in an
accompanying PR. If `checkpnts` is an empty string, this will result in
a single-segment forecast.

To accommodate the new segment metatasks that must be run serially, the
capability of `create_task()` was expanded to allow a dictionary key of
`is_serial`, which controls whether a metatask is parallel or serial
using pre-existing capability in rocoto. The default when not given is
parallel (i.e. most metatasks).

Resolves NOAA-EMC#2274
Refs NOAA-EMC/wxflow#39
Refs NOAA-EMC/wxflow#40

This PR fixes a couple issues that arise when replay initial conditions
are used. These issues only occur when `REPLAY_ICS` is set to `YES` and
`OFFSET_START_HOUR` is greater than `0`. The following items are
addressed in this PR.

1. Fix issue that causes ocean_prod tasks not to be triggered (issue
[NOAA-EMC#2725](NOAA-EMC#2725)). A new
diag_table was added (called `diag_table_replay`) that is used only when
`REPLAY_ICS` is set to `YES`. This diag_table accounts for the offset
that occurs when using replay IC's.
2. Fix issue that causes atmos_prod tasks not to be triggered for the
first lead time (e.g. f003) (issue
[NOAA-EMC#2754](NOAA-EMC#2754)). When
`OFFSET_START_HOUR` is greater than `0`, the first `fhr` is
`${OFFSET_START_HOUR}+(${DELTIM}/3600)`, which is defined in
`forecast_predet.sh` and will allow data for the first lead time to be
generated. The filename with this lead time will still be labelled with
`OFFSET_START_HOUR`.
3. Minor modifications were made to the extractvars task so that atmos
data from replay cases can be processed.

This PR was split from PR NOAA-EMC#2680. 

Refs NOAA-EMC#2725, NOAA-EMC#2754 

---------

Co-authored-by: Walter Kolczynski - NOAA <[email protected]>

The purpose of this PR it to merge code that allowing global-workflow
run on AWS, first focus on ATM forecast only.

Resolves: NOAA-EMC#2709
---------

Co-authored-by: David Huber <[email protected]>

CI Testing and Jenkinsfile for Gaea C5

Pipeline updates:
- get `gh` location on remote machine and define global bash env
variable `$GH` for GitHub CLI
- Failed cases are now displayed accordingly in the Jenkins dashboard
(see NOTE below)
-  Added the Build # in messaging for clarity when running from re-runs.
- Replaced Matrix construct for concurrency with the parallel method
that can use dynamic case lists
- With removing of the hard coded list of cases we now get list of cases
dynamically from the PR case directory
- See new look of dashboard below (has more annotations and displays
only used cases)

NOTE: **failFast** (quitting all cases on failing of one) still does not
work because it isn't quitting the running remote shells. We can make
this a configurable capability in feature request with some custom code.
The current behavior has the remaining cases continuing to run after a
FAIL label has been issued and it is incumbent of the code manager to
kill the CI job in the controller before resetting another Ready label.

---------

Co-authored-by: David Huber <[email protected]>

…-EMC#2820)

# Description
From time to time, PBS pro cannot return a `qstat` response within a
given time limit set by `rocoto` (default is 45 seconds). If that
happens, then an `UNAVAILABLE` status will be returned for the given
job. This PR adds checking for this status to allow CI processing to
continue.

Update the test for UNAVAILABLE from inclusion to count: `if
['UNAVAILABLE'] > 0`
in `rocotocheck.py `script.

This PR converts the staging job from shell to python and introduces the
use of yaml.

Changes in this PR:

1. Rename `scripts/exglobal_stage_ic.sh` to
`scripts/exglobal_stage_ic.py`.
2. Update `jobs/JGLOBAL_STAGE_IC` to use `.py` script extension. Move
`COM*` variable declarations and member loop down into yaml and python
respectively. Move `GDATE/gPDY/gcyc` settings up to JJOB from ex-script
and replace with newer cycle variables (as done in forecast job).
3. Create `parm/stage` folder to hold newly created `stage.yaml.j2`,
which both mimics forecast-only functionality in existing
`scripts/exglobal_stage_ic.sh` and adds functionality for cycled mode.
4. Create `ush/python/pygfs/task/stage.py` to house staging job python
functions for call from `scripts/exglobal_stage_ic.py`.
5. Remove `stage_ic` job rocoto dependencies from xml. Do not need and
removes area of duplicate maintenance.
6. Add cycled staging jobs for gdas and enkf suites.
7. Rename `model_data` to `model` for issue NOAA-EMC#2686

There will now be distinct `stage_ic` jobs for each `RUN`:
`gdasstage_ic`, `gfsstage_ic`, `enkfgdasstage_ic`, `stage_ic` (for
gefs).

Related work was done to set up new symlink folder structure under
supported platform `ICSDIR` folder for use by updated staging job.

Resolves NOAA-EMC#2475
Resolves NOAA-EMC#2650
Resolves NOAA-EMC#2686

---------

Co-authored-by: Rahul Mahajan <[email protected]>
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>
Co-authored-by: David Huber <[email protected]>

This PR will support ATM forecast only run on Azure

)

This PR adds the capability to update the ensemble of snow states 
by recentering the ensemble mean to the deterministic snow analysis
and applying increments as appropriate.

Resolves NOAA-EMC#2585

---------

Co-authored-by: David Huber <[email protected]>
Co-authored-by: Rahul Mahajan <[email protected]>
Co-authored-by: Guillaume Vernieres <[email protected]>
Co-authored-by: AntonMFernando-NOAA <[email protected]>
Co-authored-by: Anil Kumar <[email protected]>
Co-authored-by: TerrenceMcGuinness-NOAA <[email protected]>

Add a parameter "pass_full_omega_to_physics_in_non_hydrostatic_mode" in
FV3 namelist. It was set to "true" to use a new method to diagnose
omega. This PR is based on the /ufs-community/ufs-weather-model#2327)

Corresponding parameter changed in GFSv17 related regression tests
ufs-community/ufs-weather-model#2373))

Changes to make GEFS C48 case run on AWS.

After C48 ATM forecast only runs on AWs, the next step is to make GEFS
C48 run on AWS.
Changes to AWS env, and yaml files.

Resolves NOAA-EMC#2817
Refs NOAA-EMC#2711

Support global-workflow ATM forecast only runs on Google.

Add/Modify env, yaml, and python scripts changes to make global-workflow
ATM forecast only runs on GSP.

  Resolves NOAA-EMC#2831
  Refs NOAA-EMC#2826
  Refs NOAA-EMC#2711

Module git-lfs is required to run CI test on gaea machine and added
gaea in the Jenkinsfile
- Module (git-lfs) added in the modulefiles/module_gwsetup.gaea.lua
- Gaea added (Jenkinsfile)

This PR adds JEDI ATM lgetkf observer and solver jobs to
global-workflow. This is approach is akin GSI-based eobs and eupd.
Splitting the single JEDI ATM lgetkf job into separate observer and
solver jobs improves memory and computational efficiency.

Resolves NOAA-EMC#2415

Make ATM-OCN-ICE coupling model run on AWS.

This adds capability to run UFS atm-ocn-ice coupling on AWS.

Resolves NOAA-EMC#2858

This PR corrects a bug in the staging job for ocean `MOM.res_#` IC
files. The `OCNRES` value was coming in as an integer (e.g. `25`) but
the `ocean.yaml.j2` file was checking for `"025"`. Correct to now set
OCNRES to be three digits in staging script and also correct the for
loop range to include third file.

Resolves NOAA-EMC#2864

…AA-EMC#2738)

This PR adds in support for computing files needed for the aerosol
analysis **B**. This includes a new task, `aeroanlgenb`. This work was
performed by both me and @andytangborn

Resolves NOAA-EMC#2501
Resolves NOAA-EMC#2737

---------

Co-authored-by: Andrew.Tangborn <[email protected]>
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>

Adds files `atmi009.nc`, `atmi003.nc`, `ratmi009.nc`, and `ratmi003.nc`
to list of files to be staged for ICs, if available. These are necessary
for starting an IAU run, and are currently missing.

Resolves NOAA-EMC#2874

# Description

Support global-worflow GEFS C48 on Google Cloud.

Make env. var. and yaml file changes, so global-workflow GEFS C48 case
can run properly on Google Cloud.

Resolves NOAA-EMC#2860

Add the ability to run CI test C96_atm3DVar on Gaea-C5

Resolves NOAA-EMC#2766 
Refs NOAA-EMC/prepobs#32
Refs NOAA-EMC/Fit2Obs#28

Use the updated 2013 to 2024 mean MERRA2 climatology instead of 2003 to
2014 mean

Depends on NOAA-EMC#2887 
Refs: ufs-community/ufs-weather-model#2272
Refs: ufs-community/ufs-weather-model#2273

…MC#2893)

This changes the order of the cleanup job so that the working directory
is deleted at the end. It also adds the `-ignore_readdir_race` flag to
`find` to prevent errors if a file was deleted after the list of files
was collected. This can happen if two consecutive cycles run the cleanup
job at the same time.

This updates the model hash to include the UPP update needed to be able
to run the post processor on Orion, thus reenabling support on that
system.

A note on the UPP: it is using a newer version of g2tmpl that requires a
separate spack-stack 1.6.0 installation. This version of g2tmpl will be
standard in spack-stack 1.8.0, but for now requires loading separate
modules for the UPP.

A note on running analyses on Orion: due to a yet-unknown issue causing
the BUFR library to run much slower on Orion when compared with Rocky 8,
the GSI and GDASApp are expected to run significantly slower than on any
other platform (on the order of an hour longer).

Lastly, I made adjustments to the build_all.sh script to send more cores
to compiling the UFS and GDASApp. Under this configuration, the GSI,
UPP, UFS_Utils, and WW3 pre/post executables finish compiling before the
UFS when run with 20 cores.

Resolves NOAA-EMC#2694 
Resolves NOAA-EMC#2851 

---------

Co-authored-by: Rahul Mahajan <[email protected]>
Co-authored-by: Walter.Kolczynski <[email protected]>

…#2816)

- This task is an extension of the empty arch job previously merged. 
- This feature adds an archive task to GEFS system to archive files
locally.
- This feature archives files in ensstat directory. 

Resolves NOAA-EMC#2698
Refs NOAA-EMC#832 NOAA-EMC#2772

The current operational BUFR job begins concurrently with the GFS model
run. This PR updates the script and ush to process all forecast hour
data simultaneously, then combines the temporary outputs to create BUFR
sounding products for each station. The updated job will now start
processing data only after the GFS model completes its 180-hour run,
handling all forecast files from 000hr to 180hr at a time. The new
version job running will need 7 nodes instead of the current operational
4 nodes.

This PR depends on the GFS bufr code update NOAA-EMC/gfs-utils#75

With the updates of bufr codes and scripts, there is no need to add
restart capability to GFS post-process job JGFS_ATMOS_POSTSND.

This PR includes the other changes:

Rename the following table files:

parm/product/bufr_ij13km.txt to parm/product/bufr_ij_gfs_C768.txt
parm/product/bufr_ij9km.txt to parm/product/bufr_ij_gfs_C1152.txt

Add a new table file: parm/product/bufr_ij_gfs_C96.txt for GFSv17 C96
testing.

Added a new capability to the BUFR package. The job priority is to read
bufr_ij_gfs_${CASE}.txt. If the table file is not available, the code
will automatically find the nearest neighbor grid point (i, j).

Refs NOAA-EMC#1257
Refs NOAA-EMC/gfs-utils#75

This PR creates a PyGFS class called JEDI, which is to be instantiated
everytime a JEDI application is run. The AtmAnalysis and AtmEnsAnalysis
classes are no longer children of the Analysis class, but rather direct
children of the Task class. They each have a JEDI object as an
attribute, which is used to run either the variational/ensemble DA JEDI
applications or the FV3 increment converter JEDI application, depending
on which job they are created for (e.g. atmanlvar vs. atmanlfv3inc). The
intention is that a later PR will apply this framework to all analysis
task, and the PyGFS Analysis class will be removed.