JCB-based obs+bias staging, Jedi class updates, and marine B-matrix refactoring

[DavidNew-NOAA]

Description

This PR is a companion to GDASApp PR #1312 (merged) and JCB-GDAS PR #31 (merged).

This PR does four things:

1. It changes the observation and bias staging for the atmospheric analysis tasks to use JCB templates instead of reading the full JEDI input configuration dictionary in order to construct a list of files to stage. This is cleaner and places fewer constraints on how to initialize the analysis.
2. It removes the initialize_jedi and execute methods from the AtmAnalysis and AtmEnsAnalysis classes and has that functionality handled directly by the Jedi class. This removes a lot of redundant code from the analysis classes.
3. The Jedi constructor now takes as input a dictionary that is a subset of the task_config dictionary. This makes the code clearer and less opaque and makes debugging easier.
4. All JEDI applications and their input YAMLs are now initialized in the initialize job of the AtmAnalysis and AtmEnsAnalysis. Before, in the atmensanl* jobs for example, the LETKF solver was initialized in the atmensanlinitcjob, but the LETKF solver and FV3 increment converter were both initialized and executed in the atmensanlobs and atmensanlfv3inc jobs respectively. This was because the Jedi class was looking for the JEDIEXE, JCB_ALGO_YAML, etc environment variables, which could only hold information for one application per job. Now, there are variables called JEDIEXE_OBS, JEDIEXE_SOL, JEDIEXE_FV3INC, etc which are specified in the input dictionary for the Jedi class constructor, all performed in atmensanlinit. This not only makes more sense in terms of resources allocation for jobs, but it also makes the code less opaque.

Addendum:

I'm now rolling in the refactoring of the marine B-matrix task into this PR. That makes it also a companion of GDASApp PR #1346 and JCB-GDAS PR #36.

These new changes introduce the Jedi class and JCB into the marine B-matrix job.

Type of change

• Bug fix (fixes something broken)
• New feature (adds functionality)
• Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

• Is this a breaking change (a change in existing functionality)? YES
• Does this change require a documentation update? NO
• Does this change require an update to any of the following submodules? NO
  • EMC verif-global
  • GDAS #1346
  • GFS-utils
  • GSI
  • GSI-monitor
  • GSI-utils
  • UFS-utils
  • UFS-weather-model
  • wxflow

How has this been tested?

C96C48_ufs_hybatmDA CI runs successfully
GDASApp jjob tests pass successfully

Checklist

• Any dependent changes have been merged and published
• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have documented my code, including function, input, and output descriptions
• My changes generate no new warnings
• New and existing tests pass with my changes
• This change is covered by an existing CI test or a new one has been added
• I have made corresponding changes to the system documentation if necessary

[DavidNew-NOAA]

@guillaumevernieres and @AndrewEichmann-NOAA , FYI, I'm rolling the marine B-matrix refactoring into this PR since that refactoring depends on the changes made to the Jedi class in this PR.

The relevant files changed for the B-matrix refactoring are:
parm/config/gfs/config.marineanl
parm/config/gfs/config.marinebmat
scripts/exglobal_marinebmat.py
ush/python/pygfs/task/marine_analysis.py (very minor JCB-related change)
ush/python/pygfs/task/marine_bmat.py

[guillaumevernieres]

Looks good. Thanks @DavidNew-NOAA .