Workflow for maintaining GEOS‐Chem external library (may be used as a template for other external libraries)

Workflow which delineates the separation of duties between Harvard and NCAR in maintaining the GEOS-Chem external library in CAM

1. If NCAR encounters a breaking change:
   1. NCAR will open an issue on GEOS-Chem GitHub using label “CESM”
   2. If NCAR knows a fix:
      1. Develop fix
      2. Test on derecho
      3. Create a PR targeting GEOS-Chem GitHub organization repo branch “cesm/release” using label “CESM”
   3. If NCAR does not easily see a fix:
      1. Harvard will work with NCAR to develop the fix via GitHub issue discussion
      2. Who tests on derecho will be determined on a case-by-case basis. For example, if the issue is a typo or similarly small problem that Harvard can provide the solution for on the GitHub issue, then NCAR can implement and resume their testing already in progress.
   4. Harvard reviews, merges, and provides a hash to use in CAM via a PR into ESCOMP/CAM
2. If an urgent science bug is identified by the GEOS-Chem community
   1. Harvard will open an issue in ESCOMP/CAM describing the bug
   2. Either Harvard or the bug reporter will develop a fix and make a PR
   3. If fix is urgent:
      1. Harvard will review and merge PR into "cesm/release" and test on derecho
      2. Harvard will give NCAR the new hash via a PR into ESCOMP/CAM
   4. If fix can wait weeks to months:
      1. Update will be brought in with next version release (see #4)
3. If a science bug is identified by someone at NCAR
   1. NCAR will open an issue on GEOS-Chem GitHub with label “CESM”
   2. Will discuss on GitHub:
      1. If the fix is urgent or can wait for next GEOS-Chem release
      2. Who will develop a fix, test, and create PR
   3. If fix is urgent:
      1. Harvard will review and merge PR into "cesm/release" and test on derecho
      2. Harvard will give NCAR the new hash via a PR into ESCOMP/CAM
   4. If fix can wait weeks to months:
      1. Update will be brought in with next version release (see #4)
4. For each new GEOS-Chem version we would like to bring into CAM:
   1. Harvard will open an issue in ESCOMP/CAM announcing the new GEOS-Chem version
   2. Harvard will make the updates using latest cam_development
   3. Harvard will test on derecho
   4. Harvard will give NCAR the new tag via a PR into ESCOMP/CAM

Notes on externals:

1. CAM will use official release tags as much as possible
2. Urgent bug fixes will go into GEOS-Chem branch "cesm/release" and hash provided to NCAR