Merge pull request #50 from MAAP-Project/docs-ops-rem-sk

API URL Adjustments, workflow update

CHANGELOG.md

@@ -5,6 +5,13 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog], and this project adheres to
 [Semantic Versioning].
 
+## [0.6.1] - 2023-09-26
+
+### Fixed
+
+- [#49](https://github.com/MAAP-Project/gedi-subsetter/issues/49) Remove all
+  API urls that contain ops as they have now been retired (eg. api.ops.maap-project.org).
+
 ## [0.6.0] - 2023-06-02
 
 ### Fixed

README.md

@@ -282,9 +282,9 @@ inputs = dict(
 
 ### Submitting a DPS Job
 
-The GEDI Subsetting DPS Job is named `gedi-subset_ubuntu`, and may be executed
-from your ADE Workspace by opening the **DPS/MAS Operations** menu, choosing
-the **Execute DPS Job** menu option, and selecting `gedi-subset_ubuntu:<VERSION>`
+The GEDI Subsetting DPS Job is named `gedi-subset`, and may be executed
+from your ADE Workspace by opening the **Jobs** menu, choosing
+the **View & Submit Jobs** menu option, and selecting `gedi-subset:<VERSION>`
 from the dropdown.  You will be prompted for the inputs as described in the
 previous section.
 
@@ -294,7 +294,7 @@ MAAP API from a Notebook (or a Python script), as follows:
 ```python
 from maap.maap import MAAP
 
-maap = MAAP(maap_host='api.ops.maap-project.org')
+maap = MAAP(maap_host='api.maap-project.org')
 
 # See "Algorithm Inputs" section as well as "Specifying a DOI"
 inputs = dict(
@@ -307,11 +307,12 @@ inputs = dict(
    query="<QUERY>",
    limit=0,  # 0 implies no limit to the number of granules downloaded
    temporal="<RANGE>",  # or exclude this input for unlimited temporal range
+   output="<OUTPUT FILE>",
 )
 
 result = maap.submitJob(
     identifier="<DESCRIPTION>",
-    algo_id="gedi-subset_ubuntu",
+    algo_id="gedi-subset",
     version="<VERSION>",
     queue="maap-dps-worker-32gb",
     username="<USERNAME>",  # Your Earthdata Login username
@@ -324,59 +325,54 @@ job_id or result
 
 ### Checking the DPS Job Status
 
-To check the status of your job via the ADE UI, open the **DPS/MAS Operations**
-menu, choose **Get DPS Job Status**, and enter the value of the `job_id` to
-obtain the status, just as if you had submitted the job from the menu (rather
-than programmatically).
+To check the status of your job via the ADE UI, open the **Jobs**
+menu, choose **View & Submit Jobs**, and view the list of running jobs to see
+their statuses.
 
 Alternatively, to programmatically check the status of the submitted job, you
 may run the following code.  If using a notebook, use a separate cell so you can
 run it repeatedly until you get a status of either `'Succeeded'` or `'Failed'`:
 
 ```python
-import re
-
 # Should evaluate to 'Accepted', 'Running', 'Succeeded', or 'Failed'
-re.search(
-   r"Status>(?P<status>.+)</wps:Status>",
-   maap.getJobStatus(job_id).text
-).group('status')
+from maap.maap import MAAP
+maap = MAAP(maap_host='api.maap-project.org')
+
+maap.getJobStatus(job_id)
 ```
 
 ### Getting the DPS Job Results
 
 Once the job status is either **Succeeded** or **Failed**, you may obtain the
-job results either via the UI.
+job results either via the **View & Submit Jobs** page within the ADE or program-
+matically via the MAAP API.
 
 To obtain the results programmatically, you can use the following to see the
 details.  Note the use of `print`, which is necessary within a Jupyter notebook
 because the output may contain multiple lines.  Without wrapping everything in a
 call to `print`, the output might be very hard to read:
 
 ```python
-print(
-   re.search(
-      r"Data>(?P<data>.+)</wps:Data>",
-      maap.getJobResult(job_id).text,
-      re.DOTALL
-   ).group('data')
-)
+from maap.maap import MAAP
+maap = MAAP(maap_host='api.maap-project.org')
+
+maap.getJobResult(job_id)
 ```
 
 If the jobs status is **Failed**, the job results should show failure details.
 
-If the job status is **Succeeded**, the job results should show 3 URLs, with the
+If the job status is **Succeeded**, the job results should show 2 URLs, with the
 first URL of the following form:
 
 ```plain
-http://.../<USERNAME>/dps_output/gedi-subset_ubuntu/<VERSION>/<DATETIME_PATH>
+http://.../<USERNAME>/dps_output/gedi-subset/<VERSION>/<DATETIME_PATH>
 ```
 
 Based upon this URL, the `gedi_subset.gpkg` file generated by the job should be
 available at the following path within the ADE:
 
 ```plain
-~/my-private-bucket/dps_output/gedi-subset_ubuntu/<VERSION>/<DATETIME_PATH>/gedi_subset.gpkg
+~/my-private-bucket/dps_output/gedi-subset/<VERSION>/<DATETIME_PATH>/gedi_subset.gpkg
 ```
 
 ## Getting the GeoJSON URL for a geoBoundary
@@ -460,13 +456,6 @@ To prepare for contributing, do the following in an ADE workspace:
 
 1. Clone this GitHub repository.
 1. Change directory to the cloned repository.
-1. Add the GitLab repository as another remote (named `ade` here, but you may
-   specify a different name for the remote):
-
-   ```bash
-   git remote add --tags -f ade https://repo.ops.maap-project.org/data-team/gedi-subsetter.git
-   ```
-
 1. Create the `gedi_subset` virtual environment (**NOTE:** _you will need to
    repeat this step whenever your restart your ADE workspace_):
 
@@ -499,9 +488,6 @@ explained below.
 1. Create a new branch based on an appropriate existing branch (typically based
    on `main`).
 1. Add your desired code and/or configuration changes.
-1. Update the value of `version` in `algorithm_config.yaml`
-   according to the versioning convention referenced at the top of the
-   [Changelog](./CHANGELOG.md).
 1. Add appropriate entries to the [Changelog](./CHANGELOG.md), according to
    the [Keep a Changelog] convention.  In particular:
    - Add a new, second-level section of the following form:
@@ -535,42 +521,44 @@ explained below.
 
 ### Registering an Algorithm Release
 
-Once a release is published in the GitHub repository (see above), the code from
-the GitHub repository must be pushed to the GitLab repository in order to be
-able to register the new version of the algorithm, as follows, within the ADE:
-
-1. Open a Terminal tab (if necessary) and change directory to the repository.
-1. Pull the latest code from GitHub (to obtain merged PR, if necessary):
-
-   ```bash
-   git checkout main
-   git pull origin
-   ```
-
-1. Push the latest code to GitLab (replace `ade` with the appropriate remote
-   name, if you didn't use `ade` earlier):
-
-   ```bash
-   git push --all ade
-   git push --tags ade
-   ```
-
-   **NOTE:** On occassion, you might get a "server certificate verification
-   failed" error attempting to push to GitLab.  If so, simply prefix the
-   preceding commands with `GIT_SSL_NO_VERIFY=1`
-1. In the ADE's File Browser, navigate to `gedi-subsetter`.
-1. Right-click on `algorithm_config.yaml` and choose **Register as MAS
-   Algorithm** from the context menu.
-1. Confirm that the value of the **version** field matches the GitHub release
-   version you created above.  If not, click **Cancel** and review earlier
-   steps.  If so, click **Ok**, which will trigger a build job that will take
-   about 30 minutes.
+Once a release is published in the GitHub repository (see above), the algorithm
+must be registered in the ADE.  To do so, follow these steps:
+1. Within an ADE workspace, open a **New Launcher** tab and underneath the
+   **MAAP Extensions** section, select **Register Algorithm**.
+   **Terminal** launcher.
+1. On the main registration page:
+   1. Under Repository Information:
+      - Enter the **Repository URL** as
+         `https://github.com/MAAP-Project/gedi-subsetter.git`.
+      - Enter the **Repository Branch** as the name of the release tag (e.g., `1.0.0`).
+      - Enter the **Run Command** as `gedi-subsetter/subset.sh`.
+      - Enter the **Build Command** as `gedi-subsetter/build.sh`.
+   1. Under General Information:
+      - Enter the **Algorithm Name** as `gedi-subsetter`.
+      - Enter the **Algorithm Description** as `Subset GEDI L1B, L2A, L2B, or L4A granules within an area of interest (AOI)`.
+      - Enter the **Disk Space** as `20GB`.
+      - Enter the **Resource Allocation** as `maap-dps-worker-32gb`.
+      - Enter the **Container URL** as `mas.dit.maap-project.org/root/maap-workspaces/base_images/r:dit`.
+   1. Under Input Types, add the following Name, Description pairs:
+      1. For File Inputs:
+         - `aoi`: Area of Interest GeoJSON file
+      1. For Positional Inputs:
+         - `doi`: Digital Object Identifier (DOI) of the GEDI collection to subset, or a logical name representing such a DOI
+         - `temporal`: (Default: full temporal range): Temporal range to subset.
+         - `lat`: Name of the dataset used for latitude.
+         - `lon`: Name of the dataset used for longitude.
+         - `beams`: (Default: `all`): `all`, `coverage`, or `power`, or a comma-separated list of specific beam names, with or without the `BEAM` prefix (e.g., `BEAM0000,BEAM0001` or `0000,0001`).
+         - `columns`: One or more column names, separated by commas, to include in the output file.
+         - `query`: (Default: no query): Query expression for subsetting the rows in the output file.
+         - `limit`: (Default: 1_000): Maximum number of GEDI granule data files to download from the CMR
+         - `output`: Name to use for the output file.
+1. Once finished, click **Register Algorithm**
 1. Check the build job status at
-   <https://repo.ops.maap-project.org/root/register-job/-/jobs>.  If the job
+   <https://repo.maap-project.org/root/register-job-hysds-v4/-/jobs>.  If the job
    fails, you will need to correct the issue (and likely create a patch release,
    following the release steps again).  Otherwise, you should now be able to
-   open the **DPS/MAS Operations** menu, choose **Execute DPS Job**, and find
-   the new version of the algorithm in the dropdown list for confirmation.
+   open the **Jobs** menu, choose **View & Submit Jobs**, and find
+   the new version of the algorithm in the dropdown list of the **Submit** tab.
 
 ## Citations
 
@@ -592,7 +580,7 @@ administrative boundaries.  PLoS ONE 15(4): e0231866.
   https://keepachangelog.com/en/1.0.0/
 [MAAP GEDI Subsetter hosted on GitHub]:
   https://github.com/MAAP-Project/gedi-subsetter.git
-[MAAP GEDI Subsetter hosted on GitLab]:
-  https://repo.ops.maap-project.org/data-team/gedi-subsetter.git
+[MAAP GitLab]:
+  https://repo.maap-project.org/
 [NASA MAAP]:
-  https://ops.maap-project.org/
+  https://maap-project.org/

src/gedi_subset/subset.py

@@ -352,7 +352,7 @@ def main(
     # output directory.
     osx.remove(f"{dest}")
 
-    maap = MAAP("api.ops.maap-project.org")
+    maap = MAAP("api.maap-project.org")
     cmr_host = "cmr.earthdata.nasa.gov"
 
     IOResult.do(