diff --git a/docs/how_to_publish_to_cmr_with_earthdata_varinfo.ipynb b/docs/how_to_publish_to_cmr_with_earthdata_varinfo.ipynb index 6444a97..e83778d 100644 --- a/docs/how_to_publish_to_cmr_with_earthdata_varinfo.ipynb +++ b/docs/how_to_publish_to_cmr_with_earthdata_varinfo.ipynb @@ -5,32 +5,10 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## How to use `earthdata-varinfo` to publish UMM-Var records to CMR" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Version 2.0.0" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ + "## How to use `earthdata-varinfo` to publish UMM-Var records to CMR\n", "### Overview:\n", "\n", - "This notebook demonstrates how to create and publish, Unified Metadata Model-Variable (UMM-Var) records to NASA's Common Metadata Repository (CMR) with, `earthdata-varinfo` >= 2.0.0. `earthdata-varinfo` utilizes [`python-cmr`](https://github.com/nasa/python_cmr) to query CMR for collection granules to download locally. The `VarInfoFromNetCDF4` class in `earhdata-varinfo` is used to create CMR compliant UMM-Var entries. Lastly, the `requests` library is used to publish UMM-Var records to a given CMR environment (`OPS`, `UAT`, and `SIT`)." - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ + "This notebook demonstrates how to create and publish, Unified Metadata Model-Variable (UMM-Var) records to NASA's Common Metadata Repository (CMR) with, `earthdata-varinfo` >= 2.0.0. `earthdata-varinfo` utilizes [`python-cmr`](https://github.com/nasa/python_cmr) to query CMR for collection granules to download locally. The `VarInfoFromNetCDF4` class in `earhdata-varinfo` is used to create CMR compliant UMM-Var entries. Lastly, the `requests` library is used to publish UMM-Var records to a given CMR environment (`OPS`, `UAT`, and `SIT`).\n", "### Setting up your environment to run this notebook\n", "\n", "Create and activate your `pyenv` or conda environment, then:\n", @@ -45,47 +23,29 @@ "git clone https://github.com/nasa/earthdata-varinfo\n", "cd earthdata-varinfo\n", "pip install -e .\n", - "```" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ + "```\n", "### Other notebook requirements:\n", "\n", - "Required packages for `earthdata-varinfo` should be installed as dependencies of PyPI. For local development, without a standard pip installation, third party requirements can be installed from the following files:\n", + "Required packages should be installed as dependencies of `earthdata-varinfo` when doing a pip install from PyPI. \n", + "For local development, without a standard pip installation, third party requirements can be installed from the following files:\n", "\n", "```\n", "pip install -r requirements.txt -r dev-requirements.txt\n", "pip install notebook\n", - "```" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ + "```\n", "### Example usage:\n", "\n", "* [GLDAS_NOAH10_3H](https://cmr.uat.earthdata.nasa.gov/search/collections.umm_json?concept-id=C1256543837-EEDTEST)\n", - "* [M2I1NXASM](https://cmr.uat.earthdata.nasa.gov/search/collections.umm_json?concept-id=C1256535511-EEDTEST)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ + "* [M2I1NXASM](https://cmr.uat.earthdata.nasa.gov/search/collections.umm_json?concept-id=C1256535511-EEDTEST)\n", + "\n", "### Authorization:\n", "\n", - "* Launchpad or Bearer tokens must be used in order query and publish to CMR.\n", - "* Bearer tokens are tokens that contain the header `Bearer` before the token\n", - " * For example: `Bearer tHiS iS mY tOKen`\n", - "* Launchpad tokens are tokens that do **NOT** contain any headers\n", - " * For example: `tHiS iS mY tOKen`\n", + "* Launchpad or EDL tokens must be used in order query and publish to CMR.\n", + "* Authorization headers for EDL tokens contain the header prefix `Bearer` before the token\n", + " * For example: `Bearer `\n", + "* Authorization headers for Launchpad tokens do **NOT** contain any prefixes in the header\n", + " * For example: ``\n", + "\n", "To request a Launchpad Token visit:\n", "* [Launchpad Authentication User's Guide](https://wiki.earthdata.nasa.gov/display/CMR/Launchpad+Authentication+User%27s+Guide)" ] @@ -105,7 +65,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Update `auth_header` to include your Bearer (e.g. `'Bearer tHiS iS mY tOKen'`) or Launchpad token (e.g. `tHiS iS mY tOKen`)" + "Update `auth_header` to include your EDL token (e.g. `Bearer `) or Launchpad token (e.g. ``)" ] }, { @@ -114,7 +74,7 @@ "metadata": {}, "outputs": [], "source": [ - "auth_header = 'Header + Token or Only token'" + "auth_header = 'Bearer or '" ] }, { @@ -161,8 +121,12 @@ "\n", "* Download the most recent granule for **GLDAS_NOAH10_3H**\n", "* Generate the UMM-Var records for this granule\n", - "* Publish these records to CMR if `publish=True` and return a list of published variable concept-ids\n", - "(e.g. `['V1259971755-EEDTEST', 'V1259971757-EEDTEST' ...]`)" + "* Publish these records to CMR if `publish=True`. \n", + "* If `publish=True`, a list of ingested variable concept-ids or the error(s) from an unsucessful ingest is returned\n", + " * `['V1259971755-EEDTEST', 'V1259971757-EEDTEST', ...]` \n", + " * `['V1259971755-EEDTEST', '#: CMR error 1\\n #: CMR error 2', ...]`\n", + "* If `publish=False` (default) a list of UMM-Var entries is returned:\n", + " * `[...{'Name': 'lat', 'LongName': 'lat', ...}, {'Name': 'time', 'LongName': 'time', ...}...]`" ] }, { @@ -180,8 +144,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Publishing and creating UMM-Var entries for **M2I1NXASM** with:\n", - "\n", + "### Publishing and creating UMM-Var entries for **M2I1NXASM**:\n", + "This example is an alternative to using `generate_collection_umm_var`. It demonstrates the individual components of `generate_collection_umm_var` with:\n", "* `varinfo.cmr_search`: queries CMR for a granule download link and downloads granules locally\n", "* `VarInfoFromNetCDF4`: varinfo parent class that represents the contents of a granule\n", "* `varinfo.umm_var`: contains functions for creating and publishing UMM-Var records to CMR\n", @@ -230,7 +194,8 @@ "source": [ "Get the granule record and granule download URL with `get_granules` and `get_granule_link`\n", "\n", - "* `get_granules`: queries `CMR_OPS` for a UMM-G record (granule record) given a collection or granule concept-id\n", + "* `get_granules`: queries `CMR_OPS` (default) for a UMM-G record (granule record) given a collection or granule concept-id\n", + " * you can query any CMR environment by adding `cmr_env=CMR_UAT` or `cmr_env=CMR_SIT`\n", "* `get_granule_link`: parses the UMM-G record from `get_granules` for a data download URL" ] }, @@ -241,7 +206,6 @@ "outputs": [], "source": [ "granule_response = get_granules(concept_id=collection_concept_id_merra,\n", - " cmr_env=CMR_UAT,\n", " auth_header=auth_header)\n", "url = get_granule_link(granule_response)\n", "url" @@ -290,7 +254,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Retrieve a dictionary of UMM-Var JSON records." + "Retrieve a dictionary of UMM-Var JSON records\n", + "* Returns a nested dictionary of UMM-Var records with full variable paths as keys and their UMM-Var records as values\n", + "* e.g. `{'/lon': {'Name': 'lon', 'LongName': 'lon', ...}, '/lat': {'Name': 'lat', 'LongName': 'lat', ...}...}`" ] }, { @@ -308,7 +274,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Publish all UMM-Var records for **M2I1NXASM** to CMR_UAT with `publish_all_umm_var`;\n", + "Publish all UMM-Var records for **M2I1NXASM** to CMR_UAT with `publish_all_umm_var`\n", "* Returns a dictionary of variable names and variable concept-ids as key value pairs respectively.\n", "* Example output: ```{'/lon': 'V1259972387-EEDTEST', '/lat': 'V1259972389-EEDTEST'...}```" ] @@ -331,6 +297,7 @@ "metadata": {}, "source": [ "### Publish one UMM-Var record with the `var_info.get_variable()` object\n", + "This example is another alternative to using `generate_collection_umm_var`. In this example we use the granule we have already download locally (**M2I1NXASM**) to create and ingest a single UMM-Var record.\n", "* Use `var_info.get_variable()` to retrieve the variable object from `var_info`\n", "* Keys are the full variable paths (e.g. `'/TROPPV'`)" ]