Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename yearly NO2 collection from OMI_trno2-COG to no2-yearly #141

Open
1 task
j08lue opened this issue Jun 7, 2024 · 3 comments
Open
1 task

Rename yearly NO2 collection from OMI_trno2-COG to no2-yearly #141

j08lue opened this issue Jun 7, 2024 · 3 comments

Comments

@j08lue
Copy link
Contributor

j08lue commented Jun 7, 2024

We currently have three NO2 products in our catalog:

  • no2 - monthly data
  • no2-monthly-difference - monthly anomaly (difference from some baseline)
  • OMI_trno2-COG - yearly data

They present the same type of property (NO2 concentrations), just different derived products. Would it make sense to rename the OMI_trno2-COG collection to no2-yearly or so, to harmonize the naming pattern between the three?

We would need to update it in VEDA Earthdata Dashboard and the EO Dashboard, but I think that is easy enough to do.

Acceptance criteria

  • Renamed OMI_trno2-COG to no2-yearly
@anayeaye
Copy link
Contributor

anayeaye commented Jun 7, 2024

@j08lue This change of collection id is better handled by creating a new collection because the id is an index and the database and the collection id maps 1:1 to the s3 prefix in the production data store. If we decide to do this, et's instead:

  1. duplicate all object in veda-data-store/OMI_trno2-COG to veda-data-store/no2-yearly
  2. (a) create a new ingest-items/production/collection with id no2-yearly.json here in veda-data along with a (b) discovery items config file (it should be easy to produce following how the original collection and discovery were configured. (c) We will also need a transfer-config json that shows how the UAH covid-eo-dashboard/OMI_trno2-COG collection maps to no2-yearly in production (I believe this is an ongoing collection?)
  3. publish the new collection and ingest all items
  4. announce that the old collection_id is deprecated and coordinate with the dashboard team, EOX, and EIC
  5. finally when all three dashboards are updated for the new collection_id we can begin the deprecation process of deleting the stac records and setting a lifecycle policy on the deprecated prefix in s3

@j08lue
Copy link
Contributor Author

j08lue commented Jun 13, 2024

Great to see the process laid out this clearly, @anayeaye, and sounds like a very solid approach. Btw, we should copy this to operations docs somewhere - like "How to rename collections" and "How to deprecate collections"!

Not sure it is worth the effort for the OMI rename, though? What do others think - @anayeaye, @slesaad, @smohiudd?

@anayeaye
Copy link
Contributor

Btw, we should copy this to operations docs somewhere - like "How to rename collections" and "How to deprecate collections"!

Yes! We do also have some work planned on covering how to remove VEDA data safely #116, #117
Maybe we could sneak in some differentiation on collection id vs name in that documentation, too? It is easy to change the collection name but that index is a more significant change. On the same note, maybe we should add waaaaay more emphasis on choosing a collection id in our contributing docs https://nasa-impact.github.io/veda-docs/contributing/dataset-ingestion/. I think it is not initially clear how valuable a human readable meaningful STAC Collection id is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants