Getting GEOS cubed-sphere History output into uxarray easily

[mathomp4]

Back in April I attended the ISS Conference in Boulder and one of the subjects I was most intrigued by was on UXarray by @philipc2 and @erogluorhan (tutorial and talk).

Now, our cubed-sphere format is in a way an unstructured grid. Well, more unstructured than boring old lat-lon, enough that most tools for lat-lon viewing can't understand it. (The one major exception other than home-grown tools is Panoply which @rschmunk was nice enough to work with us to get it to recognize our format.)

UXarray already sort of has an "easy" mode for MPAS, SCRIP, and ESMF grid (and other) formats. I believe internally it can recognize these formats and then converts to UGRID format.

So, my thought is in order to get GEOS CS support in UXarray, our main challenge is figuring out how to get GEOS CS into UGRID format (a la UXARRAY/uxarray#773 for ICON).

Good thing: we have people like @bena-nasa who knows SCRIP and ESMF pretty well and so the fact that there is SCRIP-to-UGRID conversion somewhere in there, might mean it can be sort of "reverse engineered" to a GEOS-to-UGRID converter.

[philipc2]

Hi @mathomp4

I'm glad you found the talks during ISS interesting! Always happy to hear from the community.

Good thing: we have people like @bena-nasa who knows SCRIP and ESMF pretty well and so the fact that there is SCRIP-to-UGRID conversion somewhere in there, might mean it can be sort of "reverse engineered" to a GEOS-to-UGRID converter.

Yeah, ESMF grids are very similar to UGRID, with minor modifications needed to get them represent in the UGRID conventions.

If you have a sample grid that I could take a look at, I should be able to get a rough idea on how we can move forward with implementing a GEOS CS grid reader.

[mathomp4]

Note: these aren't stretched grid. I'm still looking at trying to get you something from that, but I've encountered a few runtime...oddities in trying to generate some on my own.

[philipc2]

Sweet!! I've worked with some data from the DAMOND initiative before (a 3.75km MPAS mesh), excited to try this one out!

[philipc2]

Woah, this is definitely the largest grid I've ever worked with.

<uxarray.Grid>
Original Grid Type: GEOS-CS
Grid Dimensions:
  * n_node: 199134726
  * n_face: 199065600
  * n_max_face_nodes: 4
  * n_nodes_per_face: (199065600,)
Grid Coordinates (Spherical):
  * node_lon: (199134726,)
  * node_lat: (199134726,)
  * face_lon: (199065600,)
  * face_lat: (199065600,)
Grid Coordinates (Cartesian):
Grid Connectivity Variables:
  * face_node_connectivity: (199065600, 4)

Rasterizing the cell-centers is extremely quick (roughly a few seconds) for generating a global plot

However, zooming in, much of the detail is lost.

Using a larger pixel ratio helps, at the cost of execution time.

Much better local fidelity

We also support polygon plotting with UXarray, but it is very impractical for the scale of this dataset. Going to look into taking a regional subset and doing a polygon plot on that region.

[mathomp4]

Very cool! And, yes, big. Many tools seem to choke on files this size. I suppose if there could be some...dynamic way of doing things? Raster at global scale but somehow transition to polygons at a finer scale? You are the experts!

Luckily, this is an extreme case. Our current operational run is at C720 and maybe someday we will move to C1440. But even at C720 you are near the size of pixels in most PNGs one produces.

[philipc2]

I suppose if there could be some...dynamic way of doing things? Raster at global scale but somehow transition to polygons at a finer scale? You are the experts!

Yeah this is something that Datashader & HoloViews enables us to do with our UXarray visualization API. It does however require some processing to get the grid represented as polygons.

This is what a 3.75km MPAS mesh looks like (from our Cookbook)

This grid is about 42 million faces and 84 million nodes, which is smaller than the C1440 grid. Takes about 93s to process the grid into polygons (a one-time operation), and then subsequent visualizations can be rendered in a few seconds (including dynamic rasterization when zooming / panning).

[rschmunk]

In something of a coincidence, I have been working on getting Panoply to plot UGRID netCDF data. Still at the incremental stage where, okay, I need an another example file in order to move on to the next improvement. But getting there.

[mathomp4]

It's all coming together!

from Imgflip Meme Generator