Julia NetCDF getting negative values from nc file

[fabiogdy]

Hello!

I have been trying to read a NetCDF file with Julia, and this file has mostly positive values, however when I read the contents of a variable, the output is mostly negative. I have also tried reading the same file with R, Python and GRADS, and their output, which is positive, are the same. I am not sure if I am missing something.

The variables have the following dimensions: 23,376 time steps, longitude that range from 306.625 to 316 (X size = 75) and latitude that range from -26.625 to -19.625 (Y size = 48).

I tried both methods posted here in this git (high.jl and low.jl), and both give the same result.

Thank you in advance for any help!

[visr]

Hi, is it possible to share your file? If you could do that with a short example of how you check the values in Python and how you do it in Julia, we might be able to spot something.

[fabiogdy]

Hello!

Github does not recognize the file format, therefore I uploaded it here: https://www.dropbox.com/s/cnuoylm9azzsv4x/solar_fc.nc?dl=0

Hope you are able to access it.

Regarding the Python code, I did the following:

from netCDF4 import Dataset
dataset = Dataset("solar_fc.nc", "r")
ssrd = dataset.variables["ssrd"]

In Python the dimension order looks a bit different from what we have in Julia. In Python the dimension is [time steps, y, x], whereas in Julia it is [x, y, time steps].

In Julia, I did:

using NetCDF
nc = NetCDF.open("solar_fc.nc")
ssrd = nc.vars["ssrd"]

In this case, I just fixed x = 0 and y = 0, and changed the time steps parameter, and I was expecting to see the same values for position [20, 0, 0] (Python) and [0, 0, 21] (Julia), for example. I double-checked to see if x = 0 and y = 0 give the same longitude and latitude both in Python and Julia, and it does, I also checked that time step 20 is the same as time step 21 in Julia. That being said, when I check ssrd value at the positions I mentioned, I get a value of 9,249,791.43 in Python, and a value of -12,583 in Julia.

Any idea why am I seeing this? I am almost sure I am messing something up, but I just can't figure out.

Thank you again!

[visr]

The difference in value is due to the scale_factor and offset_value that your NetCDF has. In Python they are automatically applied, in this package they are (currently) not. See also #39.

So when applying these corrections, we get the same value:

>>> ssrd[0,0,0]
0.0
>>> ssrd[-1,-1,-1]
11535283.475119406

julia> ssrd[1] * ssrd.atts["scale_factor"] + ssrd.atts["add_offset"]
0.0

julia> ssrd[end] * ssrd.atts["scale_factor"] + ssrd.atts["add_offset"]
1.1535283475119406e7

The dimension order indeed appears differently here, but this is intentional and not an issue. It has to do with the memory layout, and preventing having to copy data. See the comment here: JuliaIO/Zarr.jl#1 (comment).