Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare for the release (changelog, README, copyright, etc) #221

Merged
merged 10 commits into from
Apr 8, 2021
12 changes: 10 additions & 2 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,18 @@ Changelog for cfgrib
0.9.9.0 (unreleased)
--------------------

- Depend on the ECMWF `eccodes python package <https://pypi.org/project/eccodes>`_ to access
the low level ecCodes C-library, dropping all other GRIB decoding options.
See: `#95 <https://github.com/ecmwf/cfgrib/issues/95>`_,
`#14 <https://github.com/ecmwf/cfgrib/issues/14>`_.
`#204 <https://github.com/ecmwf/cfgrib/issues/204>`_,
`#147 <https://github.com/ecmwf/cfgrib/issues/147>`_ and
`#141 <https://github.com/ecmwf/cfgrib/issues/141>`_.
- Many performance improvements during the generation of the index and during data access.
See: `#142 <https://github.com/ecmwf/cfgrib/issues/142>`_.
See: `#142 <https://github.com/ecmwf/cfgrib/issues/142>`_ and
`#197 <https://github.com/ecmwf/cfgrib/issues/197>`_.
- ``filter_by_keys`` now can select on all keys known to *ecCodes* without the need to
add non default ones to ``read_keys`` explicitely.
add non default ones to ``read_keys`` explicitly.
See: `#187 <https://github.com/ecmwf/cfgrib/issues/187>`_.
- Fixed issue where could not load a GRIB message that has only one grid point.
See: `#199 <https://github.com/ecmwf/cfgrib/issues/199>`_.
Expand Down
27 changes: 14 additions & 13 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,29 +11,30 @@ The high level API is designed to support a GRIB engine for `xarray <http://xarr
and it is inspired by `netCDF4-python <http://unidata.github.io/netcdf4-python/>`_
and `h5netcdf <https://github.com/shoyer/h5netcdf>`_.
Low level access and decoding is performed via the
`ECMWF ecCodes library <https://software.ecmwf.int/wiki/display/ECC/>`_.
`ECMWF ecCodes library <https://software.ecmwf.int/wiki/display/ECC/>`_ and
the `eccodes python package <https://pypi.org/project/eccodes>`_.

Features with development status **Beta**:

- enables the ``engine='cfgrib'`` option to read GRIB files with *xarray*,
- reads most GRIB 1 and 2 files including heterogeneous ones with ``cfgrib.open_datasets``,
- supports all modern versions of Python 3.9, 3.8, 3.7, 3.6 and PyPy3,
- supports all modern versions of Python 3.9, 3.8, 3.7 and PyPy3,
- the 0.9.6.x series with support for Python 2 will stay active and receive critical bugfixes,
- works on *Linux*, *MacOS* and *Windows*, the *ecCodes* C-library is the only binary dependency,
- works wherever *eccodes-python* does: *Linux*, *MacOS* and *Windows*
- conda-forge package on all supported platforms,
- reads the data lazily and efficiently in terms of both memory usage and disk access,
- allows larger-than-memory and distributed processing via *dask*,
- allows larger-than-memory and distributed processing via *xarray* and *dask*,
- supports translating coordinates to different data models and naming conventions,
- supports writing the index of a GRIB file to disk, to save a full-file scan on open.

Work in progress:

- **Alpha** install a ``cfgrib`` utility that can convert a GRIB file ``to_netcdf``
- **Beta** install a ``cfgrib`` utility that can convert a GRIB file ``to_netcdf``
with a optional conversion to a specific coordinates data model,
see `#40 <https://github.com/ecmwf/cfgrib/issues/40>`_.
- **Alpha** support writing carefully-crafted ``xarray.Dataset``'s to a GRIB1 or GRIB2 file,
see the *Advanced write usage* section below and
`#18 <https://github.com/ecmwf/cfgrib/issues/18>`_.
- **Alpha/Broken** support writing carefully-crafted ``xarray.Dataset``'s to a GRIB1 or GRIB2 file,
see the *Advanced write usage* section below, `#18 <https://github.com/ecmwf/cfgrib/issues/18>`_
and `#156 <https://github.com/ecmwf/cfgrib/issues/156>`_.

Limitations:

Expand All @@ -58,14 +59,14 @@ Python package from *PyPI* with::
Binary dependencies
-------------------

The Python module depends on the `eccodes python package <https://pypi.org/project/eccodes/>`_
*cfgrib* depends on the `eccodes python package <https://pypi.org/project/eccodes>`_
to access the ECMWF *ecCodes* binary library,
when not using *conda* please follow the *System dependencies* section there.

You may run a simple selfcheck command to ensure that your system is set up correctly::

$ python -m cfgrib selfcheck
Found: ecCodes v2.19.0.
Found: ecCodes v2.20.0.
Your system is ready.


Expand All @@ -82,9 +83,9 @@ Read-only *xarray* GRIB engine
------------------------------

Most of *cfgrib* users want to open a GRIB file as a ``xarray.Dataset`` and
need to have *xarray>=0.12.0* installed::
need to have *xarray* installed::

$ pip install xarray>=0.12.0
$ pip install xarray

In a Python interpreter try:

Expand Down Expand Up @@ -827,7 +828,7 @@ See also the list of `contributors <https://github.com/ecmwf/cfgrib/contributors
License
=======

Copyright 2017-2020 European Centre for Medium-Range Weather Forecasts (ECMWF).
Copyright 2017-2021 European Centre for Medium-Range Weather Forecasts (ECMWF).

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cf2cdm/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2017-2020 European Centre for Medium-Range Weather Forecasts (ECMWF).
# Copyright 2017-2021 European Centre for Medium-Range Weather Forecasts (ECMWF).
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cf2cdm/cfcoords.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2017-2020 European Centre for Medium-Range Weather Forecasts (ECMWF).
# Copyright 2017-2021 European Centre for Medium-Range Weather Forecasts (ECMWF).
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cf2cdm/cfunits.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2017-2020 European Centre for Medium-Range Weather Forecasts (ECMWF).
# Copyright 2017-2021 European Centre for Medium-Range Weather Forecasts (ECMWF).
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cf2cdm/datamodels.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2017-2020 European Centre for Medium-Range Weather Forecasts (ECMWF).
# Copyright 2017-2021 European Centre for Medium-Range Weather Forecasts (ECMWF).
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cfgrib/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2017-2020 European Centre for Medium-Range Weather Forecasts (ECMWF).
# Copyright 2017-2021 European Centre for Medium-Range Weather Forecasts (ECMWF).
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
37 changes: 36 additions & 1 deletion cfgrib/__main__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2017-2020 European Centre for Medium-Range Weather Forecasts (ECMWF).
# Copyright 2017-2021 European Centre for Medium-Range Weather Forecasts (ECMWF).
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -61,11 +61,46 @@ def to_netcdf(inpaths, outpath, cdm, engine):
ds = xr.open_dataset(inpaths[0], engine=engine) # type: ignore
else:
ds = xr.open_mfdataset(inpaths, engine=engine, combine="by_coords") # type: ignore

if cdm:
coord_model = getattr(cf2cdm, cdm)
ds = cf2cdm.translate_coords(ds, coord_model=coord_model)

ds.to_netcdf(outpath)


@cfgrib_cli.command("dump")
@click.argument("inpaths", nargs=-1)
@click.option("--variable", "-v", default=None)
@click.option("--cdm", "-c", default=None)
@click.option("--engine", "-e", default="cfgrib")
def dump(inpaths, variable, cdm, engine):
# type: (T.List[str], str, str, str) -> None
import xarray as xr

import cf2cdm

# NOTE: noop if no input argument
if len(inpaths) == 0:
return

if len(inpaths) == 1:
# avoid to depend on dask when passing only one file
ds = xr.open_dataset(inpaths[0], engine=engine) # type: ignore
else:
ds = xr.open_mfdataset(inpaths, engine=engine, combine="by_coords") # type: ignore

if cdm:
coord_model = getattr(cf2cdm, cdm)
ds = cf2cdm.translate_coords(ds, coord_model=coord_model)

if variable:
ds_or_da = ds[variable]
else:
ds_or_da = ds

print(ds_or_da)


if __name__ == "__main__": # pragma: no cover
cfgrib_cli()
2 changes: 1 addition & 1 deletion cfgrib/cfmessage.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2017-2020 European Centre for Medium-Range Weather Forecasts (ECMWF).
# Copyright 2017-2021 European Centre for Medium-Range Weather Forecasts (ECMWF).
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
2 changes: 1 addition & 1 deletion cfgrib/dataset.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2017-2020 European Centre for Medium-Range Weather Forecasts (ECMWF).
# Copyright 2017-2021 European Centre for Medium-Range Weather Forecasts (ECMWF).
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
Loading