'hdf5.h' file not found during install on Apple Silicon

[aidanmontare-edu]

I am unable to install digital_rf on my M1 Mac. Has anyone else run into issues on Apple Silicon? (I am using Python directly on arm, not through Rosetta).

When I run pip install digital_rf, it fails with fatal error: 'hdf5.h' file not found.

The first time I tried this, it failed during the install of a dependency, h5py. I was able to get h5py installed and working by first installing hdf5 from homebrew and then providing this path when calling pip: HDF5_DIR=/opt/homebrew/opt/hdf5 pip install --no-binary=h5py h5py. I verified the h5py was installed correctly by running the tests as described in its documentation.

Even though hdf5 and h5py are now installed, installing digital_rf still fails with the same file not found error. The full output of pip install digital_rf is below. My best guess is that even though I have the dependencies somewhere on the system, it is not something that the install can find. However, I'm not well-versed in these things, so any help would be appreciated.

A few system details:

CPU: M1 Pro
Mac OS 12.3
python 3.10.2 (installed using pyenv)
pip 22.0.4
hdf5 1.13.0 (via homebrew)
h5py 3.6.0

Collecting digital_rf
  Using cached digital_rf-2.6.7.tar.gz (1.2 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: packaging in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (21.3)
Requirement already satisfied: six in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (1.16.0)
Requirement already satisfied: python-dateutil in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (2.8.2)
Collecting watchdog
  Using cached watchdog-2.1.6-cp310-cp310-macosx_11_0_arm64.whl (86 kB)
Requirement already satisfied: h5py in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (3.6.0)
Requirement already satisfied: numpy in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (1.22.2)
Requirement already satisfied: pytz in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from digital_rf) (2021.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./.pyenv/versions/3.10.2/lib/python3.10/site-packages (from packaging->digital_rf) (3.0.7)
Building wheels for collected packages: digital_rf
  Building wheel for digital_rf (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for digital_rf (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [37 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-12.2-arm64-3.10
      creating build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/ringbuffer.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/mirror.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/digital_rf_deprecated_hdf5.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/_version.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/util.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/drf_command.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/__init__.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/list_drf.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/watchdog_drf.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/digital_metadata.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      copying digital_rf/digital_rf_hdf5.py -> build/lib.macosx-12.2-arm64-3.10/digital_rf
      creating build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      copying gr_digital_rf/vector.py -> build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      copying gr_digital_rf/__init__.py -> build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      copying gr_digital_rf/digital_rf_sink.py -> build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      copying gr_digital_rf/raster.py -> build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      copying gr_digital_rf/digital_rf_source.py -> build/lib.macosx-12.2-arm64-3.10/gr_digital_rf
      UPDATING build/lib.macosx-12.2-arm64-3.10/digital_rf/_version.py
      set build/lib.macosx-12.2-arm64-3.10/digital_rf/_version.py to '2.6.7'
      running build_ext
      INFO: pkgconfig cannot find HDF5. Defaulting to HDF5_ROOT="/Users/aidan/.pyenv/versions/3.10.2"
      building 'digital_rf._py_rf_write_hdf5' extension
      creating build/temp.macosx-12.2-arm64-3.10
      creating build/temp.macosx-12.2-arm64-3.10/lib
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/private/var/folders/7w/6t8zqxtj03n_hhb4jmx6knh00000gn/T/pip-install-2wrim69t/digital-rf_a3d1bf96baeb4f93a17ee50fbdaa775b/include -I/Users/aidan/.pyenv/versions/3.10.2/include/python3.10 -I/private/var/folders/7w/6t8zqxtj03n_hhb4jmx6knh00000gn/T/pip-build-env-atbhlnz4/overlay/lib/python3.10/site-packages/numpy/core/include -I/Users/aidan/.pyenv/versions/3.10.2/include -I/opt/local/include -I/usr/local/include -I/usr/include -c lib/py_rf_write_hdf5.c -o build/temp.macosx-12.2-arm64-3.10/lib/py_rf_write_hdf5.o
      In file included from lib/py_rf_write_hdf5.c:24:
      /private/var/folders/7w/6t8zqxtj03n_hhb4jmx6knh00000gn/T/pip-install-2wrim69t/digital-rf_a3d1bf96baeb4f93a17ee50fbdaa775b/include/digital_rf.h:46:10: fatal error: 'hdf5.h' file not found
      #include "hdf5.h"
               ^~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for digital_rf
Failed to build digital_rf
ERROR: Could not build wheels for digital_rf, which is required to install pyproject.toml-based projects

[ryanvolz]

@jswoboda Can you take a look at this on your M1?

@aidanmontare-edu Have you ever used the conda package manager before? You might have better luck installing digital_rf with that since it will already be compiled. At least until we can see about making the source build work better for the M1.

[aidanmontare-edu]

@ryanvolz I'm not currently a conda user, but that's something I could try

[aidanmontare-edu]

@ryanvolz I installed miniconda on my system and was able to install digital_rf! I ran one or two of the examples, which finished fine, so this seems like a success.

I'd still be interested in a way to install / build with pip, if that becomes possible down the road. I am happy to be a tester for work on this.