Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Protobuf to v21.12 #2626

Merged
merged 3 commits into from
Nov 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Detailed instructions are provided below.
```
python >= 3.8
gcc >= 6.4
protobuf >= 3.20.3
protobuf >= 4.21.12
cmake >= 3.13.4
make >= 4.2.1 or ninja >= 1.10.2
java >= 1.11 (optional)
Expand Down
2 changes: 1 addition & 1 deletion docker/Dockerfile.llvm-project
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ RUN distro=$(cat /etc/os-release|grep -Po '(?<=^ID=").*(?=")|(?<=^ID=)[^"].*[^"]
&& ln -sf /usr/bin/python3 /usr/bin/python

# Install protobuf
ARG PROTOBUF_VERSION=3.20.3
ARG PROTOBUF_VERSION=21.12
RUN git clone -b v${PROTOBUF_VERSION} --recursive https://github.com/protocolbuffers/protobuf.git \
&& cd protobuf && ./autogen.sh \
&& ./configure --enable-static=no \
Expand Down
6 changes: 3 additions & 3 deletions docs/BuildOnLinuxOSX.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,13 +99,13 @@ make: *** [Makefile:146: all] Error 2.

The suggested workaround until jsoniter is fixed is as follows: install maven (e.g. `brew install maven`) and run `alias nproc="sysctl -n hw.logicalcpu"` in your shell.

#### Protobuf issue (Mac M1, specific to protobuf 3.20.3 which is currently required)
#### Protobuf issue (Mac M1, specific to protobuf 4.21.12 which is currently required)

On Mac M1, you may have some issues building protobuf. In particular, you may fail to install onnx (via `pip install -e third_party/onnx`) or you may fail to compile `onnx-mlir` (no arm64 symbol for `InternalMetadata::~InternalMetadata`).

The first failure is likely an issue with having multiple versions of protobuf.
Installing a version with `brew` was not helpful (version 3.20.3 because of a known bug that can be corrected with a patch below).
Uninstall the brew version, and make sure you install the right one with pip: `pip install protobuf==3.20.3`.
Installing a version with `brew` was not helpful (version 4.21.12 because of a known bug that can be corrected with a patch below).
Uninstall the brew version, and make sure you install the right one with pip: `pip install protobuf== 4.21.12`.

The second failure can be remediated by downloading protobuf source code, applying a patch, and installing it on the local machine.
See [Dockerfile.llvm-project](../docker/Dockerfile.llvm-project) on line 66 for cloning instructions. After cloning the right version, you should apply a patch [patch](https://github.com/protocolbuffers/protobuf/commit/0574167d92a232cb8f5a9107aabda0aefbc39e8b) by downloading from the link above and applying it.
Expand Down
6 changes: 3 additions & 3 deletions docs/BuildOnWindows.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ Build protobuf as a static library.

[same-as-file]: <> (utils/install-protobuf.cmd)
```shell
REM Check out protobuf v3.20.3
set protobuf_version=3.20.3
REM Check out protobuf v21.12
set protobuf_version=21.12
git clone -b v%protobuf_version% --recursive https://github.com/protocolbuffers/protobuf.git

set root_dir=%cd%
Expand All @@ -42,7 +42,7 @@ set PATH=%root_dir%\protobuf_install\bin;%PATH%

If you wish to be able to run all the ONNX-MLIR tests, you will also need to install the matching version of protobuf through pip. Note that this is included in the requirements.txt file at the root of onnx-mlir, so if you plan on using it, you won't need to explicitly install protobuf.
```shell
python3 -m pip install protobuf==3.20.3
python3 -m pip install protobuf==4.21.12
```

#### MLIR
Expand Down
2 changes: 1 addition & 1 deletion docs/Prerequisite.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
```
python >= 3.8
gcc >= 6.4
protobuf >= 3.20.3
protobuf >= 4.21.12
cmake >= 3.13.4
make >= 4.2.1 or ninja >= 1.10.2
java >= 1.11 (optional)
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ lit~=15.0
# numpy 1.24 deprecates np.object, np.bool, np.float, np.complex, np.str,
# and np.int which are used heavily in onnx-mlir.
numpy~=1.22.2, <=1.23.5
protobuf~=3.20
protobuf==4.21.12
pytest~=7.2
pytest-xdist~=3.0
4 changes: 2 additions & 2 deletions utils/install-protobuf.cmd
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
REM Check out protobuf v3.20.3
set protobuf_version=3.20.3
REM Check out protobuf v21.12
set protobuf_version=21.12
git clone -b v%protobuf_version% --recursive https://github.com/protocolbuffers/protobuf.git

set root_dir=%cd%
Expand Down
4 changes: 2 additions & 2 deletions utils/install-protobuf.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Check out protobuf (mac x86 seems to need an older version)
PROTOBUF_VERSION=3.18.3
# Check out protobuf
PROTOBUF_VERSION=21.12
git clone -b v${PROTOBUF_VERSION} --depth 1 --recursive https://github.com/protocolbuffers/protobuf.git

cd protobuf
Expand Down
Loading