[WIP] DEBUG only {2023.06,2023a} PyTorch-bundle v2.1.2 #603
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- PR to help debugging various issues when building PyTorch-bundle - includes a fix for `find_library` provided by `ctypes.util` which prevented importing `sndfile` - includes a fix for `aarch64/generic` where importing `sentencepiece` lead to the error `libtcmalloc_minimal.so.4: cannot allocate memory in static TLS block` - includes a fix for the extension `torchvision` where some library was not compiled with `jpeg` support, hence some tests failed
trz42
added
help wanted
|
Instance
|
|
Instance
|
|
Initially we'll build only for bot: build arch:x86_64/amd/zen2 repo:eessi.io-2023.06-software |
Updates by the bot instance
|
Updates by the bot instance
|
|
New job on instance
|
|
New job on instance
|
|
The two jobs (12607 and 12608) that did not include any fixes failed both in the sanity check for
we repeat the building for the same architectures bot: build arch:x86_64/amd/zen2 repo:eessi.io-2023.06-software |
Updates by the bot instance
|
Updates by the bot instance
|
|
New job on instance
|
|
New job on instance
|
|
Rebuilding for bot: build arch:x86_64/amd/zen2 repo:eessi.io-2023.06-software |
Updates by the bot instance
|
Updates by the bot instance
|
Updates by the bot instance
|
|
New job on instance
|
Maybe related to: |
…-layer into debug-2023.06-software.eessi.io-PyTorch-2.1.2-foss-2023a
- PR EESSI#655 implements a general fix for the import error
|
Rebuilding after #655 got merged to verify if the bot: build arch:x86_64/amd/zen2 repo:eessi.io-2023.06-software |
Updates by the bot instance
|
Updates by the bot instance
|
|
New job on instance
|
|
New job on instance
|
This was referenced Aug 6, 2024
…-layer into debug-2023.06-software.eessi.io-PyTorch-2.1.2-foss-2023a
|
Rebuilding after changes have been minimised (only hook for SentencePiece kept for now) and #660 has been ingested... bot: build arch:x86_64/amd/zen2 repo:eessi.io-2023.06-software |
Updates by the bot instance
|
Updates by the bot instance
|
|
New job on instance
|
…-layer into debug-2023.06-software.eessi.io-PyTorch-2.1.2-foss-2023a
|
Revisit switching off TCMALLOC... bot: build arch:aarch64/generic repo:eessi.io-2023.06-software |
Updates by the bot instance
|
Updates by the bot instance
|
|
New job on instance
|
|
Maybe switch off the following |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The main purpose of this PR is to facilitate debugging various issues when building PyTorch-bundle and demonstrating approaches that could solve the issues. It is expected that the fixes provided here are not final.
includes a fix forfind_libraryprovided byctypes.utilwhich prevented importingsoundfileaarch64/{generic,neoverse_n1,neoverse_v1}where importingsentencepiecelead to the errorlibtcmalloc_minimal.so.4: cannot allocate memory in static TLS blockincludes a fix for the extension$\rightarrow$ torchvisionwhere some library was not compiled withjpegsupport, hence some tests failedInitially we will disable all fixes, build for selected architectures and document the errors. We then enable fixes one-by-one and document the results (some error fixed, some new errors, ...).
Note, see the original PR for PyTorch-bundle (#585) for additional discussion about some of the issues listed above.