Update to latest wgpu-native

[almarklein]

For context, the last update was #289 and finished January 2023.

Preparations:

• Any outstanding issues or PR's in wgpu-py or pygfx that we want to apply first? -> Nope!
• Make a release of wgpu-py: Bump version #368
• Make a release of pygfx against the latest wgpu-py version before the update: Bump version pygfx#593
• Update wgpu-native to lates wgpu-core, if necessary. (done 2 days ago)
• Update wgpu-native to lates webgpu.h, if necessary. (no need, is just about 2 weeks old)
• Release wgpu-native, if necessary. Latest release (0.17.0.2) is from 25-07-2023. -> Already done: 0.17.2.1

Updating:

• Checkout the kind of changes required and see if it makes sense to apply the update in multiple steps.
• Update API to latest IDL #369
• Update to wgpu-native 0.17.0.2 #370
• Update pygfx pygfx#595
• Release wgpu-py.
• Update to wgpu-native 0.17.2.1 #373
• Release wgpu-py.
• Release pygfx.
• Mind the update for handling of strings for uncaptured error callbacks, see refactor gfx-rs/wgpu-native#262 (comment).

More improvements after-care:

• Any outstanding issues that make sense to apply now? Perhaps implement some of the stuff that is available in wgpu-native but not yet in wgpu-py?
  • See Post-update wgpu sprint #387

---

wgpu-native versions

• 0.14.2.3 (25-01-2023) This is what we're currently at.
• 0.14.2.4 (30-01-2023)
• 0.15.1.1 (14-02-2023)
• 0.15.1.2 (27-02-2023)
• 0.16.0.1 (28-04-2023)
• 0.16.1.1 (06-06-2023)
• 0.16.2.1 (12-07-2023)
• 0.17.0.1 (24-07-2023)
• 0.17.0.2 (25-07-2023)
• 0.17.2.1 (06-10-2023

[zk-tarts]

Hey! I'm interested in having wgpu-py working with the latest version of wgpu-native.

I tried running the update script(s), Ran into another issue that needed fixing.
This pr gfx-rs/wgpu-native#243 changed the api for getting present modes.

Also its worth noting somewhere that the WGSL language has changed since the release
type became alias and top level let is disallowed in favour of const https://github.com/gfx-rs/wgpu/releases

[almarklein]

Yeah, the update process typically comes with added complexities :) - We have the update planned after the current work on mesh morphing.

[hmaarrfk]

I would be interesting in helping out to some extent. I've managed to compile wgpu-native for conda forge so hopefully this should make this a "python only exercise".

https://github.com/conda-forge/wgpu-native-feedstock/

I'm happy to compile a few intermediate versions as well if that helps.

Second, there would be the challenge of updating your CIs for PyPI. I know that isn't always trivial.

[almarklein]

Thanks. Well one thing that would be great, since the wgpu-native binaries are now on conda, is to enable wgpu-py to use these. I.e. check whether the binary is bundled or detectable on conda, and select one.

Second, there would be the challenge of updating your CIs for PyPI.

Sorry, what do you mean? We already publish binary wheels to pypi. Not sure if its related to your comment, but the wgpu-native repo has CD setup to produce binaries (using manylinux).

[hmaarrfk]

wgpu-py to use these

wgpu-py, as installed from conda-forge will use these today. it will pin to the specific version you use since there were some breaks when i tried to use version 0.17

Second, there would be the challenge of updating your CIs for PyPI.

I mean that there work going from version 0.14 to 0.17 and sometimes compilation fails. Not sure if it really is as easy as updating a single line in a configuration file.

[Korijn]

You appear to misunderstand the setup here. Let me try to clarify:

• wgpu-native is compiled here: https://github.com/gfx-rs/wgpu-native
  • the build pipeline produces binaries which are published here: https://github.com/gfx-rs/wgpu-native/releases
• wgpu-py (this repo) holds a script to download those binaries, so there is no compilation step in this repo
  • wgpu-py is 99% python wrappers for the c++ interface of wgpu-native

So updating wgpu-py to a newer version of wgpu-native is a three step process:

• Pinning to a new version of wgpu-native
• Determining what portion of the wrappers need to be updated
• Updating and testing the wrappers

[almarklein]

Not sure if it really is as easy as updating a single line in a configuration file.

Hihi, I was it was 😄 But the "Determining what portion of the wrappers need to be updated" step that Korijn mentions above is a pretty big step that will cost me multiple days, especially now that we're behind wgpu-native by quite a bit.

[hmaarrfk]

Thanks for the explanation. In your experience, would it make sense to update from

• 0.14 to 0.15, then from 0.15 to 0.16, then from 0.16 to 0.17

or to just try to do the leap from

• 0.14 to 0.17

Is there a blocker, perhaps meshmorphing, that you still want to accomplish before this work is undertaken (potentially by others)

[almarklein]

It depends on the situation, i.e. what changes are made in each release. But it might indeed be easier to do it in 2 steps (or more).

Is there a blocker, perhaps meshmorphing, that you still want to accomplish before this work is undertaken

Yeah, I'm still wrapping up the meshmorphing now.

[Korijn]

Just letting all the subscribers here know @almarklein is working on this now, see #370

If you're curious to see what the work entails, this is your moment to observe

[almarklein]

Also see the top post in this issue, that I use for planning the update.