Buffer mapping API

[almarklein]

This closes #114, also updated that issue's top comment to explain the issue better.

Repeating the advantages of the WebGPU mapping API for clarity:

• Allows reading and writing data in a buffer.
• Allows doing that with subranges.
• Allows doing that with multiple subranges in one go, i.e. without mapping/unmapping multiple times.
• Without unnecessary data copies.

I've had another look at disabling the base array so that the views don't work anymore, but without success.

Therefore this proposal does not expose a mapped array to read from / write to. Instead it has two methods for this: read_mapped() and write_mapped(). The former has a kwarg copy that can be set to False to get the actually mapped data. Users who know what they're doing can then benefit from more performance.

With this new API we support all the four points above. It's also a lot closer to the WebGPU spec: we have map() and unmap() in between which the data can be read or written multiple times (fast!). But instead of obtaining a piece of the mapped memory (getMappedRange()), we have read_mapped() and write_mapped(), so we don't have to expose the mapped memory (except whith read_mapped(..., copy=False).

[almarklein]

I moved the test file with cases that do not work to a Gist: https://gist.github.com/almarklein/a7b59d5cf18ae979f79602115807b0b2

[Korijn]

This may be ignorant but how bad is it really that you could have a view on released memory? It could just be a documented risk that users of wgpu-py need to be aware of.

[almarklein]

Well we could document it carefully, but it's so easy to forget, also because this is not something people are used to having to care about in Python. What makes it worse is that these views can easily end up in a different place unrelated to wgpu. That, plus the fact that segfaults are sometimes not reproducible (sometimes it will crash, sometimes you get bogus data), can result in some nasty bug tracking ...

I also realized just now that the data-copy downside of only an issue when reading from the buffer. When writing it's not because Python/numpy makes use of views so much.

[almarklein]

Just pushed a proposal for an API. The buffer can be mapped and unmapped like the WebGPU spec, but instead of requesting an array-like object (that represents the mapped memory) to read from or write to, the buffer has methods read_mapped() and write_mapped().

Still have to adjust examples and tests.

[Korijn]

Is it guaranteed that the buffer will not be unmapped until you call the unmap function?

If so, a context manager might be very nice in combination with the proposed copy=True kwarg, to create a "safe space" where you can work with a zero-copy guaranteed-mapped buffer?

with buffer.map(copy=False) as view:
    view.do_stuff()  # safe?

[almarklein]

Is it guaranteed that the buffer will not be unmapped until you call the unmap function?

The other way is that the buffer gets destroyed.

[almarklein]

Ran into an interesting edge-case. Filed an issue upstream: gfx-rs/wgpu-native#305 For now I fixed it here.

[almarklein]

This change does not actually require changes in PyGfx - it looks like we only used the parts of the API that have not changed.

[almarklein]

This change does not actually require changes in PyGfx - it looks like we only used the parts of the API that have not changed.

Woops, using it in the picking logic. Will prepare a pr for pygfx shortly.