Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CreateFromDepthImage uint16 confusion #6962

Open
3 tasks done
rurban opened this issue Sep 11, 2024 · 1 comment
Open
3 tasks done

CreateFromDepthImage uint16 confusion #6962

rurban opened this issue Sep 11, 2024 · 1 comment
Labels
bug Not a build issue, this is likely a bug.

Comments

@rurban
Copy link
Contributor

rurban commented Sep 11, 2024

Checklist

Describe the issue

I get strange result from my realsense Z16 depth images, which has the depths stored as uint16_t (little-endian).
open3d PointCloud::CreateFromDepthImage only accepts num_channels == 1 and bytes_per_channel == 2 and converts the values then via Image::ConvertDepthToFloatImage. The other accepted format is float32 as bytes_per_channel_ == 4. This is fine.

But ConvertDepthToFloatImage does float *p = output->PointerAt<float>(x, y); which accesses the data as float32 array for an uint16 image. Which looks wrong.
The final image looks good though. There must be some confusion somewhere, which I cannot find.

It should really convert the uint16 to float per value first.

Steps to reproduce the bug

    vector<uint16_t> depths;
    // ... read depths from a tiff
    auto image = make_shared<geometry::Image>();
    image->Prepare(width, height, /* num_channels */ 1, /* bytes_per_channel */ 2);
    memcpy(image->data_.data(), depths.data(), depths.size() * 2);
    open3d::camera::PinholeCameraIntrinsic intrinsics(
        width, height, intr->f[0], intr->f[1], intr->pp[0], intr->pp[1]);
    pcd = geometry::PointCloud::CreateFromDepthImage(*image, intrinsics);

Error message

No response

Expected behavior

No response

Open3D, Python and System information

- Operating system: Ubuntu 24.04
- Open3D version: latest main 553ca86cc11d57324b84eb96ae7cda5c90011091
- System architecture: x86
- Compiler version (if built from source): gcc 13

Additional information

No response

@rurban rurban added the bug Not a build issue, this is likely a bug. label Sep 11, 2024
@rurban
Copy link
Contributor Author

rurban commented Sep 12, 2024

As workaround I have to create a FloatImage and create the pixels by myself.

    geometry::Image gimage;
    auto image = gimage.CreateFloatImage();
    //o3d can only handle float images, not others. despite documented otherwise
    image->Prepare(width, height, /* num_channels */ 1, /* bytes_per_channel */ 4);
    for (int k = 0; k < depths.size(); k++) {
      const int x = k % width;
      const int y = k / width;
      float *p = image->PointerAt<float>(x, y);
      *p = depths[k] * 0.001; // mm to m
    }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Not a build issue, this is likely a bug.
Projects
None yet
Development

No branches or pull requests

1 participant