Map Space View and GPS Coordinates archetype (Mapbox/OSM support)

[tfoldi]

What

Map Space View for GNSS/GPS data logged through new GPS Coordinates archetype.

rerun_mapview.mp4

• Added a new re_space_view_map crate for visualizing Points3D with gps coordinates.
  • map_visualizer_system.rs implements the VisualizerSystem trait. It has only one meaningful function, that takes the Positions3D components and turns into vec<MapEntry> array along with optional Color and Radius components. This vec<MapEntry> is the only field of the MapVisualizerSystem.
  • map_space_view.rs is where the magic happens. It uses walkers slippy map implementation to show an OpenStreetMap or Mapbox map. On the selection_ui you can change the map providers. To view Mapbox maps, you need to set mapbox tokens (you can get free tokens from Mapbox's site) and pass it via the env variable (MAPBOX_ACCESS_TOKEN) or via the blueprint
  • map_windows.rs is a small helper to show the zoom controls and acknowledgment on the UIs.
• For blueprints I made MapProvider component (enum with providers+styles), and a MapOptions archetype.

The logging of data is performed as follows:

//! Log some GPS coordinates

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let rec = rerun::RecordingStreamBuilder::new("rerun_example_gps_coordinates").spawn()?;

    rec.log(
        "points",
        &rerun::Points3D::new([(47.6344, 19.1397, 0.0), (47.6344, 19.1399, 1.0)]),
    )?;

    Ok(())
}

The blueprint is defined as follows:

"""Use a blueprint to show a map."""

import rerun as rr
import rerun.blueprint as rrb

rr.init("rerun_example_gps_coordinates", spawn=True)

rr.log("points", Points3D([[47.6344, 19.1397, 0], [47.6334, 19.1399, 1]]))

# Create a map view to display the chart.
blueprint = rrb.Blueprint(
    rrb.MapView(
        origin="points",
        name="MapView",
        map_options=rrb.archetypes.MapOptions(provider=rrb.components.MapProvider.MapboxStreets),
    ),
    collapse_panels=True,
)

rr.send_blueprint(blueprint)

The PR is not perfect, but the view is usable.

Please review the changes and evaluate their potential utility for your needs. If you don't like, prefer not to merge just close it - no hard feelings.

To be discussed

• Check with you guys to see if the name is right at all. Maybe instead of GPS we can go with something better
• Position3D is limited to f32 instead of f64. This makes it less precise than /NavSatFix

Checklist

• I have read and agree to Contributor Guide and the Code of Conduct
• I've included a screenshot or gif (if applicable)
• I have tested the web demo (if applicable):
  • Using my own lame demo app which is not part of the PR.
• The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG

• PR Build Summary
• Recent benchmark results
• Wasm size tracking

To run all checks from main, comment on the PR with @rerun-bot full-check.

[k-bx]

Awesome work! Sorry if it's a noob question, but how do I check this out?

ubuntu@localhost:~/workspace/rerun-tfoldi$ pixi run rerun-web-release
 Pixi task (rerun-build-web-release in default): rustup target add wasm32-unknown-unknown && cargo run -p re_dev_tools -- build-web-viewer --release -g
info: component 'rust-std' for target 'wasm32-unknown-unknown' is up to date
    Finished dev [optimized + debuginfo] target(s) in 0.10s
     Running `target/debug/re_dev_tools build-web-viewer --release -g`
Building web viewer…

Compiling Rust to wasm in /home/ubuntu/workspace/rerun-tfoldi/target_wasm…
/home/ubuntu/workspace/rerun-tfoldi> CARGO_ENCODED_RUSTFLAGS="--cfg=web_sys_unstable_apis" RUSTFLAGS="--cfg=web_sys_unstable_apis" "cargo" "build" "--quiet" "--package" "re_viewer" "--lib" "--target" "wasm32-unknown-unknown" "--target-dir" "/home/ubuntu/workspace/rerun-tfoldi/target_wasm" "--no-default-features" "--features=analytics" "--release"
error: package `bitstream-io v2.4.2` cannot be built because it requires rustc 1.79 or newer, while the currently active rustc version is 1.76.0
Either upgrade to rustc 1.79 or newer, or use
cargo update [email protected] --precise ver
where `ver` is the latest version of `bitstream-io` supporting rustc 1.76.0
Error: Failed to build Wasm

[tfoldi]

Either upgrade to rustc 1.79 or newer, or use
cargo update [email protected] --precise ver
where ver is the latest version of bitstream-io supporting rustc 1.76.0
Error: Failed to build Wasm

New rerun needs rust version 1.79 while you have only 1.76. Update your rust toolchain and try it again.

[k-bx]

Ah, needed to update the rust-toolchain file. All good, compiled fine!

[tfoldi]

Few changes we discussed:

• Remove GpsCoordinate archetype
• Add ViewCoordinates.LAT_LON_ALT enum value to indicate that a Position3D represents lat/lon/alt values
• Use ViewCoordinates in view heuristics to select the right heuristics for view
• due to walkers' reqwest dependency it should be an optional feature

[jleibs]

* Add `ViewCoordinates.LAT_LON_ALT` enum value to indicate that a `Position3D` represents lat/lon/alt values 
* Use `ViewCoordinates` in view heuristics to select the right heuristics for view

I talked this one over with some folks and the desire is to keep ViewCoordinates separate and somewhat dedicated to interpretation of camera-orientation, as distinct from data semantics.

I'm going to put together a proposal for a new archetype, probably named something like CRS (CoordinateReferenceSystem). We can start with this somewhat dedicated to the mapping use-cases, but I think it could be helpful for other things like CAD import where it might be useful to carry context like unit-sizes.

[jleibs]

Here's my proposal for adding a very simple CRS type as an indicator: #6601

[tfoldi]

The new image crate depends on a newer version of Rust by default unless you disable it by setting default-features = false. Since walkers enables default features, this is causing the Rust build to fail on CI. I will look into how this can be addressed in the original repository.

The current cargo tree:

    ├── re_data_source v0.17.0-alpha.4 (/Users/.../rerun/crates/re_data_source)
    │   ├── re_data_loader v0.17.0-alpha.4 (/Users/.../rerun/crates/re_data_loader)
    │   │   ├── image v0.25.1
    │   │   │   ├── ravif v0.11.5
    │   │   │   │   ├── rav1e v0.7.1
    │   │   │   │   │   ├── bitstream-io v2.4.2  <--- this requires 1.79

In image's Cargo.toml:

ravif = { version = "0.11.2", default-features = false, optional = true }

[tfoldi]

• Removed GpsCoordinates in favor of pure Points3D.
• Removed default features from image in walkers repo so now the code works with older rustc.
• Formatted/linted/removed all gps position related stuff
• Added ZoomLevel and Secret components for the map options archetype
  • Secret component displayed as password field in selection_ui

What is left from what we discussed:

1. add feature flags to make the map view optional

For 1, I'm open to your suggestions. Should the feature be enabled or disabled by default? Either way, how should the CI/CD integration work? Should we test all features by default or test combinations of features?

[patrickelectric]

@tfoldi nice work! Excited to see it merged :)

[tfoldi]

Extended the nuScenes example with MapView to have at least one example using the new view:

nuscenes_mapview.mp4

[journaux]

would this support plotting a scatter of points across the map (instead of a single point/track) ? from the PR seems yes a la rr.log("points", Points3D([[47.6344, 19.1397, 0], [47.6334, 19.1399, 1]])) however the visual confused me (centers on a single point)

[tfoldi]

would this support plotting a scatter of points across the map (instead of a single point/track) ? from the PR seems yes a la rr.log("points", Points3D([[47.6344, 19.1397, 0], [47.6334, 19.1399, 1]])) however the visual confused me (centers on a single point)

yes, it does support multiple points

[journaux]

good to know ! curious if team/egui has evaluated integrating cesium

-> its rendering pipeline for large set of N points may be faster
-> it has 3d terrain capability

[k-bx]

Hey, still very keen to see this. What's the current blocker?

[samorr]

The same from my side, will it be continued?
Is there any plan to make GPS coordinates using float64? With 32 bits it's very 'rough'

[tfoldi]

The same from my side, will it be continued?

I’m open to continuing this, but it's largely dependent on the Rerun team's priorities. As you may have seen from the PR history, I tried to keep it up-to-date for a few versions but eventually paused due to the complexity involved. There are many moving parts in Rerun, and with so much auto-generated code and changing internals, maintaining consistency across view-related PRs isn’t straightforward.

I received two key comments from @jleibs:

1. Introduce new archetype for minimal CoordinateReferenceSystem (CRS) #6601: Suggestion to introduce a new CoordinateReferenceSystem type as type indicator. Personally, I’m not sure this is necessary on day one. We could potentially move forward by logging Position3D types and setting the view to Map in the blueprint.
2. The walker egui component brings in the reqwest crate, so making it an optional feature might be a cleaner approach.

On June 22nd, I asked a few questions regarding these points, but since I didn’t receive any responses, so I paused further updates. That said, I’m happy to rebase the PR to the latest main branch if there’s a realistic chance it could be merged.

[abey79]

@tfoldi First, thanks again for your contribution, and for putting up with our slow responses so far. With 0.19 out and the huge push on the dataframe and video fronts, we are now ready to focus on other areas—and this one is high up our list. Starting on Monday, I'll make it my priority to help you land this PR.

Would you be willing to rebase it to latest main? My plan would then be to dive in, give it a thorough review, and help (including hands-on) to land it sooner than later.

As for the "feature flag": yes we want one, that is disabled by default (at least initially). This will give us more confidence to land stuff before they are 100% polished. To be clear, by "feature flag", we don't necessarily mean a Cargo-style feature, but a global application setting which dynamically enable/disable the map view in the viewer. That's often how we signify to users that a particular view is "in beta". (For illustration, I recently removed the feature flag we had on the dataframe view in this PR).

[abey79]

This generally looks in excellent shape. Thanks for this contribution! I'm not yet completely done with my review, here are already some comments.

[abey79]

Very nice addition to this example. However, please open a separate PR for this, as I'm a bit worried this will create CI issue due to how we're initially going to deal with cargo features.

[abey79]

Note to self: update the architecture diagram accordingly

[abey79]

As discussed, this is probably no the right archetype to initially support. Either Point2D or some custom type.

[abey79]

I think selected_provider and access_token are not really necessary here. We usually try to keep view states as lean as possible. My suggestion:

• remove these two fields
• make map_memory an Option
• update map_memory from MapViewClass::ui()

Also, I'd either inline ensure_and_get_mut_refs or somehow move it to MapViewClass.

[tfoldi]

selected_provider and access_token are used to detect if I need to invalidate the tiles cache. what is the best way to check in ui() what parameters have been changed if not comparing the state with the new blueprint values?

[abey79]

You're right. Then let's just add a comment above selected_provider to indicate that it is used for invalidation.

[abey79]

Our usual semantics for double clicking is zoom to the "bounding box", e.g. whatever area that contains the entire view content.

I haven't looked into it yet. If it is involved, we can keep that for a follow-up PR though.

[tfoldi]

double click reset the view (re-enable follow mode). We can change the zoom level as well, but if you have only point finding what zoom level the user expects is tricky imu

[abey79]

In addition to zoom level, it would be extremely valuable to store in the blueprint the scroll position (coordinate of center point?). In particular, that would make it settable from code.

If that's too involved, we can keep it for a follow-up PR.

[tfoldi]

absolutely possible, but the map center can be more than a position:

• it can be in follow mode, which is useful if you have one or interconnected points. here the first logged point is always in the center (like in the embedded video on the PR description)
• we can fix the map at a certain lat, long position

what archetypes should I use to store this? in the lib if the map center is None then it is in the following mode; otherwise, it is fixed. but I assume this approach is not idiomatic in rerun

[tfoldi]

According to your last comment, the center_position should be DVec2D, and follow_mode should be a new enum or a boolean with a well-named entry to keep the number of components down.

[abey79]

My gut feeling would be to minimise the feature set for this PR and skip any sort of "auto-scroll" mode. The MVP would be:

• CenterPosition view property (datatype: DVec2D) to store the center location along with the zoom level
• Best effort implementation for the fallback provider of CenterPosition based on whatever data is logged

In the future, we indeed need to devise a set of more advanced auto-scroll/follow mode, but I think the hindsight we'll gain from landing this PR will help designing that properly. (Ideally, it'd be somewhat consistent with the 2D view too.)

[abey79]

see my other note, would be great to store the map position in the blueprint as well

[abey79]

I just discussed the data model aspect of this PR with @jleibs and we reached the following decision.

We introduce a new GeoPoints archetype as per this outline:

table GeoPoints  {
  /// All the 3D positions at which the point cloud shows points.
  latlon: [rerun.components.LatLon];  // datatype: DVec2D

  // --- Recommended ---

  /// Optional radii for the points, effectively turning them into circles.
  radii: [rerun.components.Radius];

  /// Optional colors for the points.
  colors: [rerun.components.Color];

  // --- Optional ---

  /// Optional text labels for the points.
  labels: [rerun.components.Text];

  /// Optional choice of whether the text labels should be shown by default.
  show_labels: rerun.components.ShowLabels;
}

This PR will have to introduce the DVec2D datatype, identical to the Vec2D but with double precision floats.

For the scope of this PR, the recommended/optional components should be included only so long as they are actually supported by the view. If they are not, then it's fine to drop them—we can add them in a subsequent PR.

In the future, we will further add a Altitude recommended component to carry the height information. How we define the kind of altitude we support (ground height, ellipsoid, amsl, whatnot) is yet to be defined. We currently lean towards having that setting as a view property, but that's for another day to decide.

Motivation for this decision:

• Using Points2D is problematic precision-wise. f32 means ~1.7m real world accuracy. GeoPoints has f64.
• Splitting the latlon part from the (future) Altitude component clarifies that these numbers dont have the same units (angles for latlon, likely meters for Altitude).
• More generally, we will try to adhere to the standard GIS data model going forward (in particular the GeoArrow model). The proposed GeoPoints archetype fits the MultiPoint case (by virtue of our components always being batch), and it's likely that MultiLineString and MultiPolygon will eventually follow.
• Having a distinct archetype from Points2D also has the advantage to disambiguate things for the heuristics.

[tfoldi]

For the scope of this PR, the recommended/optional components should be included only so long as they are actually supported by the view. If they are not, then it's fine to drop them—we can add them in a subsequent PR.

we can support text labels. However, shouldn't you be able to change their fonts, sizes, and colors?

[abey79]

For the scope of this PR, the recommended/optional components should be included only so long as they are actually supported by the view. If they are not, then it's fine to drop them—we can add them in a subsequent PR.

we can support text labels. However, shouldn't you be able to change their fonts, sizes, and colors?

No, I don't think this is required for this PR (and the short term future tbh). IWe don't support styling of labels in other views either.

[abey79]

Let's make sure we reference https://epsg.io/4326 in the GeoPoint archetype and LatLon component.