Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update aggregation layers documentation #9135

Merged
merged 2 commits into from
Sep 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 20 additions & 36 deletions docs/api-reference/aggregation-layers/aggregation-layer.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,38 @@
# AggregationLayer
# AggregationLayer (Experimental)

All of the layers in `@deck.gl/aggregation-layers` module perform some sort of data aggregation. All these layers perform aggregation with different parameters (CPU vs GPU, aggregation to rectangular bins vs hexagon bins, world space vs screen space, aggregation of single weight vs multiple weights etc).
`AggregationLayer` is the base class for all layers in `@deck.gl/aggregation-layers` module. It implements the most common tasks for aggregation with flexibility of customizations.

`AggregationLayer` and `GridAggregationLayer` perform most of the common tasks for aggregation with flexibility of customizations. This document describes what `AggregationLayer` does and how to use it in other aggregation layers.
`AggregationLayer` extends [CompositeLayer](../core/composite-layer.md).

## Methods

`AggregationLayer` is subclassed form `CompositeLayer` and all layers in `@deck.gl/aggregation-layers` are subclassed from this Layer.
Any layer subclassing the `AggregationLayer` must implement the following methods:

## Integration with `AttributeManager`
#### `getAggregatorType` {#getaggregatortype}

This layer creates `AttributeManager` and makes it available for its subclasses. Any aggregation layer can add attributes to the `AttributeManager` and retrieve them using `getAttributes` method. This enables using `AttributeManager`'s features and optimization for using attributes. Also manual iteration of `data` prop can be removed and attributes can be directly set on GPU aggregation models or accessed directly for CPU aggregation.
Returns a string that indicates the type of aggregator that this layer uses, for example `'gpu'`. The aggregator type is re-evaluated every time the layer updates (usually due to props or state change). If the type string does not match its previous value, any existing aggregator will be disposed,and `createAggregator` is called to create a new instance.

Example: Adding attributes to an aggregation layer
#### `createAggregator` {#createaggregator}

```
const attributeManager = this.getAttributeManager();
attributeManager.add({
positions: {size: 3, accessor: 'getPosition'},
color: {size: 3, accessor: 'getColorWeight'},
elevation: {size: 3, accessor: 'getElevationWeight'}
});
```
Arguments:
- `type` (string) - return value from `getAggregatorType()`

## updateState()
Returns a [Aggregator](./aggregator.md) instance. This instance will be accessible via `this.state.aggregator`.

During update state, Subclasses of `AggregationLayer` must first call 'super.updateState()', which calls
#### `onAttributeChange` {#onattributechange}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thought for future: Could this be a base layer callback?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably require Aggregtor to be defined with a map of dependencies/update triggers. I'll play with some options.

- `updateShaders(shaders)` : Subclasses can override this if they need to update shaders, for example, when performing GPU aggregation, aggregation shaders must be merged with argument of this function to correctly apply `extensions`.
Arguments:
- `attributeId` (string) - the id of an attribute that has been updated

- `_updateAttributes`: This checks and updates attributes based on updated props.
This event handler should be used to update the props of the aggregator, if needed, and call `aggregator.setNeedsUpdate` to request an update.

## Checking if aggregation is dirty
#### `renderLayers` {#renderlayers}

### Dimensions
Returns a list of sub layers.

Typical aggregation, involves :
1. Group the input data points into bins
2. Compute the aggregated value for each bin
Aggregation results can be obtained here with `aggregator.getBins`, `aggregator.getResult` and `aggregator.getResultDomain`.

For example, when `cellSize` or `data` is changed, layer needs to perform both `1` and `2` steps, when a parameter affecting a bin's value is changed (like `getWeight` accessor), layer only need to perform step `2`.

When doing CPU Aggregation, both above steps are performed individually. But for GPU aggregation, both are merged into single render call.
## Source

To support what state is dirty, constructor takes `dimensions` object, which contains, several keyed dimensions. It must contain `data` dimension that defines, when re-aggregation needs to be performed.

### isAggregationDirty()

This helper can be used if a dimension is changed. Sublayers can defined custom dimensions and call this method to check if a dimension is changed.


### isAttributeChanged()

`AggregationLayer` tracks what attributes are changed in each update cycle. Super classes can use `isAttributeChanged()` method to check if a specific attribute is changed or any attribute is changed.
[modules/aggregation-layers/src/common/aggregation-layer.ts](https://github.com/visgl/deck.gl/tree/master/modules/aggregation-layers/src/common/aggregation-layer.ts)
168 changes: 168 additions & 0 deletions docs/api-reference/aggregation-layers/aggregator.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# Aggregator Interface

The `Aggregator` interface describes a type of class that performs aggregation.

## Terminology

_Aggregation_ is a 2-step process:

1. **Sort**: Group a collection of _data points_ by some property into _bins_.
2. **Aggregate**: for each _bin_, calculate a numeric output (_result_) from some metrics (_values_) from all its members. Multiple results can be obtained independently (_channels_).

An implementation of the _Aggregator_ interface takes the following inputs:
- The number of data points
- The group that each data point belongs to, by mapping each data point to a _binId_ (array of integers)
- The values to aggregate, by mapping each data point in each channel to one _value_ (number)
- The method (_operation_) to reduce a list of values to one number, such as SUM

And yields the following outputs:
- A list of _binIds_ that data points get sorted into
- The aggregated values (_result_) as a list of numbers, comprised of one number per bin per channel
- The [min, max] among all aggregated values (_domain_) for each channel

### Example

Consider the task of making a [histogram](https://en.wikipedia.org/wiki/Histogram) that shows the result of a survey by age distribution.

1. The _data points_ are the list of participants, and we know the age of each person.
2. Suppose we want to group them by 5-year intervals. A 21-year-old participant is assigned to the bin of age 20-25, with _binId_ `[20]`. A 35-year-old participant is assigned to the bin of age 35-40, with _binId_ `[35]`, and so on.
3. For each bin (i.e. age group), we calculate 2 _values_:
+ The first _channel_ is "number of participants". Each participant in this group yields a _value_ of 1, and the result equals all values added together (_operation_: SUM).
+ The second _channel_ is "average score". Each participant in this group yields a _value_ that is their test score, and the result equals the sum of all scores divided by the number of participants (_operation_: MEAN).
4. As the outcome of the aggregation, we have:
+ Bins: `[15, 20, 25, 30, 35, 40]`
+ Channel 0 result: `[1, 5, 12, 10, 8, 3]`
+ Channel 0 domain: `[1, 12]`
+ Channel 1 result: `[6, 8.2, 8.5, 7.9, 7.75, 8]`
+ Channel 1 domain: `[6, 8.5]`


## Methods

An implementation of `Aggregator` should expose the following methods:

#### `setProps` {#setprops}

Set runtime properties of the aggregation.

```ts
aggregator.setProps({
pointCount: 10000,
attributes: {...},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One day we should call this columns. Not sure if we could go that far now....

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the same format as props.data.attributes and we have a bunch of classes/types along that line such as AttributeManager, BinaryAttribute etc. At least for now I don't think we should add more concepts to the mix.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. But I would love to give this an overhaul in deck.gl v10 as part of deep Arrow integration etc.

operations: ['SUM', 'MEAN'],
binOptions: {groupSize: 5}
});
```

Arguments:
- `pointCount` (number) - number of data points.
- `attributes` ([Attribute](../core/attribute.md)[]) - the input data.
- `operations` (string[]) - How to aggregate the values inside a bin, defined per channel.
- `binOptions` (object) - arbitrary settings that affect bin sorting.
- `onUpdate` (Function) - callback when a channel has been recalculated. Receives the following arguments:
+ `channel` (number) - the channel that just updated

#### `setNeedsUpdate` {#setneedsupdate}

Flags a channel to need update. This could be a result of change in the input data or bin options.

```ts
aggregator.setNeedsUpdate(0);
```

Arguments:
- `channel` (number, optional) - mark the given channel as dirty. If not provided, all channels will be updated.

#### `update` {#update}

Called after all props are set and before results are accessed. The aggregator should allocate resources and redo aggregations if needed at this stage.

```ts
aggregator.update();
```

#### `preDraw` {#predraw}

Called before the result buffers are drawn to screen. Certain types of aggregations are dependent on render time context and this is alternative opportunity to update just-in-time.

```ts
aggregator.preDraw({moduleSettings: ...});
```

#### `getBin` {#getbin}

Get the information of a given bin.

```ts
const bin = aggregator.getBin(100);
```

Arguments:
- `index` (number) - index of the bin to locate it in `getBins()`

Returns:
- `id` (number[]) - Unique bin ID.
- `value` (number[]) - Aggregated values by channel.
- `count` (number) - Number of data points in this bin.
- `pointIndices` (number[] | undefined) - Indices of data points in this bin if available. This field may not be populated when using GPU-based implementations.

#### `getBins` {#getbins}

Get an accessor to all bin IDs.

```ts
const binIdsAttribute = aggregator.getBins();
```

Returns:
- A [binary attribute](../core/layer.md#dataattributes) of the output bin IDs, or
- null, if `update` has never been called

#### `getResult` {#getresult}

Get an accessor to the aggregated values of a given channel.

```ts
const resultAttribute = aggregator.getResult(0);
```

Arguments:
- `channel` (number) - the channel to retrieve results from

Returns:
- A [binary attribute](../core/layer.md#dataattributes) of the output values of the given channel, or
- null, if `update` has never been called

#### `getResultDomain` {#getresultdomain}

Get the [min, max] of aggregated values of a given channel.

```ts
const [min, max] = aggregator.getResultDomain(0);
```

Arguments:
- `channel` (number) - the channel to retrieve results from

Returns the domain ([number, number]) of the aggregated values of the given channel.

#### `destroy` {#destroy}

Dispose all allocated resources.

```ts
aggregator.destroy();
```


## Members

An implementation of `Aggregator` should expose the following members:

#### `binCount` (number) {#bincount}

The number of bins in the aggregated result.

## Source

[modules/aggregation-layers/src/common/aggregator/aggregator.ts](https://github.com/visgl/deck.gl/tree/master/modules/aggregation-layers/src/common/aggregator/aggregator.ts)
35 changes: 25 additions & 10 deletions docs/api-reference/aggregation-layers/contour-layer.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ npm install @deck.gl/core @deck.gl/layers @deck.gl/aggregation-layers

```ts
import {ContourLayer} from '@deck.gl/aggregation-layers';
import type {ContourLayerProps} from '@deck.gl/aggregation-layers';
import type {ContourLayerProps, ContourLayerPickingInfo} from '@deck.gl/aggregation-layers';

new ContourLayer<DataT>(...props: ContourLayerProps<DataT>[]);
```
Expand All @@ -171,7 +171,7 @@ new deck.ContourLayer({});

Inherits from all [Base Layer](../core/layer.md) properties.

### Render Options
### Aggregation Options

#### `cellSize` (number, optional) ![transition-enabled](https://img.shields.io/badge/transition-enabled-green.svg?style=flat-square") {#cellsize}

Expand All @@ -183,19 +183,27 @@ Size of each cell in meters

* Default: true

When set to true and browser supports GPU aggregation, aggregation is performed on GPU. GPU aggregation can be 2 to 3 times faster depending upon number of points and number of cells.
When set to `true` and the browser supports it, aggregation is performed on GPU.

In the right context, enabling GPU aggregation can significantly speed up your application. However, depending on the nature of input data and required application features, there are pros and cons in leveraging this functionality. See [CPU vs GPU aggregation](./overview.md#cpu-vs-gpu-aggregation) for an in-depth discussion.


#### `aggregation` (string, optional) {#aggregation}

* Default: 'SUM'
* Default: `'SUM'`

Defines the operation used to aggregate all data object weights to calculate a cell's value. Valid values are:

- `'SUM'`: The sum of weights across all points that fall into a cell.
- `'MEAN'`: The mean weight across all points that fall into a cell.
- `'MIN'`: The minimum weight across all points that fall into a cell.
- `'MAX'`: The maximum weight across all points that fall into a cell.
- `'COUNT'`: The number of points that fall into a cell.

Defines the type of aggregation operation, valid values are 'SUM', 'MEAN', 'MIN' and 'MAX'. When no value or an invalid value is set, 'SUM' is used as aggregation.
`getWeight` and `aggregation` together determine the elevation value of each cell.

* SUM : Grid cell contains sum of all weights that fall into it.
* MEAN : Grid cell contains mean of all weights that fall into it.
* MIN : Grid cell contains minimum of all weights that fall into it.
* MAX : Grid cell contains maximum of all weights that fall into it.

### Render Options

#### `contours` (object[], optional) {#contours}

Expand Down Expand Up @@ -240,11 +248,18 @@ The weight of each object.
* If a function is provided, it is called on each object to retrieve its weight.


## Picking

The [PickingInfo.object](../../developer-guide/interactivity.md#the-pickinginfo-object) field returned by hover/click events of this layer represents a path (isoline) or a polygon (isoband). The object contains the following fields:

- `contour` (object) - one of the contour configurations passed to the `contours` prop.


## Sub Layers

The `ContourLayer` renders the following sublayers:

* `lines` - For Isolines, rendered by [LineLayer](../layers/line-layer.md)
* `lines` - For Isolines, rendered by [PathLayer](../layers/path-layer.md)
* `bands` - For Isobands, rendered by [SolidPolygonLayer](../layers/solid-polygon-layer.md)


Expand Down
Loading
Loading