Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bridges V2 - dynamic lanes #4427

Closed
wants to merge 16 commits into from
Closed

Conversation

bkontur
Copy link
Contributor

@bkontur bkontur commented May 10, 2024

Migrated from the branch bridges-v2-migration/

TBD: more description when migration is done

Closes: paritytech/parity-bridges-common#2976

TODO

  • search and fix for comment: // TODO:(bridges-v2) - most of that stuff was introduced with free header execution: https://github.com/paritytech/polkadot-sdk/pull/4102 - more info in the comment bellow

@bkontur

This comment was marked as resolved.

@svyatonik

This comment was marked as resolved.

@bkontur

This comment was marked as resolved.

@bkontur bkontur force-pushed the bko-bridges-v2 branch 3 times, most recently from bcc6204 to 69ad2f8 Compare June 5, 2024 10:07
@bkontur bkontur force-pushed the bko-bridges-v2 branch 2 times, most recently from 4d3758f to d1e9479 Compare June 7, 2024 10:17
@bkontur bkontur changed the base branch from master to bko-bridges-v2-compact-proofs June 7, 2024 10:23
@bkontur bkontur force-pushed the bko-bridges-v2-compact-proofs branch 2 times, most recently from 46f3c36 to 79e3a79 Compare June 13, 2024 10:23
@bkontur bkontur force-pushed the bko-bridges-v2-compact-proofs branch from 79e3a79 to f0f50af Compare June 14, 2024 10:46
@bkontur bkontur force-pushed the bko-bridges-v2 branch 2 times, most recently from fdaca45 to eae0d8c Compare June 28, 2024 13:54
svyatonik and others added 16 commits July 1, 2024 14:12
* add LaneState enum and a field to lanes data

* use OptionQuery in InboundLanes and OutboundLanes

* ensure that lanes are opened when accessing them

* an option to specify opened lanes in genesis config + fixed tests

* added PR reference to TODO

* fix failing benchmarks

* spelling

* get_or_init_data -> data

bump Substrate, Polkadot and Cumulus (#2223)
* change LaneId underlying type from [u8; 4] to H256

* fixed typo

* added some tests

* spelling

* started fixing testnets

* uncommented call size test

* changed RewardsAccountParams encoding + added values separator when computing LaneId

* review suggestions
* some useful stuff like LanesManager

* clippy

* more clippy

* Error::LanesManager

* {in, out}bound_lane -> active_{in, out}bound_lane

* merge two impl blocks in one

* fmt
* Updated version according to Cumulus

* Updated docs
* allow delivery confirmations on closed outbound lanes

* fix benchmarks compilation
* XCM over bridge pallet (won't ever be merged to this repo): initial commit

* try both bridge-id and lane-id

* flush

* flush

* flush

* flush

* first prototype

* started working on xcm-over-bridge tests

* proper open_bridge_works test

* more open_bridge tests

* request_bridge_closure tests

* tests for close_bridge

* report_misbehavior tests

* renaming xcm-over-bridge ad xcm-bridge-hub: we'll have two similar pallets for dynamic lanes/fees support. One will be deployed at the bridge hub (xcm-bridge-hub) and another one (xcm-bridge-hub-router) at sibling/parent chains to send messages to remote network over the bridge hub

* added couple of TODOs

* moved BridgeQueuesState here + TODO for implementing report_bridge_queues_state + fmt

* fixing TODOs

* fixing TODOs

* moved bridge locations to primitives

* added a couple of TODOs

* ref issue from TODO

* clippy and spelling

* fix tests

* another TODOs

* typo

* LaneState -> force_close_bridge

* start_closing_the_bridge -> start_closing_bridge

* removed Closing bridge state, so now we only have the Opened -> optional temporary Closed state -> None

* spelling

* started prototyping suspend+resume on misbehavior

* continue prototyping

* more tests for open_bridge

* more tests for close_bridge

* more tests for report_misbehavior

* started tests for resume_misbehaving_bridges

* implemented tests for resume_misbehaving_bridges

* remove UnsuspendedChannelWithMisbehavingOwner because now, when all bridges are resumed at once + we assume that the inbound XCM channel is suspended immediately it is no longer actual

* added TODO

* add comment re misbehavior reporter if he's associated with the bridge owner

* ignored clippy

* fmt

* use versioned XCM structs in public interfaces and storage + Box XCM locations where possible

* spelling

* removed TODO in favor of paritytech/parity-bridges-common#2257

* LocalChannelManager -> LocalXcmChannelManager

* removed code specific to #2233, because it isn't the only option now

* removemisbehavior mentions

* also temporary (?) remove BridgesByLocalOrigin because the storage format will likely change to be able to resume bridges from the on_iniitalize/on_idle

* AllowedOpenBridgeOrigin -> OpenBridgeOrigin

* - TooManyBridgesForLocalOrigin

* use bridge_destination_relative_location in args

* better impl of strip_low_level_junctions

* get rid of xcm_into_latest

* clippy

* added #![warn(missing_docs)] to new crates
)

* implement ExportXcm and MessageDispatch for pallet-xcm-bridge-hub

* spelling
… pallet-xcm-bridge-hub (#2261)

Xcm bridge hub router v2 (backport to master branch) (#2312)

* copy new pallet (palle-xcm-bridge-hub-router) from dynamic-fees-v1 branch

* added remaining traces of pallet-xcm-bridge-hub-router

* added comment about sharing delivery fee factor between all bridges, opened by this chain

* spelling

* clippy

Implement additional require primitives for dynamic fees directly for pallet-xcm-bridge-hub (#2261)

* added backoff mechanism to inbound bridge queue

* impl backpressure in the XcmBlobHaulerAdapter

* leave TODOs

* BridgeMessageProcessor prototype

* another TODO

* Revert "also temporary (?) remove BridgesByLocalOrigin because the storage format will likely change to be able to resume bridges from the on_iniitalize/on_idle"

This reverts commit bdd7ae11a8942b58c5db6ac6d4e7922aa28cece4.

* prototype for QueuePausedQuery

* implement ExportXcm and MessageDispatch for pallet-xcm-bridge-hub

* spelling

* flush

* small comments to myself

* more backports from dynamic-fees-v1

* use new pallet as exporter and dispatcher in Millau

* use new pallet as exporter and dispatcher in Rialto

* use new pallet as exporter and dispatcher in RialtoParachain

* flush

* fix remaining compilation issues

* warnings + fmt

* fix tests

* LocalXcmChannelManager

* change lane ids

* it works!

* remove bp-xcm-bridge-hub-router and use LocalXcmChannelManager everywhere

* removed commented code

* cleaning up

* cleaning up

* cleaning up

* - separated BridgeId and LaneId
- BridgeId now uses versioned universal locations
- added missing stuff to exporter.rs

* OnMessagesDelivered is back

* start using bp-xcm-bridge-hub as OnMessagesDelivered

* cleaning up

* spelling

* fix stupid issues

* Backport latest relevant dynamic fees changes from v1 to v2 (#2372)

* backport latest relevant dynamic fees changes from v1 to v2

* fix comment

Added remaining unit tests for pallet-xcm-bridge-hub (#2499)

* added backoff mechanism to inbound bridge queue

* impl backpressure in the XcmBlobHaulerAdapter

* leave TODOs

* BridgeMessageProcessor prototype

* another TODO

* Revert "also temporary (?) remove BridgesByLocalOrigin because the storage format will likely change to be able to resume bridges from the on_iniitalize/on_idle"

This reverts commit bdd7ae11a8942b58c5db6ac6d4e7922aa28cece4.

* prototype for QueuePausedQuery

* implement ExportXcm and MessageDispatch for pallet-xcm-bridge-hub

* spelling

* flush

* small comments to myself

* more backports from dynamic-fees-v1

* use new pallet as exporter and dispatcher in Millau

* use new pallet as exporter and dispatcher in Rialto

* use new pallet as exporter and dispatcher in RialtoParachain

* flush

* fix remaining compilation issues

* warnings + fmt

* fix tests

* LocalXcmChannelManager

* change lane ids

* it works!

* remove bp-xcm-bridge-hub-router and use LocalXcmChannelManager everywhere

* removed commented code

* cleaning up

* cleaning up

* cleaning up

* - separated BridgeId and LaneId
- BridgeId now uses versioned universal locations
- added missing stuff to exporter.rs

* OnMessagesDelivered is back

* start using bp-xcm-bridge-hub as OnMessagesDelivered

* cleaning up

* spelling

* fix stupid issues

* added remaining unit tests for pallet-xcm-bridge-hub

fixed benchmarks (#2504)

Remove pallet_xcm_bridge_hub::SuspendedBridges (#2505)

* remove pallet_xcm_bridge_hub::SuspendedBridges

* apply review suggestions
* moved all call-related definitions to primitives

* moving refund extension to relayers pallet

* cleaning up

* fix compilation + spelling

fix compilation

remove priority_calculator.rs

fix extensions
* fiox overflow when computing priority boost (#2587)

* Backport changes from polkadot-sdk (#2588)

* backport #1543

* another backport

* https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/3833103
@paritytech-cicd-pr
Copy link

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: test-linux-stable 3/3
Logs: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6595759

@bkontur bkontur closed this Jul 4, 2024
@bkontur bkontur deleted the bko-bridges-v2 branch July 4, 2024 22:35
github-merge-queue bot pushed a commit that referenced this pull request Jul 12, 2024
…cations (#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <[email protected]>
Co-authored-by: Serban Iorga <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: command-bot <>
TomaszWaszczyk pushed a commit to TomaszWaszczyk/polkadot-sdk that referenced this pull request Jul 13, 2024
…cations (paritytech#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](paritytech#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
paritytech#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <[email protected]>
Co-authored-by: Serban Iorga <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: command-bot <>
TarekkMA pushed a commit to moonbeam-foundation/polkadot-sdk that referenced this pull request Aug 2, 2024
Contains mainly changes/nits/refactors related to the relayer code
(`client-substrate` and `lib-substrate-relay`) migrated from the Bridges
V2 [branch](paritytech#4427).

Relates to:
paritytech/parity-bridges-common#2976
Companion: paritytech/parity-bridges-common#2988


## TODO
- [x] fix comments

## Questions
- [x] Do we need more testing for client V2 stuff? If so, how/what is
the ultimate test? @svyatonik
- [x] check
[comment](paritytech#4494 (comment))
for more testing

---------

Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: Serban Iorga <[email protected]>
TarekkMA pushed a commit to moonbeam-foundation/polkadot-sdk that referenced this pull request Aug 2, 2024
…cations (paritytech#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](paritytech#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
paritytech#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <[email protected]>
Co-authored-by: Serban Iorga <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: command-bot <>
github-merge-queue bot pushed a commit that referenced this pull request Sep 2, 2024
Relates to:
paritytech/parity-bridges-common#2451
Closes: paritytech/parity-bridges-common#2500

## Summary

Now, the bridging pallet supports only static lanes, which means lanes
that are hard-coded in the runtime files. This PR fixes that and adds
support for dynamic, also known as permissionless, lanes. This means
that allowed origins (relay chain, sibling parachains) can open and
close bridges (through BridgeHubs) with another bridged (substrate-like)
consensus using just `xcm::Transact` and `OriginKind::Xcm`.

_This PR is based on the migrated code from the Bridges V2
[branch](#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2)._

## Explanation

Please read
[bridges/modules/xcm-bridge-hub/src/lib.rs](https://github.com/paritytech/polkadot-sdk/blob/149b0ac2ce43fba197988f2642032fa24dd8289a/bridges/modules/xcm-bridge-hub/src/lib.rs#L17-L136)
to understand how managing bridges works. The basic concepts around
`BridgeId` and `LaneId` are also explained there.


## TODO

- [x] search and fix for comment: `// TODO:(bridges-v2) - most of that
stuff was introduced with free header execution:
https://github.com/paritytech/polkadot-sdk/pull/4102` - more info in the
comment
[bellow](#4427 (comment))
- [x] TODO: there's only one impl of `EnsureOrigin<Success = Location>`

## TODO - not blocking review

**benchmarking:**
- [x] regenerate all relevant weights for BH/AH runtimes
- [ ] regenerate default weights for bridging pallets e.g.
`modules/messages/src/weights.rs`
- [ ] add benchmarks for `xcm-bridge-hub` pallet
#5550

**testing:**
- [ ] add xcm-emulator tests for Rococo/Penpal to Westend/Penpal with
full opening channel and sending/receiving `xcm::Transact`

**migrations:**
- [x] add migrations for BridgeHubRococo/Westend
paritytech/parity-bridges-common#2794 (to be
reusable for P/K bridge)
  - [x] check also storage migration, if needed for pallets
  - [ ] migration for XCM type (optional)
  - [x] migration for static lanes to the dynamic (reuse for fellows)

**investigation:**
- [ ] revisit
paritytech/parity-bridges-common#2380
- [ ] check congestion around `LocalXcmChannelManager` and
`OutboundLanesCongestedSignals` impls -
#5551
  - to be reusable for polkadot-fellows
- return `report_bridge_status` was remove, so we need to `XcmpQueue`
alternative?

---------

Signed-off-by: Branislav Kontur <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <[email protected]>
x3c41a pushed a commit that referenced this pull request Sep 4, 2024
Relates to:
paritytech/parity-bridges-common#2451
Closes: paritytech/parity-bridges-common#2500

## Summary

Now, the bridging pallet supports only static lanes, which means lanes
that are hard-coded in the runtime files. This PR fixes that and adds
support for dynamic, also known as permissionless, lanes. This means
that allowed origins (relay chain, sibling parachains) can open and
close bridges (through BridgeHubs) with another bridged (substrate-like)
consensus using just `xcm::Transact` and `OriginKind::Xcm`.

_This PR is based on the migrated code from the Bridges V2
[branch](#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2)._

## Explanation

Please read
[bridges/modules/xcm-bridge-hub/src/lib.rs](https://github.com/paritytech/polkadot-sdk/blob/149b0ac2ce43fba197988f2642032fa24dd8289a/bridges/modules/xcm-bridge-hub/src/lib.rs#L17-L136)
to understand how managing bridges works. The basic concepts around
`BridgeId` and `LaneId` are also explained there.

## TODO

- [x] search and fix for comment: `// TODO:(bridges-v2) - most of that
stuff was introduced with free header execution:
https://github.com/paritytech/polkadot-sdk/pull/4102` - more info in the
comment
[bellow](#4427 (comment))
- [x] TODO: there's only one impl of `EnsureOrigin<Success = Location>`

## TODO - not blocking review

**benchmarking:**
- [x] regenerate all relevant weights for BH/AH runtimes
- [ ] regenerate default weights for bridging pallets e.g.
`modules/messages/src/weights.rs`
- [ ] add benchmarks for `xcm-bridge-hub` pallet
#5550

**testing:**
- [ ] add xcm-emulator tests for Rococo/Penpal to Westend/Penpal with
full opening channel and sending/receiving `xcm::Transact`

**migrations:**
- [x] add migrations for BridgeHubRococo/Westend
paritytech/parity-bridges-common#2794 (to be
reusable for P/K bridge)
  - [x] check also storage migration, if needed for pallets
  - [ ] migration for XCM type (optional)
  - [x] migration for static lanes to the dynamic (reuse for fellows)

**investigation:**
- [ ] revisit
paritytech/parity-bridges-common#2380
- [ ] check congestion around `LocalXcmChannelManager` and
`OutboundLanesCongestedSignals` impls -
#5551
  - to be reusable for polkadot-fellows
- return `report_bridge_status` was remove, so we need to `XcmpQueue`
alternative?

---------

Signed-off-by: Branislav Kontur <[email protected]>
Co-authored-by: Svyatoslav Nikolsky <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Francisco Aguirre <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Split and migrate origin/bridges-v2 branch
3 participants