Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partitioning strategies for patch data #335

Merged
merged 13 commits into from
Jun 20, 2024

Conversation

hannesbrandt
Copy link
Contributor

This PR adds a new interface for partitioning patch data in different modes, with one of them being already implemented.

We introduce a new workflow for partitioning patch data:

  1. The patch data is packed into send buffers by calling fclaw2d_domain_iterate_pack. The function receives a user-defined fclaw2d_pack_callback_t, which packs the data of a patch into a provided storage location. Then the function starts sending the packed data according to the new partition boundaries.
  2. While waiting for the communication to complete, the data of patches which stay local can be transferred between the old and the new domain with fclaw2d_domain_iterate_transfer. The function uses a fclaw2d_transfer_callback_t, which may be a simple copy or reassignment of the patch data. fclaw2d_domain_iterate_transfer may be called before iterate_pack or after iterate_unpack instead.
  3. Every call to fclaw2d_domain_iterate_pack has to be succeeded by a call to fclaw2d_domain_iterate_unpack. This function unpacks the repartitioned patch data it received by a user-defined fclaw2d_unpack_callback_t.
  4. Finally, a call to fclaw2d_domain_partition_free frees all partition-specific context data.

The workflow has to be called after fclaw2d_domain_partition, which repartitions the patches without the patch data and updates the partition boundaries as required.
For now we keep the old workflow of using assign_before_partition, retrieve_after_partition and iterate_partitioned. It may be useful for writing a comparison paper on different partitioning modes.

The implementation currently comes with a single mode that packs all patches into the send buffer, even if they are not sent after all. For future additional modes, we propose to add two partitioning options, skip_local and skip_refine, which can be combined arbitrarily:

  • With skip_local enabled, we pack data only for the patches that change process during repartitioning.
  • With skip_refine enabled, we change the packing behavior for the last level of newly refined patches. To achieve this, we save the local patch indices of newly refined patches during the last call to {domain,wrap}_adapt. Based on these indices, only the first child of a family of newly refined patches calls the fclaw2d_pack_callback_t. Subsequently, the fclaw2d_unpack_callback_t is called for all children with the packed data from the first child.

Does this interface with the corresponding options seem convenient to you? We would appreciate your feedback/ideas on this topic!

@scottaiton
Copy link
Member

The interface and options look good! I don't have any requests for changes.

@hannesbrandt
Copy link
Contributor Author

Great, then I will turn this into a normal PR, mainly for the interface.
The skip_local and skip_refine options will follow in future PRs.

@hannesbrandt hannesbrandt marked this pull request as ready for review June 20, 2024 09:26
@scottaiton scottaiton merged commit 0140f86 into ForestClaw:develop-3d Jun 20, 2024
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants