GH-26818: [C++][Python] Preserve order when writing dataset multi-threaded

[EnricoMi]

Rationale for this change

The order of rows in a dataset might be important for users and should be preserved when writing to a filesystem. With multi-threaded write, the order is currently not guaranteed,

What changes are included in this PR?

Preserving the dataset order of rows requires the SourceNode to use ImplicitOrdering (this gives exec batches an index), and the ConsumingSinkNode to sequence exec batches (preserve order of batches by their index).

User-facing changes:

• Add option preserve_order to FileSystemDatasetWriteOptions

Dev-facing changes:

• Add option ordering to SourceNodeOptions
• Add option implicit_ordering to ScanNodeOptions

Default behaviour is current behaviour.

Are these changes tested?

Unit tests have been added,

Are there any user-facing changes?

Users can set FileSystemDatasetWriteOptions.preserve_order = true (C++) / arrow.dataset.write_dataset(..., preserve_order=True) (Python).

• GitHub Issue: [C++][Dataset] Preserve order when writing dataset #26818

[github-actions]

⚠️ GitHub issue #26818 has been automatically assigned in GitHub to PR creator.