Skip to content

Conversation

antiguru
Copy link
Member

Currently, the partitioning logic has two separate branches for
power-of-two peers using bit masking and mod for all other configurations.
The branches duplicate the logic to distribute the data to the pushers.
This change moves the common functionality into a separate function.

We force inlining of the partition function to avoid a performance penalty.

Signed-off-by: Moritz Hoffmann [email protected]

Currently, the partitioning logic has two separate branches for
power-of-two peers using bit masking and mod for all other configurations.
The branches duplicate the logic to distribute the data to the pushers.
This change moves the common functionality into a separate function.

We force inlining of the partition function to avoid a performance penalty.

Signed-off-by: Moritz Hoffmann <[email protected]>

/// Partition data according to an index function
#[inline(always)]
fn partition<F: Fn(u64) -> usize>(&mut self, time: &T, data: &mut Vec<D>, func: F) {
Copy link
Member Author

@antiguru antiguru Dec 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename to push_partitioned and fix the doc comment.

Rename, and redocument.
@frankmcsherry
Copy link
Member

Looks great with naming nit fixed; thanks very much!

@frankmcsherry frankmcsherry merged commit dd60878 into TimelyDataflow:master Dec 8, 2021
@antiguru antiguru deleted the exchange_deduplicate branch December 8, 2021 15:50
@github-actions github-actions bot mentioned this pull request Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants