Skip to content

Conversation

frankmcsherry
Copy link
Member

This PR re-imagines how we provide access to inputs to users, through an InputSession that can (and must) enumerate all (time, data) pairs. This encourages folks to use inputs idiomatically, rather than potentially draining inputs or potentially not (with "not" being a logical error because the operator may not be scheduled again).

The InputSession has one method that groups all (time, data) by time and provides data as an iterator over mutable container references. The migration path is pretty standard, but occasionally fiddly. Each while let Some((time, data)) = input.next() loop turns in to input.for_each(|time, data| { where data is one additional layer of iterator, and needs to be flat_mapd or the like. This method consumes the input session, which addresses the #[must_use] attribute.

One alternative that could make the migration that much easier is to leave for_each as the container-at-a-time iteration that it is right now, and add a by_time or for_time or something that does the grouping. I think we want to encourage the latter, and making the migration lightly disruptive in order to do so might be fine.

@frankmcsherry frankmcsherry marked this pull request as ready for review September 20, 2025 12:09
@frankmcsherry
Copy link
Member Author

One candidate change is to have for_each(|time, data| ...) go over pairs of capability and container, but be "deprecated" or whatever it takes to whine at users, and a new by_time perform the grouping we do here. Mostly thinking it would be nice to ensure a smooth path for existing users, and I think this old for_each ensures that, even though the newer one is what we'd like folks to use.

@frankmcsherry
Copy link
Member Author

Merging to move forward, but lmk in review if there are things to fix up, or if we should revert for some reason! :D

@frankmcsherry frankmcsherry merged commit 729f75b into TimelyDataflow:master Sep 20, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant