feat: Adds Instrumented Object Store Registry to datafusion-cli #17953
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BlakeOrth
force-pushed
the
feature/cli_instrumented_registry
branch
2 times, most recently
from
9494588 to
65c980c
Compare
- Adds a new Object Store Registry wrapper to datafusion-cli to support interacting with instrumented object store instances - Adds basic tests for the new registry wrapper
BlakeOrth
force-pushed
the
feature/cli_instrumented_registry
branch
from
65c980c to
dc60619
Compare
|
I will review this later todya -- thank you @BlakeOrth |
I don't think this is necessary. Linking to the overall ticket is just great |
alamb
approved these changes
Oct 8, 2025
There was a problem hiding this comment.
looks good to me -- thank you @BlakeOrth
I'll plan to merge this later today unless others would like a chance to comment
| use object_store::ObjectStore; | ||
| use url::Url; | ||
|
|
||
| #[derive(Debug)] |
There was a problem hiding this comment.
I think it would be good to add some comments here explaining what this structure is good for
Something like
Suggested change
| #[derive(Debug)] | |
| /// Returns wrapped [`ObjectStore`] instances that record | |
| /// network requests for reporting. | |
| #[derive(Debug)] |
|
Let's keep development going 🚀 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
This does not fully close, but is an incremental building block component to:
datafusion-cli] Add a way to see what object store requests are made #17207Functionally this code currently does nothing, but it's an important component that will allow access to future instrumented object stores. The full context of how this code is likely to progress can be seen in the POC for this effort:
(note: If we need an individual issue to close for process reasons I am happy to make one and associate it with #17207 as part of that larger effort)
Rationale for this change
The full implementation for #17207 is likely too large for a single PR, so to better account for review bandwidth we would like to break the implementation into smaller pieces for easier review. This happens to be about the smallest and least intrusive atomic unit I could think of based on the POC work.
What changes are included in this PR?
instrumentedsub-module to theobject_storagemodule to house all future instrumented object store codeAre these changes tested?
Yes, new tests have been introduced and all existing tests are passing
Are there any user-facing changes?
No
cc: @alamb