HADOOP-19286: S3A: Support cross region access when S3 region/endpoint is set #7093
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
@steveloughran - branch-3.4 backport |
steveloughran
approved these changes
Oct 2, 2024
Contributor
steveloughran
left a comment
steveloughran
left a comment
There was a problem hiding this comment.
+1 pending a successful Yetus build.
|
🎊 +1 overall
This message was automatically generated. |
Contributor
|
Just noticed I got the commit message wrong on the trunk PR; I was trying to summarise it missed out the fs. In front of the property. I could revert and we could put a better message in if you like? |
steveloughran
requested changes
Oct 3, 2024
Contributor
steveloughran
left a comment
steveloughran
left a comment
There was a problem hiding this comment.
not merging; reopened trunk jira as the tests fail for me. once a followup patch is merged in, a branch-3.4 pr can be done with both changes in
Contributor
Author
|
@steveloughran - Created follow up PR: #7098 |
Contributor
|
just merge the addendum in here for a single backport patch. |
shameersss1
force-pushed
the
HADOOP-19286-branch-3.4
branch
from
481d97b to
c398350
Compare
Contributor
Author
|
@steveloughran - Raised a single commit for both the changes. |
|
🎊 +1 overall
This message was automatically generated. |
Contributor
|
where did you test this? |
Contributor
Author
Tested this in us-east-1 |
steveloughran
approved these changes
Oct 8, 2024
Contributor
|
done. I'm not going to put into 3.4.1 as I'm only putting critical fixes or things which need to go in their first. Then we can start to focus on 3.4.2 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently when S3 region nor endpoint is set, the default region is set to us-east-2 with cross region access enabled. But when region or endpoint is set, cross region access is not enabled.
The proposal here is to carves out cross region access as a separate config and enable/disable it irrespective of region/endpoint is set. This gives more flexibility to the user.
S3 cross region access can be enabled/disabled via config fs.s3a.cross.region.access.enabled which is set to true by default.
Tested with us-east-1