Skip to content

Conversation

@naveenpaul1
Copy link
Contributor

@naveenpaul1 naveenpaul1 commented Sep 26, 2025

Describe the Problem

Noobaa core throwing lots of Region is missing error and its filling log file fast.

write_usage ERROR Error: Region is missing
block_store_services.block_store_s3:: _get_blocks_usage: Error: Region is missing

Explain the Changes

  1. Update the block store to fetch sdkv3
  2. update createSTSS3SDKv3Client code

Issues: Fixed #xxx / Gap #xxx

Testing Instructions:

  1. Create AWS bucket
  2. Create Azure bucket
  3. Add replication policy to AWS bucket
nb api bucket_api put_bucket_replication '{"name":"my-bucket-claim1-9b6265f2-2d96-4525-bfdf-adbda4995d77","replication_policy":{"rules":[{"rule_id":"rule-1","destination_bucket":"azure-bucket-8e32f9ee-c04e-4a8c-80dc-9700aa49cf5e","filter":{"prefix":"log"},"sync_deletions":true,"sync_versions":false}]}}'
  1. Check the replication happening or not
  • Doc added/updated
  • Tests added

Summary by CodeRabbit

  • Refactor

    • S3 client creation unified behind a common wrapper to improve compatibility across AWS and S3‑compatible endpoints and simplify connection handling.
  • Bug Fixes

    • Signed URL generation now respects the requested or configured region instead of a hard-coded value.
    • Improved handling and connectivity for non‑AWS/S3‑compatible endpoints, including more reliable request handling for certain endpoint types.

@coderabbitai
Copy link

coderabbitai bot commented Sep 26, 2025

Walkthrough

Replace direct AWS SDK v3 S3 client instantiation with calls to noobaa_s3_client.get_s3_client_v3_params(...) in S3 code paths; adjust region selection and endpoint/requestHandler choices (including unsecured HTTP agent for some non-AWS endpoints); remove direct @aws-sdk/client-s3 imports.

Changes

Cohort / File(s) Summary
S3 client migration (block store)
src/agent/block_store_services/block_store_s3.js
Replace direct new S3(...) usage with noobaa_s3_client.get_s3_client_v3_params(...); preserve region and pathStyle logic; switch endpoint handling to NooBaa client; use unsecured HTTP agent for certain non-AWS endpoints; update imports accordingly.
S3 client migration (cloud utils)
src/util/cloud_utils.js
createSTSS3SDKv3Client now delegates to noobaa_s3_client.get_s3_client_v3_params(...); get_signed_url uses `params.region

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Caller
  participant CloudUtils as cloud_utils.createSTSS3SDKv3Client
  participant NooBaa as noobaa_s3_client
  participant S3 as S3 API

  Caller->>CloudUtils: request STS-backed S3 client params
  CloudUtils->>NooBaa: get_s3_client_v3_params(creds, region, endpoint, requestHandler)
  NooBaa-->>CloudUtils: SDKv3-compatible client params
  CloudUtils-->>Caller: return client params
  Caller->>S3: perform S3 operations using returned params
  Note over CloudUtils,NooBaa: Replaces direct `new S3(...)` construction
Loading
sequenceDiagram
  autonumber
  participant BlockStore as block_store_s3
  participant NooBaa as noobaa_s3_client
  participant S3 as S3 API

  BlockStore->>NooBaa: get_s3_client_v3_params(conn details, region, endpoint, reqHandler)
  NooBaa-->>BlockStore: client params (with region/requestHandler)
  BlockStore->>S3: execute S3 calls using returned params
  Note over BlockStore: non-AWS endpoints select unsecured HTTP agent via requestHandler
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • dannyzaken
  • jackyalbo
  • liranmauda
  • tangledbytes

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title identifies a valid change by referencing the SDKv3 client region parameter fix but fails to capture the main objective of resolving Azure bucket replication. It accurately highlights the addition of the region handling but omits the broader replication fix. It therefore only partially summarizes the most important change in the pull request. A teammate scanning the history may not immediately understand the replication context.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@liranmauda liranmauda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not seem (per this fix) as a replication issue.
It seems like a general issue that will effect anything that use createSTSS3SDKv3Client, the replication logic is not relevent here.
Update the PR subject and description

@naveenpaul1 naveenpaul1 changed the title SDK | fix replication issue SDK | SDKv3Client fix for replication issue Oct 3, 2025
@naveenpaul1 naveenpaul1 requested a review from liranmauda October 3, 2025 09:03
@naveenpaul1 naveenpaul1 marked this pull request as ready for review October 3, 2025 09:04
@naveenpaul1 naveenpaul1 requested a review from jackyalbo October 3, 2025 09:04
@naveenpaul1 naveenpaul1 force-pushed the sdk_v3_replication branch 2 times, most recently from afe9286 to 208193a Compare October 6, 2025 06:34
Signed-off-by: Naveen Paul <[email protected]>
@naveenpaul1 naveenpaul1 changed the title SDK | SDKv3Client fix for replication issue SDK | SDKv3Client region error Oct 8, 2025
@naveenpaul1 naveenpaul1 changed the title SDK | SDKv3Client region error SDK | SDKv3Client region missing error Oct 8, 2025
@naveenpaul1 naveenpaul1 dismissed liranmauda’s stale review October 8, 2025 11:49

Review changes are added to code
dismissing the review to merge it.

@naveenpaul1 naveenpaul1 merged commit 4986d5d into noobaa:master Oct 8, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants