Skip to content

Conversation

l4mby
Copy link
Contributor

@l4mby l4mby commented Apr 9, 2025

This pr adds a consumer update listener in the consumer. In this way, the end user can customize the behaviour of a single active consumer when it becomes active. An example is also added to show how a single active consumer can resume the consuming of messages from a specific offset, upon activation.

@l4mby l4mby linked an issue Apr 9, 2025 that may be closed by this pull request
@gpad gpad requested a review from Copilot April 30, 2025 08:02
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for single active consumer offset tracking by incorporating a consumer update listener in the consumer, client, and connection modules, and by updating end-to-end tests to cover the new behavior.

  • Introduces a consumerUpdateListener and singleActive flag in StreamConsumer.
  • Updates the client logic to handle consumer update queries and offset resumption.
  • Adds an end-to-end test that verifies the offset tracking mechanism when switching the active consumer.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
test/e2e/declare_consumer.test.ts Added a new test case for validating offset tracking with two single active consumers.
src/consumer.ts Integrated consumerUpdateListener and singleActive flag into consumer initialization.
src/connection.ts Introduced a debug console.log in storeOffset (likely for debugging purposes).
src/client.ts Updated consumer declaration and added getConsumerOrServerSavedOffset logic with debug logs.
Comments suppressed due to low confidence (1)

test/e2e/declare_consumer.test.ts:220

  • Remove or replace 'it.only' to ensure that the full test suite runs during CI, unless this is an intentional temporary measure.
it.only(

@l4mby l4mby force-pushed the feat/239-feature-request-automatic-offset-refresh-on-leadership-claim-in-sac-mode branch from 0891281 to 5728d8b Compare April 30, 2025 14:18
@l4mby l4mby marked this pull request as ready for review April 30, 2025 14:19
@gpad gpad requested a review from Copilot April 30, 2025 14:40
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a consumer update listener for single active consumers so that upon activation the consumer can resume message consumption from a previously saved offset. Key changes include:

  • Adding a new consumer update listener callback in both the test and production code.
  • Updating the consumer and client code to support offset updates and single active consumer behavior.
  • Providing an example demonstrating how to resume consumption from a designated offset.

Reviewed Changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

File Description
test/e2e/declare_consumer.test.ts Added a new end-to-end test case for single active consumer offset tracking.
src/consumer.ts Introduced ConsumerUpdateListener type and implementation for updating offset.
src/client.ts Added logic to call consumerUpdateListener and update consumer offset accordingly.
example/src/single_active_consumer_update_example.js Provided a usage example for the new consumer update behavior.
Files not reviewed (1)
  • example/package-lock.json: Language not supported
Comments suppressed due to low confidence (1)

test/e2e/declare_consumer.test.ts:246

  • [nitpick] Using fixed wait times in tests may lead to flaky test behavior; consider replacing fixed delays with more deterministic synchronization mechanisms (such as polling or event-based triggers) to ensure consumer state readiness.
await wait(500)

tarzacodes
tarzacodes previously approved these changes Apr 30, 2025
@l4mby l4mby requested a review from tarzacodes May 5, 2025 12:57
@l4mby l4mby merged commit 6ef1c96 into main May 5, 2025
4 of 10 checks passed
@l4mby l4mby deleted the feat/239-feature-request-automatic-offset-refresh-on-leadership-claim-in-sac-mode branch May 5, 2025 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Automatic Offset Refresh on Leadership Claim in SAC Mode

2 participants