Skip to content

fix: throw away excess data in order to avoid delivering duplicate data #1453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Jul 22, 2024
Merged

fix: throw away excess data in order to avoid delivering duplicate data #1453

merged 8 commits into from
Jul 22, 2024

Conversation

danieljbruce
Copy link
Contributor

@danieljbruce danieljbruce commented Jul 19, 2024 •
edited
Loading

Changes:

  • Throw away excess data if its value doesn't exceed the row key
  • Fix the mock in the Table class so that the row function actually returns a more realistic value

@product-auto-label product-auto-label bot added size: s Pull request size is small. api: bigtable Issues related to the googleapis/nodejs-bigtable API. labels Jul 19, 2024
@danieljbruce danieljbruce changed the title 353765540 try ignoring duplicated data 2 353765540 try ignoring data 2 Jul 19, 2024
@danieljbruce danieljbruce changed the title 353765540 try ignoring data 2 fix: throw away excess data Jul 19, 2024
@danieljbruce danieljbruce marked this pull request as ready for review July 19, 2024 14:33
@danieljbruce danieljbruce requested review from a team as code owners July 19, 2024 14:33
@danieljbruce danieljbruce added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 19, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 19, 2024
igorbernstein2
igorbernstein2 reviewed Jul 19, 2024
View reviewed changes
src/table.ts Outdated
@@ -759,7 +759,7 @@ Please use the format 'prezzy' or '${instance.name}/tables/prezzy'.`);
readableHighWaterMark: 0, // We need to disable readside buffering to allow for acceptable behavior when the end user cancels the stream early.
writableHighWaterMark: 0, // We need to disable writeside buffering because in nodejs 14 the call to _transform happens after write buffering. This creates problems for tracking the last seen row key.
transform(row, _encoding, callback) {
if (userCanceled) {
if (userCanceled || TableUtils.lessThanOrEqualTo(row.id, lastRowKey)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please split these into separate if statements and add a comment explaining why this is necessary

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@danieljbruce danieljbruce changed the title fix: throw away excess data fix: throw away excess data in order to avoid delivering duplicate data Jul 19, 2024
@danieljbruce danieljbruce added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 19, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 19, 2024
@danieljbruce danieljbruce added the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 19, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 19, 2024
@danieljbruce danieljbruce merged commit 069239d into googleapis:main Jul 22, 2024
15 of 19 checks passed
@release-please release-please bot mentioned this pull request Jul 22, 2024
Merged
@danieljbruce danieljbruce mentioned this pull request Jul 29, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@igorbernstein2 igorbernstein2 igorbernstein2 left review comments

@leahecole leahecole leahecole approved these changes

Assignees
No one assigned
Labels
api: bigtable Issues related to the googleapis/nodejs-bigtable API. size: s Pull request size is small.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@danieljbruce @leahecole @igorbernstein2 @yoshi-kokoro