-
Notifications
You must be signed in to change notification settings - Fork 62
feat: Bigtable Authorized Views - Allow checkAndMutate and ReadModifyWriteRow calls on Authorized Views #1464
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
danieljbruce
merged 43 commits into
bigtable-authorized-views
from
bigtable-authorized-views-move-row-methods
Sep 23, 2024
Merged
feat: Bigtable Authorized Views - Allow checkAndMutate and ReadModifyWriteRow calls on Authorized Views #1464
danieljbruce
merged 43 commits into
bigtable-authorized-views
from
bigtable-authorized-views-move-row-methods
Sep 23, 2024
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…github.com/googleapis/nodejs-bigtable into bigtable-authorized-views-initial-refactor
and readWriteModifyRow
…apis/nodejs-bigtable into bigtable-authorized-views-move-row-methods # Conflicts: # src/row.ts # src/table.ts # src/tabular-api-service.ts # src/utils/table.ts # test/table.ts
bhshkh
approved these changes
Sep 5, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well organized code. Thanks for adding the detailed description regarding the changes in the PR!
cb1ebd4
into
bigtable-authorized-views
13 of 17 checks passed
danieljbruce
added a commit
that referenced
this pull request
Nov 7, 2024
* Create the new tabular api service file * feat: Bigtable authorized views - move the code over to the TabularApiSurface class (#1463) * Move the constructor over to TabularApiService * Move sampleRowKeys over * Move sampleRowKeys functions over and use promisify * Adjust the proxyquire to work with TabularAPIserv * Move all the ReadRows functionality over * Solve the issue with the is dependency * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Add header for new class * Only include Table mocks that are necessary * Remove TODO * Rename TabularApiService to TabularApiSurface * Change all imports to tabular-api-surface * surface. not service --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * feat: Bigtable Authorized Views - Allow checkAndMutate and ReadModifyWriteRow calls on Authorized Views (#1464) * Move the constructor over to TabularApiService * Move sampleRowKeys over * Move sampleRowKeys functions over and use promisify * Adjust the proxyquire to work with TabularAPIserv * Move all the ReadRows functionality over * Solve the issue with the is dependency * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Add header for new class * Trying out the FilterInformation class * Add a DataUtils module for the shared row function * Outsource functionality of filter to helper * debugging * Adjust proxyquire to include mocks for moved fn * Remove imports * Outsource code to a createRulesUtil function * Change proxyquire to include createRulesUtil * Move increment over to the utils folder * Move the functions into a static class for mocks * Remove mockCreateRules and shorten mock * Stub out FakeRowDataUtil * Remove unused dependencies * Add the functions to work with checkAndMutate and readWriteModifyRow * Fix regressions from the merge * Add documentation for the class * Document the new methods of table * Change the interface of the rowUtils * Pull the generateProperties code out avoid duplica * Move duplicate code out into a getProperties object * Remove console.log * Add method for creating views * Object for making grpc calls for authorized views * Remove import * Update the documentation for the Table * More specific type * Add documentation for each of the functions * Remove TODO * Add headers * Add documentation for new class * Remove imports * Reintroduce before * Remove unused import --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * feat: Create a view on the instance, not on the table (#1476) * Move view creation from the table class to instanc * feat: Create a view on the instance, not on the table * Add table name parameter to the documentation * Remove Table * feat: Create the unit tests for Authorized Views ensuring that requests are consistent (#1501) * Create a test for authorized views * Set the ReadRows options * Add authorized view test for createreadstream * Pass authorizedViewName along for createReadStream * Add the unit test for getRows * Add the unit test for getRows * Take out the test mock for the request function * Mutate rows request - test setup * Made corrections and src code changes so unit test passes * Fix the readrows test to end the stream Also add a test for insert and make the test easier to debug. * Finish the sampleRowKeys tests * Refactor part of the test for getting request opts * Authorized views for readRows test should be one assert * Add the mock function for request * Make mutate rows use the mockRequest function * Setup the sampleRowKeys setup fn to use mockReq * Add comments to the mocking functions * Fix comments. Move functions to right place * Setup the readModifyWriteRow tests * For createRules test use new mockCallbackRequest You need to call the callback in order to end the operation to createRules. * Use the new mockCallbackRequest function * Get rid of the unused mock function * Create a test file for createRules not got * increment * Add the test for increment * Change the response value for authorized view * Add the console logs and change return type * Flesh out response so that view call works * Exclude appropriate properties using promisify. * Removed console logs * Finished the filter test * Add the view call for filter * Remove only * Change to more readable values * Remove only * Add header to the test * For tests, expect view to be excluded from promis * run linter * Add a few @params and @returns * Add a comment about making requests for auth views * Added more comments - auth views vs table Also added a comment for calling increment * feat: Bigtable authorized views integration tests (#1504) * Create a test for authorized views * Set the ReadRows options * Add authorized view test for createreadstream * Pass authorizedViewName along for createReadStream * Add the unit test for getRows * Add the unit test for getRows * Take out the test mock for the request function * Mutate rows request - test setup * Made corrections and src code changes so unit test passes * Fix the readrows test to end the stream Also add a test for insert and make the test easier to debug. * Finish the sampleRowKeys tests * Refactor part of the test for getting request opts * Authorized views for readRows test should be one assert * Add the mock function for request * Make mutate rows use the mockRequest function * Setup the sampleRowKeys setup fn to use mockReq * Add comments to the mocking functions * Fix comments. Move functions to right place * Setup the readModifyWriteRow tests * For createRules test use new mockCallbackRequest You need to call the callback in order to end the operation to createRules. * Use the new mockCallbackRequest function * Get rid of the unused mock function * Create a test file for createRules not got * increment * Add the test for increment * Change the response value for authorized view * Add the console logs and change return type * Flesh out response so that view call works * Exclude appropriate properties using promisify. * Removed console logs * Finished the filter test * Add the view call for filter * Remove only * Change to more readable values * Remove only * Add header to the test * For tests, expect view to be excluded from promis * run linter * Create the before hook for auth views * Note to self on auth views - TODO * Add an insert statement * Make a test for getRows * Add the test for the getRows function. * Finish ‘should fail when writing to a row not in v * Add another mutate test, preserve table after test Add after hook to make sure table values stay the same. Add test for modifying from different column * Remove TODO * refactor the error message in the test * Add mutate, insert and sampleRowKeys tests * Fix sampleRowKeys test to require a fix upstream * Create samplerowkeys and createreadstream tests * Surround all tests in a try block This allows for better error reporting * Fixed the column filter so it works * Reduce verbosity in tests In many cases, 3 lines can be reduced to one variable. This is crucial for making the integration tests shorter. * Replace verbose data with data variable * Add an insert for multiple rows * Fix the check and mutate test to ignore corrupt d * Add the readModifyWriteRow tests * Add awaits * wrap columnFamily and columnIdInView in brackets * Eliminate the TODO * Remove only * Add key back to filter config option --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Kevin Kim <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
api: bigtable
Issues related to the googleapis/nodejs-bigtable API.
size: l
Pull request size is large.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This PR introduces the architecture required to service
checkAndMutate
andreadModifyWriteRow
requests for Authorized views.Background:
this other PR has already been merged towards the effort of supporting data API calls for Authorized Views.
This current PR demonstrates the shape of the architecture that is required for Authorized View requests that correspond to the
checkAndMutate
andreadModifyWriteRow
calls currently serviced by calling theRow
class methods. There may be clerical mistakes, but those will be flushed out by writing unit tests and integration tests.Changes:
Row Data Utils:
increment
,filter
andcreateRules
in theRow
class is moved into new methods inrow-data-utils
. To support authorized view calls this has to be done. You see,increment
,filter
andcreateRules
on the Row class make use of a privatedata
property in that Row class. For this reason, to supportincrement
,filter
andcreateRules
for authorized views we cannot simply createincrement
,filter
andcreateRules
methods on theTabularApiSurface
class because thendata
would have to be passed into them and we do not wantdata
to be part of the method signature. Instead, we moveincrement
,filter
andcreateRules
functionality fromRow
to methods in a sharedrow-data-utils
class. These methods in therow-data-utils
class will now make requests for both tables and authorized views.Authorized View:
increment
,filter
andcreateRules
methods are added to the newAuthorizedView
class for makingcheckAndMutate
andreadModifyWriteRow
calls for an authorized view. These methods should not be exposed onTabularApiSurface
because then they will be exposed onTable
, but this functionality is already available for tables on theincrement
,filter
andcreateRules
methods onRow
.createRules
,filter
andincrement
methods on theAuthorizedView
class will have the same signature as the methods with the same name on theRow
class except they must also be provided with a row id that the request will be made for.AuthorizedView
class will store row data for authorized views just like theRow
class has adata
property that stores data for that row. ThisrowData
variable inAuthorizedView
stores data for each row it makes a request on behalf of.Row:
RowDataUtils
and replaced with one line of code that delegates the call.Table:
TabularApiSurface
constructor is protected.TabularApiSurface:
test/Row.ts
Next Steps:
increment
,createRules
andfilter
methods now available onAuthorizedView
. Make sure that these methods pass the proper parameters along and that they target an authorized view.TabularApiSurface
class.