Skip to content

feat: Bigtable Authorized Views - Allow checkAndMutate and ReadModifyWriteRow calls on Authorized Views #1464

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

danieljbruce
Copy link
Contributor

@danieljbruce danieljbruce commented Aug 28, 2024

Summary:

This PR introduces the architecture required to service checkAndMutate and readModifyWriteRow requests for Authorized views.

Background:

this other PR has already been merged towards the effort of supporting data API calls for Authorized Views.

This current PR demonstrates the shape of the architecture that is required for Authorized View requests that correspond to the checkAndMutate and readModifyWriteRow calls currently serviced by calling the Row class methods. There may be clerical mistakes, but those will be flushed out by writing unit tests and integration tests.

Bigtable - RMWR and CAM support drawio

Changes:

Row Data Utils:

  • Code from increment, filter and createRules in the Row class is moved into new methods in row-data-utils. To support authorized view calls this has to be done. You see, increment, filter and createRules on the Row class make use of a private data property in that Row class. For this reason, to support increment, filter and createRules for authorized views we cannot simply create increment, filter and createRules methods on the TabularApiSurface class because then data would have to be passed into them and we do not want data to be part of the method signature. Instead, we move increment, filter and createRules functionality from Row to methods in a shared row-data-utils class. These methods in the row-data-utils class will now make requests for both tables and authorized views.

Authorized View:

  • increment, filter and createRules methods are added to the new AuthorizedView class for making checkAndMutate and readModifyWriteRow calls for an authorized view. These methods should not be exposed on TabularApiSurface because then they will be exposed on Table, but this functionality is already available for tables on the increment, filter and createRules methods on Row.
  • The createRules, filter and increment methods on the AuthorizedView class will have the same signature as the methods with the same name on the Row class except they must also be provided with a row id that the request will be made for.
  • The AuthorizedView class will store row data for authorized views just like the Row class has a data property that stores data for that row. This rowData variable in AuthorizedView stores data for each row it makes a request on behalf of.

Row:

  • Existing method bodies are moved to RowDataUtils and replaced with one line of code that delegates the call.

Table:

  • A view method is added which will be used for creating authorized views
  • A constructor is added matching the constructor that was in Table before. This is needed now that the TabularApiSurface constructor is protected.

TabularApiSurface:

  • The constructor is now protected since the class isn't meant to be instantiated directly
  • A viewName member is added and will be undefined for tables, but provided for authorized views

test/Row.ts

  • Since the code has been moved then different files need to be mocked in proxyquire and sinon so these mocks are adjusted.

Next Steps:

  1. Write some tests that use the new increment, createRules and filter methods now available on AuthorizedView. Make sure that these methods pass the proper parameters along and that they target an authorized view.
  2. Write a PR for supporting authorized views calls for all methods of the TabularApiSurface class.

danieljbruce and others added 27 commits August 20, 2024 10:02
…apis/nodejs-bigtable into bigtable-authorized-views-move-row-methods

# Conflicts:
#	src/row.ts
#	src/table.ts
#	src/tabular-api-service.ts
#	src/utils/table.ts
#	test/table.ts
@danieljbruce danieljbruce requested review from a team as code owners August 28, 2024 19:05
@product-auto-label product-auto-label bot added the size: l Pull request size is large. label Aug 28, 2024
@product-auto-label product-auto-label bot added the api: bigtable Issues related to the googleapis/nodejs-bigtable API. label Aug 28, 2024
@danieljbruce danieljbruce changed the title feat: Expose functionality from Row to be available on the TabularApiSurface too feat: Bigtable Authorized Views - Expose functionality from Row to be available on the TabularApiSurface too Aug 28, 2024
@danieljbruce danieljbruce changed the title feat: Bigtable Authorized Views - Expose functionality from Row to be available on the TabularApiSurface too feat: Bigtable Authorized Views - Expose functionality from Row to be available on AuthorizedViews too Aug 29, 2024
@danieljbruce danieljbruce changed the title feat: Bigtable Authorized Views - Expose functionality from Row to be available on AuthorizedViews too feat: Bigtable Authorized Views - Allow checkAndMutate and ReadModifyWriteRow calls on Authorized Views Aug 29, 2024
Copy link

@bhshkh bhshkh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well organized code. Thanks for adding the detailed description regarding the changes in the PR!

@danieljbruce danieljbruce merged commit cb1ebd4 into bigtable-authorized-views Sep 23, 2024
13 of 17 checks passed
@danieljbruce danieljbruce deleted the bigtable-authorized-views-move-row-methods branch September 23, 2024 20:13
danieljbruce added a commit that referenced this pull request Nov 7, 2024
* Create the new tabular api service file

* feat: Bigtable authorized views - move the code over to the TabularApiSurface class (#1463)

* Move the constructor over to TabularApiService

* Move sampleRowKeys over

* Move sampleRowKeys functions over and use promisify

* Adjust the proxyquire to work with TabularAPIserv

* Move all the ReadRows functionality over

* Solve the issue with the is dependency

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Add header for new class

* Only include Table mocks that are necessary

* Remove TODO

* Rename TabularApiService to TabularApiSurface

* Change all imports to tabular-api-surface

* surface. not service

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* feat: Bigtable Authorized Views - Allow checkAndMutate and ReadModifyWriteRow calls on Authorized Views (#1464)

* Move the constructor over to TabularApiService

* Move sampleRowKeys over

* Move sampleRowKeys functions over and use promisify

* Adjust the proxyquire to work with TabularAPIserv

* Move all the ReadRows functionality over

* Solve the issue with the is dependency

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Add header for new class

* Trying out the FilterInformation class

* Add a DataUtils module for the shared row function

* Outsource functionality of filter to helper

* debugging

* Adjust proxyquire to include mocks for moved fn

* Remove imports

* Outsource code to a createRulesUtil function

* Change proxyquire to include createRulesUtil

* Move increment over to the utils folder

* Move the functions into a static class for mocks

* Remove mockCreateRules and shorten mock

* Stub out FakeRowDataUtil

* Remove unused dependencies

* Add the functions to work with checkAndMutate

and readWriteModifyRow

* Fix regressions from the merge

* Add documentation for the class

* Document the new methods of table

* Change the interface of the rowUtils

* Pull the generateProperties code out avoid duplica

* Move duplicate code out into a getProperties object

* Remove console.log

* Add method for creating views

* Object for making grpc calls for authorized views

* Remove import

* Update the documentation for the Table

* More specific type

* Add documentation for each of the functions

* Remove TODO

* Add headers

* Add documentation for new class

* Remove imports

* Reintroduce before

* Remove unused import

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* feat: Create a view on the instance, not on the table (#1476)

* Move view creation from the table class to instanc

* feat: Create a view on the instance, not on the table

* Add table name parameter to the documentation

* Remove Table

* feat: Create the unit tests for Authorized Views ensuring that requests are consistent (#1501)

* Create a test for authorized views

* Set the ReadRows options

* Add authorized view test for createreadstream

* Pass authorizedViewName along for createReadStream

* Add the unit test for getRows

* Add the unit test for getRows

* Take out the test mock for the request function

* Mutate rows request - test setup

* Made corrections and src code changes so unit test

passes

* Fix the readrows test to end the stream

Also add a test for insert and make the test easier to debug.

* Finish the sampleRowKeys tests

* Refactor part of the test for getting request opts

* Authorized views for readRows test should be one

assert

* Add the mock function for request

* Make mutate rows use the mockRequest function

* Setup the sampleRowKeys setup fn to use mockReq

* Add comments to the mocking functions

* Fix comments. Move functions to right place

* Setup the readModifyWriteRow tests

* For createRules test use new mockCallbackRequest

You need to call the callback in order to end the operation to createRules.

* Use the new mockCallbackRequest function

* Get rid of the unused mock function

* Create a test file for createRules not got

* increment

* Add the test for increment

* Change the response value for authorized view

* Add the console logs and change return type

* Flesh out response so that view call works

* Exclude appropriate properties using promisify.

* Removed console logs

* Finished the filter test

* Add the view call for filter

* Remove only

* Change to more readable values

* Remove only

* Add header to the test

* For tests, expect view to be excluded from promis

* run linter

* Add a few @params and @returns

* Add a comment about making requests for auth views

* Added more comments - auth views vs table

Also added a comment for calling increment

* feat: Bigtable authorized views integration tests (#1504)

* Create a test for authorized views

* Set the ReadRows options

* Add authorized view test for createreadstream

* Pass authorizedViewName along for createReadStream

* Add the unit test for getRows

* Add the unit test for getRows

* Take out the test mock for the request function

* Mutate rows request - test setup

* Made corrections and src code changes so unit test

passes

* Fix the readrows test to end the stream

Also add a test for insert and make the test easier to debug.

* Finish the sampleRowKeys tests

* Refactor part of the test for getting request opts

* Authorized views for readRows test should be one

assert

* Add the mock function for request

* Make mutate rows use the mockRequest function

* Setup the sampleRowKeys setup fn to use mockReq

* Add comments to the mocking functions

* Fix comments. Move functions to right place

* Setup the readModifyWriteRow tests

* For createRules test use new mockCallbackRequest

You need to call the callback in order to end the operation to createRules.

* Use the new mockCallbackRequest function

* Get rid of the unused mock function

* Create a test file for createRules not got

* increment

* Add the test for increment

* Change the response value for authorized view

* Add the console logs and change return type

* Flesh out response so that view call works

* Exclude appropriate properties using promisify.

* Removed console logs

* Finished the filter test

* Add the view call for filter

* Remove only

* Change to more readable values

* Remove only

* Add header to the test

* For tests, expect view to be excluded from promis

* run linter

* Create the before hook for auth views

* Note to self on auth views - TODO

* Add an insert statement

* Make a test for getRows

* Add the test for the getRows function.

* Finish ‘should fail when writing to a row not in v

* Add another mutate test, preserve table after test

Add after hook to make sure table values stay the same.

Add test for modifying from different column

* Remove TODO

* refactor the error message in the test

* Add mutate, insert and sampleRowKeys tests

* Fix sampleRowKeys test to require a fix upstream

* Create samplerowkeys and createreadstream tests

* Surround all tests in a try block

This allows for better error reporting

* Fixed the column filter so it works

* Reduce verbosity in tests

In many cases, 3 lines can be reduced to one variable. This is crucial for making the integration tests shorter.

* Replace verbose data with data variable

* Add an insert for multiple rows

* Fix the check and mutate test to ignore corrupt d

* Add the readModifyWriteRow tests

* Add awaits

* wrap columnFamily and columnIdInView in brackets

* Eliminate the TODO

* Remove only

* Add key back to filter config option

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Kevin Kim <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the googleapis/nodejs-bigtable API. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants