Skip to content

feat: update execute_query to use PrepareQuery API #1095

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

jackdingilian
Copy link
Contributor

@jackdingilian jackdingilian commented Mar 10, 2025

Update execute_query to use PrepareQuery for each request.

In there future there will also be an option to prepare once and reuse a PreparedStatement across requests. This updates the metadata to come from the PrepareResponse, but does so in a way that will be compatible with a future version that refreshes the PrepareResponse periodically (which can lead to schema change for e.g 'Select *' queries.

This also creates a sql_helpers test class and replaces old, more minimal, sql helpers in __testing

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@jackdingilian jackdingilian requested review from a team as code owners March 10, 2025 22:02
@product-auto-label product-auto-label bot added the size: xl Pull request size is extra large. label Mar 10, 2025
@product-auto-label product-auto-label bot added the api: bigtable Issues related to the googleapis/python-bigtable API. label Mar 10, 2025
@jackdingilian jackdingilian added the kokoro:run Add this label to force Kokoro to re-run the tests. label Mar 11, 2025
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Mar 11, 2025
Copy link
Contributor

@daniel-sanche daniel-sanche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. I added a couple small comments, and a question around retries

Metadata will not be set until the first row has been yielded or response with no rows
completes.
"""
return self._final_metadata
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

returning None seems fine, but have you considered raising an exception if this is called in the wrong state?

I could imagine arguments for either one, depending on the use-case

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont have a strong opinion between those options. Do you have a preference? or is there anything else like this in the client we should be consistent with?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I might have a slight preference towards an exception, because then we can attach a message explaining why it's empty. and it makes the types stronger, since we can avoid Optionals.

But I'm fine with either way, and it probably comes down to the context we expect people to use it in

(We have precedent for raising exceptions for something like this in firestore, but I can't think of any for bigtable)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Updated it to raise an exception

@@ -1116,6 +1122,7 @@ async def test_execute_query_params(self, client, table_id, instance_id):
"tsArrayParam": SqlType.Array(SqlType.Timestamp()),
"dateArrayParam": SqlType.Array(SqlType.Date()),
}

result = await client.execute_query(
query, instance_id, parameters=parameters, parameter_types=param_types
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are there any new system tests that can be added? Maybe using the new prepare_* arguments? Or accessing iterator.metadata?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to add at least one against a real table, and I want to refactor the Metadata to expose the fields in ProtoMetadata. I was planning both of those for a follow on if that's ok (still to the branch before we merge).

For prepare_* it's a bit tricky since those are all retry/timeout params

@jackdingilian jackdingilian force-pushed the prepare-rebase branch 2 times, most recently from 0073253 to 441d71a Compare March 12, 2025 16:41
Copy link
Contributor

@daniel-sanche daniel-sanche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@daniel-sanche daniel-sanche merged commit 5527282 into googleapis:prepare-query Mar 13, 2025
8 of 9 checks passed
jackdingilian added a commit that referenced this pull request Mar 18, 2025
* feat: update execute_query to use PrepareQuery API (#1095)

* feat: Implement updated execute query protocol (#1096)

* feat: Refactor Metadata, add system tests, remove preview warning (#1099)

* Fix  setup.py merge

* fix: skip sql tests for emulator
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the googleapis/python-bigtable API. size: xl Pull request size is extra large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants