pydeephaven.query¶
This module implements the Query class that can be used to execute a chained set of Table operations in one batch.
- class Query(session, table)[source]¶
Bases:
TableInterface[Query]A Query object is used to define and execute a sequence of Deephaven table operations on the server.
When the query is executed, the table operations specified for the Query object are batched together and sent to the server in a single request, thus avoiding multiple round trips between the client and the server. The result of executing the query is a new Deephaven table.
Note, an application should always use the factory method on the Session object to create a Query instance as the constructor is subject to future changes to support more advanced features already planned.
- agg_all_by(agg, by)¶
Adds this operation to the query.
The agg_all_by method creates a new table containing grouping columns and grouped data. The resulting grouped data is defined by the aggregation specified.
Note, because agg_all_by applies the aggregation to all the columns of the table, it will ignore any column names specified for the aggregation.
- Parameters:
agg (
Aggregation) – the aggregation to be appliedby (
str|list[str]) – the group-by column name(s)
- Return type:
- Returns:
self
- Raises:
DHError –
- agg_by(aggs, by)¶
Adds this operation to the query.
The agg_by method creates a new table containing grouping columns and grouped data. The resulting grouped data is defined by the aggregation(s) specified.
- Parameters:
aggs (
Aggregation|list[Aggregation]) – the aggregation(s) to be appliedby (
str|list[str]) – the group-by column name(s)
- Return type:
- Returns:
self
- Raises:
DHError –
- aj(table, on, joins=None)¶
Adds this operation to the query.
The aj (as-of join) method creates a new table containing all the rows and columns of the left table, plus additional columns containing data from the right table. For columns appended to the left table (joins), row values equal the row values from the right table where the keys from the left table most closely match the keys from the right table without going over. If there is no matching key in the right table, appended row values are NULL.
- Parameters:
table (
Table) – the right-table of the joinon (
str|list[str]) – the column(s) to match, can be a common name or a match condition of two columns, e.g. ‘col_a = col_b’. The first ‘N-1’ matches are exact matches. The final match is an inexact match. The inexact match can use either ‘>’ or ‘>=’. If a common name is used for the inexact match, ‘>=’ is used for the comparison.joins (
str|list[str] |None) – the column(s) to be added from the right table to the result table, can be renaming expressions, i.e. “new_col = col”; default is None, which means all the columns from the right table, excluding those specified in ‘on’
- Return type:
- Returns:
self
- Raises:
DHError –
- avg_by(by=None)¶
Adds this operation to the query.
The avg_by method creates a new table containing the average for each group. Columns not used in the grouping must be of numeric types.
- Parameters:
by (
str|list[str] |None) – the group-by column name(s), default is None, meaning grouping all the rows into one group- Return type:
- Returns:
self
- count_by(col, by=None)¶
Adds this operation to the query.
The count_by method creates a new table containing the number of rows for each group. The count of each group is stored in a new column named after the ‘col’ parameter.
- Parameters:
col (
str) – the name of the column to store the countsby (
str|list[str] |None) – the group-by column name(s), default is None, meaning grouping all the rows into one group
- Return type:
- Returns:
self
- drop_columns(cols)¶
Adds this operation to the query.
The drop_column method creates a new table with the same size as this table but omits any of the specified columns.
- Parameters:
cols (
str|list[str]) – the column name(s)- Return type:
- Returns:
self
- exact_join(table, on, joins=None)¶
Adds this operation to the query.
The exact_join method creates a new table containing all the rows and columns of this table plus additional columns containing data from the right table. For columns appended to the left table (joins), row values equal the row values from the right table where the key values in the left and right tables are equal.
- Parameters:
table (
Table) – the right-table of the joinon (
str|list[str]) – the column(s) to match, can be a common name or an equal expression, i.e. “col_a = col_b” for different column namesjoins (
str|list[str] |None) – the column(s) to be added from the right table to the result table, can be renaming expressions, i.e. “new_col = col”; default is None, which means all the columns from the right table, excluding those specified in ‘on’
- Return type:
- Returns:
self
- first_by(by=None)¶
Adds this operation to the query.
The first_by method creates a new table which contains the first row of each distinct group.
- Parameters:
by (
str|list[str] |None) – the group-by column name(s), default is None, meaning grouping all the rows into one group- Return type:
- Returns:
self
- group_by(by=None)¶
Adds this operation to the query.
The group_by method creates a new table containing grouping columns and grouped data, column content is grouped into arrays.
If no group-by column is given, the content of each column is grouped into its own array.
- Parameters:
by (
str|list[str] |None) – the group-by column name(s), default is None, meaning grouping all the rows into one group- Return type:
- Returns:
self
- head(num_rows)¶
Adds this operation to the query.
The head method creates a new table with a specific number of rows from the beginning of the table.
- Parameters:
num_rows (
int) – the number of rows at the head of table- Return type:
- Returns:
self
- head_by(num_rows, by)¶
Adds this operation to the query.
The head_by method creates a new table containing the first number of rows for each group.
- Parameters:
num_rows (
int) – the number of rows at the beginning of each groupby (
str|list[str]) – the group-by column name(s)
- Return type:
- Returns:
self
- join(table, on=None, joins=None, reserve_bits=10)¶
Adds this operation to the query.
The join method creates a new table containing rows that have matching values in both tables. Rows that do not have matching criteria will not be included in the result. If there are multiple matches between a row from the left table and rows from the right table, all matching combinations will be included. If no columns to match (on) are specified, every combination of left and right table rows is included.
- Parameters:
table (
Table) – the right-table of the joinon (
str|list[str] |None) – the column(s) to match, can be a common name or an equal expression, i.e. “col_a = col_b” for different column names, default is Nonejoins (
str|list[str] |None) – the column(s) to be added from the right table to the result table, can be renaming expressions, i.e. “new_col = col”; default is None, which means all the columns from the right table, excluding those specified in ‘on’reserve_bits (
int) – the number of bits of key-space to initially reserve per group; default is 10
- Return type:
- Returns:
self
- last_by(by=None)¶
Adds this operation to the query.
The last_by method creates a new table which contains the last row of each distinct group.
- Parameters:
by (
str|list[str] |None) – the group-by column name(s), default is None, meaning grouping all the rows into one group- Return type:
- Returns:
self
- lazy_update(formulas)¶
Adds this operation to the query.
The lazy_update method creates a new table containing a new, cached, formula column for each formula.
- Parameters:
formulas (
str|list[str]) – the column formula(s)- Return type:
- Returns:
self
- max_by(by=None)¶
Adds this operation to the query.
The max_by method creates a new table containing the maximum value for each group. Columns not used in the grouping must be of numeric types.
- Parameters:
by (
str|list[str] |None) – the group-by column name(s), default is None, meaning grouping all the rows into one group- Return type:
- Returns:
self
- median_by(by=None)¶
Adds this operation to the query.
The median_by method creates a new table containing the median for each group. Columns not used in the grouping must be of numeric types.
- Parameters:
by (
str|list[str] |None) – the group-by column name(s), default is None, meaning grouping all the rows into one group- Return type:
- Returns:
self
- min_by(by=None)¶
Adds this operation to the query.
The min_by method creates a new table containing the minimum value for each group. Columns not used in the grouping must be of numeric types.
- Parameters:
by (
str|list[str] |None) – the group-by column name(s), default is None, meaning grouping all the rows into one group- Return type:
- Returns:
self
- natural_join(table, on, joins=None, type=NaturalJoinType.ERROR_ON_DUPLICATE)¶
Adds this operation to the query.
The natural_join method creates a new table containing all the rows and columns of this table, plus additional columns containing data from the right table. For columns appended to the left table (joins), row values equal the row values from the right table where the key values in the left and right tables are equal. If there is no matching key in the right table, appended row values are NULL.
- Parameters:
table (
Table) – the right-table of the joinon (
str|list[str]) – the column(s) to match, can be a common name or an equal expression, i.e. “col_a = col_b” for different column namesjoins (
str|list[str] |None) – the column(s) to be added from the right table to the result table, can be renaming expressions, i.e. “new_col = col”; default is None, which means all the columns from the right table, excluding those specified in ‘on’type (
NaturalJoinType) – the action to be taken when duplicate right hand rows are encountered; default is ERROR_ON_DUPLICATE
- Return type:
- Returns:
self
- raj(table, on, joins=None)¶
Adds this operation to the query.
The raj (reverse as-of join) method creates a new table containing all the rows and columns of the left table, plus additional columns containing data from the right table. For columns appended to the left table ( joins), row values equal the row values from the right table where the keys from the left table most closely match the keys from the right table without going under. If there is no matching key in the right table, appended row values are NULL.
- Parameters:
table (
Table) – the right-table of the joinon (
str|list[str]) – the column(s) to match, can be a common name or a match condition of two columns, e.g. ‘col_a = col_b’. The first ‘N-1’ matches are exact matches. The final match is an inexact match. The inexact match can use either ‘<’ or ‘<=’. If a common name is used for the inexact match, ‘<=’ is used for the comparison.joins (
str|list[str] |None) – the column(s) to be added from the right table to the result table, can be renaming expressions, i.e. “new_col = col”; default is None, which means all the columns from the right table, excluding those specified in ‘on’
- Return type:
- Returns:
self
- Raises:
DHError –
- select(formulas=None)¶
Adds this operation to the query.
The select method creates a new in-memory table that includes one column for each formula. If no formula is specified, all columns will be included.
- Parameters:
formulas (
str|list[str] |None) – the column formula(s), default is None- Return type:
- Returns:
self
- select_distinct(cols=None)¶
Adds this operation to the query.
The select_distinct method creates a new table containing all the unique values for a set of key columns. When the selectDistinct method is used on multiple columns, it looks for distinct sets of values in the selected columns.
- Parameters:
cols (
str|list[str] |None) – the column name(s), default is None- Return type:
- Returns:
self
- slice(start, stop)¶
Adds this operation to the query.
Extracts a subset of a table by row positions into a new Table.
If both the start and the stop are positive, then both are counted from the beginning of the table. The start is inclusive, and the stop is exclusive. slice(0, N) is equivalent to
head()(N) The start must be less than or equal to the stop.If the start is positive and the stop is negative, then the start is counted from the beginning of the table, inclusively. The stop is counted from the end of the table. For example, slice(1, -1) includes all rows but the first and last. If the stop is before the start, the result is an empty table.
If the start is negative, and the stop is zero, then the start is counted from the end of the table, and the end of the slice is the size of the table. slice(-N, 0) is equivalent to
tail()(N).If the start is negative and the stop is negative, they are both counted from the end of the table. For example, slice(-2, -1) returns the second to last row of the table.
- Parameters:
start (
int) – the first row position to include in the resultstop (
int) – the row position to stop before, exclusive
- Return type:
- Returns:
self
Examples
>>> table.slice(0, 5) # first 5 rows >>> table.slice(-5, 0) # last 5 rows >>> table.slice(2, 6) # rows from index 2 to 5 >>> table.slice(6, 2) # ERROR: cannot slice start after end >>> table.slice(-6, -2) # rows from 6th last to 2nd last (exclusive) >>> table.slice(-2, -6) # ERROR: cannot slice start after end >>> table.slice(2, -3) # all rows except the first 2 and the last 3 >>> table.slice(-6, 8) # rows from 6th last to index 8 (exclusive)
- snapshot()¶
Adds this operation to the query.
The snapshot method creates a static snapshot table.
- Return type:
- Returns:
self
- snapshot_when(trigger_table, stamp_cols=None, initial=False, incremental=False, history=False)¶
Adds this operation to the query.
The snapshot_when creates a table that captures a snapshot of this table whenever trigger_table updates.
When trigger_table updates, a snapshot of this table and the “stamp key” from trigger_table form the resulting table. The “stamp key” is the last row of the trigger_table, limited by the stamp_cols. If trigger_table is empty, the “stamp key” will be represented by NULL values.
- Parameters:
trigger_table (
Table) – the trigger tablestamp_cols (
str|list[str] |None) – The column(s) from trigger_table that form the “stamp key”, may be renames, default is None, meaning that all columns from trigger_table form the “stamp key”.initial (
bool) – Whether to take an initial snapshot upon construction, default is False. When False, the resulting table will remain empty until trigger_table first updates.incremental (
bool) – Whether the resulting table should be incremental, default is False. When False, all rows of this table will have the latest “stamp key”. When True, only the rows of this table that have been added or updated will have the latest “stamp key”.history (
bool) – Whether the resulting table should keep history, default is False. A history table appends a full snapshot of this table and the “stamp key” as opposed to updating existing rows. The history flag is currently incompatible with initial and incremental: when history is True, incremental and initial must be False.
- Return type:
- Returns:
self
- sort(order_by, order=None)¶
Adds this operation to the query.
The sort method creates a new table where the rows are ordered based on values in the specified set of columns.
- Parameters:
order_by (
str|list[str]) – the column(s) to be sorted onorder (
SortDirection|list[SortDirection] |None) – the corresponding sort direction(s) for each sort column, default is None. In the absence of explicit sort directions, data will be sorted in the ascending order.
- Return type:
- Returns:
self
- std_by(by=None)¶
Adds this operation to the query.
The std_by method creates a new table containing the sample standard deviation for each group. Columns not used in the grouping must be of numeric types.
Sample standard deviation is computed using Bessel’s correction, which ensures that the sample variance will be an unbiased estimator of population variance.
- Parameters:
by (
str|list[str] |None) – the group-by column names(s), default is None, meaning grouping all the rows into one group- Return type:
- Returns:
self
- sum_by(by=None)¶
Adds this operation to the query.
The sum_by method creates a new table containing the sum for each group. Columns not used in the grouping must be of numeric types.
- Parameters:
by (
str|list[str] |None) – the group-by column name(s), default is None, meaning grouping all the rows into one group- Return type:
- Returns:
self
- tail(num_rows)¶
Adds this operation to the query.
The tail method creates a new table with a specific number of rows from the end of the table.
- Parameters:
num_rows (
int) – the number of rows at the end of table- Return type:
- Returns:
self
- tail_by(num_rows, by)¶
Adds this operation to the query.
The tail_by method creates a new table containing the last number of rows for each group.
- Parameters:
num_rows (
int) – the number of rows at the end of each groupby (
str|list[str]) – the group-by column name(s)
- Return type:
- Returns:
self
- ungroup(cols=None, null_fill=True)¶
Adds this operation to the query.
The ungroup method creates a new table in which array columns from the source table are unwrapped into separate rows. The ungroup columns should be of array types.
- Parameters:
cols (
str|list[str] |None) – the array column(s), default is None, meaning all array columns will be ungrouped, default is None, meaning all array columns will be ungroupednull_fill (
bool) – indicates whether null should be used to fill missing cells, default is True
- Return type:
- Returns:
self
- update(formulas)¶
Adds this operation to the query.
The update method creates a new table containing a new, in-memory column for each formula.
- Parameters:
formulas (
str|list[str]) – the column formula(s)- Return type:
- Returns:
self
- update_by(ops, by=None)¶
Adds this operation to the query.
The update_by method creates a table with additional columns calculated from window-based aggregations of columns in this table. The aggregations are defined by the provided operations, which support incremental aggregations over the corresponding rows in the table. The aggregations will apply position or time-based windowing and compute the results over the entire table or each row group as identified by the provided key columns.
- Parameters:
ops (
UpdateByOperation|list[UpdateByOperation]) – the UpdateByOperation(s) to be appliedby (
str|list[str] |None) – the group-by column name(s), defaults to None, meaning all calculations are performed over the entire table
- Return type:
- Returns:
self
- update_view(formulas)¶
Adds this operation to the query.
The update_view method creates a new table containing a new, formula column for each formula.
- Parameters:
formulas (
str|list[str]) – the column formula(s)- Return type:
- Returns:
self
- var_by(by=None)¶
Adds this operation to the query.
The var_by method creates a new table containing the sample variance for each group. Columns not used in the grouping must be of numeric types.
Sample variance is computed using Bessel’s correction, which ensures that the sample variance will be an unbiased estimator of population variance.
- Parameters:
by (
str|list[str] |None) – the group-by column name(s), default is None, meaning grouping all the rows into one group- Return type:
- Returns:
self
- view(formulas)¶
Adds this operation to the query.
The view method creates a new formula table that includes one column for each formula.
- Parameters:
formulas (
str|list[str]) – the column formula(s)- Return type:
- Returns:
self
- where(filters)¶
Adds this operation to the query.
The where method creates a new table with only the rows meeting the filter criteria in the column(s) of the table.
- where_in(filter_table, cols)¶
Adds this operation to the query.
The where_in method creates a new table containing rows from the source table, where the rows match values in the filter table.
- where_not_in(filter_table, cols)¶
Adds this operation to the query.
The where_not_in method creates a new table containing rows from the source table, where the rows do not match values in the filter table.