| title | The Python-Java boundary and how it affects query efficiency |
|---|
This guide discusses the relationship between Python and Java in Deephaven Python queries and how it affects query efficiency. Calls pass between the two languages in both directions:
- Python code calls into the Java engine when you run a table operation.
- The engine calls back into Python when a query string uses a Python function.
The short version: table operations you call from Python are inexpensive. A Python function inside a query string, however, can run once for every row. So prefer built-in functions, and write the Python calls you keep so that the engine can vectorize them.
The first section shows how the Python API wraps Java table operations. Why crossing the boundary affects performance covers the other direction and explains why it can dominate query cost. The remaining sections cover how to reduce boundary crossings and the memory that Python functions in query strings use.
Deephaven's backend is written almost entirely in Java. Over 90% of the code in the Deephaven Community Core repository is Java.
Deephaven's Python API wraps the engine's powerful Java table operations in idiomatic Python that follows PEP 8 conventions. Python developers don't have to write Java or call Java classes directly. The wrappers rely on jpy, a powerful bidirectional Python-Java bridge.
The Python-Java boundary is the point where a call passes from the Python interpreter to the Java engine, or back. Each crossing takes time, because jpy has to convert the arguments and the return value between Python and Java types.
The update table operation shows the pattern. It creates a new table containing a new, in-memory column for each formula given in a list. Below is a simplified version of the Python Table class and its update method.
from __future__ import annotations
from collections.abc import Sequence
import jpy
_JTable = jpy.get_type("io.deephaven.engine.table.Table")
class Table:
def __init__(self, j_table: jpy.JType):
self.j_table = jpy.cast(j_table, _JTable)
def update(self, formulas: Sequence[str]) -> Table:
return Table(j_table=self.j_table.update(*formulas))deephaven.table.Tableis the Python class that wraps the Java classio.deephaven.engine.table.Tablein a Pythonic interface._JTableis jpy's reference to the Java class.jpy.JTypeis the type hint the Deephaven Python API uses for a Java object.jpy.castcastsj_tableto the JavaTabletype so that jpy has the correct signatures for making calls into Java.j_table.update(*formulas)calls the Javaupdatemethod through jpy.Table(j_table=...)wraps the Java result in a new PythonTable, whichupdatereturns.
Much of Deephaven's Python API looks like this under the hood. This thin-wrapper design serves two purposes: convenience and speed. The thin wrappers make the API feel Pythonic and add only a small overhead for each operation. The Java query engine still does the actual work, so table operations keep the concurrency and efficiency it offers.
A table operation called from Python, like update in the wrapper above, crosses the boundary only a few times, however many rows the table has. A Python function called inside a query string is different. The engine calls it from Java once per row. When the engine can vectorize the call, it calls it once per chunk (a block of rows the engine processes together) instead. Either way, a single query can cross the boundary many times.
Each crossing takes time, so the number of boundary crossings affects the query's efficiency. This matters most with real-time and large data, where speed can make or break an application.
Calling Python also costs parallelism. Unless the Python build is free-threaded, the engine doesn't split a column or filter that calls Python across threads. Parallelization explains how the engine parallelizes queries.
The following code times the built-in sin function against numpy.sin on 100,000 rows:
from deephaven import empty_table
import numpy as np
from time import time
n_tries = 100
source = empty_table(100_000).update(["X = 0.1 * ii"])
start = time()
for _ in range(n_tries):
result_builtin = source.update(["Y = sin(X)"])
end = time()
elapsed = (end - start) / n_tries
print(f"Built-in sine function - {elapsed:.3f} seconds.")
start = time()
for _ in range(n_tries):
result_numpy = source.update(["Y = (double)np.sin(X)"])
end = time()
elapsed = (end - start) / n_tries
print(f"NumPy sine function - {elapsed:.3f} seconds.")One run printed the following times:
Built-in sine function - 0.045 seconds.
NumPy sine function - 1.341 seconds.
That's a significant difference: the NumPy version takes about 30 times as long.
Why is the NumPy version so much slower?
- The query string that creates
result_builtinuses only the built-insinfunction, so evaluating it never crosses the Python-Java boundary. - The query string that creates
result_numpycalls into Python at least twice for every row:- It looks up
sinon thenpmodule. - It calls
sinon the value ofX, and jpy converts the argument to Python and the result back to Java.
- It looks up
The engine also can't vectorize this call. Vectorize the Python calls you keep explains why.
Efficient queries make as few Python-Java boundary crossings as possible. There are two ways to get there:
- Use Java methods and variables built into the query language. When a function has a Java equivalent, use it, and that call never crosses the boundary. What's built into the query language? lists where to find them.
- Vectorize the Python calls you keep. When a function has no Java equivalent, write the call so the engine can run it once per chunk of rows instead of once per row. See Vectorize the Python calls you keep.
Start with Deephaven's own built-in query language functions, which include the sin function from the timing example.
Not every function in the Deephaven Python API is built into the query language. The date-time conversion helpers in deephaven.time are Python functions, so calling them in a query string crosses the boundary on every call. For built-in alternatives, see Time in Deephaven.
Because Deephaven is written in Java, a query string can also use any public Java class on the server's classpath. See Java objects and classes in query strings.
- Query strings can use classes in
java.langandjava.utilwithout importing them. - To use a class from another package, such as
java.math.BigDecimal, write its fully qualified name or import it withquery_library.import_class.
The Java SE API class index lists every class in the Java SE API. These packages and classes are especially useful in queries:
java.langjava.lang.Math— Standard math functions. Most of them, includingcbrt,hypot, andlog10, are also built-in query language functions that handle Deephaven null values, so prefer the built-in. CallMathdirectly only for a method with no built-in, such asfma.java.lang.Number— Methods for convertingBigDecimalandBigIntegervalues to primitive types.java.lang.String— Methods such ascontains,startsWith, andsubstringthat you can call directly on a non-nullStringcolumn value.
java.mathjava.math.BigDecimal— Arbitrary-precision signed decimal values.java.math.BigInteger— Arbitrary-precision integer values.
java.utiljava.util.Arrays— Methods for searching and manipulating arrays.java.util.Collections— Methods that operate on or return collections.java.util.Random— A seeded pseudorandom generator. For unseeded values, built-in functions such asrandomandrandomIntneed no setup.
When a query must call a Python function, write the call so the engine can vectorize it. A vectorized call crosses into Python once per chunk of rows instead of once per row.
This is not NumPy vectorization:
- Inside each chunk, Deephaven still calls the Python function once per value, even when the function is a NumPy ufunc (a function that NumPy applies element by element to an array) such as
numpy.sin. - Vectorizing changes only how many times the engine crosses the boundary to make those calls.
- A Numba
@vectorizefunction is the exception: Deephaven passes it the whole chunk as arrays.
To be vectorized, a call must meet several requirements, including these:
- It is a plain function call, not one made through a module or object.
- Its result is not explicitly cast.
- Its result is not used inside a larger expression.
- Each argument is a column, a variable, or a constant, not an expression such as
X + 1. - It passes exactly as many arguments as the function has parameters, so it doesn't use
*argsor leave out parameters that have defaults.
The np.sin(X) call in the timing example fails the first two requirements: it goes through the np module, and its result is cast to double.
To make the call vectorizable, wrap np.sin in a plain Python function with type hints. The float return type hint tells Deephaven the result type, so the query string doesn't need a cast:
from deephaven import empty_table
import numpy as np
source = empty_table(100_000).update(["X = 0.1 * ii"])
def np_sin(x: float) -> float:
return np.sin(x)
# Not vectorized: the call goes through the np module, and its result is cast.
result_numpy = source.update(["Y = (double)np.sin(X)"])
# Vectorized: a plain function call with a column argument and no cast.
result_vectorized = source.update(["Y = np_sin(X)"])Both tables hold the same values. The engine crosses into Python far fewer times for result_vectorized. Inside each chunk, np_sin still runs once per row, so it remains slower than the built-in sin.
Vectorizing reduces the number of boundary crossings, but it doesn't remove them. Whether the overhead of the Python calls you keep is acceptable depends on your data size and update frequency.
Python user-defined functions (UDFs) are Python functions called in query strings. They cost memory as well as time:
- Each call runs in Python, so the objects it creates live in Python's memory, outside the Java heap.
- When a UDF returns a Python object that the engine stores in a column, the Java side holds a reference to it. Python can't free that object until the JVM garbage collects the reference.
Resident memory is the physical memory the whole process occupies, including both the Java heap and Python's memory. Python UDFs can make resident memory grow, and that growth has a cost:
- Memory growth beyond the Java heap — Python objects allocated during UDF execution can accumulate over time, especially in long-running or high-throughput queries, so resident memory can grow far beyond the configured Java heap size.
- Out-of-memory (OOM) risk — When resident memory exceeds the available system or container memory, the Linux OOM killer may terminate processes.
To minimize memory risks when using Python UDFs:
- Convert performance-critical UDFs to Java — Implement frequently called functions in Java, or use built-in query language functions instead. To call your own Java code, install it as a Java package and call its classes in query strings. See also How to minimize the number of boundary crossings.
- Avoid returning large Python objects in UDFs — Large Python objects can stay in Python memory for a long time. If Python doesn't free them soon enough, Python can raise a
MemoryErrorthat fails the query, or resident memory can grow until the OOM killer terminates the Deephaven server process. Where possible, have UDFs return only the data the table columns need, which is typically primitive values and strings. - Trigger garbage collection when needed — If the JVM hasn't yet freed unused table columns that hold Python objects, call
deephaven.garbage_collectto run Python garbage collection and request a JVM garbage collection. The JVM treats the request as advisory, so the effect is not guaranteed. - Monitor resident memory — Track total process memory, not just Java heap usage, for queries using Python UDFs.
- Time in Deephaven
- Parallelization
- Query string overview
- Python functions in query strings
- Built-in query language functions
- Java objects and classes in query strings
- Install and use Java packages
- Use jpy
- Use Numba in Deephaven queries
- Run garbage collection
- Auto-imported functions
empty_tablegarbage_collectupdatequery_library.import_classPydocTablePydocdeephaven.timePydocjpyPydoc