Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
4.2.0
Description
For now, if Python workers raise an exception, it will be passed to JVM and became a RuntimeError - Python driver has to capture it and check the message to reconstruct the exception.
There are a few issues with it - first of all, we have special code to deal with such a common issue. Then there's no way to associate a SQLSTATE with the exception.
By making PythonException a normal SparkThrowable, we can use the shared code path for it, which supports SQLSTATE natively.
We can also polish the exception message a bit.