Align scalar UDF literal args with coerced return fields

[Kevin-Li-2025]

Which issue does this PR close?

• Closes ReturnFieldArgs.scalar_arguments type doesn't match with arg_fields #19982.

Rationale for this change

ReturnFieldArgs exposes both coerced arg_fields and optional scalar literal arguments to scalar UDFs. For coerced literal calls such as test_udf(1) where the UDF signature coerces the argument to Int16, the field metadata is Int16 but the scalar argument was not aligned with that field. On current main, the coerced expression can also appear as Cast(Literal), causing scalar_arguments to be None even though the argument is still a literal.

What changes are included in this PR?

• Build scalar UDF ReturnFieldArgs.scalar_arguments from literals after aligning them to the verified/coerced argument fields.
• Recognize literals wrapped in Cast / TryCast so coerced constants are still reported as scalar arguments.
• Add a regression test that asserts arg_fields[0].data_type() matches the scalar argument data type for a coercible scalar UDF.

Are these changes tested?

• cargo fmt --all --check
• CARGO_HOME=/Users/yinxiaogou/Documents/github/.cargo-home CARGO_TARGET_DIR=/Users/yinxiaogou/Documents/github/datafusion-target cargo test -p datafusion --test user_defined_integration user_defined::user_defined_scalar_functions::test_return_field_args_scalar_argument_types_match_arg_fields -- --exact

I also ran cargo test -p datafusion --test user_defined_integration user_defined::user_defined_scalar_functions; the new test and 22 other tests passed, while 3 existing tests failed locally because ARROW_TEST_DATA / PARQUET_TEST_DATA submodule data was unavailable in this checkout.

[Jefffrey]

This seems to follow a similar approach to a previous PR that attempted this issue:

• fix(expr): coerce literal arguments in return_field_from_args for UDFs #20012

There was some good discussion on there, perhaps can double check that since we found some difficulties with such an approach

[Kevin-Li-2025]

Thanks for pointing me back to #20012. I went through that discussion and tightened this PR so it does not take the broad approach that caused the earlier test churn.

The latest commit narrows scalar literal recovery to the implicit-cast shape inserted by type coercion, Cast(Literal -> expected arg type). It deliberately does not recursively unwrap user-written casts/try_casts, so cases such as arrow_cast(1, cast('Utf8' as varchar)) still fail as before instead of treating the cast expression as a constant type argument. It also preserves the original scalar value when the planning-time scalar cast fails, so builtins that validate literal values themselves keep their existing error path instead of surfacing an early Arrow cast error.

Local checks run:

• cargo test -p datafusion --test user_defined_integration test_return_field_args_scalar_argument_types_match_arg_fields
• cargo test -p datafusion --test core_integration dataframe::dataframe_functions::test_fn_arrow_cast_requires_literal_type_arg
• cargo run -p datafusion-examples --example builtin_functions -- date_time
• git diff --check

New CI is now queued on head 053b1c0e8.