[SPARK-47781][SQL] Handle negative scale decimals for JDBC data sources

[yaooqinn]

What changes were proposed in this pull request?

SPARK-30252 has disabled the definition of the negative scale for decimals. It has a regression that also disabled reading negative scale decimals from JDBC data sources. Although there is a legacy config to restore the old behavior, it seemed neither designed for such a case nor convenient in a data pipeline that extracts negative scale decimals from a database such as Oracle to Parquet files w/o negative scale decimal support.

In addition, Postgres has the negative scale decimals support since v15, which was one of the supporters for disabling negative scale decimals on our side.

In this PR, we change the schema from decimal(p,s) to decimal(p-s,0) if s<0.

Why are the changes needed?

Does this PR introduce any user-facing change?

Negative scale decimals have many supporters for rounding the internal parts, such as Oracle, Postgres, etc.

• Oracle

Negative scale is the number of significant digits to the left of the decimal point, to but not including the least significant digit. For negative scale the least significant digit is on the left side of the decimal point, because the actual data is rounded to the specified number of places to the left of the decimal point. For example, a specification of (10,-2) means to round to hundreds.

• Postgres

Beginning in PostgreSQL 15, it is allowed to declare a numeric column with a negative scale. Then values will be rounded to the left of the decimal point. The precision still represents the maximum number of non-rounded digits. Thus, a column declared as
NUMERIC(2, -3)
will round values to the nearest thousand and can store values between -99000 and 99000, inclusive. It is also allowed to declare a scale larger than the declared precision. Such a column can only hold fractional values, and it requires the number of zero digits just to the right of the decimal point to be at least the declared scale minus the declared precision. For example, a column declared as

How was this patch tested?

new tests

Was this patch authored or co-authored using generative AI tooling?

no

[yaooqinn]

cc @cloud-fan

[yaooqinn]

cc @dongjoon-hyun @cloud-fan, PTAL when you have some time

[dongjoon-hyun]

+1 LGTM

[yaooqinn]

Merged to master

Thank you. @cloud-fan @dongjoon-hyun