Skip to content

Conversation

@MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Jun 16, 2020

What changes were proposed in this pull request?

Replace CAST(... AS TIMESTAMP by TIMESTAMP_SECONDS in the following benchmarks:

  • ExtractBenchmark
  • DateTimeBenchmark
  • FilterPushdownBenchmark
  • InExpressionBenchmark

Why are the changes needed?

The benchmarks fail w/o the changes:

[info] Running benchmark: datetime +/- interval
[info]   Running case: date + interval(m)
[error] Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve 'CAST(`id` AS TIMESTAMP)' due to data type mismatch: cannot cast bigint to timestamp,you can enable the casting by setting spark.sql.legacy.allowCastNumericToTimestamp to true,but we strongly recommend using function TIMESTAMP_SECONDS/TIMESTAMP_MILLIS/TIMESTAMP_MICROS instead.; line 1 pos 5;
[error] 'Project [(cast(cast(id#0L as timestamp) as date) + 1 months) AS (CAST(CAST(id AS TIMESTAMP) AS DATE) + INTERVAL '1 months')#2]
[error] +- Range (0, 10000000, step=1, splits=Some(1))

Does this PR introduce any user-facing change?

No

How was this patch tested?

By running the affected benchmarks.

@cloud-fan
Copy link
Contributor

thanks for fixing it! It's benchmark only so we don't need to wait for jenkins.

merging to master!

@cloud-fan cloud-fan closed this in 3643565 Jun 16, 2020
@SparkQA
Copy link

SparkQA commented Jun 16, 2020

Test build #124125 has finished for PR 28843 at commit 97b5f97.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk MaxGekk deleted the GuoPhilipse-31710-fix-compatibility-followup branch December 11, 2020 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants