[SPARK-40201][SQL][TESTS] Improve v1 write test coverage #37641
Conversation
There was a problem hiding this comment.
@ulysses-you this is great! I do see we have a few test failures. Have you looked into them yet?
| // V1 write command will be empty). | ||
| if (Utils.isTesting) outputOrderingMatched = orderingMatched | ||
| if (Utils.isTesting) { | ||
| if (SQLConf.get.plannedWriteEnabled && !orderingMatched) { |
There was a problem hiding this comment.
I think we also need to check the concurrent writer is not enabled.
|
@allisonwang-db thank you for review. There are two cases that the ordering does not match:
I have a pr for case 1 about foldable see #37831, but need to simplify a bit more. I think pr #37525 can save the case 2, although I do not looked it deeply. |
|
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. |
2 participants
What changes were proposed in this pull request?
It should be a bug if
spark.sql.optimizer.plannedWrite.enabledenabled, so throw exception in test mode if the ordering does not match to help improve test coverage.Why are the changes needed?
Make v1 write test work on all SQL tests
Does this PR introduce any user-facing change?
no, only affect tests
How was this patch tested?
pass CI