Skip to content

The filters expressions in TableScan may contain fields not included in schema. #4793

@HaoYang670

Description

@HaoYang670

Describe the bug
A clear and concise description of what the bug is.
https://github.com/apache/arrow-datafusion/blob/54ae4323a730a4a34c6b5f638e2083f0c3fe5dcc/datafusion/expr/src/logical_plan/plan.rs#L1396-L1409

The Exprs in filter may contain fields which are not included in schema. This can cause the logical optimizer failing.

To Reproduce
Set skip_failing_rules to false, and run the test filter_pushdown_view.
The logical optimizer will fail and the error message says:

SchemaError(FieldNotFound { field: Column { relation: Some("t1"), name: "id" }, valid_fields: Some([Column { relation: Some("t1"), name: "bool_col" }, Column { relation: Some("t1"), name: "int_col" }]) })', datafusion/optimizer/src/unwrap_cast_in_comparison.rs:106:42

The failure is caused by that the schema of TableScan doesn't include the fields in filters:

TableScan: t1 projection=[bool_col, int_col], full_filters=[t1.id = Int32(1)]

Expected behavior
The optimizer should not fail.

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions