Skip to content

Commit d148e9b

Browse files
committed
[SPARK-52877][PYTHON][FOLLOW-UP] Use columns instead of itercolumns in RecordBatch
### What changes were proposed in this pull request? This PR proposes to use `columns` instead of `itercolumns` in RecordBatch, which does not exist in the old version of PyArrow. ### Why are the changes needed? To recover the build https://github.com/apache/spark/actions/runs/16507806777/job/46682838114 This is just a temporary workaround. ### Does this PR introduce _any_ user-facing change? No, test-only. ### How was this patch tested? Manually. ### Was this patch authored or co-authored using generative AI tooling? No, Closes #51661 from HyukjinKwon/SPARK-52877. Authored-by: Hyukjin Kwon <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent 8c53892 commit d148e9b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

python/pyspark/sql/pandas/serializers.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -793,7 +793,7 @@ def load_stream(self, stream):
793793
for batch in super().load_stream(stream):
794794
columns = [
795795
[conv(v) for v in column.to_pylist()] if conv is not None else column.to_pylist()
796-
for column, conv in zip(batch.itercolumns(), converters)
796+
for column, conv in zip(batch.columns, converters)
797797
]
798798
if len(columns) == 0:
799799
yield [[pyspark._NoValue] * batch.num_rows]

0 commit comments

Comments
 (0)