Skip to content

Commit 6613046

Browse files
Srinivasa Reddy VundelaFelix Cheung
authored andcommitted
[MINOR][DOCS][PYTHON] Adding missing boolean type for replacement value in fillna
## What changes were proposed in this pull request? Currently pyspark Dataframe.fillna API supports boolean type when we pass dict, but it is missing in documentation. ## How was this patch tested? >>> spark.createDataFrame([Row(a=True),Row(a=None)]).fillna({"a" : True}).show() +----+ | a| +----+ |true| |true| +----+ Please review http://spark.apache.org/contributing.html before opening a pull request. Author: Srinivasa Reddy Vundela <[email protected]> Closes #17688 from vundela/fillna_doc_fix.
1 parent ae3df4e commit 6613046

File tree

2 files changed

+5
-1
lines changed

2 files changed

+5
-1
lines changed

python/pyspark/sql/dataframe.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1247,7 +1247,7 @@ def fillna(self, value, subset=None):
12471247
Value to replace null values with.
12481248
If the value is a dict, then `subset` is ignored and `value` must be a mapping
12491249
from column name (string) to replacement value. The replacement value must be
1250-
an int, long, float, or string.
1250+
an int, long, float, boolean, or string.
12511251
:param subset: optional list of column names to consider.
12521252
Columns specified in subset that do not have matching data type are ignored.
12531253
For example, if `value` is a string, and subset contains a non-string column,

python/pyspark/sql/tests.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1711,6 +1711,10 @@ def test_fillna(self):
17111711
self.assertEqual(row.age, None)
17121712
self.assertEqual(row.height, None)
17131713

1714+
# fillna with dictionary for boolean types
1715+
row = self.spark.createDataFrame([Row(a=None), Row(a=True)]).fillna({"a": True}).first()
1716+
self.assertEqual(row.a, True)
1717+
17141718
def test_bitwise_operations(self):
17151719
from pyspark.sql import functions
17161720
row = Row(a=170, b=75)

0 commit comments

Comments
 (0)