pyspark.sql.DataFrame.drop_duplicates

DataFrame.drop_duplicates(subset=None)

drop_duplicates() is an alias for dropDuplicates().

New in version 1.4.

previous

pyspark.sql.DataFrame.dropDuplicatesWithinWatermark

next

pyspark.sql.DataFrame.dropna