Fast Filtering with Spark PartitionFilters and PushedFilters
Spark can use the disk partitioning of files to greatly speed up certain filtering operations. This post explains the difference between memory and disk partitioning, describes how to analyze physical […]