This post explains how to create DataFrames with ArrayType columns and how to perform common data processing operations. Array columns are one of the most useful column types, but they’re […]

This post explains how Spark registers native functions internally and the public facing APIs for you to register your own functions. Registering native functions is important if you want to […]