Type 2 Slowly Changing Dimension Upserts with Delta Lake
This post explains how to perform type 2 upserts for slowly changing dimension tables with Delta Lake. We’ll start out by covering the basics of type 2 SCDs and when […]
This post explains how to perform type 2 upserts for slowly changing dimension tables with Delta Lake. We’ll start out by covering the basics of type 2 SCDs and when […]
Datasets are available to Spark Scala/Java users and offer more type safety than DataFrames. Python and R infer types during runtime, so these APIs cannot support the Datasets. This post […]
This post shows how to create beginningOfMonthDate and endOfMonthDate functions by leveraging the native Spark datetime functions. The native Spark datetime functions are not easy to use, so it’s important […]