https://hackingandslacking.com/pyspark-macro-dataframe-methods-join-and-groupby-477a57836ff

Todd BirchardBlockedUnblockFollowFollowingJun 24We’ve had quite a journey exploring the magical world of PySpark together. After covering DataFrame transformations, structured streams, and RDDs, there are only so many things left to cross off the list before we’ve gone too deep.

To round things up for this series, we’re going to take a look back at some powerful DataFrame operations we missed. In particular, we’ll be focusing on operations which modify DataFrames as a whole, such as Joins and Aggregations. Let’s start with Joins then we can visit Aggregation and close out with some Visualization thoughts.