Scaling Data with Pandas API on Spark

Pandas DataFrame vs. PySpark DataFrame

Role of the InternalFrame in PySpark Pandas

<aside> 💡

Connecting the two APIs

</aside>

InternalFrame in Pandas API on Spark

Unlocking Data Capabilities

• Reasons to do this if something isn’t supported (like matplotlib)

Performance of Pandas, Spark, and PySpark Pandas