Scaling with Pandas APIs

Pandas UDFs and Function APIs

User-defined functions (UDFs)

image.png

Apache Arrow

<aside> 💡

Improving Performance in When Working With Pandas UDFs

</aside>

image.png

Pandas UDFs

<aside> 💡

Scale Out with Vectorized Operations

</aside>

image.png

Pandas UDFs

<aside> 💡

4 Types of Pandas UDFs

</aside>

image.png

Examples

image.png

Pandas Function APIs in Spark

<aside> 💡

Optimizing Table-Level Operations in Spark

</aside>

image.png

Pandas Function APIs

<aside> 💡

Three types: Grouped map, Map, Cogrouped MAP

</aside>