Core concept
import org.apache.spark.sql.functions._
df.select(sum($"amount").as("total"))Advertisement
How it works
UDFs slower than built-ins. Prefer built-ins.
Advertisement
Trade-offs + gotchas
Every DataFrame op.
import org.apache.spark.sql.functions._
df.select(sum($"amount").as("total"))UDFs slower than built-ins. Prefer built-ins.
Every DataFrame op.