org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow...
Spark异常：Kryo serialization failed: Buffer overflow.
1、问题描述
SparkSQL任务报错如下：
org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow. Available: 0, required: xxx. To avoid this, increase spark.kryoserializer.buffer.max value.
根据报错信息可知，Spark在Kryo序列化时发生缓存溢出
2、问题解决
根据提示信息，可以适当增加spark.kryoserializer.buffer.max的值，例如256m或512m，该参数的默认值为128m
更多信息参考：https://study.sf.163.com/documents/read/service_support/dsc-p-a-0067