FAQ-spark执行sql报错Unknown message type: 9

问题描述/异常栈
java.lang.RuntimeException: java.lang.IllegalArgumentException: Unknown message type: 9
    at org.apache.spark.network.shuffle.protocol.BlockTransferMessage$Decoder.fromByteBuffer(BlockTransferMessage.java:71)
    at org.apache.spark.network.shuffle.ExternalShuffleBlockHandler.receive(ExternalShuffleBlockHandler.java:80)
    at org.apache.spark.network.server.TransportRequestHandler.processRpcRequest(TransportRequestHandler.java:184)
    at org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:103)
    at org.apache.spark.network.server.TransportChannelHandler.channelRead(TransportChannelHandler.java:118)
    at org.spark_project.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
发现版本
spark3.3.16
解决方案
临时方案：sql任务自定义参数添加参数：conf.spark.shuffle.useOldFetchProtocol=true
问题原因
spark3兼容问题：https://issues.apache.org/jira/browse/SPARK-33093

作者：常惠渊