FAQ-key(xxx) already exists

问题描述/异常栈

设置并行度时任务报错
key(xxx) already exists

解决方案

ndi.spark.spark-conf.spark.stage.maxConsecutiveAttempts=1
ndi.spark.spark-conf.spark.yarn.max.executor.failures=1

问题原因

重试导致数据重复设置上述参数确保任务不重试

作者：林帅