FAQ-number of dynamic partitions created is xxxx

问题描述/异常栈

SPARK SQL节点已调整动态分区上限，仍然提示`number of dynamic partitions created is xxxx `

解决方案

需通过节点更多参数属性添加以下参数：
conf.spark.hadoop.hive.exec.dynamic.partition                                 true
conf.spark.hadoop.hive.exec.dynamic.partition.mode                       nonstrict
conf.spark.hadoop.hive.exec.max.dynamic.partitions                       2000
conf.spark.hadoop.hive.exec.max.dynamic.partitions.pernode         100000

问题原因

通过猛犸平台SPARK SQL节点进行动态分区处理时，因Spark SQL源码Bug导致`set hive.exec.max.dynamic.partitions = XXXXX;`参数无效

作者：林帅