功能说明

适用模块

hivesql的mr任务中，reduce任务启动不了。

具体说明

/mnt/dfs/3/yarn/local/nmPrivate/application_1635839340375_1782387/container_e86_1635839340375_1782387_01_028266/container_e86_1635839340375_1782387_01_028266.pid.tmp Writing to cgroup task files... Creating local dirs... Launching container... Getting exit code file... Creating script paths... [2021-12-17 01:57:26.885]Container exited with a non-zero exit code 134. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : /bin/bash: line 1: 29807 Aborted (core dumped) /usr/jdk64/jdk1.8.0_152/bin/java -server -XX:NewRatio=8 -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx18000M -Djava.io.tmpdir=/mnt/dfs/6/yarn/local/usercache/ndir/appcache/application_1635839340375_1782387/container_e86_1635839340375_1782387_01_028266/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/mnt/dfs/7/yarn/logs/application_1635839340375_1782387/container_e86_1635839340375_1782387_01_028266 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dyarn.app.mapreduce.shuffle.logger=INFO,shuffleCLA -Dyarn.app.mapreduce.shuffle.logfile=syslog.shuffle -Dyarn.app.mapreduce.shuffle.log.filesize=0 -Dyarn.app.mapreduce.shuffle.log.backups=0 org.apache.hadoop.mapred.YarnChild 10.196.81.51 37248 attempt_1635839340375_1782387_r_001814_0 94558000017002 > /mnt/dfs/7/yarn/logs/application_1635839340375_1782387/container_e86_1635839340375_1782387_01_028266/stdout 2> /mnt/dfs/7/yarn/logs/application_1635839340375_1782387/container_e86_1635839340375_1782387_01_028266/stderr Last 4096 bytes of stderr : SLF4J: Class path contains multiple

问题原因 遇到该类情况可判断是数据量太大，reduce端OOM导致。此时可通过加内存，手动调整设置更多reduce个数解决该维问题。

作者：刘思伟