FAQ-数据库传输Oracle2Hive任务运行报错：Connection reset

问题描述/异常栈

21-12-2022 10:37:44 CST Error executing statement: java.sql.SQLRecoverableException: Io exception: Connection reset
java.sql.SQLRecoverableException: Io exception: Connection reset
    at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:101)
    at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:133)
    at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:199)
    at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:263)
    at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:521)
    at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:418)
    at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:508)
    at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:203)
    at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:33)
    at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:510)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:247)
    at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:328)
    at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
    at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:748)
    at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:771)
    at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)
    at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
    at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
    at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:332)
    at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1933)
    at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1727)
    at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:112)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:482)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:646)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
    at azkaban.jobtype.misc.SqoopViaAzkaban.run(SqoopViaAzkaban.java:157)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at azkaban.jobtype.HadoopJavaJobRunnerMain2.runMethod(HadoopJavaJobRunnerMain2.java:214)
    at azkaban.jobtype.HadoopJavaJobRunnerMain2.<init>(HadoopJavaJobRunnerMain2.java:156)
    at azkaban.jobtype.HadoopJavaJobRunnerMain2.main(HadoopJavaJobRunnerMain2.java:81)
Caused by: java.net.SocketException: Connection reset
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
    at oracle.net.ns.DataPacket.send(DataPacket.java:150)
    at oracle.net.ns.NetOutputStream.flush(NetOutputStream.java:180)
    at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:169)
    at oracle.net.ns.NetInputStream.read(NetInputStream.java:117)
    at oracle.net.ns.NetInputStream.read(NetInputStream.java:92)
    at oracle.net.ns.NetInputStream.read(NetInputStream.java:77)
    at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1034)
    at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1010)
    at oracle.jdbc.driver.T4CTTIoauthenticate.receiveOauth(T4CTTIoauthenticate.java:760)
    at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:368)
    ... 31 more

发现版本

老版本中存在数据库传输节点

解决方案

任务节点参数中添加(或参数组中添加)：
jvm.args = -Djava.security.egd=file:/dev/../dev/urandom

FAQ-数据库传输Oracle2Hive任务运行报错：Connection reset - 图1

问题原因

连接ORACLE服务器，客户端要生成随机密钥用于客户端认证，JDK默认使用/dev/random，random采用阻塞方式生成随机数，生成的速度很慢，需要调整为/dev/urandom。
在Linux操作系统中，有一个特殊的设备文件，可以用作随机数发生器或伪随机数发生器。

/dev/random

在读取时，/dev/random设备会返回小于熵池噪声总数的随机字节。/dev/random可生成高随机性的公钥或一次性密码本。若熵池空了，对/dev/random的读操作将会被阻塞，直到从别的设备中收集到了足够的环境噪声为止。

当然你也可以设置成不堵塞，当你在open 的时候设置参数O_NONBLOCK， 但是当你read的时候，如果熵池空了，会返回-1

/dev/urandom

/dev/random的一个副本是/dev/urandom （"unlocked"，非阻塞的随机数发生器[4]），它会重复使用熵池中的数据以产生伪随机数据。这表示对/dev/urandom的读取操作不会产生阻塞，但其输出的熵可能小于/dev/random的。它可以作为生成较低强度密码的伪随机数生成器，不建议用于生成高强度长期密码。

作者：xxx