FAQ-离线传输clickhouse-hive报错Unsupported type OTHER

问题描述/异常栈

ERROR ApplicationMaster: User class threw exception: java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaVanillaMethodMirror4.jinvokeraw(JavaMirrors.scala:444)
    at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaMethodMirror.jinvoke(JavaMirrors.scala:380)
    at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaVanillaMethodMirror.apply(JavaMirrors.scala:396)
    at com.netease.bdms.ndi.common.util.ReflectionUtils$.scalaObjectMethod(ReflectionUtils.scala:14)
    at org.apache.spark.sql.jdbc.SparkJdbcUtils$.getCatalystType(SparkJdbcUtils.scala:59)
    at org.apache.spark.sql.jdbc.SparkJdbcUtils$.$anonfun$getSchema$1(SparkJdbcUtils.scala:273)
    at scala.Option.getOrElse(Option.scala:189)
    at org.apache.spark.sql.jdbc.SparkJdbcUtils$.getSchema(SparkJdbcUtils.scala:273)
    at com.netease.music.da.transfer.jdbc.reader.JDBCReader.$anonfun$getSchemaByQuery$1(JDBCReader.scala:719)
    at com.netease.music.da.transfer.jdbc.connection.DBConnection.executeQueryForSchema(DBConnection.scala:133)
    at com.netease.music.da.transfer.jdbc.reader.JDBCReader.getSchemaByQuery(JDBCReader.scala:718)
    at com.netease.music.da.transfer.jdbc.reader.JDBCReader.$anonfun$checkAndExtractSchema$1(JDBCReader.scala:210)
    at scala.collection.immutable.List.map(List.scala:293)
    at com.netease.music.da.transfer.jdbc.reader.JDBCReader.checkAndExtractSchema(JDBCReader.scala:195)
    at com.netease.music.da.transfer.jdbc.reader.JDBCReader.doRead(JDBCReader.scala:381)
    at com.netease.music.da.transfer.common.reader.AbstractDataReader.read(AbstractDataReader.scala:24)
    at com.netease.music.da.transfer.common.Worker$.main(Worker.scala:85)
    at com.netease.music.da.transfer.common.Worker.main(Worker.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:723)
Caused by: java.sql.SQLException: Unsupported type OTHER
    at org.apache.spark.sql.errors.QueryExecutionErrors$.unsupportedJdbcTypeError(QueryExecutionErrors.scala:836)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.getCatalystType(JdbcUtils.scala:230)
    ... 27 more

发现版本

解决方案

查看ck数据类型，一些识别的类型会被转换成other，可以把一些特殊的类型进行转换,用toString(字段名）as 字段名进行转换，要用到ck的语法，cast函数不生效。

也可以在自主分析查询字段数据类型，把特殊的字段类型进行替换。

作者：楚展航