hive 2.3.2中kryo库中的stackoverflow错误

时间:2018-05-25 05:05:36

标签: hive emr kryo

我正在将emr作业从emr 3.7.0迁移到5.13.0。我们在hive脚本中使用brickhouse udf函数。在EMR 5.13.0上,我在kryo库中遇到 stackoverflow错误。潜水深入后,我发现这个问题(link)已于2015年修复到kryo图书馆。 我在hive 2.3.2中遇到以下异常。

Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.IllegalArgumentException: Unable to create serializer "org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer" for class: java.util.concurrent.atomic.AtomicReference
Serialization trace:
_tableInfo (org.codehaus.jackson.sym.BytesToNameCanonicalizer)
_rootByteSymbols (org.codehaus.jackson.JsonFactory)
jsonFactory (brickhouse.udf.json.ToJsonUDF)
genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
colExprMap (org.apache.hadoop.hive.ql.exec.SelectOperator)

其次是

Caused by: java.lang.StackOverflowError
at java.util.HashMap.hash(HashMap.java:339)
at java.util.HashMap.get(HashMap.java:557)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:61)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)

此堆栈跟踪包含代码(link),这些代码作为修复程序从kryo库中删除。

0 个答案:

没有答案