如何修复KafkaStream中的RocksDBException错误

时间:2019-08-18 18:06:28

标签: apache-kafka-streams

我正在尝试为WordsCount实现我的第一个Kafka Streams Java App。 Kafka客户端。
引发的错误

  

java.lang.NoClassDefFoundError:org / rocksdb / RocksDBException

我看到了一些有关同一错误的资源,该错误指出Kafka中存在一些错误。是真的吗?

使用的jar文件:: Kafka-streams-0.11.0.0.jar, Kafka-clients-0.11.0.0.jar。

定义流后。配置这是我编写的代码。

KStreamBuilder builder = new KStreamBuilder();
    // 1- create a stream from kafka
    KStream<String,String> wordCountInput = 
builder.stream("input_topic");

    KTable<String,Long> wordCountsz = wordCountInput.mapValues(textline-> textline.toLowerCase())
                                                    .flatMapValues(loweredcaseTextLine-> Arrays.asList(loweredcaseTextLine.split(" ")))
                                                    .selectKey((ignoredKey,word)->word)
                                                    .groupByKey()
                                                    .count("stotre");
    wordCountsz.to(Serdes.String(),Serdes.Long(),"output_topic");
    KafkaStreams streams = new KafkaStreams(builder,config);
    streams.start();
    System.out.println(streams.toString());

    Runtime.getRuntime().addShutdownHook(new Thread(streams::close));

错误:

 Exception in thread "word-count-app-698a6ff3-ded7-43c4-ada0-d51263f04f33-StreamThread-1" java.lang.NoClassDefFoundError: org/rocksdb/RocksDBException
        at org.apache.kafka.streams.state.internals.RocksDBKeyValueStoreSupplier.get(RocksDBKeyValueStoreSupplier.java:56)
        at org.apache.kafka.streams.state.internals.RocksDBKeyValueStoreSupplier.get(RocksDBKeyValueStoreSupplier.java:35)
        at org.apache.kafka.streams.processor.TopologyBuilder.build(TopologyBuilder.java:1236)
        at org.apache.kafka.streams.processor.TopologyBuilder.build(TopologyBuilder.java:1179)
    at org.apache.kafka.streams.processor.internals.StreamThread.createStreamTask(StreamThread.java:1234)
    at org.apache.kafka.streams.processor.internals.StreamThread$TaskCreator.createTask(StreamThread.java:294)
    at org.apache.kafka.streams.processor.internals.StreamThread$AbstractTaskCreator.retryWithBackoff(StreamThread.java:254)
    at org.apache.kafka.streams.processor.internals.StreamThread.addStreamTasks(StreamThread.java:1313)
    at org.apache.kafka.streams.processor.internals.StreamThread.access$1100(StreamThread.java:73)
    at org.apache.kafka.streams.processor.internals.StreamThread$RebalanceListener.onPartitionsAssigned(StreamThread.java:183)
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:265)
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:363)
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:310)
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:297)
    at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1078)
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1043)
    at org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:582)
    at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:553)
    at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:527)
    Caused by: java.lang.ClassNotFoundException: org.rocksdb.RocksDBException
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

1 个答案:

答案 0 :(得分:1)

这是一个愚蠢的问题,因为该解决方案太天真了,无法提及! 但是由于可能有些人和我一样,所以请跟随一些在线资源并对其进行编码,并在执行中断时感到沮丧。

我只需要添加rocksdb jar!我将罐子更改为1.0.2

实际上,我研究的资源没有提及这个rocksdb! 虽然我理解了这个问题,但是由于有一个演示了互联网上的代码如何正常工作的演示,而没有提到rockdb! 它迫使我认为也许发生了其他事情!

抱歉,这个不必要的故事!希望对大家有帮助! 不要像我一样盲目地追踪资源!

相关问题