汇总KStreams中的实时数据

时间:2018-10-10 08:41:25

标签: scala apache-spark apache-kafka kafka-consumer-api apache-kafka-streams

我想基于指定的键求和一列数据。流就像id(String)键,value(Long)。

var scrollViewer = GetScrollViewer(listView);

//在这里失败

获取

val aggtimelogs: KTable[String, java.lang.Long] = stream
      .groupByKey()
      .aggregate(
               () => 0L,
              (key: String, value: java.lang.Long, aggregate: java.lang.Long) => value + aggregate)

如何在Scala中做什么?

Kafka版本是

Unspecified value parameters: : Materialized[K, VR, KeyValueStore[Bytes, Array[Byte]]

即使我尝试了

compile "org.apache.kafka:kafka-clients:2.0.0"
compile (group: "org.apache.kafka", name: "kafka-streams", version: "2.0.0"){
    exclude group:"com.fasterxml.jackson.core"
}

也是这个

val reducer = new Reducer[java.lang.Long]() {
      def apply(value1: java.lang.Long, value2: java.lang.Long): java.lang.Long = value1 + value2
    }

    val agg = stream
          .groupByKey()
          .reduce(reducer)

val reducer : Reducer[Long] = (value1: Long, value2: Long) => value1 + value2

1 个答案:

答案 0 :(得分:1)

我是这样做的

val aggVal = streams.groupByKey().reduce(new Reducer[Double]() {
    def apply(val1: Double, val2: Double): Double = val1 + val2
  })