Question

每天我们都会在Student-Topic-In中获得以下消息

Message 1: {"StudentID": "1", "StudentName":"aaa","fatherName":"aaa1",  "class":"1"}
Message 2: {"StudentID": "2", "StudentName":"bbb","fatherName":"bbb1",  "class":"1"}
Message 3: {"StudentID": "3", "StudentName":"ccc","fatherName":"ccc1",  "class":"2"}
Message 4: {"StudentID": "4", "StudentName":"ddd","fatherName":"ddd1",  "class":"2"}
Message 5: {"StudentID": "5", "StudentName":"eee","fatherName":"eee1",  "class":"2"}

根据每个班级的一天结束（一天一次），我们必须合并所有消息并以以下格式发布到“ Student-Topic-Out”。

Message 1:{"Class":"1"
          {"StudentID": "1", "StudentName":"aaa","fatherName":"aaa1"},
          {"StudentID": "2", "StudentName":"bbb","fatherName":"bbb1"}
       }
Message 2:{"Class":"2" 
          {"StudentID": "3", "StudentName":"ccc","fatherName":"ccc1"},
          {"StudentID": "4", "StudentName":"ddd","fatherName":"ddd1"},
          {"StudentID": "5", "StudentName":"eee","fatherName":"eee1"}
       }

我尝试了以下操作，但是不知道如何创建没有班级名称的学生名单？

KStream<String, Object> sampleStream = builder.stream("Student-Topic-in");
    sampleStream
            .filter((k, v) -> v != null)
            .mapValues(v -> (Student) v)
            .groupBy((k, v) -> KeyValue.pair(v.getClass_name(), v))
            .windowedBy(TimeWindows.of(5000))
            //I am not sure how to create a student list without Classname
            .aggregate(Student::new, (k, v, list) -> (Student)list.add((Student)v)

能否让我知道，如何构造输出JSON消息Kafka流？

Answer 1

您可以使用“ class”属性进行KStream.groupBy(...).windowedBy().aggregate().mapValues分组。

在Aggregator()中，您可以聚集List的{{1}}个学生，然后在JSON

Answer 2

您可以通过以下方式将邮件汇总到列表中：

next

一旦获得KStream<String, Object> sampleStream = builder.stream("Student-Topic-in"); KTable<Windowed<key>,List<Student>> aggregatedTable = sampleStream .filter((k, v) -> v != null) .mapValues(v -> (Student) v) .groupBy((k, v) -> KeyValue.pair(v.getClass_name(), v)) .windowedBy(TimeWindows.of(5000)) //I am not sure how to create a student list without Classname .aggregate(ArrayList::new, (k, v, list) -> list.add((Student)v, Materialized.with(keySerde(), arrayListSerde()) )，就可以使用List<Student>函数将其转换为任何所需的格式。

如何将多封邮件转换为单封邮件？

2 个答案: