Question

我的需求：我使用本地IDEA将消息发送到远程Kafka集群。我的工作：在Windows上使用IDEA Java编程，读取本地csv文件数据并将其发送到远程kafka群集。然后处理数据。但无法连接到远程kafka群集，请提供提示。我的Java代码kafka配置

Properties props = new Properties();
props.put("metadata.broker.list", "slaver1:9092");   // 远程kafka 集群配置
props.put("serializer.class", "kafka.serializer.StringEncoder");
// key.serializer.class默认为serializer.class
props.put("key.serializer.class", "kafka.serializer.StringEncoder");
props.put("request.required.acks", "1");
ProducerConfig config = new ProducerConfig(props);

远程kafka集群slaver1节点server.properties配置：

broker.id=1
port=9092
host.name=slaver1

delete.topic.enable=true

############################套接字服务器设置################ ##############

    # The address the socket server listens on. It will get the value returned from 
    # java.net.InetAddress.getCanonicalHostName() if not configured.
    #   FORMAT:
    #     listeners = listener_name://host_name:port
    #   EXAMPLE:
    #     listeners = PLAINTEXT://your.host.name:9092
    #listeners=PLAINTEXT://:9092
    listeners=PLAINTEXT://slaver1:9092

    # Hostname and port the broker will advertise to producers and consumers. If 
not set, 
    # it uses the value for "listeners" if configured.  Otherwise, it will use the value
    # returned from java.net.InetAddress.getCanonicalHostName().
    #advertised.listeners=PLAINTEXT://your.host.name:9092
    advertised.listeners=PLAINTEXT://slaver1:9092

正在运行的本机代码无法连接到远程kafka集群。

    java.nio.channels.ClosedChannelException
    at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
    at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
    atkafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
    at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
    at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
    at kafka.producer.async.DefaultEventHandler$$anonfun$handle$1.apply$mcV$sp(DefaultEventHandler.scala:67)
    at kafka.utils.Utils$.swallow(Utils.scala:172)
    at kafka.utils.Logging$class.swallowError(Logging.scala:106)
    at kafka.utils.Utils$.swallowError(Utils.scala:45)
    at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:67)
    at kafka.producer.Producer.send(Producer.scala:77)
    at kafka.javaapi.producer.Producer.send(Producer.scala:33)
    at kafka.SendDatatToKafka.send(SendDatatToKafka.java:121)
    at kafka.SendDatatToKafka.main(SendDatatToKafka.java:50)
10:51:36,353 INFO  kafka.producer.SyncProducer                                   - Disconnecting from slaver1:9092
10:51:36,353 ERROR kafka.utils.Utils$                                            - fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:slaver1,port:9092)] failed
kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:slaver1,port:9092)] failed

请指点

Windows IDEA连接远程kafka集群

0 个答案: