卡夫卡经纪人认为没有经纪人（甚至没有经纪人）

Question

问题

我正在尝试在docker中运行单个Kafka实例。我可以创建主题，但不能从主题中消费或生产主题。当我尝试使用Kafka代理时，会记录一条错误消息。

kafka_1      | [2019-03-24 16:45:05,263] ERROR [KafkaApi-5] Number of alive brokers '0' does not meet the required replication factor '1' for the offsets topic (configured via 'offsets.topic.replication.factor'). This error can be ignored if the cluster is starting up and not all brokers are up yet. (kafka.server.KafkaApis)

我不了解Number of alive brokers '0'位，因为它是一个记录此信息的Kafka经纪人，因此确定至少要有一个经纪人（本身）吗？

我使用默认的server.properties文件并进行了以下更改：

listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners=PLAINTEXT://192.168.0.109:9092
zookeeper.connect=zookeeper:2181

其中192.168.0.109是我的主机IP。

文件

docker-compose

version: '3.4'
services:
  zookeeper:
    build:
      context: ./
      dockerfile: zookeeper.dockerfile
      network: host
    image: zookeeper
    ports:
      - "2181:2181"
  kafka:
    build:
      context: ./
      dockerfile: kafka.dockerfile
      network: host
    image: kafka
    ports:
      - "9092:9092"

kafka.dockerfile

FROM alpine:3.7

# Set the name for a non-priliaged user
ENV APPUSER=appuser

# Install dependnacies
RUN apk add bash
RUN apk add openjdk8-jre

# Add a non-privilaged user and change into it's home directory
RUN adduser -D $APPUSER
WORKDIR /home/$APPUSER

# Download kafka and set its owner to $APPUSER
RUN wget https://www-eu.apache.org/dist/kafka/2.1.0/kafka_2.11-2.1.0.tgz
RUN chown $APPUSER kafka_2.11-2.1.0.tgz

# Change to the non-privilaged user
USER $APPUSER

# Extract kafka
RUN tar -xzf kafka_2.11-2.1.0.tgz

# copy custom server.properties into image 
COPY ./server.properties ./

# Set working directory to kafka dir
WORKDIR /home/$APPUSER/kafka_2.11-2.1.0

# start kafka with customer server.properties file
CMD ["bin/kafka-server-start.sh", "../server.properties"]

zookeeper.dockerfile

FROM alpine:3.7

# Set the name for a non-priliaged user
ENV APPUSER=appuser

# Install dependnacies
RUN apk add bash
RUN apk add openjdk8-jre

# Add a non-privilaged user and change into it's home directory
RUN adduser -D $APPUSER
WORKDIR /home/$APPUSER

# Download kafka and set its owner to $APPUSER
RUN wget https://www-eu.apache.org/dist/kafka/2.1.0/kafka_2.11-2.1.0.tgz
RUN chown $APPUSER kafka_2.11-2.1.0.tgz

# Change to the non-privilaged user
USER $APPUSER

# Extract kafka
RUN tar -xzf kafka_2.11-2.1.0.tgz

# Set working directory to kafka dir
WORKDIR /home/$APPUSER/kafka_2.11-2.1.0

# Run zookeeper
CMD ["bin/zookeeper-server-start.sh", "config/zookeeper.properties"]

Answer 1

这取决于侦听器配置。

当您指定advertised.listeners=PLAINTEXT://192.168.0.109:9092时，Kafka将尝试从Docker容器内的连接到192.168.0.109。但是，如果这是一个主机地址，并且您尚未设置适当的Docker网络向导，它将失败。

当您指定advertised.listeners=PLAINTEXT://localhost:9092时，Kafka将连接到Docker容器内的localhost，这当然会解析为本身。这里的要点是，您将无法在Docker容器之外（在您的主机或另一个Docker容器上）使用Kafka客户端，因为它们会尝试本地连接到localhost- ，那里不会有Kafka经纪人。

This article更详细地说明了该问题以及要使用的适当配置。如果您想要在Docker中预先构建Kafka的示例，那么this Docker Compose会显示它。