一、#创作灵感#
公司使用Kafka的软件项目较多，故写技术笔记巩固知识要点
二、软件环境
- Kafka 3.9.0 官方下载地址：Kafka 3.9.0
- Docker Desktop 4.37 容器图形化工具 官方下载地址：Docker Desktop 4.37
特别说明
- Docker Desktop仅仅是Windows下镜像用，如有能爬楼的服务器可不安装此软件
- 本文以单机模式为例
三、常规方式安装
非容器化Kraft Mode（模式）安装
1、下载Kafka
参见上面的软件环境章节的下载地址
2、修改Kafka配置
# Kraft模式启动
vi $KAFKA_HOME/config/kraft/server.properties
关键设置包括
# 指定此节点是否应该成为控制平面的一部分
process.roles=controller,kafka
# KRaft模式下的节点ID，每个节点的ID必须唯一
node.id=1
# 监听器地址，用于集群内通信
listeners=PLAINTEXT://localhost:9092
# 控制平面监听器地址
controller.listeners=PLAINTEXT://localhost:9090
# 用于集群通信的安全协议以及相关的认证和授权配置
inter.broker.listener.name=CONTROLLER
# 控制平面的ADVERTISED.LISTENERS，用于其他节点发现
controller.listener.names=CONTROLLER
controller.advertised.listeners=PLAINTEXT://localhost:9090
# 用于KRaft模式的元数据复制因子，通常设置为集群中控制器的数量
metadata.max.age=300000
# 日志存储配置，指定日志的位置和格式
log.dirs=/tmp/kraft-logs
log.retention.hours=168
# 控制平面的选举配置和集群成员配置
control.plane.listener.name=CONTROLLER
control.plane.advertised.listeners=PLAINTEXT://localhost:9090
# 集群的初始/bootstrap状态
process.roles=controller
node.id=1
listeners=CONTROLLER://localhost:9090
inter.broker.listener.name=CONTROLLER3、Kraft模式启动Kafka
# Start the Kafka broker service
$ bin/kafka-server-start.sh config/kraft/server.properties四、docker方式安装
容器化Kraft Mode（模式）安装
1、容器启动Kafka
- 下载镜像
下载一体化官方镜像
$ docker pull confluentinc/cp-kafka:7.0.1windows下载镜像（内网安装）方式见作者其他docker安装其他组件的文章，如：docker安装MySQL8:docker离线安装MySQL、docker在线安装MySQL、MySQL镜像下载、MySQL配置、MySQL命令-CSDN博客
- 创建 Docker 网络
docker network create kafka-network
# docker查看所有网络
docker network ls- (Kraft模式)使用镜像启动容器
docker run -d \
--name=kafka \
--network=kafka-network \
-e KAFKA_KRAFT_MODE=true \
-e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 \
-v $(pwd)/kafka.properties:/etc/kafka/kafka.properties \
-e KAFKA_CONFIG_FILE=/etc/kafka/kafka.properties \
-p 2181:2181 \
-p 9092:9092 \
-p 19092:19092 \
confluentinc/cp-kafka:7.0.1这条命令的各个部分功能如下：
-d: 在后台运行容器。--name=cp-kafka: 为容器命名为的cp-kafka。--network=kafka-network: 将容器连接到之前创建的网络。-e KAFKA_KRAFT_MODE=true: 启用KRaft模式。-e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092: 设置监听地址。-v $(pwd)/kafka.properties:/etc/kafka/kafka.properties: 将工作目录下的kafka.properties文件挂载到容器中。-e KAFKA_CONFIG_FILE=/etc/kafka/kafka.properties: 指定使用的Kafka配置文件。confluentinc/cp-kafka:latest: 使用最新的Confluent Kafka镜像。- -p 做端口映射
# *********** ～～～～～ 重点：上面的命令很多版本不好使 ～～～～～ **********
下面的命令针对3.8.1亲测可用，没错Kraft 就是这么简单，启动成功就完事了，就有服务了～
docker run \
--name=smet-kafka \
-v /data1/kafka/docker-kafka/log:/kafka-logs \
-p 19092:9092 \
-p 19093:9093 \
-d 1756e53f57e1
按官方的说法，不增加任何配置，也可以直接启动
[docker@GZ***-PM153160 ~]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
apache/kafka 3.8.1 1756e53f57e1 3 months ago 385MB[docker@GZ***-PM153160 ~]$ docker run -p 9092:9092 c2b75a96e17e
ZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg
2025-01-27 03:55:46,842 [myid:] - INFO [main:o.a.z.s.q.QuorumPeerConfig@177] - Reading configuration from: /conf/zoo.cfg
2025-01-27 03:55:46,845 [myid:] - INFO [main:o.a.z.s.q.QuorumPeerConfig@431] - clientPort is not set
2025-01-27 03:55:46,845 [myid:] - INFO [main:o.a.z.s.q.QuorumPeerConfig@444] - secureClientPort is not set
2025-01-27 03:55:46,845 [myid:] - INFO [main:o.a.z.s.q.QuorumPeerConfig@460] - observerMasterPort is not set
2025-01-27 03:55:46,845 [myid:] - INFO [main:o.a.z.s.q.QuorumPeerConfig@477] - metricsProvider.className is org.apache.zookeeper.metrics.impl.DefaultMetricsProvider
2025-01-27 03:55:46,851 [myid:] - ERROR [main:o.a.z.s.q.QuorumPeerConfig@702] - Invalid configuration, only one server specified (ignoring)
2025-01-27 03:55:46,852 [myid:1] - INFO [main:o.a.z.s.DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2025-01-27 03:55:46,852 [myid:1] - INFO [main:o.a.z.s.DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2025-01-27 03:55:46,852 [myid:1] - INFO [main:o.a.z.s.DatadirCleanupManager@101] - Purge task is not scheduled.
2025-01-27 03:55:46,853 [myid:1] - WARN [main:o.a.z.s.q.QuorumPeerMain@139] - Either no config or no quorum defined in config, running in standalone mode
2025-01-27 03:55:46,854 [myid:1] - INFO [main:o.a.z.j.ManagedUtil@46] - Log4j 1.2 jmx support not found; jmx disabled.
2025-01-27 03:55:46,854 [myid:1] - INFO [main:o.a.z.s.q.QuorumPeerConfig@177] - Reading configuration from: /conf/zoo.cfg
2025-01-27 03:55:46,855 [myid:1] - INFO [main:o.a.z.s.q.QuorumPeerConfig@431] - clientPort is not set
2025-01-27 03:55:46,855 [myid:1] - INFO [main:o.a.z.s.q.QuorumPeerConfig@444] - secureClientPort is not set
2025-01-27 03:55:46,855 [myid:1] - INFO [main:o.a.z.s.q.QuorumPeerConfig@460] - observerMasterPort is not set
2025-01-27 03:55:46,855 [myid:1] - INFO [main:o.a.z.s.q.QuorumPeerConfig@477] - metricsProvider.className is org.apache.zookeeper.metrics.impl.DefaultMetricsProvider
2025-01-27 03:55:46,855 [myid:1] - ERROR [main:o.a.z.s.q.QuorumPeerConfig@702] - Invalid configuration, only one server specified (ignoring)
2025-01-27 03:55:46,855 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServerMain@123] - Starting server
2025-01-27 03:55:46,865 [myid:1] - INFO [main:o.a.z.s.ServerMetrics@64] - ServerMetrics initialized with provider org.apache.zookeeper.metrics.impl.DefaultMetricsProvider@4550bb58
2025-01-27 03:55:46,867 [myid:1] - INFO [main:o.a.z.s.a.DigestAuthenticationProvider@51] - ACL digest algorithm is: SHA1
2025-01-27 03:55:46,867 [myid:1] - INFO [main:o.a.z.s.a.DigestAuthenticationProvider@65] - zookeeper.DigestAuthenticationProvider.enabled = true
2025-01-27 03:55:46,869 [myid:1] - INFO [main:o.a.z.s.p.FileTxnSnapLog@124] - zookeeper.snapshot.trust.empty : false
2025-01-27 03:55:46,877 [myid:1] - INFO [main:o.a.z.ZookeeperBanner@42] -
2025-01-27 03:55:46,877 [myid:1] - INFO [main:o.a.z.ZookeeperBanner@42] - ______ _
2025-01-27 03:55:46,877 [myid:1] - INFO [main:o.a.z.ZookeeperBanner@42] - |___ / | |
2025-01-27 03:55:46,877 [myid:1] - INFO [main:o.a.z.ZookeeperBanner@42] - / / ___ ___ | | __ ___ ___ _ __ ___ _ __
2025-01-27 03:55:46,877 [myid:1] - INFO [main:o.a.z.ZookeeperBanner@42] - / / / _ \ / _ \ | |/ / / _ \ / _ \ | '_ \ / _ \ | '__|
2025-01-27 03:55:46,877 [myid:1] - INFO [main:o.a.z.ZookeeperBanner@42] - / /__ | (_) | | (_) | | < | __/ | __/ | |_) | | __/ | |
2025-01-27 03:55:46,877 [myid:1] - INFO [main:o.a.z.ZookeeperBanner@42] - /_____| \___/ \___/ |_|\_\ \___| \___| | .__/ \___| |_|
2025-01-27 03:55:46,877 [myid:1] - INFO [main:o.a.z.ZookeeperBanner@42] - | |
2025-01-27 03:55:46,877 [myid:1] - INFO [main:o.a.z.ZookeeperBanner@42] - |_|
2025-01-27 03:55:46,878 [myid:1] - INFO [main:o.a.z.ZookeeperBanner@42] -
2025-01-27 03:55:46,879 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:zookeeper.version=3.9.3-c26634f34490bb0ea7a09cc51e05ede3b4e320ee, built on 2024-10-17 23:21 UTC
2025-01-27 03:55:46,879 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:host.name=501da54a316f
2025-01-27 03:55:46,879 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:java.version=17.0.13
2025-01-27 03:55:46,879 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:java.vendor=Eclipse Adoptium
2025-01-27 03:55:46,879 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:java.home=/opt/java/openjdk
2025-01-27 03:55:46,879 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:java.class.path=/apache-zookeeper-3.9.3-bin/bin/../zookeeper-metrics-providers/zookeeper-prometheus-metrics/target/classes:/apache-zookeeper-3.9.3-bin/bin/../zookeeper-server/target/classes:/apache-zookeeper-3.9.3-bin/bin/../build/classes:/apache-zookeeper-3.9.3-bin/bin/../zookeeper-metrics-providers/zookeeper-prometheus-metrics/target/lib/*.jar:/apache-zookeeper-3.9.3-bin/bin/../zookeeper-server/target/lib/*.jar:/apache-zookeeper-3.9.3-bin/bin/../build/lib/*.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/zookeeper-prometheus-metrics-3.9.3.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/zookeeper-jute-3.9.3.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/zookeeper-3.9.3.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/snappy-java-1.1.10.5.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/slf4j-api-1.7.30.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/simpleclient_servlet-0.9.0.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/simpleclient_hotspot-0.9.0.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/simpleclient_common-0.9.0.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/simpleclient-0.9.0.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-transport-native-unix-common-4.1.113.Final.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-transport-native-epoll-4.1.113.Final-linux-x86_64.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-transport-classes-epoll-4.1.113.Final.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-transport-4.1.113.Final.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-tcnative-classes-2.0.66.Final.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-tcnative-boringssl-static-2.0.66.Final-windows-x86_64.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-tcnative-boringssl-static-2.0.66.Final-osx-x86_64.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-tcnative-boringssl-static-2.0.66.Final-osx-aarch_64.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-tcnative-boringssl-static-2.0.66.Final-linux-x86_64.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-tcnative-boringssl-static-2.0.66.Final-linux-aarch_64.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-tcnative-boringssl-static-2.0.66.Final.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-resolver-4.1.113.Final.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-handler-4.1.113.Final.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-common-4.1.113.Final.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-codec-4.1.113.Final.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/netty-buffer-4.1.113.Final.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/metrics-core-4.1.12.1.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/logback-core-1.2.13.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/logback-classic-1.2.13.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/jline-2.14.6.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/jetty-util-ajax-9.4.56.v20240826.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/jetty-util-9.4.56.v20240826.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/jetty-servlet-9.4.56.v20240826.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/jetty-server-9.4.56.v20240826.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/jetty-security-9.4.56.v20240826.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/jetty-io-9.4.56.v20240826.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/jetty-http-9.4.56.v20240826.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/javax.servlet-api-3.1.0.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/jackson-databind-2.15.2.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/jackson-core-2.15.2.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/jackson-annotations-2.15.2.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/commons-io-2.17.0.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/commons-cli-1.5.0.jar:/apache-zookeeper-3.9.3-bin/bin/../lib/audience-annotations-0.12.0.jar:/apache-zookeeper-3.9.3-bin/bin/../zookeeper-*.jar:/apache-zookeeper-3.9.3-bin/bin/../zookeeper-server/src/main/resources/lib/*.jar:/conf:
2025-01-27 03:55:46,879 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
2025-01-27 03:55:46,879 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:java.io.tmpdir=/tmp
2025-01-27 03:55:46,879 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:java.compiler=<NA>
2025-01-27 03:55:46,879 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:os.name=Linux
2025-01-27 03:55:46,880 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:os.arch=amd64
2025-01-27 03:55:46,880 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:os.version=4.19.90-2102.2.0.0062.ctl2.x86_64
2025-01-27 03:55:46,880 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:user.name=zookeeper
2025-01-27 03:55:46,880 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:user.home=/home/zookeeper
2025-01-27 03:55:46,880 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:user.dir=/apache-zookeeper-3.9.3-bin
2025-01-27 03:55:46,880 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:os.memory.free=981MB
2025-01-27 03:55:46,880 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:os.memory.max=1000MB
2025-01-27 03:55:46,880 [myid:1] - INFO [main:o.a.z.Environment@98] - Server environment:os.memory.total=1000MB
2025-01-27 03:55:46,880 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@149] - zookeeper.enableEagerACLCheck = false
2025-01-27 03:55:46,880 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@162] - zookeeper.digest.enabled = true
2025-01-27 03:55:46,880 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@166] - zookeeper.closeSessionTxn.enabled = true
2025-01-27 03:55:46,881 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@2242] - zookeeper.serializeLastProcessedZxid.enabled = true
2025-01-27 03:55:46,881 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@1585] - zookeeper.flushDelay = 0 ms
2025-01-27 03:55:46,881 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@1594] - zookeeper.maxWriteQueuePollTime = 0 ms
2025-01-27 03:55:46,881 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@1603] - zookeeper.maxBatchSize=1000
2025-01-27 03:55:46,881 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@287] - zookeeper.intBufferStartingSizeBytes = 1024
2025-01-27 03:55:46,882 [myid:1] - INFO [main:o.a.z.s.BlueThrottle@141] - Weighed connection throttling is disabled
2025-01-27 03:55:46,882 [myid:1] - INFO [main:o.a.z.s.AuthenticationHelper@66] - zookeeper.enforce.auth.enabled = false
2025-01-27 03:55:46,883 [myid:1] - INFO [main:o.a.z.s.AuthenticationHelper@67] - zookeeper.enforce.auth.schemes = []
2025-01-27 03:55:46,883 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@1395] - minSessionTimeout set to 4000 ms
2025-01-27 03:55:46,883 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@1404] - maxSessionTimeout set to 40000 ms
2025-01-27 03:55:46,884 [myid:1] - INFO [main:o.a.z.s.ResponseCache@45] - getData response cache size is initialized with value 400.
2025-01-27 03:55:46,885 [myid:1] - INFO [main:o.a.z.s.ResponseCache@45] - getChildren response cache size is initialized with value 400.
2025-01-27 03:55:46,885 [myid:1] - INFO [main:o.a.z.s.u.RequestPathMetricsCollector@109] - zookeeper.pathStats.slotCapacity = 60
2025-01-27 03:55:46,886 [myid:1] - INFO [main:o.a.z.s.u.RequestPathMetricsCollector@110] - zookeeper.pathStats.slotDuration = 15
2025-01-27 03:55:46,886 [myid:1] - INFO [main:o.a.z.s.u.RequestPathMetricsCollector@111] - zookeeper.pathStats.maxDepth = 6
2025-01-27 03:55:46,886 [myid:1] - INFO [main:o.a.z.s.u.RequestPathMetricsCollector@112] - zookeeper.pathStats.initialDelay = 5
2025-01-27 03:55:46,886 [myid:1] - INFO [main:o.a.z.s.u.RequestPathMetricsCollector@113] - zookeeper.pathStats.delay = 5
2025-01-27 03:55:46,886 [myid:1] - INFO [main:o.a.z.s.u.RequestPathMetricsCollector@114] - zookeeper.pathStats.enabled = false
2025-01-27 03:55:46,888 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@1622] - The max bytes for all large requests are set to 104857600
2025-01-27 03:55:46,888 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@1636] - The large request threshold is set to -1
2025-01-27 03:55:46,888 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@385] - Created server with tickTime 2000 ms minSessionTimeout 4000 ms maxSessionTimeout 40000 ms clientPortListenBacklog -1 dataLogdir /datalog/version-2 snapdir /data/version-2
2025-01-27 03:55:46,905 [myid:1] - INFO [main:o.e.j.u.l.Log@170] - Logging initialized @481ms to org.eclipse.jetty.util.log.Slf4jLog
2025-01-27 03:55:46,961 [myid:1] - WARN [main:o.e.j.s.h.ContextHandler@1662] - o.e.j.s.ServletContextHandler@52af26ee{/,null,STOPPED} contextPath ends with /*
2025-01-27 03:55:46,961 [myid:1] - WARN [main:o.e.j.s.h.ContextHandler@1673] - Empty contextPath
2025-01-27 03:55:46,978 [myid:1] - INFO [main:o.e.j.s.Server@375] - jetty-9.4.56.v20240826; built: 2024-08-26T17:15:05.868Z; git: ec6782ff5ead824dabdcf47fa98f90a4aedff401; jvm 17.0.13+11
2025-01-27 03:55:47,001 [myid:1] - INFO [main:o.e.j.s.s.DefaultSessionIdManager@334] - DefaultSessionIdManager workerName=node0
2025-01-27 03:55:47,001 [myid:1] - INFO [main:o.e.j.s.s.DefaultSessionIdManager@339] - No SessionScavenger set, using defaults
2025-01-27 03:55:47,002 [myid:1] - INFO [main:o.e.j.s.s.HouseKeeper@132] - node0 Scavenging every 660000ms
2025-01-27 03:55:47,006 [myid:1] - WARN [main:o.e.j.s.ConstraintSecurityHandler@759] - ServletContext@o.e.j.s.ServletContextHandler@52af26ee{/,null,STARTING} has uncovered http methods for path: /*
2025-01-27 03:55:47,015 [myid:1] - INFO [main:o.e.j.s.h.ContextHandler@921] - Started o.e.j.s.ServletContextHandler@52af26ee{/,null,AVAILABLE}
2025-01-27 03:55:47,027 [myid:1] - INFO [main:o.e.j.s.AbstractConnector@333] - Started ServerConnector@67080771{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2025-01-27 03:55:47,027 [myid:1] - INFO [main:o.e.j.s.Server@415] - Started @603ms
2025-01-27 03:55:47,027 [myid:1] - INFO [main:o.a.z.s.a.JettyAdminServer@201] - Started AdminServer on address 0.0.0.0, port 8080 and command URL /commands
2025-01-27 03:55:47,031 [myid:1] - INFO [main:o.a.z.s.ServerCnxnFactory@172] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
2025-01-27 03:55:47,032 [myid:1] - WARN [main:o.a.z.s.ServerCnxnFactory@337] - maxCnxns is not configured, using default value 0.
2025-01-27 03:55:47,033 [myid:1] - INFO [main:o.a.z.s.NIOServerCnxnFactory@652] - Configuring NIO connection handler with 10s sessionless connection timeout, 4 selector thread(s), 80 worker threads, and 64 kB direct buffers.
2025-01-27 03:55:47,034 [myid:1] - INFO [main:o.a.z.s.NIOServerCnxnFactory@660] - binding to port /0.0.0.0:2181
2025-01-27 03:55:47,045 [myid:1] - INFO [main:o.a.z.s.w.WatchManagerFactory@42] - Using org.apache.zookeeper.server.watch.WatchManager as watch manager
2025-01-27 03:55:47,046 [myid:1] - INFO [main:o.a.z.s.w.WatchManagerFactory@42] - Using org.apache.zookeeper.server.watch.WatchManager as watch manager
2025-01-27 03:55:47,046 [myid:1] - INFO [main:o.a.z.s.ZKDatabase@134] - zookeeper.snapshotSizeFactor = 0.33
2025-01-27 03:55:47,046 [myid:1] - INFO [main:o.a.z.s.ZKDatabase@154] - zookeeper.commitLogCount=500
2025-01-27 03:55:47,051 [myid:1] - INFO [main:o.a.z.s.p.SnapStream@61] - zookeeper.snapshot.compression.method = CHECKED
2025-01-27 03:55:47,051 [myid:1] - INFO [main:o.a.z.s.p.FileTxnSnapLog@480] - Snapshotting: 0x0 to /data/version-2/snapshot.0
2025-01-27 03:55:47,054 [myid:1] - INFO [main:o.a.z.s.ZKDatabase@291] - Snapshot loaded in 8 ms, highest zxid is 0x0, digest is 1371985504
2025-01-27 03:55:47,055 [myid:1] - INFO [main:o.a.z.s.p.FileTxnSnapLog@480] - Snapshotting: 0x0 to /data/version-2/snapshot.0
2025-01-27 03:55:47,055 [myid:1] - INFO [main:o.a.z.s.ZooKeeperServer@589] - Snapshot taken in 0 ms
2025-01-27 03:55:47,062 [myid:] - INFO [ProcessThread(sid:0 cport:2181)::o.a.z.s.PrepRequestProcessor@138] - PrepRequestProcessor (sid:0) started, reconfigEnabled=false
2025-01-27 03:55:47,062 [myid:1] - INFO [main:o.a.z.s.RequestThrottler@75] - zookeeper.request_throttler.shutdownTimeout = 10000 ms
2025-01-27 03:55:47,076 [myid:1] - INFO [main:o.a.z.s.ContainerManager@83] - Using checkIntervalMs=60000 maxPerMinute=10000 maxNeverUsedIntervalMs=0
2025-01-27 03:55:47,077 [myid:1] - INFO [main:o.a.z.a.ZKAuditProvider@42] - ZooKeeper audit is disabled.
五、k8s方式安装
k8s环境容器化Kraft Mode（模式）安装
1、容器启动Kafka
- 创建ConfigMap资源
用于定义Kafka配置
apiVersion: v1
kind: ConfigMap
metadata:
name: kafka-config
data:
config.properties: |
process.roles=broker
node.id=1
listeners=PLAINTEXT://:9092
inter.broker.listener.name=PLAINTEXT
...- 创建Deployment资源
用于部署Kafka Pod
apiVersion: apps/v1
kind: Deployment
metadata:
name: kafka-deployment
spec:
replicas: 3
selector:
matchLabels:
app: kafka
template:
metadata:
labels:
app: kafka
spec:
containers:
- name: kafka
image: confluentinc/cp-kafka:7.0.1
ports:
- containerPort: 9092
env:
- name: CONFIG_FILE
value: "/etc/kafka/conf/config.properties"
...
volumeMounts:
- name: config-volume
mountPath: /etc/kafka/conf
volumes:
- name: config-volume
configMap:
name: kafka-config- 创建Service资源
用于对外暴露Kafka服务
apiVersion: v1
kind: Service
metadata:
name: kafka-service
spec:
selector:
app: kafka
ports:
- protocol: TCP
port: 9092
targetPort: 9092- 应用到你的Kubernetes集群
这些YAML文件定义了Kafka的配置、部署和服务。你需要将它们保存为文件，然后使用kubectl 应用到你的Kubernetes集群
kubectl apply -f configmap.yaml
kubectl apply -f deployment.yaml
kubectl apply -f service.yaml- （或）使用镜像启动容器
# 假设打好的镜像id是fe7e6a65afa5
docker run \
--name=cp-kafka \
-p 2181:2181 \
-p 9092:9092 \
-p 19092:19092 \
-d fe7e6a65afa5六、要点总结
- Zookeeper Mode使用的配置是$KFAKA_HOME/config/server.properties
- Kraft Mode使用的配置是$KFAKA_HOME/config/kraft/server.properties
- Zookeeper Mode启动见作者的文章：Zookeeper模式安装Kafka