Zookeeper集群部署

Zookeeper简介

Zookeeper是一个开源的分布式协调服务，由知名互联网公司Yahoo创建，它是Chubby的开源实现；换句话讲，Zookeeper是一个典型的分布式数据一致性解决方案，分布式应用程序可以基于它实现数据的发布/订阅、负载均衡、名称服务、分布式协调/通知、集群管理、Master选举、分布式锁和分布式队列。

官方网站：http://zookeeper.apache.org

部署zookeeper集群

三个ZooKeeper服务器是整体的最小建议大小。

集群节点规划

配置主机名

hostnamectl set-hostname zookeeper01
hostnamectl set-hostname zookeeper02
hostnamectl set-hostname zookeeper03

配置hosts解析

cat > /etc/hosts <<EOF
192.168.92.56 zookeeper01
192.168.92.57 zookeeper02
192.168.92.58 zookeeper03
EOF

安装java依赖

yum install -y epel-release
yum install -y java-11-openjdk-devel

创建相关目录

mkdir -p /opt/zookeeper/{data,logs}

安装zookeeper

wget https://mirrors.aliyun.com/apache/zookeeper/zookeeper-3.7.0/apache-zookeeper-3.7.0-bin.tar.gz -P /tmp
tar -zxvf /tmp/apache-zookeeper-*-bin.tar.gz -C /opt/zookeeper --strip=1

配置环境变量

cat > /etc/profile.d/zookeeper.sh <<'EOF'
export ZOOKEEPER_HOME=/opt/zookeeper
export PATH=$ZOOKEEPER_HOME/bin:$PATH
EOF

source /etc/profile

复制zookeeper配置文件

cp /opt/zookeeper/conf/{zoo_sample.cfg,zoo.cfg}

修改zookeeper配置文件

cat > /opt/zookeeper/conf/zoo.cfg <<EOF
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/opt/zookeeper/data
dataLogDir=/opt/zookeeper/logs
clientPort=2181
server.1=zookeeper01:2888:3888
server.2=zookeeper02:2888:3888
server.3=zookeeper03:2888:3888
EOF

创建myid文件，id在整体中必须是唯一的，并且应该具有1到255之间的值，主配置文件中的server id 要和其当前主节点中的myid保持一致。

[root@zookeeper01 ~]# echo '1' >/opt/zookeeper/data/myid
[root@zookeeper02 ~]# echo '2' >/opt/zookeeper/data/myid
[root@zookeeper03 ~]# echo '3' >/opt/zookeeper/data/myid

创建zookeeper用户

useradd -r -s /bin/false zookeeper
chown -R zookeeper: /opt/zookeeper

使用systemd管理zookeeper服务

cat > /usr/lib/systemd/system/zookeeper.service << EOF
[Unit]
Description=Zookeeper Service

[Service]
Type=simple
WorkingDirectory=/opt/zookeeper/
PIDFile=/opt/zookeeper/data/zookeeper_server.pid
SyslogIdentifier=zookeeper
User=zookeeper
Group=zookeeper
ExecStart=/opt/zookeeper/bin/zkServer.sh start
ExecStop=/opt/zookeeper/bin/zkServer.sh stop
Restart=always
TimeoutSec=20
SuccessExitStatus=130 143
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

启动zookeeper服务并配置为开机启动

systemctl enable --now zookeeper

查看zookeeper服务运行状态

[root@zookeeper01 ~]# systemctl status zookeeper
● zookeeper.service - Zookeeper Service
   Loaded: loaded (/usr/lib/systemd/system/zookeeper.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2021-05-06 22:15:17 CST; 4s ago
  Process: 5630 ExecStop=/opt/zookeeper/bin/zkServer.sh stop (code=exited, status=0/SUCCESS)
 Main PID: 5668 (java)
   CGroup: /system.slice/zookeeper.service
           └─5668 java -Dzookeeper.log.dir=/opt/zookeeper/bin/../logs -Dzookeeper.log.file=zookeeper-zookeeper-server-hadoop03.log -Dzookeeper.root.logger=INFO,CONSOLE -XX:+HeapDumpOnOutOfMemoryError -...

May 06 22:15:17 hadoop03 systemd[1]: Stopped Zookeeper Service.
May 06 22:15:17 hadoop03 systemd[1]: Started Zookeeper Service.
May 06 22:15:17 hadoop03 zookeeper[5652]: /usr/bin/java
May 06 22:15:17 hadoop03 zookeeper[5652]: ZooKeeper JMX enabled by default
May 06 22:15:17 hadoop03 zookeeper[5652]: Using config: /opt/zookeeper/bin/../conf/zoo.cfg

查看zookeeper主备状态

zkServer.sh status

查看端口监听状态，只有leader节点在监听2888端口：

[root@zookeeper02 ~]# ss -antlp | grep 888
LISTEN     0      50        ::ffff:192.168.92.57:3888                    :::*                   users:(("java",pid=2481,fd=56))
LISTEN     0      50        ::ffff:192.168.92.57:2888                    :::*                   users:(("java",pid=2481,fd=58))

[root@zookeeper01 ~]# ss -antlp | grep 888
LISTEN     0      50        ::ffff:192.168.92.56:3888                    :::*                   users:(("java",pid=2554,fd=56))

关于故障转移：主节点关闭后，会选举新的主节点，原主节点启动后会作为从节点，在现有集群中三个节点的情况下，关闭两个节点后，第三个节点的状态会出现异常，启动一个节点（两个节点）状态又可以恢复。

[root@zookeeper02 ~]# zkServer.sh restart
[root@zookeeper02 ~]# zkServer.sh status

在集群任意一个节点使用zkCli.sh即可连接当前节点的2181端口；连接以后会有一个zk的控制台：

zkCli.sh -server zookeeper01:2181

zkCli.sh -server zookeeper01:2181,zookeeper02:2181,zookeeper03:2181

查看命令帮助

[zk: zookeeper01:2181(CONNECTED) 0] ?

查看集群成员配置信息

[zk: zookeeper01:2181(CONNECTED) 16] config
server.1=zookeeper01:2888:3888:participant
server.2=zookeeper02:2888:3888:participant
server.3=zookeeper03:2888:3888:participant
version=0

docker部署zookeeper集群

节点规划

配置主机名

hostnamectl set-hostname zookeeper01
hostnamectl set-hostname zookeeper02
hostnamectl set-hostname zookeeper03

配置主机名解析

cat > /etc/hosts <<EOF
192.168.92.80 zookeeper01
192.168.92.81 zookeeper02
192.168.92.82 zookeeper03
EOF

部署集群，3个节点分别执行以下命令，三台服务器的ZOO_MY_ID不一样，分别为1、2、3，其他配置参数相同：

docker run -d --name zookeeper \
    --restart always \
    --net=host \
    -v /data/zookeeper/data:/data \
    -v /data/zookeeper/datalog:/datalog \
    -v /data/zookeeper/logs:/logs \
    -e TZ=Asia/Shanghai \
    -e ZOO_LOG4J_PROP="INFO,ROLLINGFILE" \
    -e ZOO_MY_ID=1 \
    -e ZOO_SERVERS="server.1=zookeeper01:2888:3888;2181 server.2=zookeeper02:2888:3888;2181 server.3=zookeeper03:2888:3888;2181" \
    zookeeper:3.7.0

查看zookeeper运行状态

[root@zookeeper02  ~]# docker exec -it zookeeper zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leader