Kolla-ansible自动化部署openstack

kolla ansible简介

kolla 的使命是为 openstack 云平台提供生产级别的、开箱即用的交付能力。kolla 的基本思想是一切皆容器，将所有服务基于 Docker 运行，并且保证一个容器只跑一个服务（进程），做到最小粒度的运行 docker。

kolla 要实现 openetack 部署总体上分为两步，第一步是制作 docker 镜像，第二步是编排部署。因此，kolla 项目又被分为两个小项目：kolla、kolla-ansible 。

一、准备工作（模拟all-in-one部署）

1、配置好网卡IP（至少2张网卡）

vm模拟环境（1张nat+1张桥接网卡）

nat网卡(ens32):192.168.108.10

桥接网卡(ens33)：10.51.40.211

2、修改主机名

hostnamectl set-hostname+主机名

3、关闭防火墙、NM服务、selinux

systemctl stop firewalld && systemctl disable firewalld 
systemctl stop NetworkManager && systemctl disable NetworkManager

setenforce 0
vi /etc/sysconfig/selinux

4、把节点IP写入hosts

vi /etc/hosts

5、做主机信任

ssh-keygen -t rsa(一直按回车)

​ ssh-copy-id kolla

6、修改DNS

vi /etc/resolv.conf
nameserver 114.114.114.114

二、环境准备

（1）部署节点

1、安装使用工具

yum -y  install vim wget net-tools 

2、更新base源和下载epel源

wget  -O  /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo

3、安装python环境相关依赖包

yum -y install python-devel libffi-devel gcc openssl-devel libselinux-python

4、安装python-pip

yum -y install python-pip

5、配置国内的pip源

mkdir ~/.pip

cat << EOF > ~/.pip/pip.conf
[global]
index-url = https://mirrors.aliyun.com/pypi/simple/

[install]
trusted-host=mirrors.aliyun.com
EOF

6、升级pip

 pip install --upgrade pip==20.3.4

7、安装ansible

pip install ansible==2.9.0

8、修改ansible配置文件

mkdir /etc/ansible/

cat << EOF > /etc/ansible/ansible.cfg
[defaults]
host_key_checking=False

pipelining=True
forks=100
EOF

（2）所有节点都做

1、安装python-pip

​ yum -y install python-pip

2、配置国内的pip源

mkdir ~/.pip

cat << EOF > ~/.pip/pip.conf
[global]
index-url = https://mirrors.aliyun.com/pypi/simple/

[install]
trusted-host=mirrors.aliyun.com
EOF

3、升级pip

​ pip install --upgrade pip==20.3.4

4、安装必要的部署工具

yum install -y yum-utils device-mapper-persistent-data lvm2
yum -y install nfs-utils

5、添加软件源信息

yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
sudo sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo

6、更新并安装Docker-CE

yum makecache fast
yum -y install docker-ce-20.10.5
systemctl enable docker && systemctl restart docker && systemctl status docker

7、配置docker共享挂载

所谓共享挂载即同一个目录或设备可以挂载到多个不同的路径并且能够保持互相之间的共享可见性，类似于 mount --shared。在
OpenStack for Kolla 中，主要解决 Neutron 的 namespace 在不同 container
中得以保持实效性的问题。

mkdir -p /etc/systemd/system/docker.service.d

cat << EOF > /etc/systemd/system/docker.service.d/kolla.conf
[Service]
MountFlags=shared
EOF

systemctl daemon-reload && systemctl restart docker && systemctl status docker

8、配置docker镜像加速器

mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://6ijb8ubo.mirror.aliyuncs.com"],
  "registry-mirrors": ["https://25ieqbb3.mirror.aliyuncs.com"]
}
EOF

sudo systemctl daemon-reload && systemctl restart docker

三、部署openstack

部署节点

1、安装kolla-ansible

 pip install kolla-ansible==9.3.0 --ignore-installed PyYAML

注：官方推荐部署环境使用 pip install kolla-ansible 方式来安装 kolla-ansible

https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html #官方安装手册

以上安装出错解决方法

pip2 install --upgrade setuptools
如果还出错，就安装以下软件
pip2 install --upgrade pip
pip2 install django-import-export

2、复制kolla-ansible相关配置文件

cp -r /usr/share/kolla-ansible/etc_examples/kolla /etc/
cp /usr/share/kolla-ansible/ansible/inventory/multinode /etc/kolla/

ls /etc/kolla/

实现效果：

ls /etc/kolla
globals.yml  multinode  passwords.yml

3、自动生成各服务的密码文件

kolla-genpwd

修改keystone和数据库的密码(163行)

vi /etc/kolla/passwords.yml
163 keystone_admin_password: HsPbEQHxTqmewKYNoRPpIOyQNdEYpHy36OX67TG3
164 keystone_database_password：7h1UDflKQmD1c6hQ9q7oyDdKfZTFlNAQYuXz148C
改为：
keystone_admin_password: 123456
keystone_database_password: 123456

注：这是登录 Dashboard，admin 使用的密码，你可以根据自己需要进行修改。密码前注意空格

4、修改全局配置文件globals.yml

vi /etc/kolla/gloabals.yml

kolla_base_distro: "centos"
kolla_install_type: "binary"
openstack_release: "train"
node_custom_config: "/etc/kolla/config"
kolla_internal_vip_address: "10.51.13.140"--->不存在主机的IP地址
docker_namespace: "kolla"
network_interface: "eth0"--->管理网络出口网卡
api_interface: "{{ network_interface }}"
neutron_external_interface: "eth1"--->外部网络网卡
neutron_plugin_agent: "openvswitch"
enable_haproxy: "yes"
enable_ceph: "no"
enable_chrony: "no"
enable_cinder: "yes"
enable_grafana: "yes"
enable_heat: "no"
enable_mariabackup: "yes"
enable_neutron_dvr: "yes"
enable_neutron_fwaas: "yes"
enable_neutron_qos: "yes"
enable_neutron_provider_networks: "yes"
glance_backend_ceph: "no"
glance_enable_rolling_upgrade: "no"
cinder_backend_ceph: "no"
cinder_backup_driver: "nfs"
nova_backend_ceph: "no"
prometheus_cmdline_extras: "-storage.local.retention 720h"
nova_console: "novnc"
nova_compute_virt_type: "kvm"
enable_cinder_backup: "yes"
enable_cinder_backend_nfs: "yes"
enable_ceilometer: "yes"
enable_gnocchi: "yes"
enable_grafana: "yes"
enable_ceilometer_ipmi: "yes"
enable_horizon_zun: "{{ enable_zun | bool }}"
enable_zun: "yes"
enable_opendaylight: "yes"
enable_opendaylight_qos: "yes"
enable_opendaylight_l3: "yes"
enable_horizon_fwaas: "{{ enable_neutron_fwaas | bool }}"
enable_kuryr: "yes"
kolla_dev_mode: "yes"
heat_dev_mode: "yes"

5、修改multinode文件

修改为所有节点的主机名或IP地址，根据hosts来

我的主机名为：kolla

# These initial groups are the only groups required to be modified. The
# additional groups are for more control of the environment.
[control]
# These hostname must be resolvable from your deployment host
kolla

# The above can also be specified as follows:
#control[01:03]     ansible_user=kolla

# The network nodes are where your l3-agent and loadbalancers will run
# This can be the same as a host in the control group
[network]
kolla

[compute]
kolla

[monitoring]
kolla

# When compute nodes and control nodes use different interfaces,
# you need to comment out "api_interface" and other interfaces from the globals.yml
# and specify like below:
#compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1

[storage]
kolla

[deployment]
localhost       ansible_connection=local

[baremetal:children]
control
network
compute
storage
monitoring
........

注：如果主机数较多可以使用正则表达式；例如：

[openstack-controller]
controller[01:03]

6、检查

（1）检查multinode配置是否正确，实现ping通各个主机

ansible -i /etc/kolla/multinode all -m ping

无报错显示

（2）部署检查

kolla-ansible -i /etc/kolla/multinode bootstrap-servers

无报错显示结果

kolla-ansible -i /etc/kolla/multinode prechecks

无报错显示结果

7、做nfs与cinder-volumes的卷组对接

1、创建好磁盘挂载目录：mkdir -p /data/openstack_data/cinder
2、把磁盘挂载到目录上：mount /dev/sdb /data
3、创建对接cinder磁盘的文件：mkdir -p /etc/kolla/config
   vi nfs_shares
   192.168.108.10:/data/openstack_data/cinder

8、安装openstack客户端

先安装版本不匹配的组件
pip install -I ipaddress

安装openstack客户端
pip install python-openstackclient

9、执行部署

kolla-ansible -i /etc/kolla/multinode deploy

无报错显示结果

10、生成openrc文件，该文件中设置了管理员用户的凭据

kolla-ansible post-deploy

cat /etc/kolla/admin-openrc.sh

11、访问openstack horizon需要使用vip地址，节点上可以看到由keepalived容器生成的vip

浏览器直接访问该地址即可登录到dashboard界面

http:vip地址

登录账号：admin

密码：123456(根据passwords.yml中的keystone_admin_password: 123456)

以上就是实现了kolla-ansible自动部署Openstack
下面为待补充对网络、安全组、镜像、磁盘的内容

12、定义init-runonce示例脚本外部网络配置

vim /usr/share/kolla-ansible/init-runonce

# 与外部连接网络 相关设置
ENABLE_EXT_NET=${ENABLE_EXT_NET:-1}
EXT_NET_CIDR=${EXT_NET_CIDR:-'192.168.199.0/24'}
EXT_NET_RANGE=${EXT_NET_RANGE:-'start=192.168.199.20,end=192.168.199.79'}
EXT_NET_GATEWAY=${EXT_NET_GATEWAY:-'192.168.199.1'}

参数说明：

EXT_NET_CIDR 指定外部网络，根据自己的网络模式，决定网段，即第二个网卡所在外网网段。

EXT_NET_RANGE 指定从外部网络取出一个地址范围，作为外部网络的地址池。

EXT_NET_GATEWAY 外部网络网关，这里与自己实际网络使用的网关一致。

去除 cirros镜像下载相关(下载速度特别慢)

注释掉与镜像下载相关
echo Checking for locally available cirros image.
# Let's first try to see if the image is available locally
# nodepool nodes caches them in $IMAGE_PATH
#if ! [ -f "${IMAGE_PATH}/${IMAGE}" ]; then
#    IMAGE_PATH='./'
#    if ! [ -f "${IMAGE_PATH}/${IMAGE}" ]; then
#        echo None found, downloading cirros image.
#        curl -L -o ${IMAGE_PATH}/${IMAGE} ${IMAGE_URL}/${IMAGE}
#    fi
#else
#    echo Using cached cirros image from the nodepool node.
#fi

#echo Creating glance image.
#openstack image create --disk-format qcow2 --container-format bare --public \
#    --property os_type=${IMAGE_TYPE} --file ${IMAGE_PATH}/${IMAGE} ${IMAGE_NAME}

E}" ]; then

IMAGE_PATH=‘./’

if ! [ -f “$IMAG{E}_{P}ATH/${IMAGE}” ]; then

echo None found, downloading cirros image.

curl -L -o $IMAG{E}_{P}ATH/${IMAGE} $IMAG{E}_{U}RL/${IMAGE}

fi

#else

echo Using cached cirros image from the nodepool node.

#fi

#echo Creating glance image.
#openstack image create --disk-format qcow2 --container-format bare --public \

–property os_type=${IMAGE_TYPE} --file $IMAG{E}_{P}ATH/${IMAGE} ${IMAGE_NAME}