Flink Docker仓库包含了Dockerfiles用于为Flink构建docker images使用，这些 Dockerfile 由 Apache Flink 社区维护，但 Docker 社区负责在 Docker Hub 上构建和托管映像。目前市面上流行的Flink镜像都是基于Ubuntu镜像构建，由于项目需求变化，需要在容器中进行HDFS文件挂载（京东云海存储，适配了HDFS，目前驱动适配CentOS8）我们需要在CentOS8的镜像中重构Flink的官方镜像，一下示例按照Flink-1.19.0版本镜像制作过程示例如下：
CentOS8 镜像
FROM centos:centos8
# Install dependencies
RUN sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
RUN sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
RUN set -ex; \
dnf update -y; \
dnf install -y epel-release; \
dnf install -y gpg snappy gettext-devel jemalloc wget java-1.8.0-openjdk bind-utils; \
yum clean all;
ENV GOSU_VERSION 1.17
ENV ARCH=arm64
RUN set -ex; \
wget -nv -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$ARCH"; \
wget -nv -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$ARCH.asc"; \
export GNUPGHOME="$(mktemp -d)"; \
for server in ha.pool.sks-keyservers.net $(shuf -e \
hkp://p80.pool.sks-keyservers.net:80 \
keyserver.ubuntu.com \
hkp://keyserver.ubuntu.com:80 \
pgp.mit.edu) ; do \
gpg --batch --keyserver "$server" --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 && break || : ; \
done && \
gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
gpgconf --kill all; \
rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
chmod +x /usr/local/bin/gosu; \
gosu nobody true
# Configure Flink version
ENV FLINK_TGZ_URL=https://www.apache.org/dyn/closer.cgi?action=download&filename=flink/flink-1.19.0/flink-1.19.0-bin-scala_2.12.tgz \
FLINK_ASC_URL=https://downloads.apache.org/flink/flink-1.19.0/flink-1.19.0-bin-scala_2.12.tgz.asc \
GPG_KEY=028B6605F51BC296B56A5042E57D30ABEE75CA06 \
CHECK_GPG=true
# Prepare environment
ENV FLINK_HOME=/opt/flink
ENV PATH=$FLINK_HOME/bin:$PATH
RUN groupadd --system --gid=9999 flink && \
useradd --system --home-dir $FLINK_HOME --uid=9999 --gid=flink flink
WORKDIR $FLINK_HOME
# Install Flink
RUN set -ex; \
wget -nv -O flink.tgz "$FLINK_TGZ_URL"; \
\
if [ "$CHECK_GPG" = "true" ]; then \
wget -nv -O flink.tgz.asc "$FLINK_ASC_URL"; \
export GNUPGHOME="$(mktemp -d)"; \
for server in ha.pool.sks-keyservers.net $(shuf -e \
hkp://p80.pool.sks-keyservers.net:80 \
keyserver.ubuntu.com \
hkp://keyserver.ubuntu.com:80 \
pgp.mit.edu) ; do \
gpg --batch --keyserver "$server" --recv-keys "$GPG_KEY" && break || : ; \
done && \
gpg --batch --verify flink.tgz.asc flink.tgz; \
gpgconf --kill all; \
rm -rf "$GNUPGHOME" flink.tgz.asc; \
fi; \
\
tar -xf flink.tgz --strip-components=1; \
rm flink.tgz; \
\
chown -R flink:flink .; \
\
# Replace default REST/RPC endpoint bind address to use the container's network interface \
CONF_FILE="$FLINK_HOME/conf/flink-conf.yaml"; \
if [ ! -e "$FLINK_HOME/conf/flink-conf.yaml" ]; then \
CONF_FILE="${FLINK_HOME}/conf/config.yaml"; \
/bin/bash "$FLINK_HOME/bin/config-parser-utils.sh" "${FLINK_HOME}/conf" "${FLINK_HOME}/bin" "${FLINK_HOME}/lib" \
"-repKV" "rest.address,localhost,0.0.0.0" \
"-repKV" "rest.bind-address,localhost,0.0.0.0" \
"-repKV" "jobmanager.bind-host,localhost,0.0.0.0" \
"-repKV" "taskmanager.bind-host,localhost,0.0.0.0" \
"-rmKV" "taskmanager.host=localhost"; \
else \
sed -i 's/rest.address: localhost/rest.address: 0.0.0.0/g' "$CONF_FILE"; \
sed -i 's/rest.bind-address: localhost/rest.bind-address: 0.0.0.0/g' "$CONF_FILE"; \
sed -i 's/jobmanager.bind-host: localhost/jobmanager.bind-host: 0.0.0.0/g' "$CONF_FILE"; \
sed -i 's/taskmanager.bind-host: localhost/taskmanager.bind-host: 0.0.0.0/g' "$CONF_FILE"; \
sed -i '/taskmanager.host: localhost/d' "$CONF_FILE"; \
fi;
# Configure container
COPY docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]
EXPOSE 6123 8081
CMD ["help"]
需要注意小编的电脑是Apple M1 Pro,因此在进行镜像编译的时候，需要将ARCH环境变量修改为arm64，如果是Intel芯片的x86架构，请将ARCH修改为amd64即可,修改结束后可以执行一下命令实现Flink镜像打包等工作：
jiangzhongzhou@ZBMac-C02CW08SM scala_2.12-java8-centos % docker build -t flink:1.19-scala_2.12-centos8-java8 .
[+] Building 311.6s (14/14) FINISHED docker:desktop-linux
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 4.89kB 0.0s
=> [internal] load metadata for docker.io/library/centos:centos8 15.3s
=> [1/9] FROM docker.io/library/centos:centos8@sha256:a27fd8080b517143cbbbab9dfb7c8571c40d67d534bbdee55bd6c473f432b177 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 42B 0.0s
=> CACHED [2/9] RUN sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-* 0.0s
=> CACHED [3/9] RUN sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-* 0.0s
=> CACHED [4/9] RUN set -ex; dnf install -y epel-release; dnf update -y; dnf install -y gpg snappy gettext-devel jemalloc wget java-1.8.0-openjdk bind-utils; yum clean all; 0.0s
=> [5/9] RUN set -ex; wget -nv -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/1.17/gosu-arm64"; wget -nv -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/1.17/gosu-arm64.asc"; export GNUPGHOME="$(mktemp -d)"; for server in ha.pool.sks-key 18.9s
=> [6/9] RUN groupadd --system --gid=9999 flink && useradd --system --home-dir /opt/flink --uid=9999 --gid=flink flink 0.2s
=> [7/9] WORKDIR /opt/flink 0.0s
=> [8/9] RUN set -ex; wget -nv -O flink.tgz "https://www.apache.org/dyn/closer.cgi?action=download&filename=flink/flink-1.19.0/flink-1.19.0-bin-scala_2.12.tgz"; if [ "true" = "true" ]; then wget -nv -O flink.tgz.asc "https://downloads.apache.org/flink/flink-1.19.0/flink-1.19.0-bin-scala_2 276.1s
=> [9/9] COPY docker-entrypoint.sh / 0.0s
=> exporting to image 0.9s
=> => exporting layers 0.9s
=> => writing image sha256:ea57baecdc1d1fa59ac697062c4a59b3d0ed3abfc698b0f4ecfa4f6692009810 0.0s
=> => naming to docker.io/library/flink:1.19-scala_2.12-centos8-java8 0.0s
What's Next?
View a summary of image vulnerabilities and recommendations → docker scout quickview
Rocky 镜像
FROM arm64v8/rockylinux:9.3
# Install dependencies
RUN set -ex; \
dnf update -y; \
dnf install -y epel-release; \
dnf install -y gpg snappy gettext-devel jemalloc wget java-1.8.0-openjdk bind-utils; \
dnf clean all;
ENV GOSU_VERSION 1.17
ENV ARCH=arm64
RUN set -ex; \
wget -nv -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$ARCH"; \
wget -nv -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$ARCH.asc"; \
export GNUPGHOME="$(mktemp -d)"; \
for server in ha.pool.sks-keyservers.net $(shuf -e \
hkp://p80.pool.sks-keyservers.net:80 \
keyserver.ubuntu.com \
hkp://keyserver.ubuntu.com:80 \
pgp.mit.edu) ; do \
gpg --batch --keyserver "$server" --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 && break || : ; \
done && \
gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
gpgconf --kill all; \
rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
chmod +x /usr/local/bin/gosu; \
gosu nobody true
# Configure Flink version
ENV FLINK_TGZ_URL=https://www.apache.org/dyn/closer.cgi?action=download&filename=flink/flink-1.19.0/flink-1.19.0-bin-scala_2.12.tgz \
FLINK_ASC_URL=https://downloads.apache.org/flink/flink-1.19.0/flink-1.19.0-bin-scala_2.12.tgz.asc \
GPG_KEY=028B6605F51BC296B56A5042E57D30ABEE75CA06 \
CHECK_GPG=true
# Prepare environment
ENV FLINK_HOME=/opt/flink
ENV PATH=$FLINK_HOME/bin:$PATH
RUN groupadd --system --gid=9999 flink && \
useradd --system --home-dir $FLINK_HOME --uid=9999 --gid=flink flink
WORKDIR $FLINK_HOME
# Install Flink
RUN set -ex; \
wget -nv -O flink.tgz "$FLINK_TGZ_URL"; \
\
if [ "$CHECK_GPG" = "true" ]; then \
wget -nv -O flink.tgz.asc "$FLINK_ASC_URL"; \
export GNUPGHOME="$(mktemp -d)"; \
for server in ha.pool.sks-keyservers.net $(shuf -e \
hkp://p80.pool.sks-keyservers.net:80 \
keyserver.ubuntu.com \
hkp://keyserver.ubuntu.com:80 \
pgp.mit.edu) ; do \
gpg --batch --keyserver "$server" --recv-keys "$GPG_KEY" && break || : ; \
done && \
gpg --batch --verify flink.tgz.asc flink.tgz; \
gpgconf --kill all; \
rm -rf "$GNUPGHOME" flink.tgz.asc; \
fi; \
\
tar -xf flink.tgz --strip-components=1; \
rm flink.tgz; \
\
chown -R flink:flink .; \
\
# Replace default REST/RPC endpoint bind address to use the container's network interface \
CONF_FILE="$FLINK_HOME/conf/flink-conf.yaml"; \
if [ ! -e "$FLINK_HOME/conf/flink-conf.yaml" ]; then \
CONF_FILE="${FLINK_HOME}/conf/config.yaml"; \
/bin/bash "$FLINK_HOME/bin/config-parser-utils.sh" "${FLINK_HOME}/conf" "${FLINK_HOME}/bin" "${FLINK_HOME}/lib" \
"-repKV" "rest.address,localhost,0.0.0.0" \
"-repKV" "rest.bind-address,localhost,0.0.0.0" \
"-repKV" "jobmanager.bind-host,localhost,0.0.0.0" \
"-repKV" "taskmanager.bind-host,localhost,0.0.0.0" \
"-rmKV" "taskmanager.host=localhost"; \
else \
sed -i 's/rest.address: localhost/rest.address: 0.0.0.0/g' "$CONF_FILE"; \
sed -i 's/rest.bind-address: localhost/rest.bind-address: 0.0.0.0/g' "$CONF_FILE"; \
sed -i 's/jobmanager.bind-host: localhost/jobmanager.bind-host: 0.0.0.0/g' "$CONF_FILE"; \
sed -i 's/taskmanager.bind-host: localhost/taskmanager.bind-host: 0.0.0.0/g' "$CONF_FILE"; \
sed -i '/taskmanager.host: localhost/d' "$CONF_FILE"; \
fi;
# Configure container
COPY docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]
EXPOSE 6123 8081
CMD ["help"]
需要注意小编的电脑是Apple M1 Pro,因此在进行镜像编译的时候，需要将ARCH环境变量修改为arm64，如果是Intel芯片的x86架构，请将ARCH修改为amd64即可,修改结束后可以执行一下命令实现Flink镜像打包等工作：
jiangzhongzhou@ZBMac-C02CW08SM scala_2.12-java8-rocky % docker build -t flink:1.19-scala_2.12-rocky-java8 .
[+] Building 297.9s (12/12) FINISHED docker:desktop-linux
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 4.72kB 0.0s
=> [internal] load metadata for docker.io/arm64v8/rockylinux:9.3 0.0s
=> [1/7] FROM docker.io/arm64v8/rockylinux:9.3 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 42B 0.0s
=> CACHED [2/7] RUN set -ex; dnf update -y; dnf install -y epel-release; dnf install -y gpg snappy gettext-devel jemalloc wget java-1.8.0-openjdk bind-utils; dnf clean all; 0.0s
=> [3/7] RUN set -ex; wget -nv -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/1.17/gosu-arm64"; wget -nv -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/1.17/gosu-arm64.asc"; export GNUPGHOME="$(mktemp -d)"; for server in ha.pool.sks-key 16.0s
=> [4/7] RUN groupadd --system --gid=9999 flink && useradd --system --home-dir /opt/flink --uid=9999 --gid=flink flink 0.3s
=> [5/7] WORKDIR /opt/flink 0.0s
=> [6/7] RUN set -ex; wget -nv -O flink.tgz "https://www.apache.org/dyn/closer.cgi?action=download&filename=flink/flink-1.19.0/flink-1.19.0-bin-scala_2.12.tgz"; if [ "true" = "true" ]; then wget -nv -O flink.tgz.asc "https://downloads.apache.org/flink/flink-1.19.0/flink-1.19.0-bin-scala_2 277.3s
=> [7/7] COPY docker-entrypoint.sh / 0.0s
=> exporting to image 4.1s
=> => exporting layers 4.1s
=> => writing image sha256:eb38aa8f37e1a920e46efd4792d5803fed9ba95698e3d465ed365f98b67cad30 0.0s
=> => naming to docker.io/library/flink:1.19-scala_2.12-rocky-java8 0.0s
What's Next?
View a summary of image vulnerabilities and recommendations → docker scout quickview