通过配置IP路由解决多网卡配置同网段IP的问题

1、环境

硬件：X86服务器一台，含四个以太网口，4个网口都接入一台交换机
操作系统：ubuntu 2204
需求：在一台服务器部署了四个服务分别为apache、git、nexus和harbor，并且都采用了80端口，通过多网卡配置同网段IP地址来使用。实际生产环境不建议采用该模式，可以采用容器化部署，不同服务采用不同的端口。
接口、IP地址和服务规划如下：

2、系统配置

编辑/etc/sysctl.conf

vim /etc/sysctl.conf

net.ipv4.ip_forward=1 
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.eno1.arp_announce = 2
net.ipv4.conf.eno1.arp_ignore = 1
net.ipv4.conf.eno2.arp_announce = 2
net.ipv4.conf.eno2.arp_ignore = 1
net.ipv4.conf.eno3.arp_announce = 2
net.ipv4.conf.eno3.arp_ignore = 1
net.ipv4.conf.eno4.arp_announce = 2
net.ipv4.conf.eno4.arp_ignore = 1

执行命令

sysctl -p

3、路由配置

3.1、解决思路

给每个网卡分配单独的路由表，并且通过 ip rule 来指定

3.2、添加路由表

添加路由表/etc/iproute2/rt_tables

echo "11 net11" >> /etc/iproute2/rt_tables
echo "12 net12" >> /etc/iproute2/rt_tables
echo "13 net13" >> /etc/iproute2/rt_tables
echo "14 net14" >> /etc/iproute2/rt_tables

3.3、添加路由规则

编辑/etc/rc.local，添加如下

ip route add 192.168.10.0/24 dev eno1 proto kernel scope link src 192.168.10.11 table net11
ip route add default via 192.168.10.254 dev eno1 table net11
ip rule add from 192.168.10.11 table net11

ip route add 192.168.10.0/24 dev eno2 proto kernel scope link src 192.168.10.12 table net12
ip route add default via 192.168.10.254 dev eno2 table net12
ip rule add from 192.168.10.12 table net12

ip route add 192.168.10.0/24 dev eno3 proto kernel scope link src 192.168.10.13 table net13
ip route add default via 192.168.10.254 dev eno3 table net13
ip rule add from 192.168.10.13 table net13

ip route add 192.168.10.0/24 dev eno4 proto kernel scope link src 192.168.10.14 table net14
ip route add default via 192.168.10.254 dev eno4 table net14
ip rule add from 192.168.10.14 table net14

# 把新增加的路由策略和路由表刷新到缓存中，即时生效
ip route flush cache

给/etc/rc.local添加执行权限

chmod +x /etc/rc.local

注意：若/etc/rc.local为链接文件，将权限给到原文件，比如是/etc/rc.d/rc.local。

3.3、重启

以上配置过程均完成后将系统重启reboot

3.4、测试

通过ip route查看路由信息
通过ip rule查看路由策略
通过同网段的服务器，使用arping 192.168.10.11/12/13/14，确认返回对应网卡的mac地址正确即可

4、参考文献

https://www.zhihu.com/question/41331151/answer/169574975
https://lartc.org/howto/
https://www.cnblogs.com/stellar-liyanbin/p/18190079
https://blog.csdn.net/blade2001/article/details/46271449
https://blog.csdn.net/weixin_46386047/article/details/126058461
https://www.cnblogs.com/dissipate/p/13741595.html