< 自用文 OS 有关 > (续)发现正在被攻击 后的自救 Fail2ban + IPset + UFW 工作流程详解

发布于:2025-09-05 ⋅ 阅读:(22) ⋅ 点赞:(0)

继上编:< 自用文 主机 USC 记录:> 发现正在被攻击 后的自救-CSDN博客

环境:

改进:

以下是把代码,懒得写,扔给了 AI ,让它出的:

Fail2ban + IPset + UFW 工作流程详解

工具概述

1. UFW (Uncomplicated Firewall)

  • 作用: iptables 的简化前端
  • 功能: 基础防火墙规则管理
  • 特点: 用户友好的规则语法

2. IPset

  • 作用: IP地址集合管理工具
  • 功能: 高效存储和匹配大量IP地址
  • 特点: 比传统iptables规则更高效

3. Fail2ban

  • 作用: 入侵检测和自动封禁系统
  • 功能: 分析日志、检测攻击、触发封禁动作
  • 特点: 智能攻击模式识别

完整工作流程

阶段1: 初始化和基础防护

[系统启动] → [UFW启动] → [建立基础防火墙规则]
                ↓
[IPset工具初始化] → [创建IP集合]
                ↓
[Fail2ban启动] → [读取配置] → [开始监控日志]

阶段2: 攻击检测流程

[攻击者连接] → [服务日志记录] → [Fail2ban监控]
                                    ↓
[模式匹配] → [计数器增加] → [达到阈值?]
     ↓              ↓              ↓
   [继续监控]    [重置计数器]     [触发封禁]

阶段3: 封禁执行流程

[Fail2ban检测到攻击] 
        ↓
[调用Action脚本]
        ↓
[IPset添加恶意IP] → [ipset add fail2ban-ssh 192.168.1.100]
        ↓
[UFW规则生效] → [DROP来自IPset集合的流量]
        ↓
[攻击者被阻断]

技术架构图

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   网络流量      │────│   UFW/iptables  │────│   目标服务       │
│                 │    │                 │    │   (SSH/HTTP)    │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                              │                         │
                              │                         │
                       ┌─────────────┐            ┌─────────────┐
                       │   IPset     │            │   日志文件   │
                       │  (IP集合)    │            │             │
                       └─────────────┘            └─────────────┘
                              │                         │
                              └──────┬──────────────────┘
                                     │
                              ┌─────────────┐
                              │  Fail2ban   │
                              │  (监控分析)  │
                              └─────────────┘

详细配置示例

1. UFW基础配置

# 启用UFW
sudo ufw enable

# 默认策略
sudo ufw default deny incoming
sudo ufw default allow outgoing

# 允许SSH (在配置fail2ban之前)
sudo ufw allow ssh

# 创建IPset规则 (UFW会自动处理)
# 实际在iptables中创建规则引用IPset

2. IPset配置

# 创建IP集合
sudo ipset create fail2ban-ssh hash:ip timeout 86400

# 手动添加IP到集合 (通常由fail2ban自动执行)
sudo ipset add fail2ban-ssh 192.168.1.100

# 查看集合内容
sudo ipset list fail2ban-ssh

3. Fail2ban配置 (/etc/fail2ban/jail.local)

[DEFAULT]
# 使用ipset作为封禁后端
banaction = iptables-ipset-proto6[name=fail2ban-ssh]

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 86400
findtime = 600

# 使用ipset action
action = iptables-ipset-proto6[name=fail2ban-ssh, port=ssh, protocol=tcp]

数据流详解

1. 正常流量处理

外部请求 → UFW检查 → 基础规则匹配 → 允许通过 → 到达服务
         ↓
    检查IPset集合
         ↓
    IP不在黑名单 → 继续处理

2. 恶意流量处理

攻击流量 → UFW检查 → IPset集合匹配 → 发现恶意IP → 直接丢弃
                              ↓
                         记录封禁日志

3. 新攻击检测

新攻击 → 到达服务 → 认证失败 → 写入日志
                              ↓
       Fail2ban监控 → 模式匹配 → 计数增加
                              ↓
      达到阈值 → 调用action → 添加IP到IPset
                              ↓
       下次请求 → UFW/IPset阻断

性能优势分析

传统iptables方式

# 每个被封IP都要一条规则
iptables -I INPUT -s 192.168.1.100 -j DROP
iptables -I INPUT -s 192.168.1.101 -j DROP
iptables -I INPUT -s 192.168.1.102 -j DROP
# ... 可能有数千条规则

问题: 规则数量多时,匹配效率低下

IPset优化方式

# 只需一条规则引用整个集合
iptables -I INPUT -m set --match-set fail2ban-ssh src -j DROP

# 所有IP存储在高效的哈希表中
ipset add fail2ban-ssh 192.168.1.100
ipset add fail2ban-ssh 192.168.1.101
ipset add fail2ban-ssh 192.168.1.102

优势: O(1) 时间复杂度查找,支持数万IP

实际工作示例

场景:SSH暴力破解攻击

  1. 攻击开始

    攻击者IP: 203.0.113.100
    连续尝试SSH登录,密码错误
    
  2. 日志记录

    /var/log/auth.log:
    Failed password for root from 203.0.113.100 port 22 ssh2
    Failed password for admin from 203.0.113.100 port 22 ssh2
    Failed password for user from 203.0.113.100 port 22 ssh2
    
  3. Fail2ban检测

    fail2ban监控到3次失败
    达到maxretry=3的阈值
    触发封禁action
    
  4. IPset执行

    # Fail2ban执行命令
    ipset add fail2ban-ssh 203.0.113.100
    
  5. UFW/iptables生效

    # 现有规则开始匹配IPset
    iptables -I INPUT -m set --match-set fail2ban-ssh src -j DROP
    
  6. 结果

    攻击者IP被立即阻断
    后续请求直接在内核层面丢弃
    不再到达SSH服务
    

监控和管理

查看当前状态

# 查看fail2ban状态
sudo fail2ban-client status sshd

# 查看IPset内容
sudo ipset list fail2ban-ssh

# 查看UFW状态
sudo ufw status verbose

# 查看iptables规则
sudo iptables -L INPUT -n -v

手动管理

# 手动解封IP
sudo fail2ban-client set sshd unbanip 203.0.113.100

# 直接从IPset移除
sudo ipset del fail2ban-ssh 203.0.113.100

# 查看封禁历史
sudo tail -f /var/log/fail2ban.log

优势总结

  1. 高效性: IPset提供O(1)查找效率,支持大规模IP封禁
  2. 自动化: Fail2ban自动检测和响应,减少人工干预
  3. 灵活性: UFW提供简单的规则管理界面
  4. 可扩展: 支持多种服务和攻击模式检测
  5. 持久化: IPset支持超时和持久化存储
  6. 资源节约: 比传统方法消耗更少系统资源

这种组合为Linux系统提供了一套完整、高效的自动化安全防护解决方案。

代码:仅 fail2ban+ipset 

这是我自用的装机 bash script, 之前有分享过,因为有两个 V 没过审核,留着自已来。

主要功能:

现在这个脚本 15千行:

这里包含几个 filter,测试脚本 ,还有硬写的日志文件名(这需要替换),参考时要记得修改。

# 13. Install Fail2Ban
install_fail2ban() {
    log_function_start "Installing Fail2Ban with IPSet integration"
    log_info "Installing Fail2Ban with IPSet and UFW integration for enhanced security..."

    # Verify nobody user exists for service isolation
    verify_nobody_user || log_warning "Nobody user not available for service isolation"

    # Safe to stop and disable Fail2Ban if already running
    safe_stop_and_disable_service "fail2ban" "Fail2Ban service"
    
    # Remove any existing Fail2Ban packages
    log_info "Removing any existing Fail2Ban packages..."
    apt remove --purge -y fail2ban || true
    log_success "Existing Fail2Ban packages removed"

    # Clean up orphaned dependencies
    log_info "Cleaning up orphaned dependencies..."
    apt autoremove -y || true
    apt autoclean -y || true
    log_success "Orphaned dependencies cleaned up"
    
    # Remove any existing Fail2Ban configuration files
    log_info "Removing any existing Fail2Ban configuration files..."
    rm -rf /etc/fail2ban/* 2>/dev/null || true
    log_info "Removed existing Fail2Ban configuration files"
    
    rm -rf /var/lib/fail2ban/* 2>/dev/null || true
    log_info "Removed existing Fail2Ban data files"

    rm -rf /var/log/fail2ban.log 2>/dev/null || true
    log_info "Removed existing Fail2Ban log file"
    
    log_success "Existing Fail2Ban configuration files removed"

    # Check system requirements for Fail2Ban
    log_info "Checking system requirements for Fail2Ban..."

    # Check log files
    if [ ! -f /var/log/auth.log ]; then
        log_warning "Auth log file not found at /var/log/auth.log"
        touch /var/log/auth.log
        chmod 640 /var/log/auth.log
        chown root:adm /var/log/auth.log
        sleep 1
        log_info "Created missing auth log file at /var/log/auth.log with correct permissions"
        log_success "Auth log file created successfully"
    else
        log_success "Auth log file found at /var/log/auth.log"
    fi

    # Check UFW status
    if ! command -v ufw >/dev/null; then
        log_error "UFW is not installed. Installing it now..."
        if configure_ufw; then
            log_success "UFW installed and configured successfully"
        else
            log_error "Failed to install UFW. Fail2Ban installation cannot proceed."
            log_function_end "Installing Fail2Ban" 1
            return 1
        fi
    else
        log_success "UFW is installed and available"
    fi

    # Install and configure IPSet
    log_info "Installing IPSet package..."
    if apt install -y ipset; then
        log_success "IPSet installed successfully"
    else
        log_error "Failed to install IPSet"
        log_function_end "Installing Fail2Ban with IPSet" 1
        return 1
    fi

    # Clean up any existing ipsets
    log_info "Cleaning up any existing fail2ban ipsets..."
    ipset destroy fail2ban-banned 2>/dev/null || true
    ipset destroy fail2ban-recidive 2>/dev/null || true
    log_info "Existing fail2ban ipsets cleaned up"

    # Create IPSet for fail2ban
    log_info "Creating IPSet for fail2ban..."
    if ipset create fail2ban-banned hash:ip timeout 0 2>/dev/null; then
        log_success "Created IPSet 'fail2ban-banned'"
    else
        log_error "Failed to create IPSet 'fail2ban-banned'"
        log_function_end "Creating IPSet for fail2ban" 1
        return 1
    fi

    # Create additional IPSet for recidive (repeat offenders)
    log_info "Creating IPSet for repeat offenders..."
    if ipset create fail2ban-recidive hash:ip timeout 0 2>/dev/null; then
        log_success "Created IPSet 'fail2ban-recidive'"
    else
        log_warning "Failed to create IPSet 'fail2ban-recidive' (may already exist)"
    fi

    # Create the initial persistent file for IPSet rules
    log_info "Creating the initial persistent file for IPSet rules..."
    if ipset save > /etc/ipset.rules; then
        log_success "Created the initial persistent file for IPSet rules"
    else
        log_error "Failed to create the initial persistent file for IPSet rules"
        log_function_end "Creating the initial persistent file for IPSet rules" 1
        return 1
    fi


    # Create IPSet persistence service
    log_info "Creating IPSet persistence service..."
    cat > /etc/systemd/system/ipset-persistent.service << 'EOF'
[Unit]
Description=IPSet persistent rule service
Before=netfilter-persistent.service network-pre.target ufw.service fail2ban.service
ConditionFileNotEmpty=/etc/ipset.rules

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/sbin/ipset restore -exist -file /etc/ipset.rules
ExecStop=/sbin/ipset save -file /etc/ipset.rules
ExecReload=/sbin/ipset save -file /etc/ipset.rules

[Install]
WantedBy=multi-user.target
EOF
    log_success "Created IPSet persistence service"

    # Enable IPSet persistence
    systemctl daemon-reload
    systemctl enable ipset-persistent.service
    log_success "Enabled IPSet persistence via systemd"

    # Create IPSet save script for graceful shutdown
    log_info "Creating IPSet save script..."
    cat > /etc/systemd/system/ipset-save.service << 'EOF'
[Unit]
Description=Save IPSet rules on shutdown
DefaultDependencies=false
Before=shutdown.target reboot.target halt.target

[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=/bin/true
ExecStop=/sbin/ipset save -file /etc/ipset.rules
TimeoutStopSec=30

[Install]
WantedBy=multi-user.target
EOF
    systemctl enable ipset-save.service
    log_success "Created IPSet save script for graceful shutdown"

    # Replace /etc/ufw/before.rules to integrate with IPSet
    log_info "Updating /etc/ufw/before.rules to integrate with IPSet..."
    cp /etc/ufw/before.rules /etc/ufw/before.rules.backup.$(date +%s)
    log_info "Backed up existing /etc/ufw/before.rules"

    cat > /etc/ufw/before.rules << 'EOF'
#
# rules.before
#
# Rules that should be run before the ufw command line added rules. Custom
# rules should be added to one of these chains:
#   ufw-before-input
#   ufw-before-output
#   ufw-before-forward
#

# Don't delete these required lines, otherwise there will be errors
*filter
:ufw-before-input - [0:0]
:ufw-before-output - [0:0]
:ufw-before-forward - [0:0]
:ufw-not-local - [0:0]

# Fail2Ban IPSet integration
# Block all traffic from fail2ban banned IPs
-A ufw-before-input -m set --match-set fail2ban-banned src -j DROP
-A ufw-before-input -m set --match-set fail2ban-recidive src -j DROP

# End required lines

# allow all on loopback
-A ufw-before-input -i lo -j ACCEPT
-A ufw-before-output -o lo -j ACCEPT

# quickly process packets for which we already have a connection
-A ufw-before-input -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-output -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-forward -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

# drop INVALID packets (logs these in loglevel medium and higher)
-A ufw-before-input -m conntrack --ctstate INVALID -j ufw-logging-deny
-A ufw-before-input -m conntrack --ctstate INVALID -j DROP

# ok icmp codes for INPUT
-A ufw-before-input -p icmp --icmp-type destination-unreachable -j ACCEPT
-A ufw-before-input -p icmp --icmp-type time-exceeded -j ACCEPT
-A ufw-before-input -p icmp --icmp-type parameter-problem -j ACCEPT
-A ufw-before-input -p icmp --icmp-type echo-request -j ACCEPT

# ok icmp code for FORWARD
-A ufw-before-forward -p icmp --icmp-type destination-unreachable -j ACCEPT
-A ufw-before-forward -p icmp --icmp-type time-exceeded -j ACCEPT
-A ufw-before-forward -p icmp --icmp-type parameter-problem -j ACCEPT
-A ufw-before-forward -p icmp --icmp-type echo-request -j ACCEPT

# allow dhcp client to work
-A ufw-before-input -p udp --sport 67 --dport 68 -j ACCEPT

#
# ufw-not-local
#
-A ufw-before-input -j ufw-not-local

# if LOCAL, RETURN
-A ufw-not-local -m addrtype --dst-type LOCAL -j RETURN

# if MULTICAST, RETURN
-A ufw-not-local -m addrtype --dst-type MULTICAST -j RETURN

# if BROADCAST, RETURN
-A ufw-not-local -m addrtype --dst-type BROADCAST -j RETURN

# all other non-local packets are dropped
-A ufw-not-local -m limit --limit 3/min --limit-burst 10 -j ufw-logging-deny
-A ufw-not-local -j DROP

# allow MULTICAST mDNS for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 224.0.0.251 --dport 5353 -j ACCEPT

# allow MULTICAST UPnP for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 239.255.255.250 --dport 1900 -j ACCEPT

# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT
EOF

    log_success "Updated /etc/ufw/before.rules with IPSet integration"

    # Install Fail2Ban
    log_info "Installing Fail2Ban package..."
    if apt install -y fail2ban; then
        log_success "Fail2Ban installed successfully"
    else
        log_error "Failed to install Fail2Ban"
        log_function_end "Installing Fail2Ban" 1
        return 1
    fi

    # Create /etc/fail2ban/action.d/ipset-block.conf
    log_info "Creating Persistent /etc/fail2ban/action.d/ipset-block.conf"
    cat > /etc/fail2ban/action.d/ipset-block.conf << 'EOF'
# Fail2Ban action file for ipset
#
# This action blocks/unblocks an IP address using ipset.
#
[INCLUDES]

before =

[Definition]

# The name of the ipset set to use.
setname = fail2ban-banned

# Action startup (creates the set if it doesn't exist)
actionstart = /usr/sbin/ipset create <setname> hash:ip timeout 0 exist

# Action shutdown (flushes the set when Fail2Ban stops)
#actionstop = ipset flush <setname>
actionstop = 

# Action to ban an IP
actionban = /usr/sbin/ipset add <setname> <ip> timeout 0 exist

# Action to unban an IP
actionunban = /usr/sbin/ipset del <setname> <ip> exist

[Init]
EOF
    log_success "Created /etc/fail2ban/action.d/ipset-block.conf"


# Create IPSet action for fail2ban
    log_info "Creating Fail2Ban IPSet action configuration..."
    mkdir -p /etc/fail2ban/action.d
    
    cat > /etc/fail2ban/action.d/ipset-ban.conf << 'EOF'
[INCLUDES]
before = iptables-common.conf

[Definition]
# Option: actionstart
# Notes.: command executed once at the start of Fail2Ban.
actionstart = ipset create <ipmset> hash:ip timeout <default-timeout> <family>

# Option: actionstop
# Notes.: command executed once at the end of Fail2Ban
actionstop = ipset destroy <ipmset>

# Option: actionban
# Notes.: command executed when banning an IP.
actionban = ipset add <ipmset> <ip> timeout <timeout> -exist

# Option: actionunban
# Notes.: command executed when unbanning an IP.
actionunban = ipset del <ipmset> <ip> -exist

[Init]
# Default name of the ipset
ipmset = fail2ban-<name>

# Option: timeout
# Notes.: timeout for the ban, in seconds
timeout = 0

# Option: default-timeout  
# Notes.: default timeout for ipset creation
default-timeout = 0

# Option: family
# Notes.: IP family, can be inet or inet6
family = inet
EOF

    # Create enhanced IPSet action for permanent bans
    cat > /etc/fail2ban/action.d/ipset-persistent.conf << 'EOF'
[INCLUDES]
before = iptables-common.conf

[Definition]
actionstart = ipset create <ipmset> hash:ip timeout 0 -exist
              ipset restore -file /etc/ipset.rules -exist

actionstop = ipset save -file /etc/ipset.rules
             ipset destroy <ipmset>

actionban = ipset add <ipmset> <ip> timeout 0 -exist
            ipset save -file /etc/ipset.rules

actionunban = ipset del <ipmset> <ip> -exist
              ipset save -file /etc/ipset.rules

[Init]
ipmset = fail2ban-banned
EOF

    # Create recidive action for repeat offenders
    cat > /etc/fail2ban/action.d/ipset-recidive.conf << 'EOF'
[INCLUDES]
before = iptables-common.conf

[Definition]
actionstart = ipset create <ipmset> hash:ip timeout 0 -exist

actionstop = ipset save -file /etc/ipset.rules

actionban = ipset add <ipmset> <ip> timeout 0 -exist
            ipset save -file /etc/ipset.rules

actionunban = ipset del <ipmset> <ip> -exist
              ipset save -file /etc/ipset.rules

[Init]
ipmset = fail2ban-recidive
EOF

    log_success "Created Fail2Ban IPSet action configurations"


    # Create jail configuration
    log_info "Configuring Fail2Ban jails with IPSet integration..."
    cat > /etc/fail2ban/jail.local << EOF
[DEFAULT]
# Ban settings
bantime = -1
findtime = 1800
maxretry = 3
backend = auto

# IPSet integration
banaction = ipset-persistent
banaction_allports = ipset-persistent

# Network settings
allowipv6 = auto
ignoreip = 127.0.0.1/8 ::1
            usw.daven.us 
            jpt.daven.us 
            bjn.daven.us 
            bjt.daven.us 
            usc.daven.us

# Email settings (optional)
# destemail = admin@yourdomain.com
# sendername = Fail2Ban
# mta = sendmail

[recidive]
enabled = true
logpath = /var/log/fail2ban.log
banaction = ipset-recidive
bantime = -1
findtime = 86400
maxretry = 3
filter = recidive

[sshd]
enabled = true
port = $SSH_PORT
filter = sshd
logpath = /var/log/auth.log
backend = auto
bantime = 72000
findtime = 600
maxretry = 5

[ssh-scanner]
enabled = true
port = $SSH_PORT
filter = ssh-scanner
logpath = /var/log/auth.log
maxretry = 5
bantime = 72000
findtime = 600

[nginx-scan]
enabled = true
port = http,https,6033
logpath = /var/log/nginx/access.log
         /var/log/nginx/*.access.log
filter = nginx-scan
maxretry = 5
bantime = -1
findtime = 600

[nginx-404-scan]
enabled = true
port = http,https,6033
logpath = /var/log/nginx/access.log
         /var/log/nginx/*.access.log
filter = nginx-404-scan
maxretry = 20
bantime = -1
findtime = 600

[nginx-login-bruteforce]
enabled = true
port = http,https,6033
logpath = /var/log/nginx/access.log
filter = nginx-login-bruteforce
maxretry = 5
bantime = -1
findtime = 600

[nginx-bad-useragent]
enabled = true
port = http,https,6033
logpath = /var/log/nginx/access.log
filter = nginx-bad-useragent
maxretry = 5
bantime = -1
findtime = 600

[nginx-ray-abuse]
enabled = true
port = http,https,6033
logpath = /var/log/nginx/access.log
filter = nginx-ray-abuse
maxretry = 12
bantime = -1
findtime = 60

[nginx-status-abuse]
enabled = true
port = http,https,6033
logpath = /var/log/nginx/access.log
         /var/log/nginx/*.access.log
filter = nginx-status-abuse
maxretry = 30
bantime = -1
findtime = 600

[nginx-attacks]
enabled = true
port = http,https
filter = nginx-attack
logpath = /var/log/nginx/access.log
maxretry = 3
bantime = -1
findtime = 60

[nginx-web-attacks]
enabled = true
port = http,https
filter = nginx-web-attacks
logpath = /var/log/nginx/*.daven.us.access.log
maxretry = 12
bantime = -1
findtime = 600
EOF
    log_success "Created /etc/fail2ban/jail.local configuration file"


    # fail2ban for sshd and ssh-scanner local filter
    log_success "Fail2Ban jails configured successfully"
    mkdir -p /etc/fail2ban/filter.d
    
    log_info "Creating Fail2Ban filter for sshd and ssh-scanner..."

    # Create sshd local filter configuration
    log_info "Creating sshd.local filter configuration..."
    cat > /etc/fail2ban/filter.d/sshd.local << 'EOF'
[Definition]
allowipv6 = auto
failregex = ^<HOST> .* "[^"]*" .*" "(?:Go-http-client|wget|curl|python-requests|masscan|zgrab|sqlmap|nikto|dirb|gobuster)" .*$
            ^<HOST> .* "[^"]*" .*" ".*(?:Mozi\.m|scamanje|stresserit)" .*$
            ^<HOST> .* "[^"]*" .*" "xfa1" .*$
ignoreregex = ^<HOST> .* "[^"]*" .*" ".*(?:Googlebot|Bingbot|YandexBot|DuckDuckBot)" .*$
EOF
    log_success "sshd.local filter configuration created"

    # Create ssh-scanner filter
    log_info "Creating Fail2Ban filter for SSH scanner..."
    cat > /etc/fail2ban/filter.d/ssh-scanner.conf << 'EOF'
[INCLUDES]
before = common.conf
[Definition]
_daemon = sshd
allowipv6 = auto
failregex = ^%(__prefix_line)sConnection from <HOST> port \d+ on \S+ port \d+$
            ^%(__prefix_line)sConnection closed by <HOST> port \d+ \[preauth\]$
            ^%(__prefix_line)sConnection closed by authenticating user .* <HOST> port \d+ \[preauth\]$
            ^%(__prefix_line)sDisconnected from .* <HOST> port \d+ \[preauth\]$
            ^%(__prefix_line)sReceived disconnect from <HOST> port \d+:\d+: .* \[preauth\]$
            ^%(__prefix_line)sbanner exchange: Connection from <HOST> port \d+: invalid format$
ignoreregex =
EOF
    log_success "ssh-scanner filter created"


    # Create Fail2Ban jail for nginx scanners
    # Create nginx-scan filter
    log_info "Creating Fail2Ban filter for Nginx scanner..."
    cat > /etc/fail2ban/filter.d/nginx-scan.conf << 'EOF'
[Definition]
allowipv6 = auto
failregex = ^<HOST> .* "(GET|POST|PUT) [^"]*(?:\.env|\.git|phpinfo|eval-stdin\.php|shell\?|admin/login\.asp|boaform/admin|login\.cgi)" .*$
            ^<HOST> .* "(GET|POST) [^"]*(?:wget\+|chmod\+|/tmp/|Mozi\.m|scamanje\.stresserit\.pro)" .*$
            ^<HOST> .* "(GET|POST) [^"]*(?:/SDK/webLanguage|/actuator|/console|/geoserver|/solr/admin)" .*$
            ^<HOST> .* "(GET|POST) [^"]*(?:phpunit|vendor/phpunit|_ignition)" .*$
             ^<HOST> .* "[^"]*(?:27;wget|mstshash=)" .*$
ignoreregex = ^<HOST> .* "GET /(?:favicon\.ico|robots\.txt|sitemap\.xml) .*$
EOF

    log_success "nginx-scan filter created"

    # Create nginx-404-scan filter
    log_info "Creating Fail2Ban filter for Nginx 404 scanner..."
    cat > /etc/fail2ban/filter.d/nginx-404-scan.conf << 'EOF'
[Definition]
allowipv6 = auto
failregex = ^<HOST> - - \[.*\] "(?:GET|POST) /(?:admin|wp-admin|\.php|\.asp|phpinfo|\.env|config|backup|uploads|test|debug|system_info|diagnostics)" .*" 404 .*$
            ^<HOST> - - \[.*\] "(?:GET|POST) [^"]*/(?:partymgr|jasperserver|solr|owncloud|geoserver|WebInterface)" .*" 404 .*$
            ^<HOST> - - \[.*\] "(?:GET|POST|HEAD|OPTIONS|PUT|DELETE|CONNECT|TRACE|PATCH) /[a-zA-Z0-9]{4,8}(?:\s|/|$)" .*" 404 .*$
ignoreregex = ^<HOST> - - \[.*\] "GET /(?:favicon\.ico|robots\.txt|sitemap\.xml|apple-touch-icon|\.well-known/)" .*" 404 .*$
EOF
    log_success "nginx-404-scan filter created"

    # Create login scanner filter
    log_info "Creating Fail2Ban filter for nginx login bruteforce scanner..."
    cat > /etc/fail2ban/filter.d/nginx-login-bruteforce.conf << 'EOF'
[Definition]
allowipv6 = auto
failregex = ^<HOST> .* "(?:GET|POST) /login\.cgi\?username=.*&psd=" .*$
            ^<HOST> .* "POST /boaform/admin/formLogin" .*$
            ^<HOST> .* "GET /boaform/admin/formLogin\?username=.*&psd=" .*$
            ^<HOST> .* "POST /admin/login\." .*$
ignoreregex =
EOF
    log_success "nginx-login-bruteforce filter created"

    # Create nginx bad User-Agent filter
    log_info "Creating Fail2Ban filter for Nginx bad User-Agent..."
    cat > /etc/fail2ban/filter.d/nginx-bad-useragent.conf << 'EOF'
[Definition]
allowipv6 = auto
failregex = ^<HOST> .* "[^"]*" .*" "(?:Go-http-client|wget|curl|python-requests|masscan|zgrab|sqlmap|nikto|dirb|gobuster)" .*$
            ^<HOST> .* "[^"]*" .*" ".*(?:Mozi\.m|scamanje|stresserit)" .*$
            ^<HOST> .* "[^"]*" .*" "xfa1" .*$
ignoreregex = ^<HOST> .* "[^"]*" .*" ".*(?:Googlebot|Bingbot|YandexBot|DuckDuckBot)" .*$
EOF
    log_success "nginx-bad-useragent filter created"

    # Create nginx ray abuse filter
    log_info "Creating Fail2Ban filter for Nginx Ray abuse..."
    cat > /etc/fail2ban/filter.d/nginx-ray-abuse.conf << 'EOF'
[Definition]
allowipv6 = auto
failregex = ^<HOST> .* "GET /api HTTP/1\.1" 101 .*$
ignoreregex =
EOF
    log_success "nginx-ray-abuse filter created"

    # Create nginx status abuse filter
    log_info "Creating Fail2Ban filter for Nginx status abuse..."
    cat > /etc/fail2ban/filter.d/nginx-status-abuse.conf << 'EOF'
[Definition]
allowipv6 = auto
failregex = ^<HOST> - - \[.*\] "(?:GET|POST|HEAD) .* HTTP/1\.[01]" 40[0-3] \d+ ".*" ".*"$
ignoreregex = ^<HOST> .* "GET /(?:favicon\.ico|robots\.txt)" 40[0-3] .*$
EOF
    log_success "nginx-status-abuse filter created"

# Create nginx status abuse filter
    log_info "Creating Fail2Ban filter for Nginx attack abuse..."
    cat > /etc/fail2ban/filter.d/nginx-attack.conf << 'EOF'
[Definition]
allowipv6 = auto
failregex = ^<HOST> -.*"GET .*(.env|.git|.well-known/security\.txt|\/phpmyadmin|\/wp-login\.php|\/wp-admin|\/cgi-bin|\/phpinfo|\.sql|\.log|\.ini|\.conf|\.bak).* HTTP.*" (40[0-4]|444|499|500)
            ^<HOST> -.*"GET .*(?:\x09hink|\x09pp).*shell_exec.*wget.*" 400.*
ignoreregex =
EOF
    log_success "nginx-attack filter created"

# Create nginx status abuse filter
    log_info "Creating Fail2Ban filter for Nginx web attacks abuse..."
    cat > /etc/fail2ban/filter.d/nginx-web-attacks.conf << 'EOF'
[Definition]
allowipv6 = auto
failregex = ^<HOST> -.*"GET .*(?:\.env|\.git|\.well-known/security\.txt|\/phpmyadmin|\/wp-admin|\/cgi-bin|\/webui|\/geoserver|\/remote/login|onvif|PSIA|boaform|owa/auth/logon\.aspx|PROPFIND|\+CSCOE\+).*HTTP.*" (40[0-4]|444|499|500)
            ^<HOST> -.*"GET .*(?:\x09hink|\x07pp).*shell_exec.*wget.*" 400.*
            ^<HOST> -.*"POST .*cgi-bin/(\.\./).*bin/sh HTTP.*" (400|404|301|302)
            ^<HOST> -.*"GET .*HTTP.*" (40[0-4]|444|499).*"(?:zgrab|CMS-Checker|libredtail-http|WanScannerBot|Odin|GenomeCrawlerd)"
ignoreregex =
EOF
    log_success "nginx-web-attacks filter created"

    # Disable conflicting default configuration files
    log_info "Disabling conflicting default configuration files..."
    if [ -f /etc/fail2ban/jail.d/defaults-debian.conf ]; then
        if mv /etc/fail2ban/jail.d/defaults-debian.conf /etc/fail2ban/jail.d/defaults-debian.disabled; then
            log_success "Conflicting default configuration files disabled"
        else
            log_warning "Failed to disable conflicting default configuration files"
        fi
    fi
    
    # Restart services in proper order
    log_info "Starting services in proper order..."
    
    # Start IPSet persistence first
    systemctl start ipset-persistent.service
    
    # Reload UFW to pick up the new before.rules
    log_info "Reloading UFW to apply IPSet integration..."
    ufw --force reload
    log_success "UFW reloaded with IPSet integration"
    
    # Enable and start Fail2Ban
    systemctl enable fail2ban
    systemctl start fail2ban
    log_info "Fail2Ban service started"

    # Wait for fail2ban to initialize
    sleep 3

    # Verify IPSet integration
    log_info "Verifying IPSet integration..."
    if ipset list fail2ban-banned >/dev/null 2>&1; then
        log_success "IPSet 'fail2ban-banned' is active"
    else
        log_error "IPSet 'fail2ban-banned' is not active"
    fi

    # Verify UFW integration
    if iptables -L ufw-before-input -n | grep -q fail2ban-banned; then
        log_success "UFW is integrated with IPSet"
    else
        log_warning "UFW integration with IPSet may not be working"
    fi

    # Create management tools
    create_ipset_management_tools

    # Create fail2ban-tester.sh
    log_info "Creating /usr/bin/fail2ban-tester.sh..."
    cat > /usr/bin/fail2ban-tester.sh << 'EOF'
#!/bin/bash
# A script to automatically test all enabled Fail2ban jails using fail2ban-regex.
# It fetches the filter and logpath for each active jail and runs the test.
# By Dave Nian on 25Aug25

# --- Configuration ---
# Colors for better readability
C_RESET='\033[0m'
C_RED='\033[0;31m'
C_GREEN='\033[0;32m'
C_YELLOW='\033[0;33m'
C_BLUE='\033[0;34m'
C_CYAN='\033[0;36m'
C_BOLD='\033[1m'

# --- Pre-flight Checks ---
# Check if running as root
if [[ $EUID -ne 0 ]]; then
   echo -e "${C_RED}This script must be run as root.${C_RESET}"
   exit 1
fi

# Check if fail2ban-client is available
if ! command -v fail2ban-client &> /dev/null; then
    echo -e "${C_RED}Error: 'fail2ban-client' command not found.${C_RESET}"
    echo "Please ensure Fail2ban is installed and you are running this on the correct server."
    exit 1
fi

# --- Main Logic ---
echo -e "${C_BOLD}${C_BLUE}--- Starting Fail2ban Filter Test ---${C_RESET}"

# Get the list of enabled jails from fail2ban-client
# The sed command cleans up the output to get just the jail names
ENABLED_JAILS=$(fail2ban-client status | grep "Jail list:" | sed -e 's/.*Jail list:\s*//' -e 's/,//g')

if [ -z "$ENABLED_JAILS" ]; then
    echo -e "${C_RED}Could not find any enabled jails. Exiting.${C_RESET}"
    exit 1
fi

echo -e "Found enabled jails: ${C_YELLOW}${ENABLED_JAILS}${C_RESET}\n"

# Loop through each enabled jail
for jail in $ENABLED_JAILS; do
    echo -e "${C_BOLD}${C_CYAN}========================================${C_RESET}"
    echo -e "${C_BOLD}${C_CYAN} Testing Jail: [${jail}] ${C_RESET}"
    echo -e "${C_BOLD}${C_CYAN}========================================${C_RESET}"

    # Get the filter name for the current jail.
    # Redirect stderr to /dev/null to keep the output clean.
    filter_name=$(fail2ban-client get "$jail" filter 2>/dev/null)

    # ** FIX **: Check if the command failed or returned a known error message.
    # Some versions of fail2ban-client output this error to stdout.
    # If it fails, we assume the filter name is the same as the jail name.
    if [ $? -ne 0 ] || [[ "$filter_name" == *"Invalid command"* ]] || [ -z "$filter_name" ]; then
        echo -e "  ${C_YELLOW}Warning: Could not dynamically get filter for [${jail}].${C_RESET}"
        echo -e "  ${C_YELLOW}Assuming filter name matches jail name.${C_RESET}"
        filter_name=$jail
    fi

    filter_file="/etc/fail2ban/filter.d/${filter_name}.conf"

    # ** FIX **: Get the log path(s) and parse the multi-line output correctly.
    # This command now specifically extracts lines that are file paths.
    log_paths_output=$(fail2ban-client get "$jail" logpath)
    log_paths=$(echo "$log_paths_output" | grep -E '^[|`]-' | sed -E 's/^[|`]- //')

    # If the above fails (for single-line outputs), use the raw output but clean it.
    if [ -z "$log_paths" ]; then
        log_paths=$(echo "$log_paths_output" | sed 's/Current monitored log file(s)://g' | tr -d '`-' | tr -d '|')
    fi


    # Check if the filter file actually exists
    if [ ! -f "$filter_file" ]; then
        echo -e "  ${C_RED}Filter file not found:${C_RESET} ${filter_file}"
        # Try to find it with a .local extension as a fallback
        filter_file_local="/etc/fail2ban/filter.d/${filter_name}.local"
        if [ -f "$filter_file_local" ]; then
            echo -e "  ${C_YELLOW}Found fallback filter:${C_RESET} ${filter_file_local}"
            filter_file=$filter_file_local
        else
            echo -e "  ${C_RED}Skipping jail [${jail}] as no valid filter was found.${C_RESET}\n"
            continue
        fi
    fi

    echo -e "  ${C_BOLD}Filter:${C_RESET} ${filter_file}"
    echo -e "  ${C_BOLD}Log(s):${C_RESET}\n${log_paths}" # Print the cleaned list
    echo ""

    # Loop through each log path (a jail can have multiple)
    for log_path in $log_paths; do
        # Expand wildcards in log paths
        expanded_logs=$(eval echo $log_path)
        for expanded_log_path in $expanded_logs; do
            # Check if the log file exists and is readable
            if [ ! -r "$expanded_log_path" ]; then
                echo -e "  -> Testing log: ${C_YELLOW}${expanded_log_path}${C_RESET}"
                echo -e "     ${C_RED}Result: Log file not found or not readable. Skipping.${C_RESET}\n"
                continue
            fi

            echo -e "  -> Testing log: ${C_GREEN}${expanded_log_path}${C_RESET}"

            # Run the fail2ban-regex command and capture the output
            test_output=$(fail2ban-regex "$expanded_log_path" "$filter_file")
            
            # ** FIX **: More robustly extract the number of matched lines.
            # This specifically finds the number preceding "matched".
            matches=0 # Default to 0
            matches_line=$(echo "$test_output" | grep "Lines:")
            if [[ -n "$matches_line" ]]; then
                # Use grep with Perl-compatible regex to extract the number
                matches=$(echo "$matches_line" | grep -oP '\d+(?=\s+matched)')
            fi

            # Print the summary
            if [ "$matches" -gt 0 ]; then
                echo -e "     ${C_GREEN}${C_BOLD}Result: Found ${matches} matching lines!${C_RESET}"
            else
                echo -e "     ${C_YELLOW}Result: Found 0 matching lines.${C_RESET}"
            fi
            echo ""
        done
    done
done

echo -e "${C_BOLD}${C_BLUE}--- Test Complete ---${C_RESET}"

EOF

    if [ -f "/usr/bin/fail2ban-tester.sh" ];then 
        log_info "fail2ban-tester.sh created in /usr/bin/"
    else
        log_warning "fail2ban-tester.sh didn't exist in /usr/bin/"
        return 1
    fi

    if chmod 777 /usr/bin/fail2ban-tester.sh;then
        log_info "/usr/bin/fail2ban-tester.sh has been given execute permission"
    else
        log_warning "Ran chmode on /usr/bin/fail2ban-tester.sh failed"
        return 1
    fi

    if ln -s /usr/bin/fail2ban-tester.sh /usr/bin/fail2ban-tester;then
        log_info "Linked /usr/bin/fail2ban-tester.sh to /usr/bin/fail2ban-tester"
    else
        log_warning "Failed to link /usr/bin/fail2ban-tester"
        return 1
    fi

    # Fixing the WARNING 'allowipv6' not defined in 'Definition'
    if fail2ban_sshd_local_allowipv6; then
        log_success "'allowipv6' configuration fixed in sshd.local"
    else
        log_warning "'allowipv6' configuration not fixed in sshd.local"
    fi

    # Final restart to apply all changes
    log_info "Performing final restart of services..."
    systemctl daemon-reload
    systemctl restart fail2ban
    
    if systemctl is-active --quiet fail2ban; then
        log_success "Fail2Ban restarted successfully with IPSet integration"
    else
        log_error "Failed to restart Fail2Ban"
        systemctl status fail2ban --no-pager -l || true
        log_function_end "Installing Fail2Ban with IPSet" 1
        return 1
    fi

    # Create Fail2Ban management tools
    log_info "Creating Fail2Ban management tools..."    
    create_fail2ban_tools
    
    # Verify installation
    if verify_fail2ban_installation; then
        log_success "🎉 Fail2Ban with IPSet integration completed successfully!"
        show_fail2ban_summary
        log_function_end "Installing Fail2Ban with IPSet" 0
        return 0
    else
        log_warning "⚠️  Fail2Ban with IPSet installed but some issues detected"
        show_fail2ban_troubleshooting
        log_function_end "Installing Fail2Ban with IPSet" 1
        return 1
    fi
}

# Function to create IPSet management tools
create_ipset_management_tools() {
    log_info "Creating IPSet management tools..."
    
    # Create IPSet status script
    cat > /usr/local/bin/ipset-status << 'EOF'
#!/bin/bash
# IPSet Status Display Script

echo "=== IPSet Status ==="
echo "Active IPSets:"
ipset list -name

echo ""
echo "=== Fail2Ban IPSets ==="
for set in $(ipset list -name | grep fail2ban); do
    echo "--- $set ---"
    echo "Members: $(ipset list $set | grep -c '^[0-9]')"
    ipset list $set | head -20
    echo ""
done

echo "=== IPSet Rules in iptables ==="
iptables -L ufw-before-input -n | grep set
EOF
    chmod +x /usr/local/bin/ipset-status
    log_success "Created IPSet status script"

    # Create IPSet backup script
    cat > /usr/local/bin/ipset-backup << 'EOF'
#!/bin/bash
# IPSet Backup Script

BACKUP_DIR="/etc/ipset-backups"
mkdir -p "$BACKUP_DIR"

BACKUP_FILE="$BACKUP_DIR/ipset-backup-$(date +%Y%m%d-%H%M%S).rules"
ipset save > "$BACKUP_FILE"

echo "IPSet rules backed up to: $BACKUP_FILE"

# Keep only last 10 backups
ls -t "$BACKUP_DIR"/ipset-backup-*.rules | tail -n +11 | xargs rm -f 2>/dev/null
EOF
    chmod +x /usr/local/bin/ipset-backup
    log_success "Created IPSet backup script"
}

fail2ban_sshd_local_allowipv6() {
    local source_file="/etc/fail2ban/filter.d/sshd.conf"
    local local_file="/etc/fail2ban/filter.d/sshd.local"
    
    if [ -f "$source_file" ]; then
        log_info "Setting up sshd.local filter configuration..."
        
        # Backup existing sshd.local if it exists
        if [ -f "$local_file" ]; then
            log_info "Backing up existing sshd.local..."
            cp "$local_file" "${local_file}.backup.$(date +%s)"
        fi

        # Copy and set permissions
        if cp -f "$source_file" "$local_file"; then
            chmod 644 "$local_file"

            # Verify copy result
            if [ -s "$local_file" ]; then
                log_success "sshd.conf copied to sshd.local successfully"
            else
                log_error "Copied file is empty"
                return 1
            fi
        else
            log_error "Failed to copy sshd.conf to sshd.local"
            return 1
        fi

        # Fix 'allowipv6' configuration
        log_info "Fixing 'allowipv6' warning in sshd.local..."
        
        if sed -n '/^\[Definition\]/,/^\[/p' "$local_file" | grep -q "^[[:space:]]*allowipv6[[:space:]]*=[[:space:]]*\(auto\|yes\)"; then
            log_success "'allowipv6' already correctly configured in sshd.local"
        else
            if sed -n '/^\[Definition\]/,/^\[/p' "$local_file" | grep -q "^[[:space:]]*allowipv6[[:space:]]*="; then
                log_info "Updating existing 'allowipv6' configuration to 'auto'"
                sed -i '/^\[Definition\]/,/^\[/{ s/^[[:space:]]*allowipv6[[:space:]]*=.*/allowipv6 = auto/; }' "$local_file"
            else
                log_info "Adding 'allowipv6 = auto' to sshd.local"
                sed -i '/^\[Definition\]/a allowipv6 = auto' "$local_file"
            fi

            # Verify modification result
            if grep -A 5 "^\[Definition\]" "$local_file" | grep -q "^allowipv6[[:space:]]*=[[:space:]]*auto"; then
                log_success "✅ allowipv6 successfully configured in sshd.local"
            else
                log_warning "⚠️  allowipv6 configuration may not be correct"
            fi
        fi
        
    else
        log_error "Source file does not exist: $source_file"
        return 1
    fi
}

# Unified verification function that handles both standard and IPSet installations
verify_fail2ban_installation() {
    log_info "Verifying Fail2Ban installation and configuration..."
    
    local verification_passed=true
    local tools_dir="/usr/local/bin/fail2ban-tools"
    local has_ipset=false
    
    # Detect if IPSet integration is being used
    if command -v ipset >/dev/null 2>&1 && ipset list fail2ban-banned >/dev/null 2>&1; then
        has_ipset=true
        log_info "IPSet integration detected, performing enhanced verification..."
    else
        log_info "Standard Fail2Ban installation detected..."
    fi
    
    # 1. Check Fail2Ban service status
    if systemctl is-active --quiet fail2ban; then
        log_success "✅ Fail2Ban service is running"
    else
        log_error "❌ Fail2Ban service is not running"
        verification_passed=false
    fi

    # 2. IPSet-specific checks (only if IPSet integration detected)
    if [ "$has_ipset" = true ]; then
        # Check IPSet persistence service
        if systemctl is-active --quiet ipset-persistent; then
            log_success "✅ IPSet persistence service is active"
        else
            log_warning "⚠️  IPSet persistence service may not be active"
        fi

        # Check IPSets exist
        if ipset list fail2ban-banned >/dev/null 2>&1; then
            log_success "✅ IPSet 'fail2ban-banned' exists"
        else
            log_error "❌ IPSet 'fail2ban-banned' does not exist"
            verification_passed=false
        fi

        # Check for recidive IPSet (optional)
        if ipset list fail2ban-recidive >/dev/null 2>&1; then
            log_success "✅ IPSet 'fail2ban-recidive' exists"
        else
            log_info "ℹ️  IPSet 'fail2ban-recidive' not found (optional)"
        fi

        # Check UFW integration with IPSet
        if iptables -L ufw-before-input -n | grep -q fail2ban-banned; then
            log_success "✅ UFW is integrated with IPSet"
        else
            log_error "❌ UFW is not integrated with IPSet"
            verification_passed=false
        fi
        
        # Check IPSet action files
        local ipset_actions=(
            "/etc/fail2ban/action.d/ipset-ban.conf"
            "/etc/fail2ban/action.d/ipset-persistent.conf"
            "/etc/fail2ban/action.d/ipset-recidive.conf"
        )
        for action_file in "${ipset_actions[@]}"; do
            if [ -f "$action_file" ]; then
                log_success "✅ IPSet action file exists: $(basename "$action_file")"
            else
                log_warning "⚠️  IPSet action file missing: $action_file"
            fi
        done

        # Check IPSet rules file
        if [ -f "/etc/ipset.rules" ]; then
            log_success "✅ IPSet rules file exists: /etc/ipset.rules"
        else
            log_warning "⚠️  IPSet rules file missing: /etc/ipset.rules"
        fi
    fi

    # 3. Check jail status
    local active_jails=$(fail2ban-client status 2>/dev/null | grep "Jail list:" | cut -d: -f2 | tr ',' '\n' | wc -w)
    if [ "$active_jails" -gt 0 ]; then
        log_success "✅ $active_jails jail(s) active"
        
        # Check specific jails
        for jail in sshd ssh-scanner; do
            if fail2ban-client status "$jail" >/dev/null 2>&1; then
                log_success "✅ $jail jail is active"
            else
                log_warning "⚠️  $jail jail is not active"
            fi
        done
    else
        log_error "❌ No active jails found"
        verification_passed=false
    fi

    # 4. Check configuration files
    local config_files=(
        "/etc/fail2ban/jail.local"
        "/etc/fail2ban/filter.d/sshd.local"
        "/etc/fail2ban/filter.d/ssh-scanner.conf"
        "/etc/fail2ban/filter.d/nginx-scan.conf"
        "/etc/fail2ban/filter.d/nginx-404-scan.conf"
        "/etc/fail2ban/filter.d/nginx-login-bruteforce.conf"
        "/etc/fail2ban/filter.d/nginx-bad-useragent.conf"
        "/etc/fail2ban/filter.d/nginx-ray-abuse.conf"
        "/etc/fail2ban/filter.d/nginx-status-abuse.conf"
        "/etc/fail2ban/filter.d/nginx-attack.conf"
        "/etc/fail2ban/filter.d/nginx-web-attacks.conf"
    )
    
    local missing_configs=0
    for config_file in "${config_files[@]}"; do
        if [ -f "$config_file" ]; then
            log_success "✅ Configuration file exists: $(basename "$config_file")"
        else
            log_error "❌ Configuration file missing: $config_file"
            verification_passed=false
            ((missing_configs++))
        fi
    done

    if [ "$missing_configs" -eq 0 ]; then
        log_success "✅ All configuration files are present"
    else
        log_error "❌ $missing_configs configuration file(s) missing"
    fi
    
    # 5. Check log file
    if [ -f "/var/log/fail2ban.log" ]; then
        local recent_errors=$(tail -50 /var/log/fail2ban.log 2>/dev/null | grep -c "ERROR" || echo "0")
        if [ "$recent_errors" -eq 0 ]; then
            log_success "✅ No recent errors in Fail2Ban log"
        else
            log_warning "⚠️  Found $recent_errors recent errors in Fail2Ban log"
        fi
    else
        log_warning "⚠️  Fail2Ban log file not found at /var/log/fail2ban.log"
    fi

    # 6. Check tool files
    if [ -d "$tools_dir" ]; then
        log_success "✅ Fail2Ban tools directory exists ($tools_dir)"
        if [ -f "$tools_dir/fail2ban-checking.sh" ]; then
            log_success "✅ Fail2Ban check script exists"
            if [ -f "$tools_dir/fail2ban-status.sh" ]; then
                log_success "✅ Fail2Ban status script exists"
            else
                log_error "❌ Fail2Ban status script missing"
                verification_passed=false
            fi
        else
            log_error "❌ Fail2Ban check script missing"
            verification_passed=false
        fi
    else
        log_error "❌ Fail2Ban tools directory missing"
        verification_passed=false
    fi
    
    # 7. Check tool symlinks
    if [ -f "/usr/local/bin/fail2ban-checking" ]; then
        log_success "✅ Fail2Ban checking symlink exists"
        if [ -f "/usr/local/bin/fail2ban-status" ]; then
            log_success "✅ Fail2Ban status symlink exists"
        else
            log_error "❌ Fail2Ban status symlink missing"
            verification_passed=false
        fi
    else
        log_error "❌ Fail2Ban checking symlink missing"
        verification_passed=false
    fi

    # 8. Check fail2ban-tester script
    if [ -f "/usr/bin/fail2ban-tester.sh" ] && [ -f "/usr/bin/fail2ban-tester" ]; then
        log_success "✅ Fail2Ban tester script exists"
    else
        log_warning "⚠️  Fail2Ban tester script missing"
    fi

    # 9. Additional IPSet management tools check (if IPSet detected)
    if [ "$has_ipset" = true ]; then
        if [ -f "/usr/local/bin/ipset-status" ]; then
            log_success "✅ IPSet status script exists"
        else
            log_warning "⚠️  IPSet status script missing"
        fi
        
        if [ -f "/usr/local/bin/ipset-backup" ]; then
            log_success "✅ IPSet backup script exists"
        else
            log_warning "⚠️  IPSet backup script missing"
        fi
    fi
    
    # Summary
    if [ "$has_ipset" = true ]; then
        log_info "Verification completed for Fail2Ban with IPSet integration"
    else
        log_info "Verification completed for standard Fail2Ban installation"
    fi
    
    return $([ "$verification_passed" = true ] && echo 0 || echo 1)
}

show_fail2ban_summary() {
    local has_ipset=false
    
    # Detect if IPSet integration is being used
    if command -v ipset >/dev/null 2>&1 && ipset list fail2ban-banned >/dev/null 2>&1; then
        has_ipset=true
    fi
    
    echo
    if [ "$has_ipset" = true ]; then
        log_info "📋 Fail2Ban with IPSet Integration Summary"
        echo "============================================="
        echo "🔐 SSH Port: $SSH_PORT"
        echo "🚫 Ban Policy: Permanent bans via IPSet"
        echo "🕒 SSH Attempts: 3 failures in 30 minutes"
        echo "🕒 Scanner Detection: 2 connections in 5 minutes"
        echo "🔧 Firewall Integration: UFW + IPSet"
        echo "💾 IPSet Persistence: Enabled"
        echo
        echo "📁 Key Configuration Files:"
        echo "  • Main config: /etc/fail2ban/jail.local"
        echo "  • IPSet actions: /etc/fail2ban/action.d/ipset-*.conf"
        echo "  • IPSet rules: /etc/ipset.rules"
        echo "  • UFW before rules: /etc/ufw/before.rules"
        echo "  • SSH filter: /etc/fail2ban/filter.d/sshd.local"
        echo "  • Scanner filter: /etc/fail2ban/filter.d/ssh-scanner.conf"
        echo "  • Nginx filters: /etc/fail2ban/filter.d/nginx-*.conf"
        echo
        echo "🔧 Management Commands:"
        echo "  • Fail2Ban status: fail2ban-client status"
        echo "  • IPSet status: ipset-status"
        echo "  • IPSet backup: ipset-backup"
        echo "  • View banned IPs: ipset list fail2ban-banned"
        echo "  • Unban IP: fail2ban-client set <jail> unbanip <IP>"
        echo "  • Manual IPSet ban: ipset add fail2ban-banned <IP>"
        echo "  • Manual IPSet unban: ipset del fail2ban-banned <IP>"
        echo
        echo "🔧 Service Management:"
        echo "  • Restart Fail2Ban: systemctl restart fail2ban"
        echo "  • Reload UFW: ufw reload"
        echo "  • Save IPSet: ipset save > /etc/ipset.rules"
        echo "============================================="
    else
        log_info "📋 Fail2Ban Configuration Summary"
        echo "=================================="
        echo "🔐 SSH Port: $SSH_PORT"
        echo "🚫 Ban Policy: Permanent bans"
        echo "🕒 SSH Attempts: 3 failures in 30 minutes"
        echo "🕒 Scanner Detection: 2 connections in 5 minutes"  
        echo "🔧 Firewall Integration: UFW"
        echo
        echo "📁 Configuration Files:"
        echo "  • Main config: /etc/fail2ban/jail.local"
        echo "  • SSH filter: /etc/fail2ban/filter.d/sshd.local"
        echo "  • Scanner filter: /etc/fail2ban/filter.d/ssh-scanner.conf"
        echo "  • Nginx filters: /etc/fail2ban/filter.d/nginx-*.conf"
        echo
        echo "🔧 Management Commands:"
        echo "  • Check status: fail2ban-client status"
        echo "  • Check SSH jail: fail2ban-client status sshd"
        echo "  • View logs: tail -f /var/log/fail2ban.log"
        echo "  • Unban IP: fail2ban-client set sshd unbanip <IP>"
        echo "  • Restart service: systemctl restart fail2ban"
        echo "=================================="
    fi
    
    echo
    echo "🔧 Additional Tools:"
    echo "  • Security check: fail2ban-checking"
    echo "  • Status display: fail2ban-status" 
    echo "  • Filter tester: fail2ban-tester"
    echo
    echo "🔧 Troubleshooting:"
    echo "  • Check service status: systemctl status fail2ban"
    echo "  • View logs: journalctl -u fail2ban -f"
    echo "  • Test configuration: fail2ban-client -d"
    echo "  • Check log files: tail -f /var/log/fail2ban.log"
    echo "  • Manual restart: systemctl restart fail2ban"
    echo "  • Check UFW integration: ufw status"
    if [ "$has_ipset" = true ]; then
        echo "  • Check IPSet rules: ipset list"
        echo "  • Check firewall rules: iptables -L ufw-before-input -n"
    else
        echo "  • Check firewall rules: iptables -L -n"
    fi
    echo "=================================="
}

总结:

因为我使用的 -1 永久封,造成内存中有几百条 iptables 记录,严重影响性能。 现在使用 IPset 做为一个池来集中管理。 也在修改之前的策略改为封20小时,这个脚本中可改。

这三个工具构成了一套层次化的网络安全防护体系,它们的协同工作过程如下:

角色分工

UFW (Uncomplicated Firewall):作为 iptables 的简化前端,负责基础防火墙规则管理和流量过滤的最终执行。

IPset:高效的 IP 地址集合管理工具,使用哈希表存储大量 IP 地址,提供 O(1) 时间复杂度的查找效率。

Fail2ban:智能入侵检测系统,监控日志文件、分析攻击模式、触发自动封禁动作。

核心工作流程

检测阶段

  1. 攻击者尝试连接服务(如SSH)
  2. 服务记录认证失败日志到 /var/log/auth.log
  3. Fail2ban 实时监控日志文件,使用正则表达式匹配攻击模式
  4. 计数器跟踪每个IP的失败次数

封禁阶段

  1. 当失败次数达到阈值(如 maxretry=3),Fail2ban 触发封禁动作
  2. 执行 IPset 命令:ipset add fail2ban-ssh 攻击者IP
  3. IP被添加到预先创建的IPset集合中
  4. UFW/iptables 规则立即生效:iptables -I INPUT -m set --match-set fail2ban-ssh src -j DROP

防护阶段

  1. 后续来自该IP的所有请求在内核层面被直接丢弃
  2. 不再消耗应用服务资源
  3. 根据配置的 bantime 自动解封或永久封禁

关键性能优势

传统方式的问题是每个被封IP需要单独的 iptables 规则,当封禁IP数量达到数千个时,规则匹配效率急剧下降。

IPset 优化方式只需要一条 iptables 规则引用整个IP集合,无论集合中有多少IP,查找效率都保持恒定。这种方式可以高效处理数万个封禁IP而不影响网络性能。

实际数据流

正常流量:外部请求 → UFW检查 → IPset查询(未命中) → 通过基础规则 → 到达服务

恶意流量:攻击请求 → UFW检查 → IPset查询(命中) → 直接丢弃数据包

这种架构的核心优势是将被动防护(UFW/iptables)与主动检测(Fail2ban)相结合,同时通过IPset解决了大规模IP封禁的性能瓶颈问题。整个系统实现了从攻击检测到自动响应的完全自动化,大大降低了管理员的工作负担。

收到 CSDN 消息:


网站公告

今日签到

点亮在社区的每一天
去签到