【kubernetes】高可用组件安装-Haproxy、Keepalived

高可用组件我们使用Haproxy和Keepalived的组合，是为了实现集群控制节点的高可用，在生产环境很有必要，防止出现节点宕机等异常造成的不可用事故。

1. 安装HAProxy

所有Master节点通过yum安装HAProxy：

yum install haproxy -y

所有Master节点的HAProxy配置相同：

cat >/etc/haproxy/haproxy.cfg<<"EOF"
global
  maxconn  2000
  ulimit-n  16384
  log  127.0.0.1 local0 err
  stats timeout 30s

defaults
  log global
  mode  http
  option  httplog
  timeout connect 5000
  timeout client  50000
  timeout server  50000
  timeout http-request 15s
  timeout http-keep-alive 15s

frontend monitor-in
  bind *:33305
  mode http
  option httplog
  monitor-uri /monitor

frontend k8s-master
  bind 0.0.0.0:16443
  bind 127.0.0.1:16443
  mode tcp
  option tcplog
  tcp-request inspect-delay 5s
  default_backend k8s-master

backend k8s-master
  mode tcp
  option tcplog
  option tcp-check
  balance roundrobin
  default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
  server k8s-master01 11.0.1.21:6443 check
  server k8s-master02 11.0.1.22:6443 check
  server k8s-master03 11.0.1.23:6443 check
EOF

高可用组件也可独立部署。

2. 安装Keepalived

每个Master节点安装Keepalived

yum install -y keepalived

每个节点的配置不相同：

# 在master01节点执行
cat >/etc/keepalived/keepalived.conf<<"EOF"
! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
    script_user root
    enable_script_security
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 5
    weight -5
    fall 2  
    rise 1
}
vrrp_instance VI_1 {
    state MASTER
    interface ens33
    mcast_src_ip 11.0.1.21
    virtual_router_id 51
    priority 101
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        11.0.1.100
    }
    track_script {
       chk_apiserver
    }
}
EOF

# 在master02节点执行
cat >/etc/keepalived/keepalived.conf<<"EOF"
! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
    script_user root
    enable_script_security
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 5
    weight -5
    fall 2  
    rise 1
}
vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    mcast_src_ip 11.0.1.22
    virtual_router_id 51
    priority 100
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        11.0.1.100
    }
    track_script {
       chk_apiserver
    }
}
EOF

# 在master03节点执行
cat >/etc/keepalived/keepalived.conf<<"EOF"
! Configuration File for keepalived
global_defs {
    router_id LVS_DEVEL
    script_user root
    enable_script_security
}
vrrp_script chk_apiserver {
    script "/etc/keepalived/check_apiserver.sh"
    interval 5
    weight -5
    fall 2  
    rise 1
}
vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    mcast_src_ip 11.0.1.23
    virtual_router_id 51
    priority 100
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass K8SHA_KA_AUTH
    }
    virtual_ipaddress {
        11.0.1.100
    }
    track_script {
       chk_apiserver
    }
}
EOF

说明：`state`有`MASTER`和`BACKUP`的区别，分别表示主节点和从节点。`11.0.1.100`为集群规划中的虚拟ip。

所有master节点配置KeepAlived健康检查文件：

cat > /etc/keepalived/check_apiserver.sh <<"EOF"
#!/bin/bash

err=0
for k in $(seq 1 3)
do
    check_code=$(pgrep haproxy)
    if [[ $check_code == "" ]]; then
        err=$(expr $err + 1)
        sleep 1
        continue
    else
        err=0
        break
    fi
done

if [[ $err != "0" ]]; then
    echo "systemctl stop keepalived"
    /usr/bin/systemctl stop keepalived
    exit 1
else
    exit 0
fi
EOF

健康检查脚本可执行权限：

chmod +x /etc/keepalived/check_apiserver.sh

启动haproxy和keepalived：

systemctl daemon-reload
systemctl enable --now haproxy
systemctl enable --now keepalived

测试VIP可用性：

ping 11.0.1.100 -c 4

telnet 11.0.1.100 16443

如果VIP测试有问题，不可再继续往下做，必须排查相应的问题后才可继续。