基于上次的单节点部署开始制作，master02的IP地址实际上在上一篇有所提及，欢迎查看上一篇博客
复制配置文件以及启动脚本
[root@localhost ~]# scp -r /opt/kubernetes/ root@20.0.0.6:/opt/ [root@localhost ~]# cd /usr/lib/systemd/system [root@localhost system]# scp kube-apiserver.service kube-controller-manager.service kube-scheduler.service root@20.0.0.6:/usr/lib/systemd/system/ root@20.0.0.6's password: kube-apiserver.service 100% 282 258.3KB/s 00:00 kube-controller-manager.service 100% 317 343.0KB/s 00:00 kube-scheduler.service 100% 281 376.5KB/s 00:00拷贝etcd的文件给master02
[root@localhost /]# scp -r /opt/etcd/ root@20.0.0.6:/opt/ root@20.0.0.6's password:修改配置文件kube-apiserver中的IP地址
[root@localhost /]# vim /opt/kubernetes/cfg/kube-apiserver --bind-address=20.0.0.6 \ ##改这行 --secure-port=6443 \ --advertise-address=20.0.0.6 \ ##还有这行 启动三个服务 [root@localhost ~]# systemctl start kube-apiserver.service [root@localhost ~]# systemctl start kube-controller-manager.service [root@localhost ~]# systemctl start kube-scheduler.service 增加环境变量 [root@localhost ~]# vim /etc/profile ##增加在末行 export PATH=$PATH:/opt/kubernetes/bin/ [root@localhost ~]# source /etc/profile [root@localhost ~]# kubectl get node NAME STATUS ROLES AGE VERSION 20.0.0.4 Ready <none> 6d10h v1.12.3 20.0.0.5 Ready <none> 6d10h v1.12.3两台ngxin，在上一篇博客里也有写道 两台nginx服务器操作一致
[root@7CentOS ~]# systemctl stop firewalld [root@7CentOS ~]# setenforce 0 [root@7CentOS ~]# vim /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/7/$basearch/ gpgcheck=0 [root@7CentOS ~]# yum -y install nginx [root@7CentOS ~]# vim /etc/nginx/nginx.conf events { worker_connections 1024; } ##在下面添加 stream { log_format main '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent'; ##日期格式 access_log /var/log/nginx/k8s-access.log main; ##日志记载位置 upstream k8s-apiserver { ##负载均衡IP地址 server 20.0.0.3:6443; server 20.0.0.6:6443; } server { listen 6443; ##nginx服务的端口为6443 proxy_pass k8s-apiserver; } } [root@7CentOS ~]# systemctl start nginx [root@7CentOS ~]# netstat -ntap |grep 6443 tcp 0 0 0.0.0.0:6443 0.0.0.0:* LISTEN 13900/nginx: master事先上传了一个keepalived.conf keepalived master
[root@7CentOS /]# yum -y install keepalived [root@7CentOS ~]# cp keepalived.conf /etc/keepalived/keepalived.conf cp：是否覆盖"/etc/keepalived/keepalived.conf"？ yes [root@7CentOS ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { # 接收邮件地址 notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } # 邮件发送地址 notification_email_from Alexandre.Cassen@firewall.loc smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id NGINX_MASTER } vrrp_script check_nginx { script "/usr/local/nginx/check_nginx.sh" } vrrp_instance VI_1 { state MASTER interface ens33 virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的 priority 100 # 优先级，备服务器设置 90 advert_int 1 # 指定VRRP 心跳包通告间隔时间，默认1秒 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 20.0.0.150/24 } track_script { check_nginx } } [root@7CentOS ~]# vim /etc/nginx/check_nginx.sh count=$(ps -ef |grep nginx |egrep -cv "grep|$$") if [ "$count" -eq 0 ];then systemctl stop keepalived fi [root@7CentOS ~]# chmod +x /etc/nginx/check_nginx.sh [root@7CentOS ~]# systemctl start keepalived [root@7CentOS ~]# ip a 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:93:2c:0b brd ff:ff:ff:ff:ff:ff inet 20.0.0.7/24 brd 20.0.0.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet 20.0.0.150/24 scope global secondary ens33 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe93:2c0b/64 scope link valid_lft forever preferred_lft foreverslave部署
[root@localhost nginx]# yum -y install keepalived master复制文件给slave [root@7CentOS ~]# scp -r /etc/keepalived/ root@20.0.0.8:/etc/ root@20.0.0.8's password: keepalived.conf 100% 778 1.0MB/s 00:00 [root@localhost nginx]# vim /etc/keepalived/keepalived.conf ##进行修改 vrrp_instance VI_1 { state BACKUP ##这行 interface ens33 virtual_router_id 51 priority 90 ##这行 [root@localhost nginx]# vim /etc/nginx/check_nginx.sh count=$(ps -ef |grep nginx |egrep -cv "grep|$$") if [ "$count" -eq 0 ];then systemctl stop keepalived fi [root@localhost nginx]# systemctl start keepalived 注意：slave的ip a s不应该有浮动ip验证浮动IP 杀掉master的nginx
[root@7CentOS ~]# pkill nginxslave 查看
[root@localhost nginx]# ip a 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:f1:9c:a7 brd ff:ff:ff:ff:ff:ff inet 20.0.0.8/24 brd 20.0.0.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet 20.0.0.150/24 scope global secondary ens33 valid_lft forever preferred_lft forever inet6 fe80::68ac:e5ec:8230:c3b8/64 scope link noprefixroute valid_lft forever preferred_lft forevermaster启动nginx，再启动keepalived
[root@7CentOS ~]# systemctl start nginx [root@7CentOS ~]# systemctl start keepalivedslave再查看
[root@localhost nginx]# ip a ##浮动IP消失 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:f1:9c:a7 brd ff:ff:ff:ff:ff:ff inet 20.0.0.8/24 brd 20.0.0.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::68ac:e5ec:8230:c3b8/64 scope link noprefixroute valid_lft forever preferred_lft forevernode1上能看到，node2上则没有 此时访问日志会报错
[root@localhost /]# kubectl logs nginx-dbddb74b8-zxpk4 Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-dbddb74b8-zxpk4)使用下面的授权，再查看就成功了
[root@localhost /]# kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous clusterrolebinding.rbac.authorization.k8s.io/cluster-system-anonymous created [root@localhost /]# kubectl logs nginx-dbddb74b8-zxpk4 /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh 10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf 10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh /docker-entrypoint.sh: Configuration complete; ready for start up去node1查看网页
[root@localhost cfg]# curl 172.17.13.2 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>此时在master上也能看到日志信息了
[root@localhost /]# kubectl logs nginx-dbddb74b8-zxpk4 /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh 10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf 10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh /docker-entrypoint.sh: Configuration complete; ready for start up 172.17.13.1 - - [06/Oct/2020:16:53:50 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"