文章目录

K8S多节点部署一、实验过程1、实验准备2、上传文件（master01）3、修改配置文件（Master02）4、启动三个组件服务（master02）5、安装nginx服务（lb01和lb02）6、修改node节点配置文件统一为VIP地址7、查看k8s日志（lb01）8、测试（master01）9、在节点上操作可以直接访问（node02）10、在k8s结点上查看日志（master01）

K8S多节点部署

一、实验过程

1、实验准备

先具备单master节点部署环境

Master01：192.168.150.128/24 kube-apiserver kube-controller-manager kube-scheduler etcd Node01： 192.168.150.179/24 kubelet kube-proxy docker flannel etcd Node02： 192.168.150.163/24 kubelet kube-proxy docker flannel etcd Master02：192.168.150.130/24 安装软件同Master01 lb01： 192.168.150.131/24 nginx keepalived lb02： 192.168.150.132/24 nginx keepalived

2、上传文件（master01）

systemctl stop firewalld.service setenforce 0 scp -r /opt/kubernetes/ root@192.168.150.130:/opt ##复制kubernetes目录到master02 yes Abc123 ##输入master02的密码 scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service root@192.168.150.130:/usr/lib/systemd/system/ ##复制master中的三个组件启动脚本kube-apiserver.service kube-controller-manager.service kube-scheduler.service Abc123 scp -r /opt/etcd/ root@192.168.150.130:/opt/ ##上传etch证书，证书一定要有 Abc123

3、修改配置文件（Master02）

cd /opt/kubernetes/cfg/

vim kube-apiserver ##配置文件修改内容如下所示

---

KUBE_APISERVER_OPTS="–logtostderr=true \

–v=4 \

–etcd-servers=https://192.168.150.128:2379,https://192.168.150.179:2379,https://192.168.150.163:2379 \

–bind-address=192.168.150.130 \

–secure-port=6443 \

–advertise-address=192.168.150.130 \

–allow-privileged=true \

–service-cluster-ip-range=10.0.0.0/24 \

–enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \

–authorization-mode=RBAC,Node \

–kubelet-https=true \

–enable-bootstrap-token-auth \

–token-auth-file=/opt/kubernetes/cfg/token.csv \

–service-node-port-range=30000-50000 \

–tls-cert-file=/opt/kubernetes/ssl/server.pem \

–tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \

–client-ca-file=/opt/kubernetes/ssl/ca.pem \

–service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \

–etcd-cafile=/opt/etcd/ssl/ca.pem \

–etcd-certfile=/opt/etcd/ssl/server.pem \

–etcd-keyfile=/opt/etcd/ssl/server-key.pem"

---

4、启动三个组件服务（master02）

systemctl start kube-apiserver.service systemctl start kube-controller-manager.service systemctl start kube-scheduler.service vim /etc/profile #末尾添加一行 export PATH=$PATH:/opt/kubernetes/bin source /etc/profile kubectl get node

5、安装nginx服务（lb01和lb02）

systemctl stop firewalld.service setenforce 0 vim /etc/yum.repos.d/nginx.repo ##文件中新增内容如下所示 [nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/7/$basearch/ gpgcheck=0 yum install nginx -y

vim /etc/nginx/nginx.conf ##添加四层转发

---

events {

worker_connections 1024;

}

stream {

log_format main ‘$remote_addr $upstrea{m}_{a}ddr-[$time_local] $status $upstream_bytes_sent’;

access_log /var/log/nginx/k8s-access.log main;

upstream k8s-apiserver {

​ server 192.168.150.128:6443;

​ server 192.168.150.130:6443;

}

server {

​ listen 6443;

​ proxy_pass k8s-apiserver;

}

}

http {

---

systemctl start nginx yum install keepalived -y ##部署keepalived服务 cp keepalived.conf /etc/keepalived/keepalived.conf ##修改配置文件 yes

vim /etc/keepalived/keepalived.conf

//注意：lb01是Mster配置如下：

---

! Configuration File for keepalived

global_defs {

# 接收邮件地址

notification_email {

acassen@firewall.loc

failover@firewall.loc

sysadmin@firewall.loc

}

# 邮件发送地址

notification_email_from Alexandre.Cassen@firewall.loc

smtp_server 127.0.0.1

smtp_connect_timeout 30

router_id NGINX_MASTER

}

vrrp_script check_nginx {

script “/etc/nginx/check_nginx.sh”

}

vrrp_instance VI_1 {

state MASTER

interface ens33

virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的

priority 100 # 优先级，备服务器设置 90

advert_int 1 # 指定VRRP 心跳包通告间隔时间，默认1秒

authentication {

​ auth_type PASS

​ auth_pass 1111

}

virtual_ipaddress {

​ 192.168.150.100/24

}

track_script {

​ check_nginx

}

}

---

//注意：lb02是Backup配置如下：

---

! Configuration File for keepalived

global_defs {

# 接收邮件地址

notification_email {

acassen@firewall.loc

failover@firewall.loc

sysadmin@firewall.loc

}

# 邮件发送地址

notification_email_from Alexandre.Cassen@firewall.loc

smtp_server 127.0.0.1

smtp_connect_timeout 30

router_id NGINX_MASTER

}

vrrp_script check_nginx {

script “/etc/nginx/check_nginx.sh”

}

vrrp_instance VI_1 {

state BACKUP

interface ens33

virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的

priority 90 # 优先级，备服务器设置 90

advert_int 1 # 指定VRRP 心跳包通告间隔时间，默认1秒

authentication {

​ auth_type PASS

​ auth_pass 1111

}

virtual_ipaddress {

​ 192.168.195.100/24

}

track_script {

​ check_nginx

}

}

---

vim /etc/nginx/check_nginx.sh ##新增内容如下所示

count=$(ps -ef |grep nginx |egrep -cv "grep|$$") if [ "$count" -eq 0 ];then systemctl stop keepalived fi chmod +x /etc/nginx/check_nginx.sh systemctl start keepalived ip a //验证地址漂移（lb01中使用pkill nginx，再在lb02中使用ip a 查看） //恢复操作（在lb01中先启动nginx服务，再启动keepalived服务） //nginx站点/usr/share/nginx/html

6、修改node节点配置文件统一为VIP地址

vim /opt/kubernetes/cfg/bootstrap.kubeconfig vim /opt/kubernetes/cfg/kubelet.kubeconfig vim /opt/kubernetes/cfg/kube-proxy.kubeconfig

//三个配置文件中统统修改为VIP

server: https://192.168.150.100:6443

systemctl restart kubelet.service systemctl restart kube-proxy.service cd /opt/kubernetes/cfg grep 100 * ##替换完成直接自检

7、查看k8s日志（lb01）

tail /var/log/nginx/k8s-access.log

8、测试（master01）

kubectl run nginx --image=nginx ##测试创建pod [root@localhost ~]# kubectl get pods ##查看状态 NAME READY STATUS RESTARTS AGE nginx-dbddb74b8-gcf9h 0/1 ContainerCreating 0 33s //正在创建中 [root@localhost ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-dbddb74b8-gcf9h 1/1 Running 0 80s //创建完成，运行中 kubectl logs nginx-dbddb74b8-gcf9h //查看日志

Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-dbddb74b8-nf9sk)

kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous kubectl get pods -o wide ##查看pod网络 nginx-dbddb74b8-gcf9h 1/1 Running 0 6m 172.17.56.2 192.168.150.163 <none>

9、在节点上操作可以直接访问（node02）

curl 172.17.56.2 ##直接访问可以看到nginx界面信息

10、在k8s结点上查看日志（master01）

kubectl logs nginx-dbddb74b8-gcf9h