部署sentinel集群

Redis的哨兵模式

环境搭建

​ 思路:redis是一个缓冲数据库服务器,它可以减轻后端服务器的压力,版主后端服务器解决客户端的问题,当客户端访问服务器的时候,不需要直接去找后端服务器,而是先经过redis来查询客户找的问题,若redis没有客户端要找的数据,这时,再去服务器查找,找到之后会经过redis缓冲服务,这时redis便会记录下来这次的数据,下次就可以直接在redis中查找,这样可以减轻服务器的压力,防止客户端的过多,造成服务器的崩溃.因此在redis中,我们为了防止一台redis,down掉,所以会给他搭建集群,增加哨兵模式,此时,哨兵的作用就是用来监视redis的主从,一旦有某一台redis,down掉,这时可以将从的redis直接替换为主,服务器正常工作,而哨兵模式就是为了监视redis,当然,如果只有一台哨兵的话,为了防止一台哨兵出现故障,所以,我们会搭建一个哨兵的集群,给哨兵也建立主从模式,这样在一个哨兵坏掉的情况下,其他的哨兵可以替换他,这时就实现了主哨兵,从哨兵的模式,每个哨兵之间都是互相监视的,同时也监视redis的主从,只要有一个坏掉,其他的便都会自动替换.所有一下便是搭建哨兵的全部过程.

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fAAArA3r-1602044054882)(/home/afang/文档/笔记/第三阶段/第二周/第五天 Redis 数据库详解/assets/1190037-20180203181035312-1532457764.png)]

这里需要了解的是,主观下线和客观下线,主观下线是指单个实例确定这个哨兵下线,而客观下线指的是多个实例确定这个哨兵下线,不能处于工作状态.

准备三台服务器,这里准备的是一台master,两台slave,服务器的允许的前提下,可以多准备两台slave.

主机的名称IPredis192.168.122.166slave192.168.122.210slave01192.168.122.181

安装redis-rpm源(三台装)

[root@redis-master ~]# yum -y install http://rpms.famillecollet.com/enterprise/remi-release-7.rpm

安装redis

[root@redis-master ~]# yum -y --enablerepo=remi install redis

设置开机自启

[root@redis-master ~]# systemctl enable redis

修改主从的redis配置文件

[root@redis-master ~]#vim /etc/redis.conf #可以将原先的配置文件备份 bind 192.168.122.166 127.0.0.1 protected-mode no port 6379 tcp-backlog 511 timeout 0 tcp-keepalive 300 daemonize no #打开之后,其他的哨兵将会连接不到 supervised no pidfile /var/run/redis_6379.pid loglevel notice logfile /var/log/redis/redis.log databases 16 always-show-logo yes save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename dump.rdb rdb-del-sync-files no dir /var/lib/redis replica-serve-stale-data yes replica-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-diskless-load disabled repl-disable-tcp-nodelay no replica-priority 100 acllog-max-len 128 lazyfree-lazy-eviction no lazyfree-lazy-expire no lazyfree-lazy-server-del no replica-lazy-flush no lazyfree-lazy-user-del no appendonly no appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes aof-use-rdb-preamble yes lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 stream-node-max-bytes 4096 stream-node-max-entries 100 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit replica 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 dynamic-hz yes aof-rewrite-incremental-fsync yes rdb-save-incremental-fsync yes jemalloc-bg-thread yes requirepass redis # 配置 master密码 [root@redis-slave ~]# vim /etc/redis.conf bind 192.168.122.210 protected-mode yes port 6379 tcp-backlog 511 timeout 0 tcp-keepalive 300 daemonize no supervised no pidfile /var/run/redis_6379.pid loglevel notice logfile /var/log/redis/redis.log databases 16 always-show-logo yes save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename dump.rdb rdb-del-sync-files no dir /var/lib/redis replica-serve-stale-data yes replica-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-diskless-load disabled repl-disable-tcp-nodelay no replica-priority 100 acllog-max-len 128 lazyfree-lazy-eviction no lazyfree-lazy-expire no lazyfree-lazy-server-del no replica-lazy-flush no lazyfree-lazy-user-del no appendonly no appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes aof-use-rdb-preamble yes lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 stream-node-max-bytes 4096 stream-node-max-entries 100 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit replica 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 dynamic-hz yes aof-rewrite-incremental-fsync yes rdb-save-incremental-fsync yes jemalloc-bg-thread yes requirepass redis replicaof 192.168.122.166 6379 # 配置为 master的从， masterauth redis # 配置mast的认证密码 [root@redis-slave01 ~]# vim /etc/redis.conf bind 192.168.122.181 protected-mode yes port 6379 tcp-backlog 511 timeout 0 tcp-keepalive 300 daemonize no supervised no pidfile /var/run/redis_6379.pid loglevel notice logfile /var/log/redis/redis.log databases 16 always-show-logo yes save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename dump.rdb rdb-del-sync-files no dir /var/lib/redis replica-serve-stale-data yes replica-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-diskless-load disabled repl-disable-tcp-nodelay no replica-priority 100 acllog-max-len 128 lazyfree-lazy-eviction no lazyfree-lazy-expire no lazyfree-lazy-server-del no replica-lazy-flush no lazyfree-lazy-user-del no appendonly no appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes aof-use-rdb-preamble yes lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 stream-node-max-bytes 4096 stream-node-max-entries 100 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit replica 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 dynamic-hz yes aof-rewrite-incremental-fsync yes rdb-save-incremental-fsync yes jemalloc-bg-thread yes requirepass redis replicaof 192.168.122.166 6379 # 配置为master的从 masterauth redis # 配置master的认证密码

部署sentinel

修改sentinel的配置文件(三台都需要修改)

[root@redis-master ~]# vim /etc/redis-sentinel.conf port 26379 daemonize no pidfile /var/run/redis-sentinel.pid logfile /var/log/redis/sentinel.log dir /tmp sentinel monitor mymaster 127.0.0.1 6379 2 sentinel down-after-milliseconds mymaster 30000 sentinel parallel-syncs mymaster 1 sentinel failover-timeout mymaster 180000 sentinel deny-scripts-reconfig yes # 端口 port 26379 # 是否后台启动 daemonize yes # 保护模式如果开启只接受回环地址的ipv4和ipv6地址链接，拒绝外部链接，而且正常应该配置多个哨兵，避免一个哨兵出现独裁情况，如果配置多个哨兵那如果开启也会拒绝其他sentinel的连接。导致哨兵配置无法生效。 protected-mode no 　 # pid文件路径 pidfile /var/run/redis-sentinel.pid # 日志文件路径 logfile "/var/log/redis/sentinel.log" # 定义工作目录 dir /tmp # 定义Redis主的别名, IP, 端口，这里的2指的是需要至少2个Sentinel认为主Redis挂了才最终会采取下一步行为 sentinel monitor mymaster 127.0.0.1 6379 2 #如果redis配置了密码，那这里必须配置认证，否则不能自动切换 sentinel auth-pass mymaster redispass 　　 # 如果mymaster 30秒内没有响应，则认为其主观失效 sentinel down-after-milliseconds mymaster 30000 # 如果master重新选出来后，其它slave节点能同时并行从新master同步数据的台数有多少个，显然该值越大，所有slave节点完成同步切换的整体速度越快，但如果此时正好有人在访问这些slave，可能造成读取失败，影响面会更广。最保守的设置为1，同一时间，只能有一台干这件事，这样其它slave还能继续服务，但是所有slave全部完成缓存更新同步的进程将变慢。 sentinel parallel-syncs mymaster 1 # 该参数指定一个时间段，在该时间段内没有实现故障转移成功，则会再一次发起故障转移的操作，单位毫秒 sentinel failover-timeout mymaster 180000 # 不允许使用SENTINEL SET设置notification-script和client-reconfig-script。 sentinel deny-scripts-reconfig yes [root@redis-slave ~]# vim /etc/redis-sentinel.conf port 26379 daemonize no pidfile "/var/run/redis-sentinel.pid" logfile "/var/log/redis/sentinel.log" dir "/tmp" sentinel monitor mymaster 192.168.122.166 6379 2 sentinel auth-pass mymaster redis sentinel down-after-milliseconds mymaster 30000 sentinel parallel-syncs mymaster 1 sentinel failover-timeout mymaster 180000 sentinel deny-scripts-reconfig yes [root@redis-slave01 ~]# vim /etc/redis-sentinel.conf port 26379 daemonize yes pidfile "/var/run/redis-sentinel.pid" logfile "/var/log/redis/sentinel.log" dir "/tmp" sentinel monitor mymaster 192.168.122.166 6379 2 sentinel auth-pass mymaster redis sentinel down-after-milliseconds mymaster 30000 sentinel parallel-syncs mymaster 1 sentinel failover-timeout mymaster 180000 sentinel deny-scripts-reconfig yes

启动sentinel

注意启动的顺序:主Redis --> 从Redis --> Sentinel1/2/3

[root@redis-master ~]# systemctl start/restart redis [root@redis-master ~]# chown -R redis.redis /etc/redis-sentinel.conf # yum 安装需要redis用户写入配置问价权限,注意每台都要执行 [root@redis-master ~]# systemctl restart redis-sentinel.service [root@redis-master ~]# systemctl status redis-sentinel.service ● redis-sentinel.service - Redis Sentinel Loaded: loaded (/usr/lib/systemd/system/redis-sentinel.service; disabled; vendor preset: disabled) Drop-In: /etc/systemd/system/redis-sentinel.service.d └─limit.conf Active: active (running) since Fri 2020-06-26 10:43:18 CST; 12s ago Process: 26934 ExecStop=/usr/libexec/redis-shutdown redis-sentinel (code=exited, status=1/FAILURE) Main PID: 27107 (redis-sentinel) Status: "Ready to accept connections" CGroup: /system.slice/redis-sentinel.service └─27107 /usr/bin/redis-sentinel *:26379 [sentinel] Jun 26 10:43:18 redis1 systemd[1]: Starting Redis Sentinel... Jun 26 10:43:18 redis1 systemd[1]: Started Redis Sentinel. [root@redis-master ~]# ps -ef |grep redis redis 26485 1 0 10:03 ? 00:00:03 /usr/bin/redis-server 192.168.152.133:6379 redis 27107 1 0 10:43 ? 00:00:00 /usr/bin/redis-sentinel *:26379 [sentinel] root 27150 25442 0 10:51 pts/0 00:00:00 grep --color=auto redis [root@redis-slave ~]# systemctl start/restart redis [root@redis-slave ~]# chown -R redis.redis /etc/redis-sentinel.conf [root@redis-slave ~]# systemctl restart redis-sentinel.service [root@redis-slave ~]# systemctl status redis-sentinel.service ● redis-sentinel.service - Redis Sentinel Loaded: loaded (/usr/lib/systemd/system/redis-sentinel.service; disabled; vendor preset: disabled) Drop-In: /etc/systemd/system/redis-sentinel.service.d └─limit.conf Active: active (running) since Fri 2020-06-26 10:45:28 CST; 13s ago Main PID: 26389 (redis-sentinel) Status: "Ready to accept connections" CGroup: /system.slice/redis-sentinel.service └─26389 /usr/bin/redis-sentinel *:26379 [sentinel] Jun 26 10:45:28 redis2 systemd[1]: Starting Redis Sentinel... Jun 26 10:45:28 redis2 systemd[1]: Started Redis Sentinel. [root@redis-slave01 ~]# ps -ef |grep redis redis 26199 1 0 10:08 ? 00:00:02 /usr/bin/redis-server 192.168.152.134:6379 redis 26389 1 0 10:45 ? 00:00:00 /usr/bin/redis-sentinel *:26379 [sentinel] root 26431 25372 0 10:52 pts/0 00:00:00 grep --color=auto redis [root@redis-slave01 ~]# systemctl start/restart redis [root@redis-slave01 ~]# chown -R redis.redis /etc/redis-sentinel.conf [root@redis-slave01 ~]# systemctl restart redis-sentinel.service [root@redis-slave01 ~]# systemctl status redis-sentinel.service ● redis-sentinel.service - Redis Sentinel Loaded: loaded (/usr/lib/systemd/system/redis-sentinel.service; disabled; vendor preset: disabled) Drop-In: /etc/systemd/system/redis-sentinel.service.d └─limit.conf Active: active (running) since Fri 2020-06-26 10:48:48 CST; 21s ago Process: 26410 ExecStop=/usr/libexec/redis-shutdown redis-sentinel (code=exited, status=1/FAILURE) Main PID: 26474 (redis-sentinel) Status: "Ready to accept connections" CGroup: /system.slice/redis-sentinel.service └─26474 /usr/bin/redis-sentinel *:26379 [sentinel] Jun 26 10:48:48 redis3 systemd[1]: Starting Redis Sentinel... Jun 26 10:48:48 redis3 systemd[1]: Started Redis Sentinel. [root@redis3 ~]# ps -ef |grep redis redis 26202 1 0 10:10 ? 00:00:02 /usr/bin/redis-server 192.168.152.135:6379 redis 26474 1 0 10:48 ? 00:00:00 /usr/bin/redis-sentinel *:26379 [sentinel] root 26494 25363 0 10:49 pts/0 00:00:00 grep --color=auto redis

检测sentinel

[root@redis-master ~]# redis-cli -p 26379 # 哨兵模式查看 127.0.0.1:26379> sentinel master mymaster # 输出被监控的主节点的状态信息 1) "name" 2) "mymaster" 3) "ip" 4) "192.168.122.166" 5) "port" 6) "6379" 7) "runid" 8) "bae06cc3bc6dcbff7c2de1510df7faf1a6eb6941" 9) "flags" 10) "master" ...... 127.0.0.1:26379> sentinel slaves mymaster # 查看mymaster的从信息，可以看到有2个从节点 1) 1) "name" 2) "192.168.122.210:6379" 3) "ip" 4) "192.168.122.210" 5) "port" 6) "6379" 7) "runid" 8) "c86027e7bdd217cb584b1bd7a6fea4ba79cf6364" 9) "flags" 10) "slave" ...... 2) 1) "name" 2) "192.168.122.181:6379" 3) "ip" 4) "192.168.122.181" 5) "port" 6) "6379" 7) "runid" 8) "61597fdb615ecf8bd7fc18e143112401ed6156ec" 9) "flags" 10) "slave" ...... 127.0.0.1:26379> sentinel sentinels mymaster # 查看其它sentinel信息 1) 1) "name" 2) "ba12e2a4023d2e9bcad282395ba6b14030920070" 3) "ip" 4) "192.168.122.210" 5) "port" 6) "26379" 7) "runid" 8) "ba12e2a4023d2e9bcad282395ba6b14030920070" 9) "flags" 10) "sentinel" ...... 2) 1) "name" 2) "14fca3f851e9e1bd3a4a0dc8a9e34bb237648455" 3) "ip" 4) "192.168.122.181" 5) "port" 6) "26379" 7) "runid" 8) "14fca3f851e9e1bd3a4a0dc8a9e34bb237648455" 9) "flags" 10) "sentinel"

哨兵模式下的主从测试

模拟停止master上的Redis，查看Redis的主从变化，如下： [root@redis-master ~]# systemctl stop redis # 停止master上的redis [root@redis-slave01 ~]# tail -n 20 /var/log/redis/sentinel.log # 查看哨兵日志 ...... 1747:X 19 Apr 2019 14:59:01.747 # +monitor master mymaster 192.168.152.133 6379 quorum 2 1747:X 19 Apr 2019 14:59:44.829 # +sdown sentinel 14fca3f851e9e1bd3a4a0dc8a9e34bb237648455 192.168.152.1343 26379 @ mymaster 192.168.152.133 6379 1747:X 19 Apr 2019 14:59:46.950 # -sdown sentinel 14fca3f851e9e1bd3a4a0dc8a9e34bb237648455 192.168.152.1343 26379 @ mymaster 192.168.152.133 6379 1747:X 19 Apr 2019 15:00:44.391 # +sdown master mymaster 192.168.152.133 6379 1747:X 19 Apr 2019 15:00:44.525 # +new-epoch 1 1747:X 19 Apr 2019 15:00:44.527 # +vote-for-leader 14fca3f851e9e1bd3a4a0dc8a9e34bb237648455 1 1747:X 19 Apr 2019 15:00:45.023 # +config-update-from sentinel 14fca3f851e9e1bd3a4a0dc8a9e34bb237648455 192.168.152.1343 26379 @ mymaster 192.168.152.133 6379 1747:X 19 Apr 2019 15:00:45.023 # +switch-master mymaster 192.168.152.133 6379 192.168.152.1343 6379 1747:X 19 Apr 2019 15:00:45.024 * +slave slave 192.168.152.1342:6379 192.168.152.1342 6379 @ mymaster 192.168.152.1343 6379 1747:X 19 Apr 2019 15:00:45.024 * +slave slave 192.168.152.133:6379 192.168.152.133 6379 @ mymaster 192.168.152.1343 6379 1747:X 19 Apr 2019 15:01:15.050 # +sdown slave 192.168.152.133:6379 192.168.152.133 6379 @ mymaster 192.168.152.1343 6379 # 从上面的日志可以看到master已经sdown，并切换为192.168.152.1343为master节点，下面查看slave01上的配置，会自动的更改replicaof配置项，如下： [root@redis-slave01 ~]# grep "replicaof" /usr/local/redis/redis.conf |grep -vE "#" replicaof 192.168.152.134 6379 [root@redis-master ~]# redis-cli -p 26379 # 哨兵模式下查看主从信息，也是可以看到主从的变化 127.0.0.1:26379> sentinel master mymaster 1) "name" 2) "mymaster" 3) "ip" 4) "192.168.122.210" 5) "port" 6) "6379" 7) "runid" 8) "61597fdb615ecf8bd7fc18e143112401ed6156ec" 9) "flags" 10) "master" ...... 127.0.0.1:26379> sentinel slaves mymaster 1) 1) "name" 2) "192.168.122.181:6379" 3) "ip" 4) "192.168.122.181" 5) "port" 6) "6379" 7) "runid" 8) "c86027e7bdd217cb584b1bd7a6fea4ba79cf6364" 9) "flags" 10) "slave" ...... 2) 1) "name" 2) "192.168.122.166:6379" 3) "ip" 4) "192.168.122.166" 5) "port" 6) "6379" 7) "runid" 8) "" 9) "flags" 10) "s_down,slave,disconnected" # 提示该节点为从，并且状态为s_down，无法链接的状态 ...... 127.0.0.1:26379> sentinel sentinels mymaster 1) 1) "name" 2) "039040a7f77cec7ec78b914fcf5ab6c65db2c77f" 3) "ip" 4) "192.168.122.210" 5) "port" 6) "26379" 7) "runid" 8) "039040a7f77cec7ec78b914fcf5ab6c65db2c77f" 9) "flags" 10) "sentinel" 11) "link-pending-commands" 12) "0" 13) "link-refcount" 14) "1" 15) "last-ping-sent" 16) "0" 17) "last-ok-ping-reply" 18) "437" 19) "last-ping-reply" 20) "437" 21) "down-after-milliseconds" 22) "30000" 23) "last-hello-message" 24) "858" 25) "voted-leader" 26) "?" 27) "voted-leader-epoch" 28) "1" 2) 1) "name" 2) "347c334916045f839632d19b04ed391973faf522" 3) "ip" 4) "192.168.122.181" 5) "port" 6) "26379" 7) "runid" 8) "347c334916045f839632d19b04ed391973faf522" 9) "flags" 10) "sentinel" 11) "link-pending-commands" 12) "0" 13) "link-refcount" 14) "1" 15) "last-ping-sent" 16) "0" 17) "last-ok-ping-reply" 18) "437" 19) "last-ping-reply" 20) "437" 21) "down-after-milliseconds" 22) "30000" 23) "last-hello-message" 24) "1979" 25) "voted-leader" 26) "?" 27) "voted-leader-epoch" 28) "1"

部署完成之后,停止redis的情况下,从的哨兵会替换主,就证明自己的部署成功了