Redis的哨兵机制

liuyong00

2020-05-15

关注关注

1.准备3台服务器，一台master ，两台 slave

Master 　　192.168.146.128

Slave 　　 192.168.146.129

slave 　 192.168.146.130

2.哨兵机制需要先实现Redis的主从复制

Redis的主从复制的实现详细参考：https://www.cnblogs.com/Amywangqing/p/12897247.html

3.到redis的安装目录下有一个runtest-sentinel配置文件将配置文件拷贝到/usr/local/redis目录下

cp sentinel.conf  /usr/local/redis

4.在/usr/local/redis目录下创建sentinel文件，用于存放日志的

mkdir sentinel

5.修改sentinel.conf文件

protected-mode no
#定的主机地址
#bind 127.0.0.1 192.168.146.128
#默认号，可以修改
port 26379
#后台启动
daemonize yes
#进程pid
pidfile /var/run/redis-sentinel.pid
#用于存放日志的
logfile /usr/local/redis/sentinel/redis-sentinel.log
#工作目录
dir /usr/local/redis/sentinel/
#告诉sentinel去监听地址为ip:port的一个master，这里的master-name可以自定义，quorum是一个数字，指明当有多少个sentinel认为一个master失效时，master才算真正失效
sentinel monitor mymaster 192.168.146.128 6379 2

设置连接master的密码，这里的密码是我们设置那台为master的redis连接密码
sentinel auth-pass mymaster 123456

这个配置项指定了需要多少失效时间，单位是毫秒，默认为30秒
sentinel down-after-milliseconds mymaster 30000


sentinel parallel-syncs mymaster 1

sentinel failover-timeout mymaster 180000

保存sentinel.conf配置文件

6.sentinel启动（注意：redis必须启动着）

redis-sentinel sentinel.conf

通过ps aux | grep redis查看

[ redis]# ps aux | grep redis
root      3576  0.1  1.0 158044 10992 ?        Ssl  Apr21   0:49 ./bin/redis-server 192.168.146.128:6379
root      9739  0.3  0.7 152408  7912 ?        Ssl  05:52   0:00 redis-sentinel 127.0.0.1:26379 [sentinel]
root      9754  0.0  0.0 103248   828 pts/0    S+   05:53   0:00 grep redis
[ redis]#

root 9739 0.3 0.7 152408 7912 ? Ssl 05:52 0:00 redis-sentinel 127.0.0.1:26379 [sentinel] 这表示启动成功端口号为26379

另外两台Slave 192.168.146.129 slave 192.168.146.130和上面一样配置

6.修改sentinel.conf配置文件都一样，不改变

#保护模式关闭
protected-mode no
#默认号，可以修改
port 26379
#后台启动
daemonize yes
#进程pid
pidfile /var/run/redis-sentinel.pid
#用于存放日志的
logfile /usr/local/redis/sentinel/redis-sentinel.log
#工作目录
dir /usr/local/redis/sentinel/
#告诉sentinel去监听地址为ip:port的一个master，这里的master-name可以自定义，quorum是一个数字，指明当有多少个sentinel认为一个master失效时，master才算真正失效
sentinel monitor mymaster 192.168.146.128 6379 2

设置连接master的密码，这里的密码是我们设置那台为master的redis连接密码
sentinel auth-pass mymaster 123456

这个配置项指定了需要多少失效时间，单位是毫秒，默认为30秒
sentinel down-after-milliseconds mymaster 30000


sentinel parallel-syncs mymaster 1

sentinel failover-timeout mymaster 180000

注意：这里我们可以sentinel.conf复制到另外两台）

使用以下命令可以传输到另外两台机器上

[ redis]# scp sentinel.conf :/usr/local/redis
The authenticity of host ‘192.168.146.129 (192.168.146.129)‘ can‘t be established.
RSA key fingerprint is 4c:23:85:d4:6d:13:3f:e5:8c:a0:ee:49:77:d8:c6:25.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘192.168.146.129‘ (RSA) to the list of known hosts.
‘s password: 
sentinel.conf                                           100% 9860     9.6KB/s   00:00    
[ redis]# scp sentinel.conf :/usr/local/redis
‘s password: 
sentinel.conf                                           100% 9860     9.6KB/s   00:00    
[ redis]#

7.分别启动

redis-sentinel sentinel.conf

通过ps aux | grep redis查看192.168.146.129

[ redis]# redis-sentinel sentinel.conf
[ redis]# ps aux | grep redis
root      2849  0.1  0.7 154972  7936 ?        Ssl  06:27   0:00 ./bin/redis-server 192.168.146.129:6379
root      2857  0.0  0.7 152408  7832 ?        Ssl  06:27   0:00 redis-sentinel 127.0.0.1:26379 [sentinel]
root      2862  0.0  0.0 103248   828 pts/0    S+   06:27   0:00 grep redis
[ redis]#

通过ps aux | grep redis查看192.168.146.130

[ redis]# redis-sentinel sentinel.conf
[ redis]# ps aux | grep redis
root      2809  0.2  0.7 154972  7944 ?        Ssl  06:29   0:00 ./bin/redis-server 192.168.146.130:6379
root      2828  0.3  0.7 152408  7832 ?        Ssl  06:30   0:00 redis-sentinel 127.0.0.1:26379 [sentinel]
root      2833  0.0  0.0 103248   832 pts/0    S+   06:30   0:00 grep redis
[ redis]#

启动完成之后

8.我们进入marter 192.168.146.128主机的usr/local/redis/sentinel目录里

cd /usr/local/redis/sentinel

使用命令动态的查看redis-sentinel.log日志

tail -f redis-sentinel.log

-rw-r--r--. 1 root root 1031 Apr 22 07:54 redis-sentinel.log
[ sentinel]# tail -f redis-sentinel.log 
11353:X 22 Apr 2020 07:54:48.634 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
11353:X 22 Apr 2020 07:54:48.634 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=11353, just started
11353:X 22 Apr 2020 07:54:48.634 # Configuration loaded
11354:X 22 Apr 2020 07:54:48.817 * Increased maximum number of open files to 10032 (it was originally set to 1024).
11354:X 22 Apr 2020 07:54:48.819 * Running mode=sentinel, port=26379.
11354:X 22 Apr 2020 07:54:48.819 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
11354:X 22 Apr 2020 07:54:48.822 # Sentinel ID is abff04b29da5863e0c5777e0a81f1751cf4fa59b
11354:X 22 Apr 2020 07:54:48.823 # +monitor master mymaster 192.168.146.128 6379 quorum 2
11354:X 22 Apr 2020 07:54:48.825 * +slave slave 192.168.146.129:6379 192.168.146.129 6379 @ mymaster 192.168.146.128 6379
11354:X 22 Apr 2020 07:54:48.828 * +slave slave 192.168.146.130:6379 192.168.146.130 6379 @ mymaster 192.168.146.128 6379
11354:X 22 Apr 2020 07:55:30.733 * +sentinel sentinel 01f3cdaf9e31ceaeeab4e3e1a680f96e2076e28e 192.168.146.129 26379 @ mymaster 192.168.146.128 6379
11354:X 22 Apr 2020 07:55:47.798 * +sentinel sentinel 89c26e09ca7e1c39641ed2958ce8b0396880a8f1 192.168.146.130 26379 @ mymaster 192.168.146.128 6379

这里11354:X 22 Apr 2020 07:55:30.733 * +sentinel sentinel 01f3cdaf9e31ceaeeab4e3e1a680f96e2076e28e 192.168.146.129 26379 @ mymaster 192.168.146.128 6379

11354:X 22 Apr 2020 07:55:47.798 * +sentinel sentinel 89c26e09ca7e1c39641ed2958ce8b0396880a8f1 192.168.146.130 26379 @ mymaster 192.168.146.128 6379可以看出

slave 192.168.146.129:6379和slave 192.168.146.130:6379是从机 mymaster 192.168.146.128 6379主机

9.我们可以通过redis客户端查看192.168.146.128

[ redis]# ./bin/redis-cli -h 192.168.146.128 -p 6379 -a 123456
Warning: Using a password with ‘-a‘ or ‘-u‘ option on the command line interface may not be safe.
192.168.146.128:6379>  info replication
# Replication
role:master
connected_slaves:2
slave0:ip=192.168.146.129,port=6379,state=online,offset=75866,lag=1
slave1:ip=192.168.146.130,port=6379,state=online,offset=75866,lag=1
master_replid:430791dbf5fe96222e992151226171f3b53fec6b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:75866
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:4198
repl_backlog_histlen:71669
192.168.146.128:6379>

我们可以通过redis客户端查看192.168.146.129

[ redis]# ./bin/redis-cli -h 192.168.146.129 -p 6379 -a 123456
Warning: Using a password with ‘-a‘ or ‘-u‘ option on the command line interface may not be safe.
192.168.146.129:6379> info replication
# Replication
role:slave
master_host:192.168.146.128
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:95148
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:430791dbf5fe96222e992151226171f3b53fec6b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:95148
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:4198
repl_backlog_histlen:90951
192.168.146.129:6379>

我们可以通过redis客户端查看192.168.146.130

[ redis]#  ./bin/redis-cli -h 192.168.146.130 -p 6379 -a 123456
Warning: Using a password with ‘-a‘ or ‘-u‘ option on the command line interface may not be safe.
192.168.146.130:6379> info replication
# Replication
role:slave
master_host:192.168.146.128
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:100264
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:430791dbf5fe96222e992151226171f3b53fec6b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:100264
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:13438
repl_backlog_histlen:86827
192.168.146.130:6379>

10.故障转移

当我们关闭master192.168.146.128主节点时

[ redis]# ps aux | grep redis
root      3576  0.1  0.9 158044  9832 ?        Ssl  Apr21   0:57 ./bin/redis-server 192.168.146.128:6379
root      9739  0.3  0.7 152408  7944 ?        Ssl  05:52   0:11 redis-sentinel 127.0.0.1:26379 [sentinel]
root     10274  0.0  0.0 100936   616 pts/0    S+   06:36   0:00 tail -f redis-sentinel.log
root     10467  0.0  0.0 103248   832 pts/1    S+   06:51   0:00 grep redis
[ redis]# kill -9 3576

通过ps aux | grep redis查看已经关闭了

[ redis]# ps aux | grep redis
root      9739  0.3  0.7 152408  7944 ?        Ssl  05:52   0:11 redis-sentinel 127.0.0.1:26379 [sentinel]
root     10274  0.0  0.0 100936   616 pts/0    S+   06:36   0:00 tail -f redis-sentinel.log
root     10479  0.0  0.0 103248   832 pts/1    S+   06:52   0:00 grep redis
[ redis]#

redis-sentinel.log日志添加了

#关闭了master mymaster 192.168.146.128
11354:X 22 Apr 2020 07:58:46.891 # +sdown master mymaster 192.168.146.128 6379
#当前配置版本被更新
11354:X 22 Apr 2020 07:58:46.921 # +new-epoch 1
#进行投票选举slave服务器
11354:X 22 Apr 2020 07:58:46.928 # +vote-for-leader 89c26e09ca7e1c39641ed2958ce8b0396880a8f1 1
#投票后有两个sentinel发现master不能用
11354:X 22 Apr 2020 07:58:46.969 # +odown master mymaster 192.168.146.128 6379 #quorum 3/2
#下一次故障转移延迟
11354:X 22 Apr 2020 07:58:46.969 # Next failover delay: I will not start a failover before Wed Apr 22 08:04:47 2020
#更新192.168.146.130和192.168.146.128配置
11354:X 22 Apr 2020 07:58:47.341 # +config-update-from sentinel 89c26e09ca7e1c39641ed2958ce8b0396880a8f1 192.168.146.130 26379 @ mymaster 192.168.146.128 6379
# master地址发生改变从192.168.146.128变成了192.168.146.129
11354:X 22 Apr 2020 07:58:47.341 # +switch-master mymaster 192.168.146.128 6379 192.168.146.129 6379
11354:X 22 Apr 2020 07:58:47.342 * +slave slave 192.168.146.130:6379 192.168.146.130 6379 @ mymaster 192.168.146.129 6379
11354:X 22 Apr 2020 07:58:47.342 * +slave slave 192.168.146.128:6379 192.168.146.128 6379 @ mymaster 192.168.146.129 6379
11354:X 22 Apr 2020 07:59:17.344 # +sdown slave 192.168.146.128:6379 192.168.146.128 6379 @ mymaster 192.168.146.129 6379

我们可以通过redis客户端查看192.168.146.129，192.168.146.129变成了master

[ redis]# ./bin/redis-cli -h 192.168.146.129 -p 6379 -a 123456
Warning: Using a password with ‘-a‘ or ‘-u‘ option on the command line interface may not be safe.
192.168.146.129:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.146.130,port=6379,state=online,offset=303685,lag=0
master_replid:f736d355a36ee407bb054b1bee8c6dee61027c1b
master_replid2:a26f2d758033d818e95eb76156e90a0ecef85395
master_repl_offset:303830
second_repl_offset:37896
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:303830
192.168.146.129:6379>

我们可以通过redis客户端查看192.168.146.130还是slave，它的mmaster变成了192.168.146.129了

[ redis]# ./bin/redis-cli -h 192.168.146.130 -p 6379 -a 123456
Warning: Using a password with ‘-a‘ or ‘-u‘ option on the command line interface may not be safe.
192.168.146.130:6379> info replication
# Replication
role:slave
master_host:192.168.146.129
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:337100
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:f736d355a36ee407bb054b1bee8c6dee61027c1b
master_replid2:a26f2d758033d818e95eb76156e90a0ecef85395
master_repl_offset:337100
second_repl_offset:37896
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:29
repl_backlog_histlen:337072
192.168.146.130:6379>

我们可以从新启动192.168.146.128，这时它变成了slave，它的master变成了192.168.146.129了

[ redis]# ./bin/redis-server ./redis.conf 
11710:C 22 Apr 2020 08:24:38.791 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
11710:C 22 Apr 2020 08:24:38.792 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=11710, just started
11710:C 22 Apr 2020 08:24:38.792 # Configuration loaded
[ redis]# ./bin/redis-cli -h 192.168.146.128 -p 6379 -a 123456
Warning: Using a password with ‘-a‘ or ‘-u‘ option on the command line interface may not be safe.
192.168.146.128:6379> info replication
# Replication
role:slave
master_host:192.168.146.129
master_port:6379
master_link_status:down
master_last_io_seconds_ago:-1
master_sync_in_progress:0
slave_repl_offset:1
master_link_down_since_seconds:1587569109
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:ea5fb876e06cd5a0f1f124471e6d8458e2e32bdb
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
192.168.146.128:6379>

这样哨兵机制就搭建完成了

redis