这篇文章是MicroAd Advent Calendar 2021的第11天的文章。

概括一下

本文将介绍在Docker环境中搭建Redis Cluster并进行操作确认的步骤。由于这是为了整理理解而进行的输出，可能会存在不完善之处，请谅解。此外，本文省略了关于Redis本身和复制的解释内容。

使用Docker准备Redis环境。

我会创建以下这样的目录结构。

❯ tree
.
├── conf
│   └── redis.conf
├── conf2
│   └── redis.conf
├── conf3
│   └── redis.conf
├── conf4
│   └── redis.conf
└── docker-compose.yml

4 directories, 5 files

在redis.conf中，只需记录打开集群功能的内容。
Redis还有许多其他设置，但由于有许多简明扼要的文章，本文仅介绍最基本的设置，并默认使用默认设置。

cluster-enabled yes

Redis将使用官方的Docker镜像2。
在Redis的Docker镜像中，可以通过在命令(command)中指定配置文件路径来在启动时加载配置文件，
通过挂载(volumes)配置文件并在命令中指定与容器内路径对应的路径。
depends_on是为了按照docker{01-04}的顺序给服务命名并分配IP地址，以便保持启动顺序一致(这样在查找IP地址时更方便)。

version: '3'
services:
  redis01:
    image: redis:6.2.6
    volumes:
      - "./conf:/etc/redis"
    command: "/etc/redis/redis.conf"
  redis02:
    image: redis:6.2.6
    volumes:
      - "./conf2:/etc/redis"
    command: "/etc/redis/redis.conf"
    depends_on:
      - redis01
  redis03:
    image: redis:6.2.6
    volumes:
      - "./conf3:/etc/redis"
    command: "/etc/redis/redis.conf"
    depends_on:
      - redis02
  redis04:
    image: redis:6.2.6
    volumes:
      - "./conf4:/etc/redis"
    command: "/etc/redis/redis.conf"
    depends_on:
      - redis03

我会启动Docker容器。

❯ docker-compose up -d
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Creating network "cluster-test_default" with the default driver
Creating cluster-test_redis01_1 ... done
Creating cluster-test_redis02_1 ... done
Creating cluster-test_redis03_1 ... done
Creating cluster-test_redis04_1 ... done

我会进入 redis01 的 Docker 环境中的 Bash 进行工作。

首先，由于需要使用redis-cli命令来指定每个Redis实例的IP地址，因此需要查找IP地址。
虽然可以使用docker inspect命令进行查找，但在这种情况下我们决定使用dig进行查找。

❯ docker-compose exec redis01 bash
root@6ba9e07d9434:/data# apt-get update
Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
...略

root@6ba9e07d9434:/data# apt install dnsutils
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
...略

root@6ba9e07d9434:/data# seq 1 4 | xargs -i dig redis0{} | grep redis
; <<>> DiG 9.16.22-Debian <<>> redis01
;redis01.                       IN      A
redis01.                600     IN      A       192.168.144.2
; <<>> DiG 9.16.22-Debian <<>> redis02
;redis02.                       IN      A
redis02.                600     IN      A       192.168.144.3
; <<>> DiG 9.16.22-Debian <<>> redis03
;redis03.                       IN      A
redis03.                600     IN      A       192.168.144.4
; <<>> DiG 9.16.22-Debian <<>> redis04
;redis04.                       IN      A
redis04.                600     IN      A       192.168.144.5

经过以上步骤，已经成功启动了四个Redis实例，并且获得了它们的IP地址。

redis01 = 192.168.144.2
redis02 = 192.168.144.3
redis03 = 192.168.144.4
redis04 = 192.168.144.5

建立Redis集群

使用redis-cli –cluster命令来创建集群。

在创建集群时，需要进行Slot的设置。
Slot是确定数据存储在集群中哪个节点的机制。
在数据存储时，被分配到0~16383之间的某个数字作为Slot，并将数据存储在具有被分配Slot的节点上，从而实现水平分割（Sharding）。

root@6ba9e07d9434:/data# redis-cli --cluster create 192.168.144.2:6379 192.168.144.3:6379 192.168.144.4:6379
>>> Performing hash slots allocation on 3 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
M: 0bd346a134981afb74a136e665a6027821b2016b 192.168.144.2:6379
   slots:[0-5460] (5461 slots) master
M: 5dbd6568e5ec7b2c15e4d5bb4988726c663fcf75 192.168.144.3:6379
   slots:[5461-10922] (5462 slots) master
M: 5549145f997afaad00dad89b5f312a044be6c5ca 192.168.144.4:6379
   slots:[10923-16383] (5461 slots) master
Can I set the above configuration? (type 'yes' to accept): yes # <-- 3ノードで均等に分割するか聞かれます
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
..
>>> Performing Cluster Check (using node 192.168.144.2:6379)
M: 0bd346a134981afb74a136e665a6027821b2016b 192.168.144.2:6379
   slots:[0-5460] (5461 slots) master
M: 5549145f997afaad00dad89b5f312a044be6c5ca 192.168.144.4:6379
   slots:[10923-16383] (5461 slots) master
M: 5dbd6568e5ec7b2c15e4d5bb4988726c663fcf75 192.168.144.3:6379
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

我们可以通过redis-cli cluster nodes命令来查看集群的状态。
通过使用-h选项来指定主机，以下示例中指定了主机redis01。
其中，3个节点被标记为master，并且连接到的主机被标记为myself。

root@6ba9e07d9434:/data# redis-cli -h redis01 cluster nodes
5549145f997afaad00dad89b5f312a044be6c5ca 192.168.144.4:6379@16379 master - 0 1639150635000 3 connected 10923-16383
0bd346a134981afb74a136e665a6027821b2016b 192.168.144.2:6379@16379 myself,master - 0 1639150632000 1 connected 0-5460
5dbd6568e5ec7b2c15e4d5bb4988726c663fcf75 192.168.144.3:6379@16379 master - 0 1639150635554 2 connected 5461-10922

数据的存储和检索

连接到cluster时，使用-c选项。这样，如果目标Slot不同，它会自动重定向。

root@6ba9e07d9434:/data# redis-cli -h redis01 -c
redis01:6379> 
redis01:6379> set key01 value01
-> Redirected to slot [13770] located at 192.168.144.4:6379
OK
192.168.144.4:6379> get key01
"value01"

没有使用-c选项进行确认。
首先从没有存储数据的redis01（192.168.144.2）获取会出错。
然后连接到显示错误的redis03（192.168.144.4）可以成功进行GET操作。

root@6ba9e07d9434:/data# redis-cli -h redis01   
redis01:6379> get key01
(error) MOVED 13770 192.168.144.4:6379
redis01:6379> 
root@6ba9e07d9434:/data# redis-cli -h redis03 
redis03:6379> get key01
"value01"

添加一个奴隶

将redis04（192.168.144.5:6379）添加为redis0（192.168.144.2）的从节点。
命令格式为redis-cli –cluster add-node <要添加的节点IP:端口> <目标节点IP:端口>，如果添加时加上-cluster-slave参数，则为从节点，不加则为主节点。

root@6ba9e07d9434:/data# redis-cli --cluster add-node 192.168.144.5:6379 192.168.144.2:6379 --cluster-slave
>>> Adding node 192.168.144.5:6379 to cluster 192.168.144.2:6379
>>> Performing Cluster Check (using node 192.168.144.2:6379)
M: 0bd346a134981afb74a136e665a6027821b2016b 192.168.144.2:6379
   slots:[0-5460] (5461 slots) master
M: 5549145f997afaad00dad89b5f312a044be6c5ca 192.168.144.4:6379
   slots:[10923-16383] (5461 slots) master
M: 5dbd6568e5ec7b2c15e4d5bb4988726c663fcf75 192.168.144.3:6379
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
Automatically selected master 192.168.144.2:6379
>>> Send CLUSTER MEET to node 192.168.144.5:6379 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 192.168.144.2:6379.
[OK] New node added correctly.

确认后，在最底部添加了一个从节点。
从节点最右边的字符串是主节点的节点ID，其中0bd…指的是redis0(192.168.144.2)。

root@6ba9e07d9434:/data# redis-cli cluster nodes
5549145f997afaad00dad89b5f312a044be6c5ca 192.168.144.4:6379@16379 master - 0 1639152407000 3 connected 10923-16383
0bd346a134981afb74a136e665a6027821b2016b 192.168.144.2:6379@16379 myself,master - 0 1639152406000 1 connected 0-5460
5dbd6568e5ec7b2c15e4d5bb4988726c663fcf75 192.168.144.3:6379@16379 master - 0 1639152408449 2 connected 5461-10922
e686a6bb888858f330218ed8c4355ef481028668 192.168.144.5:6379@16379 slave 0bd346a134981afb74a136e665a6027821b2016b 0 1639152407444 1 connected

你可以使用以下指令确认每个主节点上有几个从节点和键。可以看到有一个主节点新增了一个从节点。

root@6ba9e07d9434:/data# redis-cli --cluster info 192.168.144.2 6379
192.168.144.2:6379 (0bd346a1...) -> 0 keys | 5461 slots | 1 slaves.
192.168.144.4:6379 (5549145f...) -> 1 keys | 5461 slots | 0 slaves.
192.168.144.3:6379 (5dbd6568...) -> 0 keys | 5462 slots | 0 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.

在failover命令中，可以交替切换主节点和从节点。

root@6ba9e07d9434:/data# redis-cli -h 192.168.144.5 cluster failover
OK

redis01(192.168.144.2) 成为了从服务器，redis04(192.168.144.5) 成为了主服务器。

root@6ba9e07d9434:/data# redis-cli cluster nodes
5549145f997afaad00dad89b5f312a044be6c5ca 192.168.144.4:6379@16379 master - 0 1639155508000 3 connected 10923-16383
0bd346a134981afb74a136e665a6027821b2016b 192.168.144.2:6379@16379 myself,slave e686a6bb888858f330218ed8c4355ef481028668 0 1639155511000 4 connected
5dbd6568e5ec7b2c15e4d5bb4988726c663fcf75 192.168.144.3:6379@16379 master - 0 1639155511870 2 connected 5461-10922
e686a6bb888858f330218ed8c4355ef481028668 192.168.144.5:6379@16379 master - 0 1639155510865 4 connected 0-5460

由于确认了动作，将删除从节点redis01（192.168.144.2）。
要删除节点，请使用redis-cli –cluster del-node <删除目标节点的IP:端口> <删除目标节点的ID>。

root@6ba9e07d9434:/data# redis-cli --cluster del-node 192.168.144.2:6379 0bd346a134981afb74a136e665a6027821b2016b
>>> Removing node 0bd346a134981afb74a136e665a6027821b2016b from cluster 192.168.144.2:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> Sending CLUSTER RESET SOFT to the deleted node.

查看集群信息，发现redis01（192.168.144.2）已经消失。

root@6ba9e07d9434:/data# redis-cli -h redis02 cluster nodes
5549145f997afaad00dad89b5f312a044be6c5ca 192.168.144.4:6379@16379 master - 0 1639155750324 3 connected 10923-16383
e686a6bb888858f330218ed8c4355ef481028668 192.168.144.5:6379@16379 master - 0 1639155749319 4 connected 0-5460
5dbd6568e5ec7b2c15e4d5bb4988726c663fcf75 192.168.144.3:6379@16379 myself,master - 0 1639155749000 2 connected 5461-10922

從被刪除節點redis01(192.168.144.2)的角度來看，它只能看到自己。

root@6ba9e07d9434:/data# redis-cli -h redis01 cluster nodes
0bd346a134981afb74a136e665a6027821b2016b 192.168.144.2:6379@16379 myself,master - 0 1639155731000 1 connected

添加主控

下一步是将redis01(192.168.144.2)作为主节点进行添加。

root@6ba9e07d9434:/data# redis-cli --cluster add-node 192.168.144.2:6379 192.168.144.5:6379
>>> Adding node 192.168.144.2:6379 to cluster 192.168.144.5:6379
>>> Performing Cluster Check (using node 192.168.144.5:6379)
M: e686a6bb888858f330218ed8c4355ef481028668 192.168.144.5:6379
   slots:[0-5460] (5461 slots) master
M: 5dbd6568e5ec7b2c15e4d5bb4988726c663fcf75 192.168.144.3:6379
   slots:[5461-10922] (5462 slots) master
M: 5549145f997afaad00dad89b5f312a044be6c5ca 192.168.144.4:6379
   slots:[10923-16383] (5461 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.144.2:6379 to make it join the cluster.
[OK] New node added correctly.

可以确认有4个主节点。
然而，由于添加的节点没有分配槽位，所以右侧的connected没有任何内容。

root@6ba9e07d9434:/data# redis-cli -h redis02 cluster nodes
0bd346a134981afb74a136e665a6027821b2016b 192.168.144.2:6379@16379 master - 0 1639155981275 1 connected
5549145f997afaad00dad89b5f312a044be6c5ca 192.168.144.4:6379@16379 master - 0 1639155980271 3 connected 10923-16383
e686a6bb888858f330218ed8c4355ef481028668 192.168.144.5:6379@16379 master - 0 1639155979267 4 connected 0-5460
5dbd6568e5ec7b2c15e4d5bb4988726c663fcf75 192.168.144.3:6379@16379 myself,master - 0 1639155978000 2 connected 5461-10922

您可以使用reshard命令重新分配槽位。

命令会进行交互式操作，首先会询问要移动的Slottt数量，因为16384（总Slot数）除以4（主节点数）等于4096，所以输入4096。
接下来会询问是否要从所有节点执行移动操作，输入yes。

root@6ba9e07d9434:/data# redis-cli --cluster reshard 192.168.144.2:6379
>>> Performing Cluster Check (using node 192.168.144.2:6379)
M: 0bd346a134981afb74a136e665a6027821b2016b 192.168.144.2:6379
   slots: (0 slots) master
M: 5549145f997afaad00dad89b5f312a044be6c5ca 192.168.144.4:6379
   slots:[10923-16383] (5461 slots) master
M: 5dbd6568e5ec7b2c15e4d5bb4988726c663fcf75 192.168.144.3:6379
   slots:[5461-10922] (5462 slots) master
M: e686a6bb888858f330218ed8c4355ef481028668 192.168.144.5:6379
   slots:[0-5460] (5461 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 4096   <-- 1. 移動Slot数
What is the receiving node ID? 0bd346a134981afb74a136e665a6027821b2016b
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: all  <-- 2. 全ノードから移動
...略

    Moving slot 1363 from e686a6bb888858f330218ed8c4355ef481028668
    Moving slot 1364 from e686a6bb888858f330218ed8c4355ef481028668
Do you want to proceed with the proposed reshard plan (yes/no)? yes  <-- 3. 上記変更で実行するか

....略
Moving slot 1363 from 192.168.144.5:6379 to 192.168.144.2:6379: 
Moving slot 1364 from 192.168.144.5:6379 to 192.168.144.2:6379:

当检查集群的节点时，可以看到Slot被均等地水平分割。

root@6ba9e07d9434:/data# redis-cli -h redis02 cluster nodes
0bd346a134981afb74a136e665a6027821b2016b 192.168.144.2:6379@16379 master - 0 1639157313371 5 connected 0-1364 5461-6826 10923-12287
5549145f997afaad00dad89b5f312a044be6c5ca 192.168.144.4:6379@16379 master - 0 1639157315379 3 connected 12288-16383
e686a6bb888858f330218ed8c4355ef481028668 192.168.144.5:6379@16379 master - 0 1639157314374 4 connected 1365-5460
5dbd6568e5ec7b2c15e4d5bb4988726c663fcf75 192.168.144.3:6379@16379 myself,master - 0 1639157314000 2 connected 6827-10922

总结

此文介绍了如何利用Docker环境搭建Redis集群，并进行了集群的运行验证。

如果不使用Docker，也可以通过将Redis绑定到不同的端口来启动Redis的方式来实现。使用Docker可以省去环境设置的麻烦，方便地进行操作确认，这也是我写这篇文章的原因。

只要能对你有一点帮助，我就感到很幸福。