Redis: Cluster configuration

Petr Kostelanský | 7 November 2017
We are going to look at Redis Cluster configuration and I will show you how configure masters and slaves nodes to work together.

To create high availability solution we can use Sentinel or we can also use Redis Cluster. Today I will show you how create Redis Cluster.

You can find more also in Redis Cluster Specification

Redis configuration example

Let's assume we have 3 servers and we want to have 6 redis instances (3 masters and slave for each master).

  • 1. server - master1 (192.168.100.131:6379) and slave3 (192.168.100.131:6380)
  • 2. server - master2 (192.168.100.132:6379) and slave1 (192.168.100.132:6380)
  • 3. server - master3 (192.168.100.133:6379) and slave2 (192.168.100.133:6380)

Cluster configuration

We must configure conf file for all 6 running instances of redis. On each server is running two instances one for master and second for slave (this slave will replicate data from master from other server).

Edit Redis config file /etc/redis/redis-6379.conf (6379 in conf file name mean port on which  is redis instance running)

sudo nano /etc/redis/redis-6379.conf

Set cluster mode (uncomment those lines):

cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000

Save changes

If your redis instance running as a daemon (fyi Redis instance running as background daemon) restart service:

sudo systemctl redis1 restart

Don't forget configure all redis instances like this.

Create cluster communication

Nodes in cluster need to communicate each other.

Create communication between redis servers by using meet command:

redis-cli -c -h 192.168.100.131 -p 6379 cluster meet 192.168.100.132 6379
redis-cli -c -h 192.168.100.131 -p 6379 cluster meet 192.168.100.133 6379

redis-cli -c -h 192.168.100.131 -p 6380 cluster meet 192.168.100.132 6380
redis-cli -c -h 192.168.100.131 -p 6380 cluster meet 192.168.100.133 6380

redis-cli -c -h 192.168.100.131 -p 6379 cluster meet 192.168.100.131 6380

I don't need to create meet between ...132 and ...133 because cluster nodes communicate by goosip protocol so node ...131 tell node ...132 that exists node ...133 and than communication between ...132 and ...133 will be created automaticaly

Cluster replication

Now we will configure node as a slave of specified master. More info in Cluster replicate documentation.

Before we create slave for master we will need to know node_id of master node. You can get it by cluster nodes command:

redis-cli -h 192.168.100.131 -p 6379 cluster nodes

That will return something similar to belowe and the bold string is node_id:

67ed2db8d677e59ec4a4cefb06858cf2a1a89fa1 192.168.100.131:6379 master - 0 1426238316232 2 connected 5461-10922
292f8b365bb7edb5e285caf0b7e6ddc7265d2f4f 192.168.100.132:6379 master - 0 1426238318243 3 connected 10923-16383
...

Now we can create replication:

Slave (192.168.100.132:6380) on server2 will replicate master (192.168.100.131:6379) on server1

redis-cli -c -h 192.168.100.132 -p 6380 flushall
redis-cli -c -h 192.168.100.132 -p 6380 cluster replicate master_node_id_of_192.168.100.131_6379

Slave (192.168.100.133:6380) on server3 will replicate master (192.168.100.132:6379) on server2

redis-cli -c -h 192.168.100.133 -p 6380 flushall
redis-cli -c -h 192.168.100.133 -p 6380 cluster replicate master_node_id_of_192.168.100.132_6379

Slave (192.168.100.131:6380) on server1 will replicate master (192.168.100.133:6379) on server3

redis-cli -c -h 192.168.100.131 -p 6380 flushall
redis-cli -c -h 192.168.100.131 -p 6380 cluster replicate master_node_id_of_192.168.100.133_6379

Assign hash slots to master nodes

We must determine what hash slots will by assign to cluster node (master). Hash slots are used when client (web server) need get data from cluster node. Client must ask to correct node because data are distributed across the nodes. Redis work with 16384 hash slots (start with index 0)

for slot in {0..5461}; do redis-cli -h 192.168.100.131 -p 6379 CLUSTER ADDSLOTS $slot; done;
for slot in {5462..10922}; do redis-cli -h 192.168.100.132 -p 6379 CLUSTER ADDSLOTS $slot; done;
for slot in {10923..16383}; do redis-cli -h 192.168.100.133 -p 6379 CLUSTER ADDSLOTS $slot; done;

On client (web server) is hash slot number calculate from key by CRC16 (KEY) mod 16384. There is a lot of Redis clients for different languages that can handle it.

 

Loading ads...