Cluster

Set up and operate a FyroDB cluster — slots, nodes.conf, redirects, migration, and client examples.

FyroDB implements Redis Cluster: 16384 hash slots, MOVED/ASK redirects, CROSSSLOT detection, hash tags, and a gossip bus for heartbeats and failure detection. Any cluster-aware Redis client works unchanged.

Quick Start (Docker Compose)

Save this as docker-compose.yml and run docker compose up -d. It brings up three masters covering all 16384 slots.

services:
  cluster-config:
    image: busybox
    restart: "no"
    entrypoint:
      - /bin/sh
      - -c
      - |
        cat > /config/nodes.conf <<'EOF'
        node-1 127.0.0.1:8000@node-1:18000 master - 1 0 0 connected 0-5461
        node-2 127.0.0.1:8001@node-2:18000 master - 1 0 0 connected 5462-10922
        node-3 127.0.0.1:8002@node-3:18000 master - 1 0 0 connected 10923-16383
        EOF
        for i in 1 2 3; do cp /config/nodes.conf /config/nodes-$i.conf; done
    volumes:
      - cluster-config:/config
 
  node-1:
    image: rana718/fyrodb:latest
    depends_on:
      cluster-config:
        condition: service_completed_successfully
    environment:
      FYRODB_CLUSTER_ENABLED: "true"
      FYRODB_BIND: "0.0.0.0"
      FYRODB_PORT: "8000"
      FYRODB_CLUSTER_PORT: "18000"
      FYRODB_NODE_ID: "node-1"
      FYRODB_CLUSTER_CONFIG_FILE: "/config/nodes-1.conf"
      FYRODB_RDB_PATH: "/data/fyrodb.rdb"
    ports:
      - "8000:8000"
    volumes:
      - node-1-data:/data
      - cluster-config:/config
    restart: unless-stopped
 
  node-2:
    image: rana718/fyrodb:latest
    depends_on:
      cluster-config:
        condition: service_completed_successfully
    environment:
      FYRODB_CLUSTER_ENABLED: "true"
      FYRODB_BIND: "0.0.0.0"
      FYRODB_PORT: "8000"
      FYRODB_CLUSTER_PORT: "18000"
      FYRODB_NODE_ID: "node-2"
      FYRODB_CLUSTER_CONFIG_FILE: "/config/nodes-2.conf"
      FYRODB_RDB_PATH: "/data/fyrodb.rdb"
    ports:
      - "8001:8000"
    volumes:
      - node-2-data:/data
      - cluster-config:/config
    restart: unless-stopped
 
  node-3:
    image: rana718/fyrodb:latest
    depends_on:
      cluster-config:
        condition: service_completed_successfully
    environment:
      FYRODB_CLUSTER_ENABLED: "true"
      FYRODB_BIND: "0.0.0.0"
      FYRODB_PORT: "8000"
      FYRODB_CLUSTER_PORT: "18000"
      FYRODB_NODE_ID: "node-3"
      FYRODB_CLUSTER_CONFIG_FILE: "/config/nodes-3.conf"
      FYRODB_RDB_PATH: "/data/fyrodb.rdb"
    ports:
      - "8002:8000"
    volumes:
      - node-3-data:/data
      - cluster-config:/config
    restart: unless-stopped
 
volumes:
  node-1-data:
  node-2-data:
  node-3-data:
  cluster-config:
docker compose up -d
docker compose down

Then talk to it with any Redis client:

$ redis-cli -c -p 8000
127.0.0.1:8000> set user:1 alice
-> Redirected to slot [8676] located at 127.0.0.1:8001
OK
127.0.0.1:8001> get user:1
"alice"

The -c flag is what makes redis-cli follow redirects. Without it you see the redirect as an error, which is correct protocol behaviour:

$ redis-cli -p 8000 get user:1
(error) MOVED 8676 127.0.0.1:8001

Before you deploy this beyond your laptop, change the client-facing addresses. The 127.0.0.1:800x above only works for a client on the Docker host — see nodes.conf below.

Each node uses one worker per CPU by default. Cap it when several nodes share a machine, so they do not oversubscribe it:

      FYRODB_WORKERS: "4"     # per node

Without Docker

Three processes, one config file each:

cat > nodes.conf <<'EOF'
node-1 127.0.0.1:8000@127.0.0.1:18000 master - 1 0 0 connected 0-5461
node-2 127.0.0.1:8001@127.0.0.1:18001 master - 1 0 0 connected 5462-10922
node-3 127.0.0.1:8002@127.0.0.1:18002 master - 1 0 0 connected 10923-16383
EOF
for i in 1 2 3; do cp nodes.conf nodes-$i.conf; done
 
for i in 1 2 3; do
  FYRODB_CLUSTER_ENABLED=true \
  FYRODB_BIND=127.0.0.1 \
  FYRODB_PORT=$((7999 + i)) \
  FYRODB_CLUSTER_PORT=$((17999 + i)) \
  FYRODB_NODE_ID=node-$i \
  FYRODB_CLUSTER_CONFIG_FILE=nodes-$i.conf \
  FYRODB_RDB_PATH=node-$i.rdb \
  fyro_db &
done

Environment Variables

VariableDefaultDescription
FYRODB_CLUSTER_ENABLEDfalsetrue/1/yes turns on cluster mode
FYRODB_NODE_IDautoStable node identity; must match nodes.conf
FYRODB_CLUSTER_PORT18000Cluster bus port (clients never use it)
FYRODB_ADVERTISE_ADDRFYRODB_BINDAddress used when there is no nodes.conf
FYRODB_CLUSTER_CONFIG_FILEfyrodb-nodes.confTopology file, Redis nodes.conf format
FYRODB_CLUSTER_HEARTBEAT_MS2000Ping interval on the bus
FYRODB_CLUSTER_SUSPECT_MS6000Silence before a peer is suspect; must exceed heartbeat
FYRODB_CLUSTER_FAILURE_QUORUM2Reports needed to confirm a failure
FYRODB_CLUSTER_QUEUE_CAPACITY1024Bounded outbound frames per peer
FYRODB_CLUSTER_MAX_INBOUND1024Max inbound bus connections
FYRODB_CLUSTER_AUTH(none)Shared secret for the bus handshake
FYRODB_REPLICATION_LOG_CAPACITY100000Mutation log entries retained for replicas

nodes.conf

One line per node, Redis-compatible:

<id> <client-addr>@<bus-addr> <flags> <master> <epoch> 0 0 connected <slots...>

node-1 127.0.0.1:8000@127.0.0.1:18000 master - 1 0 0 connected 0-5461
node-2 127.0.0.1:8001@127.0.0.1:18001 master - 1 0 0 connected 5462-10922
node-3 127.0.0.1:8002@127.0.0.1:18002 master - 1 0 0 connected 10923-16383

The two addresses on either side of @ live in different reachability domains, and getting this wrong is the most common setup mistake:

  • client-addr is returned in every MOVED/ASK reply, so it must be reachable by your clients, on a port they can connect to.
  • bus-addr is only ever used node-to-node.

Under Docker Compose that means a client on the host needs 127.0.0.1:<published port> on the left, while the right side can stay on the Compose network:

node-2 127.0.0.1:8001@node-2:18000 master - 1 0 0 connected 5462-10922

Point the client-facing address at a Compose service name instead if your client runs inside the network. Bus addresses may be hostnames — they are resolved on every connect attempt, so a node that restarts with a new IP recovers on its own.

id is opaque and does not have to look like an address. Node identity is compared by id, never by address.

Verifying a Cluster

redis-cli -p 8000 cluster info
redis-cli -p 8000 cluster slots
redis-cli -p 8000 cluster nodes
redis-cli -p 8000 cluster keyslot user:1

Check both of these — a cluster serves reads correctly with the bus completely dead, so functional tests alone will not catch a broken bus:

redis-cli -p 8000 cluster info | grep cluster_state          # expect ok
redis-cli -p 8000 info | grep cluster_peer_healthy           # expect peers-1

INFO also reports cluster_peer_total, cluster_peer_suspect, cluster_peer_queue_full_total, cluster_peer_reconnect_total, cluster_replication_lag_total and cluster_snapshot_attempts.

Hash Tags and CROSSSLOT

Multi-key commands must stay within one slot. Wrap the part you want hashed in {} to force keys together:

$ redis-cli -c -p 8000 mset '{cart:42}:items' 3 '{cart:42}:total' 99
OK
$ redis-cli -c -p 8000 mget '{cart:42}:items' '{cart:42}:total'
1) "3"
2) "99"
 
$ redis-cli -p 8000 mget a b
(error) CROSSSLOT Keys in request don't hash to the same slot

Slot Migration

Migration follows Redis's handshake. SETSLOT <slot> NODE <target> has to be sent to every master — only the source has a pending migration record, and a node that never hears about the new owner will keep redirecting to the old one.

SLOT=$(redis-cli -p 8000 cluster keyslot user:1 | tr -dc 0-9)
 
redis-cli -p 8001 cluster setslot $SLOT importing node-1   # on the target
redis-cli -p 8000 cluster setslot $SLOT migrating node-2   # on the source
 
# writes to the slot now ASK-redirect to the target
redis-cli -p 8000 set user:1 v
# (error) ASK 8676 127.0.0.1:8001
 
for p in 8000 8001 8002; do
  redis-cli -p $p cluster setslot $SLOT node node-2
done
redis-cli -p 8001 cluster setslot $SLOT stable

This moves ownership. Key data moves separately.

Cluster Commands

CLUSTER INFO, MYID, SLOTS, SHARDS, NODES, KEYSLOT, COUNTKEYSINSLOT, GETKEYSINSLOT, MEET, FORGET, ADDSLOTS, DELSLOTS, SETSLOT, REPLICATE, RESET, SAVECONFIG, plus ASKING.

nodes.conf is rewritten whenever the topology changes, so a restarted node comes back with the slot map it last agreed on.

Client Examples

Node.js (ioredis)

import { Cluster } from "ioredis";
 
const cluster = new Cluster([
  { host: "127.0.0.1", port: 8000 },
  { host: "127.0.0.1", port: 8001 },
  { host: "127.0.0.1", port: 8002 },
]);
 
await cluster.set("user:1", "alice");
console.log(await cluster.get("user:1"));

Python (redis-py)

from redis.cluster import RedisCluster
 
rc = RedisCluster(host="127.0.0.1", port=8000)
rc.set("user:1", "alice")
print(rc.get("user:1"))

Go (go-redis)

rdb := redis.NewClusterClient(&redis.ClusterOptions{
    Addrs: []string{"127.0.0.1:8000", "127.0.0.1:8001", "127.0.0.1:8002"},
})
rdb.Set(ctx, "user:1", "alice", 0)

Benchmarking a Cluster

The benchmark tool lives in the repository, so point it at your running cluster from a clone:

git clone https://github.com/Rana718/FyroDB && cd FyroDB/bench
go run . -cluster 127.0.0.1:8000,127.0.0.1:8001,127.0.0.1:8002

Flush between runs so the second run does not start with the first one's data:

for p in 8000 8001 8002; do redis-cli -p $p flushall; done

Give every node the same total core count as whatever you are comparing against. Redis is single-threaded per master, so its master count is its core count — a 6-master Redis Cluster on a 12-thread machine only gets half the box. See Benchmarks for the paired configurations and measured numbers.

Single Node or Cluster?

A single FyroDB node already uses every core, so clustering is about capacity and fault isolation rather than throughput on one box. Measured at matched thread count on a 6-core/12-thread machine, three nodes with four workers each land within a few percent of one node with twelve workers on most workloads, and ahead on single-key contention because sharding across processes reduces cross-core traffic. Reach for a cluster when the dataset outgrows one machine or you want independent failure domains.