Linux 知识库

Appearance

Kubernetes 部署 Redis 集群 ​

Redis 集群是一种分布式缓存解决方案，支持数据分片和高可用。在 Kubernetes 环境中可以使用 Redis Cluster 或 Sentinel 模式实现高可用架构。本文详细介绍两种方案的部署配置。

架构介绍 ​

Redis Sentinel 模式 ​

Redis Sentinel 是 Redis 官方提供的高可用解决方案，通过 sentinel 进程监控主节点状态，自动进行故障转移。Sentinel 模式适合对数据安全性要求较高的场景。

架构特点：主节点故障时自动选举从节点为新主节点；客户端通过 Sentinel 获取当前主节点地址；支持故障自动转移和服务发现；最少需要 3 个 Sentinel 实例保证高可用。

Redis Cluster 模式 ​

Redis Cluster 是 Redis 官方提供的分布式解决方案，通过_slot 分片实现数据分布存储。Cluster 模式适合需要横向扩展的大规模缓存场景。

架构特点：数据自动分片到 16384 个 slot；每个分片可以有主从复制；支持自动故障转移和负载均衡；最少需要 6 个节点（3 主 3 从）。

架构选择建议 ​

选择 Sentinel 模式的场景：需要完整的 ACID 事务支持；数据量较小但需要高可用；需要单实例透明访问。

选择 Cluster 模式的场景：数据量超过单节点内存容量；需要横向扩展能力；对性能要求极高。

部署资源清单（Redis Sentinel 模式） ​

1. 命名空间和 ServiceAccount ​

yaml
apiVersion: v1
kind: Namespace
metadata:
  name: redis-cluster
  labels:
    name: redis-cluster
    environment: production
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: redis
  namespace: redis-cluster

2. ConfigMap 配置 ​

yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-config
  namespace: redis-cluster
data:
  redis.conf: |
    # 基础配置
    bind: 0.0.0.0
    port: 6379
    protected-mode: no
    timeout: 0
    tcp-keepalive: 300
    
    # 持久化配置
    save: 900 1
    save: 300 10
    save: 60 10000
    stop-writes-on-bgsave-error: yes
    rdbcompression: yes
    rdbchecksum: yes
    dbfilename: dump.rdb
    dir: /data
    
    # 内存配置
    maxmemory: 2gb
    maxmemory-policy: allkeys-lru
    maxmemory-samples: 5
    
    # 复制配置
    repl-diskless-sync: yes
    repl-diskless-sync-delay: 5
    repl-disable-tcp-nodelay: no
    
    # 安全配置
    requirepass: Redis@2024
    masterauth: Redis@2024
    
    # 日志配置
    logfile: ""
    syslog-enabled: no
    
    # 性能优化
    lazyfree-lazy-eviction: yes
    lazyfree-lazy-expire: yes
    lazyfree-lazy-server-del: yes
    replica-lazy-flush: yes
    
    # 客户端配置
    timeout: 0
    tcp-keepalive: 300
    maxclients: 10000

  sentinel.conf: |
    port: 26379
    sentinel announce-ip: 0.0.0.0
    sentinel monitor mymaster 127.0.0.1 6379 2
    sentinel down-after milliseconds mymaster 5000
    sentinel parallel-syncs mymaster 1
    sentinel failover-timeout mymaster 180000
    sentinel auth-pass mymaster Redis@2024

3. Headless Service ​

yaml
apiVersion: v1
kind: Service
metadata:
  name: redis-headless
  namespace: redis-cluster
  labels:
    app: redis
spec:
  clusterIP: None
  ports:
    - name: redis
      port: 6379
    - name: sentinel
      port: 26379
  selector:
    app: redis

4. Redis Master StatefulSet ​

yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-master
  namespace: redis-cluster
spec:
  serviceName: redis-headless
  replicas: 3
  selector:
    matchLabels:
      app: redis
      role: master
  template:
    metadata:
      labels:
        app: redis
        role: master
    spec:
      terminationGracePeriodSeconds: 10
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: redis
            topologyKey: kubernetes.io/hostname
      containers:
      - name: redis
        image: redis:7.2.4-alpine
        imagePullPolicy: IfNotPresent
        command:
        - redis-server
        - /usr/local/etc/redis/redis.conf
        - --requirepass
        - Redis@2024
        - --masterauth
        - Redis@2024
        ports:
        - name: redis
          containerPort: 6379
        resources:
          requests:
            cpu: 100m
            memory: 512Mi
          limits:
            cpu: 2000m
            memory: 4Gi
        volumeMounts:
        - name: data
          mountPath: /data
        - name: config
          mountPath: /usr/local/etc/redis
        livenessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 5
          periodSeconds: 5
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: redis-data
      - name: config
        configMap:
          name: redis-config
          items:
          - key: redis.conf
            path: redis.conf
  volumeClaimTemplates:
  - metadata:
      name: redis-data
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: fast
      resources:
        requests:
          storage: 10Gi

5. Redis Replica StatefulSet ​

yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-replica
  namespace: redis-cluster
spec:
  serviceName: redis-headless
  replicas: 3
  selector:
    matchLabels:
      app: redis
      role: replica
  template:
    metadata:
      labels:
        app: redis
        role: replica
    spec:
      terminationGracePeriodSeconds: 10
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: redis
            topologyKey: kubernetes.io/hostname
      containers:
      - name: redis
        image: redis:7.2.4-alpine
        imagePullPolicy: IfNotPresent
        command:
        - redis-server
        - /usr/local/etc/redis/redis.conf
        - --replicaof
        - 127.0.0.1
        - 6379
        - --requirepass
        - Redis@2024
        - --masterauth
        - Redis@2024
        ports:
        - name: redis
          containerPort: 6379
        resources:
          requests:
            cpu: 100m
            memory: 512Mi
          limits:
            cpu: 1000m
            memory: 2Gi
        volumeMounts:
        - name: data
          mountPath: /data
        - name: config
          mountPath: /usr/local/etc/redis
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: redis-replica-data
      - name: config
        configMap:
          name: redis-config
          items:
          - key: redis.conf
            path: redis.conf
  volumeClaimTemplates:
  - metadata:
      name: redis-replica-data
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: fast
      resources:
        requests:
          storage: 10Gi

6. Redis Services ​

yaml
apiVersion: v1
kind: Service
metadata:
  name: redis-master-svc
  namespace: redis-cluster
spec:
  type: ClusterIP
  ports:
  - name: redis
    port: 6379
    targetPort: 6379
  selector:
    app: redis
    role: master
---
apiVersion: v1
kind: Service
metadata:
  name: redis-replica-svc
  namespace: redis-cluster
spec:
  type: ClusterIP
  ports:
  - name: redis
    port: 6379
    targetPort: 6379
  selector:
    app: redis
    role: replica

7. Redis Cluster 模式配置（可选） ​

如果选择 Redis Cluster 模式，使用以下 StatefulSet 配置：

yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-cluster
  namespace: redis-cluster
spec:
  serviceName: redis-cluster
  replicas: 6
  selector:
    matchLabels:
      app: redis-cluster
  template:
    metadata:
      labels:
        app: redis-cluster
    spec:
      terminationGracePeriodSeconds: 30
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                app: redis-cluster
            topologyKey: kubernetes.io/hostname
      containers:
      - name: redis
        image: redis:7.2.4-alpine
        imagePullPolicy: IfNotPresent
        command:
        - redis-server
        - /usr/local/etc/redis/redis.conf
        - --cluster-enabled
        - yes
        - --cluster-config-file
        - /data/nodes.conf
        - --cluster-node-timeout
        - 5000
        ports:
        - name: redis
          containerPort: 6379
        resources:
          requests:
            cpu: 100m
            memory: 512Mi
          limits:
            cpu: 2000m
            memory: 4Gi
        volumeMounts:
        - name: data
          mountPath: /data
        - name: config
          mountPath: /usr/local/etc/redis
        livenessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          exec:
            command:
            - redis-cli
            - ping
          initialDelaySeconds: 5
          periodSeconds: 5
      volumes:
      - name: data
        emptyDir: {}
      - name: config
        configMap:
          name: redis-config
          items:
          - key: redis.conf
            path: redis.conf

8. StorageClass ​

yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-ssd
allowVolumeExpansion: true
reclaimPolicy: Retain

集群初始化（Cluster 模式） ​

Redis Cluster 初始化脚本 ​

yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: redis-cluster-init
  namespace: redis-cluster
spec:
  template:
    metadata:
      labels:
        app: redis-cluster-init
    spec:
      restartPolicy: OnFailure
      containers:
      - name: init
        image: redis:7.2.4-alpine
        command:
        - sh
        - -c
        - |
          #!/bin/bash
          set -e
          
          # 等待所有 Redis 节点就绪
          echo "Waiting for Redis nodes..."
          for i in 0 1 2 3 4 5; do
            host="redis-cluster-$i.redis-cluster"
            until redis-cli -h $host ping; do
              echo "Waiting for $host..."
              sleep 2
            done
          done
          
          # 创建集群
          redis-cli --cluster create \
            redis-cluster-0.redis-cluster:6379 \
            redis-cluster-1.redis-cluster:6379 \
            redis-cluster-2.redis-cluster:6379 \
            redis-cluster-3.redis-cluster:6379 \
            redis-cluster-4.redis-cluster:6379 \
            redis-cluster-5.redis-cluster:6379 \
            --cluster-replicas 1 \
            --cluster-yes \
            --pass Redis@2024
          
          echo "Redis Cluster initialized successfully"
        env:
        - name: REDIS_PASSWORD
          value: "Redis@2024"

部署步骤 ​

1. 创建命名空间和配置 ​

bash
kubectl apply -f 00-namespace.yaml
kubectl apply -f 01-configmap.yaml
kubectl apply -f 02-storageclass.yaml

2. 创建 Service ​

bash
kubectl apply -f 03-services.yaml

3. 创建 StatefulSet ​

bash
# Sentinel 模式
kubectl apply -f 04-redis-master.yaml
kubectl apply -f 05-redis-replica.yaml

# Cluster 模式
kubectl apply -f 04-redis-cluster.yaml
kubectl apply -f 06-cluster-init-job.yaml

4. 验证部署 ​

bash
# 查看 Pod 状态
kubectl get pods -n redis-cluster

# 查看服务
kubectl get svc -n redis-cluster

# 验证 Redis 连接
kubectl exec -it redis-master-0 -n redis-cluster -- redis-cli -a Redis@2024 ping

# 查看集群状态（Cluster 模式）
kubectl exec -it redis-cluster-0 -n redis-cluster -- redis-cli -a Redis@2024 cluster info

应用配置示例 ​

Java 应用配置 ​

java
// Jedis 配置
JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxTotal(20);
poolConfig.setMaxIdle(10);
poolConfig.setMinIdle(5);

JedisPool jedisPool = new JedisPool(poolConfig, 
    "redis-master-svc.redis-cluster.svc.cluster.local",
    6379,
    2000,
    "Redis@2024");

// Spring Boot 配置
spring:
  redis:
    host: redis-master-svc.redis-cluster.svc.cluster.local
    password: Redis@2024
    port: 6379
    lettuce:
      pool:
        max-active: 20
        max-idle: 10
        min-idle: 5

Python 应用配置 ​

python
import redis

# Sentinel 模式
from redis.sentinel import Sentinel

sentinel = Sentinel([
    ('redis-sentinel-0.redis-cluster.svc.cluster.local', 26379),
    ('redis-sentinel-1.redis-cluster.svc.cluster.local', 26379),
    ('redis-sentinel-2.redis-cluster.svc.cluster.local', 26379)
], sentinel_kwargs={'password': 'Redis@2024'})
master = sentinel.master_for('mymaster', password='Redis@2024')

# Cluster 模式
cluster = redis.Cluster(
    nodes=[
        'redis-cluster-0.redis-cluster:6379',
        'redis-cluster-1.redis-cluster:6379',
    ],
    password='Redis@2024'
)

监控和告警 ​

Prometheus Exporter 配置 ​

yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-monitoring
  namespace: redis-cluster
data:
  redis-exporter.yaml: |
    - job_name: redis_exporter
      static_configs:
      - targets:
        - redis-master-0.redis-cluster:9121
        - redis-master-1.redis-cluster:9121
        - redis-master-2.redis-cluster:9121

告警规则 ​

yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: redis-alerts
  namespace: redis-cluster
spec:
  groups:
  - name: redis.rules
    rules:
    - alert: RedisDown
      expr: up{job="redis_exporter"} == 0
      for: 2m
      labels:
        severity: critical
    - alert: RedisMemoryHigh
      expr: redis_memory_used_bytes / redis_memory_max_bytes > 0.9
      for: 5m
      labels:
        severity: warning
    - alert: RedisReplicationLag
      expr: redis_replication_seconds_since_source_repl > 30
      for: 5m
      labels:
        severity: warning

运维操作 ​

备份和恢复 ​

bash
# 备份（RDB 快照）
kubectl exec redis-master-0 -n redis-cluster -- \
  redis-cli -a Redis@2024 SAVE

# 恢复
kubectl cp dump.rdb redis-master-0:/data/dump.rdb -n redis-cluster
kubectl exec redis-master-0 -n redis-cluster -- \
  redis-cli -a Redis@2024 DEBUG RELOAD

扩缩容 ​

bash
# 扩容副本
kubectl scale statefulset redis-replica -n redis-cluster --replicas=5

# 手动故障转移
kubectl exec redis-replica-0 -n redis-cluster -- \
  redis-cli -a Redis@2024 REPLICAOF NO ONE

常见问题排查 ​

连接失败 ​

检查安全组规则和网络策略；验证密码配置是否正确；检查服务发现配置。

复制延迟 ​

优化慢查询；增加网络带宽；调整 repl-diskless-sync 配置。

内存不足 ​

调整 maxmemory 配置；选择合适的淘汰策略；增加节点资源。

总结 ​

本文提供了在 Kubernetes 环境中部署 Redis Sentinel 集群和 Redis Cluster 集群的完整方案。生产环境中建议配合使用监控告警系统，定期检查集群状态，并设置合理的备份策略。根据业务数据量和可用性要求选择合适的部署模式。