• Istio + k8s 安装 Redis 集群

    • 2023-03-27 21:47
    • 字数 972
    • 阅读 398

    一、安装 NFS 存储

    创建 NFS 存储主要是为了给 Redis 提供稳定的后端存储,当 Redis 的 Pod 重启或迁移后,依然能获得原先的数据。这里,我们先要创建 NFS,然后通过使用 PV 为 Redis 挂载一个远程的 NFS 路径。

    1.1 安装 NFS

    yum -y install nfs-utils
    yum -y install rpcbind

    1.2 新增 /etc/exports 文件,设置共享路径

    cat << EOF > /etc/exports
    /data/redis/cluster0 *(rw,sync,no_root_squash)
    /data/redis/cluster1 *(rw,sync,no_root_squash)
    /data/redis/cluster2 *(rw,sync,no_root_squash)
    /data/redis/cluster3 *(rw,sync,no_root_squash)
    /data/redis/cluster4 *(rw,sync,no_root_squash)
    /data/redis/cluster5 *(rw,sync,no_root_squash)
    EOF

    1.3 创建相应目录

    mkdir -p /data/redis/cluster{0..5}

    1.4 启动 NFS 和 rpcbind 服务

    systemctl restart rpcbind
    systemctl restart nfs
    systemctl enable nfs
    exportfs -v

    1.5 客户端

    yum -y install nfs-utils

    1.6 查看存储共享

    showmount -e 192.168.20.100

    二、配置 Redis 集群

    2.1 创建命名空间

    namespace.yaml 内容如下:

    apiVersion: v1
    kind: Namespace
    metadata:
      name: scut-tech

    命令如下:

    kubectl apply -f namespace.yaml

    2.2 创建 PV

    每一个 Redis Pod 都需要一个独立的 PV 来存储自己的数据,因此可以创建一个 pv.yaml 文件,包含6个PV(具体内容请查看 pv.yaml)。

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: nfs-pv0
    spec:
      capacity:
        storage: 2Gi
      accessModes:
      - ReadWriteMany
      nfs:
        server: 192.168.20.100
        path: /data/redis/cluster0
    
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: nfs-pv1
    spec:
      capacity:
        storage: 2Gi
      accessModes:
      - ReadWriteMany
      nfs:
        server: 192.168.20.100
        path: /data/redis/cluster1
    
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: nfs-pv2
    spec:
      capacity:
        storage: 2Gi
      accessModes:
      - ReadWriteMany
      nfs:
        server: 192.168.20.100
        path: /data/redis/cluster2
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: nfs-pv3
    spec:
      capacity:
        storage: 2Gi
      accessModes:
      - ReadWriteMany
      nfs:
        server: 192.168.20.100
        path: /data/redis/cluster3
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: nfs-pv4
    spec:
      capacity:
        storage: 2Gi
      accessModes:
      - ReadWriteMany
      nfs:
        server: 192.168.20.100
        path: /data/redis/cluster4
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: nfs-pv5
    spec:
      capacity:
        storage: 2Gi
      accessModes:
      - ReadWriteMany
      nfs:
        server: 192.168.20.100
        path: /data/redis/cluster5
    
    

    命令如下:

    kubectl create -f pv.yaml

    2.3 创建 Configmap

    2.3.1 创建 redis.conf

    cat << EOF > redis.conf
    appendonly yes
    cluster-enabled yes
    cluster-config-file /var/lib/redis/nodes.conf
    cluster-node-timeout 5000
    dir /var/lib/redis
    port 6379
    EOF

    2.3.2 根据 redis.conf 创建 Configmap

    kubectl create configmap redis-conf -n scut-tech --from-file=redis.conf

    2.3.3 查看创建的 Configmap

    kubectl describe cm redis-conf -n scut-tech

    三、创建 Redis 集群

    3.1 创建 redis.yaml

    apiVersion: v1
    kind: Service
    metadata:
      name: redis
      namespace: scut-tech
      labels:
        app: redis
    spec:
      selector:
        app: redis
        appCluster: redis-cluster
      ports:
      - name: redis
        port: 6379
      clusterIP: None
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: redis-access
      namespace: scut-tech
      labels:
        app: redis
    spec:
      selector:
        app: redis
        appCluster: redis-cluster
      ports:
      - name: redis-access
        protocol: TCP
        port: 6379
        targetPort: 6379
    ---
    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: redis
      namespace: scut-tech
    spec:
      serviceName: redis
      replicas: 6
      selector:
        matchLabels:
          app: redis
          appCluster: redis-cluster
      template:
        metadata:
          labels:
            app: redis
            appCluster: redis-cluster
        spec:
          terminationGracePeriodSeconds: 20
          affinity:
            podAntiAffinity:
              preferredDuringSchedulingIgnoredDuringExecution:
              - weight: 100
                podAffinityTerm:
                  labelSelector:
                    matchExpressions:
                    - key: app
                      operator: In
                      values:
                      - redis
                  topologyKey: kubernetes.io/hostname
          containers:
          - name: redis
            image: redis:latest
            command:
            - "redis-server"
            args:
            - "/etc/redis/redis.conf"
            - "--protected-mode"
            - "no"
            - "--cluster-announce-ip"
            - "$(POD_IP)"
            env:
            - name: POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            resources:
              requests:
                cpu: "1000m"
                memory: "1Gi"
            ports:
            - containerPort: 6379
              name: redis
              protocol: TCP
            - containerPort: 16379
              name: cluster
              protocol: TCP
            volumeMounts:
            - name: conf
              mountPath: /etc/redis
            - name: data
              mountPath: /var/lib/redis
          volumes:
          - name: conf
            configMap:
              name: redis-conf
              items:
              - key: redis.conf
                path: redis.conf
      volumeClaimTemplates:
      - metadata:
          name: data
          namespace: scut-tech
        spec:
          accessModes: ["ReadWriteMany"]
          resources:
            requests:
              storage: 2Gi
    
    

    命令如下:

    kubectl apply -f redis.yaml

    3.2 初始化集群

    查看集群节点状态:

    kubectl get pods -n scut-tech

    所有节点(redis-0 ~ redis-5)的 STATUS 都是 Running 状态后,再执行如下命令初始化集群:

    kubectl exec -it redis-0 -n scut-tech -- redis-cli --cluster create `kubectl get pods -n scut-tech -l app=redis -o jsonpath='{range.items[*]}{.status.podIP}:6379 '|awk '{print $1" "$2" "$3" "$4" "$5" "$6" "}'` --cluster-replicas 1

    3.3 验证集群部署

    kubectl exec -it redis-0 -n scut-tech -- redis-cli cluster info 
    for x in $(seq 0 5); do echo "redis-$x"; kubectl exec redis-$x -n scut-tech -- redis-cli role; echo; done

    3.4 查看pod对应的DNS域名

    for x in $(seq 0 5); do kubectl exec redis-$x -n scut-tech -- hostname -f; done

    四、问题解答

    4.1 redis 集群出错怎样处理?

    4.1.1 删除集群

    kubectl delete -f redis.yaml


    4.1.2删除 nfs 配置

    rm -rf /data/redis/cluster*/**


    4.1.3 重启集群

    kubectl apply -f redis.yaml


    4.1.4 重新初始化集群

    查看集群状态,所有的 redis 节点都正常启动后,再执行如下命令初始化集群:

    kubectl exec -it redis-0 -n scut-tech -- redis-cli --cluster create `kubectl get pods -n scut-tech -l app=redis -o jsonpath='{range.items[*]}{.status.podIP}:6379 '|awk '{print $1" "$2" "$3" "$4" "$5" "$6" "}'` --cluster-replicas 1
back_to_top