• Istio + k8s 安装部署过程

    • 2023-03-26 23:28
    • 字数 1,349
    • 阅读 374

    由于云服务器机房迁移,周末重新部署了 istio + k8s,使用版本为 kubeadm v1.25.8 及 istio v1.15.6,由于每次部署总遇到些问题,记录下来以供参考。

    本次 k8s 环境采用三台服务器搭建一主两从架构,hostname 分别设置命名为 k8s-master、k8s-node1、k8s-node2。

    服务器名称 服务器IP 操作系统
    k8s-master 192.168.20.100 CentOS 7.6
    k8s-node1 192.168.20.101 CentOS 7.6
    k8s-node2 192.168.20.102 CentOS 7.6

    一、环境准备

    1.1 安装相关工具

    yum install yum-utils -y
    yum install wget -y
    yum install vim -y
    yum upgrade

    1.2 设置 yum 国内源

    wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo

    1.3 关闭防火墙

    防火墙一定要提前关闭,否则在后续安装K8S集群的时候是个trouble maker。执行下面语句关闭,并禁用开机启动:

    systemctl stop firewalld & systemctl disable firewalld

    1.4 关闭Swap

    在安装 k8s 集群时,Linux 的 Swap 内存交换机制是一定要关闭的,否则会因为内存交换而影响性能以及稳定性。这里,我们可以提前进行设置。

    sed -ri 's/.*swap.*/#&/' /etc/fstab
    swapoff -a

    二、安装 docker

    安装kubernetes前,必须要先安装Docker。

    2.1 添加阿里云的Docker仓库

    yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
    yum makecache

    2.2 执行以下命令,安装最新版Docker

    yum install docker-ce -y

    运行docker --version,可以看到安装了截止目前最新版本。

    2.3 启动Docker服务并激活开机启动

    systemctl start docker & systemctl enable docker

    三、安装 kubernrtes

    3.1 执行以下命令添加kubernetes.repo仓库

    cat > /etc/yum.repos.d/kubernetes.repo << EOF
    [kubernetes]
    name=Kubernetes
    baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
    enabled=1
    gpgcheck=0
    repo_gpgcheck=0
    gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
    EOF

    3.2 关闭swap、防火墙

    环境准备时已经关闭,这里略过。

    3.3 关闭 SeLinux

    setenforce 0

    3.4 将桥接的 IPv4 流量传递到 iptables 的链

    cat > /etc/sysctl.d/k8s.conf << EOF
    net.bridge.bridge-nf-call-ip6tables = 1
    net.bridge.bridge-nf-call-iptables = 1
    EOF
    
    echo 1 > /proc/sys/net/ipv4/ip_forward
    
    # 生效
    sysctl --system

    3.5 安装 kubelet、kubeadm、kubectl

    yum install -y kubelet-1.25.8 kubeadm-1.25.8 kubectl-1.25.8

    如需安装最新版本,可以执行如下命令:

    yum install -y kubelet kubeadm kubectl

    如需卸载,可执行如下命令:

    yum remove -y kubelet kubeadm kubectl

    如果节点加入集群失败,通过 kubeadm reset 也未能解决,可以尝试卸载 k8s,并清空相关文件目录,然后重新安装。

    卸载并清理 k8s 命令如下:

    kubeadm reset 
    yum remove kubeadm kubectl kubelet kubernetes-cni kube*
    yum autoremove
    rm -rf ~/.kube
    rm -rf /etc/cni /etc/kubernetes /var/lib/dockershim /var/lib/etcd /var/lib/kubelet /var/lib/etcd2/ /var/run/kubernetes ~/.kube/* 
    rm -rf /var/lib/docker /etc/docker /var/run/docker.sock
    rm -f /etc/apparmor.d/docker /etc/systemd/system/etcd* 

    3.6 启动 kubelet

    systemctl enable kubelet && systemctl start kubelet

    3.7 列出需要的镜像

    kubeadm config images list

    得到所有需要的组件,也就是以下七个组件:

    registry.k8s.io/kube-apiserver:v1.25.8
    registry.k8s.io/kube-controller-manager:v1.25.8
    registry.k8s.io/kube-scheduler:v1.25.8
    registry.k8s.io/kube-proxy:v1.25.8
    registry.k8s.io/pause:3.8
    registry.k8s.io/etcd:3.5.4-0
    registry.k8s.io/coredns/coredns:v1.9.3

    3.8 设置 hosts

    分别登录k8s-master、k8s-node1、k8s-node2,执行 hostnamectl set-hostname hostname 设置 hosts,如 k8s-master 命令如下:

    hostnamectl set-hostname k8s-master

    然后在 k8s-master 设置 /etc/hosts 如下:

    cat >> /etc/hosts << EOF
    # 185.199.111.133 raw.githubusercontent.com
    192.168.20.100 k8s-master
    192.168.20.101 k8s-node1
    192.168.20.102 k8s-node2
    EOF
    
    systemctl restart network

    3.9 配置 containerd

    注:如果 config.toml 不存在,可以使用如下命令生成:

    [root@k8s-master ~]# cd /etc/containerd/
    [root@k8s-master containerd]# containerd config default | sudo tee /etc/containerd/config.toml

    替换 sandbox_image 镜像

    vim /etc/containerd/config.toml
    sandbox_image="registry.aliyuncs.com/google_containers/pause:3.8"

    启动服务 

    [root@k8s-master ~]# systemctl restart containerd && systemctl enable containerd

    3.10 初始化 k8s-master

    kubeadm init \
    --apiserver-advertise-address=192.168.20.100 \
    --image-repository registry.aliyuncs.com/google_containers \
    --kubernetes-version v1.25.8 \
    --service-cidr=10.96.0.0/12 \
    --pod-network-cidr=10.244.0.0/16

    初始成功后,将打印如下日志:

    Your Kubernetes control-plane has initialized successfully!
    
    To start using your cluster, you need to run the following as a regular user:
    
      mkdir -p $HOME/.kube
      sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
      sudo chown $(id -u):$(id -g) $HOME/.kube/config
    
    Alternatively, if you are the root user, you can run:
    
      export KUBECONFIG=/etc/kubernetes/admin.conf
    
    You should now deploy a pod network to the cluster.
    Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
      https://kubernetes.io/docs/concepts/cluster-administration/addons/
    
    Then you can join any number of worker nodes by running the following on each as root:
    
    kubeadm join 192.168.20.100:6443 --token s3t3oe.4d7iiye9bsnzmo7k \
            --discovery-token-ca-cert-hash sha256:7ec91d6152705878a07fe418542f46fcbdd3eef8175433b933aaa5269a0d1dfe

    按日志要求,先在 k8s-master 执行如下命令:

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config
    kubectl get nodes

    3.11 部署 CNI 网络插件(k8s-master)

    部署CNI网络插件(如果有网络问题,需要/etc/hosts增加域名解析如:199.232.68.133 raw.githubusercontent.com),然后执行:

    kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

    或者直接下载 https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

    然后执行:

    kubectl apply -f kube-flannel.yml

    查看运行状态:

    kubectl get pods -n kube-system
    kubectl get nodes

    3.12 初始化 node(初始化 master 时提示内容)

    分别登录各节点,执行如下命令加入集群:

    kubeadm join 192.168.20.100:6443 --token s3t3oe.4d7iiye9bsnzmo7k \
            --discovery-token-ca-cert-hash sha256:7ec91d6152705878a07fe418542f46fcbdd3eef8175433b933aaa5269a0d1dfe

    注1:如果报错

    [ERROR CRI]: container runtime is not running: output: E0624 18:07:53.735108   17940 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"

    执行如下命令,注释掉 disabled_plugins = ["cri"] :

    vim /etc/containerd/config.toml

    如果出现该错误,需要全部节点处理,包括 k8s-master、k8s-node1、k8s-node2 等
    然后重启 containerd 和 kubeadm:

    systemctl restart containerd
    kubeadm reset

    注2:使用kubeadm创建集群失败报Unable to register node with API server,查看日志发现例如:failed to pull image \"k8s.gcr.io/pause:3.6\"

    可分别登录各节点,执行如下命令:

    ctr -n k8s.io i pull registry.aliyuncs.com/google_containers/pause:3.6
    ctr -n k8s.io i tag registry.aliyuncs.com/google_containers/pause:3.6 k8s.gcr.io/pause:3.6

    四、安装 istio

    4.1 下载 istio

    wget https://github.com/istio/istio/releases/download/1.15.6/istio-1.15.6-linux-amd64.tar.gz

    解压到 /usr/local

    tar zxvf istio-1.15.6-linux-amd64.tar.gz -C /usr/local/

    4.2 设置环境变量

    echo 'export ISTIO_HOME=/usr/local/istio-1.15.6' >> /etc/profile
    echo 'export PATH=$PATH:$ISTIO_HOME/bin' >> /etc/profile
    source /etc/profile

    4.3 查看版本

    istioctl version

    如上环境准备好后,就可以根据官方文档安装 istio 了。

    istioctl install --set profile=demo

    https://istio.io/latest/docs/setup/install/istioctl/

back_to_top