{"id":4149,"date":"2021-06-29T21:10:34","date_gmt":"2021-06-29T13:10:34","guid":{"rendered":"http:\/\/blog.ywdevops.cn\/?p=4149"},"modified":"2026-05-18T11:08:17","modified_gmt":"2026-05-18T03:08:17","slug":"k8s-2","status":"publish","type":"post","link":"https:\/\/blog.ywdevops.cn\/index.php\/2021\/06\/29\/k8s-2\/","title":{"rendered":"k8s\u4e8c\u8fdb\u5236\u65b9\u5f0f\u90e8\u7f72"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><strong>\u751f\u4ea7\u73af\u5883\u90e8\u7f72 K8s \u96c6\u7fa4\u7684\u4e24\u79cd\u65b9\u5f0f \uff1a<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Kubeadm\u65b9\u5f0f\uff1a Kubeadm \u662f\u4e00\u4e2a K8s \u90e8\u7f72\u5de5\u5177\uff0c\u63d0\u4f9b kubeadm init \u548c kubeadm join\uff0c\u7528\u4e8e\u5feb\u901f\u90e8\u7f72 Kubernetes \u96c6\u7fa4\u3002 <\/li>\n\n\n\n<li>\u4e8c\u8fdb\u5236\u65b9\u5f0f\uff1a \u4ece github \u4e0b\u8f7d\u53d1\u884c\u7248\u7684\u4e8c\u8fdb\u5236\u5305\uff0c\u624b\u52a8\u90e8\u7f72\u6bcf\u4e2a\u7ec4\u4ef6\uff0c\u7ec4\u6210 Kubernetes \u96c6\u7fa4\u3002 <\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">\u603b\u7ed3\uff1a Kubeadm \u964d\u4f4e\u90e8\u7f72\u95e8\u69db\uff0c\u4f46\u5c4f\u853d\u4e86\u5f88\u591a\u7ec6\u8282\uff0c\u9047\u5230\u95ee\u9898\u5f88\u96be\u6392\u67e5\u3002\u5982\u679c\u60f3\u66f4\u5bb9\u6613\u53ef\u63a7\uff0c\u63a8\u8350\u4f7f\u7528\u4e8c\u8fdb\u5236\u5305\u90e8\u7f72 Kubernetes \u96c6\u7fa4\uff0c\u867d\u7136\u90e8\u7f72\u9ebb\u70e6\uff0c\u671f\u95f4\u53ef\u4ee5\u5b66\u4e60\u5f88\u591a\u5de5\u4f5c\u539f\u7406\uff0c\u4e5f\u5229\u4e8e\u540e\u671f\u7ef4\u62a4\u3002 <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u73af\u5883\u51c6\u5907\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td class=\"has-text-align-center\" data-align=\"center\">\u8282\u70b9\u6570<\/td><td class=\"has-text-align-center\" data-align=\"center\">\u7cfb\u7edf<\/td><td class=\"has-text-align-center\" data-align=\"center\">docker\u7248\u672c<\/td><td class=\"has-text-align-center\" data-align=\"center\">kubernetes\u7248\u672c<\/td><\/tr><tr><td class=\"has-text-align-center\" data-align=\"center\">master(3\u53f0)<\/td><td class=\"has-text-align-center\" data-align=\"center\">centos7<\/td><td class=\"has-text-align-center\" data-align=\"center\">docker-ce 19.03.9<\/td><td class=\"has-text-align-center\" data-align=\"center\">1.20.8<\/td><\/tr><tr><td class=\"has-text-align-center\" data-align=\"center\">worker(2\u53f0)<\/td><td class=\"has-text-align-center\" data-align=\"center\">centos7<\/td><td class=\"has-text-align-center\" data-align=\"center\"> docker-ce 19.03.9 <\/td><td class=\"has-text-align-center\" data-align=\"center\">1.20.8<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"has-black-color has-luminous-vivid-amber-background-color has-text-color has-background wp-block-paragraph\">#\u6ce8\u610f\uff1amaster\u8282\u70b9\u81f3\u5c11\u89812\u4e2acpu\uff0c\u5426\u5219\u62a5\u9519 error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR NumCPU]: the number of available CPUs 1 is less than the required 2<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u64cd\u4f5c\u7cfb\u7edf\u521d\u59cb\u5316\u8bbe\u7f6e\uff1a<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>1\u3001\u4fee\u6539\/etc\/hosts\uff0c\u8bbe\u7f6e\u4e3b\u673a\u540d\uff0c\u6dfb\u52a0\u5185\u5bb9\u5982\u4e0b\uff1a\n192.168.15.215 k8s-master01\n192.168.15.216 k8s-master02\n192.168.15.217 k8s-master03\n192.168.15.218 k8s-worker1\n192.168.15.219 k8s-worker2<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>2\u3001 \u5173\u95ed\u9632\u706b\u5899\nsystemctl stop firewalld\nsystemctl disable firewalld\nsystemctl stop NetworkManager        #\u5173\u95ed\u540ecalico\u624d\u53ef\u4ee5\u7ba1\u7406\u7f51\u5361\uff0c\u5426\u5219\u6709\u5f71\u54cd\nsystemctl disable NetworkManager<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>3\u3001\u5173\u95ed selinux\nsed -i 's\/enforcing\/disabled\/' \/etc\/selinux\/config # \u6c38\u4e45\nsetenforce 0                                       # \u4e34\u65f6<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>4\u3001\u5173\u95ed swap\n#swap\u5f00\u542f\u4f1a\u5f71\u54cdk8s\u7f16\u6392\u548c\u8c03\u5ea6\u5e94\u7528\u7a0b\u5e8f\u8fd0\u884c\u7684\u6548\u679c\uff0c\u4f1a\u964d\u4f4e\u6027\u80fd\nswapoff -a                                         # \u4e34\u65f6\nsed -ri '\/^&#91;^#]*swap\/s@^@#@' \/etc\/fstab            # \u6c38\u4e45<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>5\u3001\u5c06\u6865\u63a5\u7684 IPv4 \u6d41\u91cf\u4f20\u9012\u5230 iptables \u7684\u94fe\ncat &gt; \/etc\/sysctl.d\/k8s.conf &lt;&lt; EOF\nnet.bridge.bridge-nf-call-ip6tables = 1\nnet.bridge.bridge-nf-call-iptables = 1\nEOF\nsysctl --system                                     # \u751f\u6548<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>6\u3001\u5b89\u88c5\u5e38\u7528\u5de5\u5177\uff1a\nyum -y install wget jq psmisc vim net-tools  telnet yum-utils device-mapper-persistent-data lvm2 git network-scripts tar curl<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>7\u3001\u65f6\u95f4\u540c\u6b65\n#\u5404\u4e2a\u8282\u70b9\u9700\u8981\u901a\u8fc7\u65f6\u95f4\u670d\u52a1\u5668\u4fdd\u6301\u65f6\u95f4\u540c\u6b65\nyum install ntpdate -y\nntpdate time.windows.com  \u6216\u8005 ntpdate -u cn.pool.ntp.org(\u4e2d\u56fdntp\u670d\u52a1\u5668)<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>8\u3001\u4fee\u6539\u5185\u6838\u53c2\u6570\u6587\u4ef6\/etc\/security\/limits.conf\uff0c\u6dfb\u52a0\u5185\u5bb9\u5982\u4e0b\uff1a\ncat &gt;&gt; \/etc\/security\/limits.conf &lt;&lt;EOF\n* soft nofile 65535\n* hard nofile 65535\n* soft noproc 65535\n* hard noproc 65535\n* soft memlock unlimited\n* hard memlock unlimited\nEOF<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>9\u3001\u5347\u7ea7\u4e00\u4e0b\u7cfb\u7edf\uff0c\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a\nyum update -y  --exclude=kernel* &amp;&amp; reboot    #\u6392\u9664\u5185\u6838\uff0c\u4e0b\u9762\u4f1a\u5355\u72ec\u5347\u7ea7\u5185\u6838<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>10\u3001\u5347\u7ea7\u5185\u6838\uff0c\u81f3\u5c11\u8981\u8fbe\u52304.18+ \u624d\u53ef\u4ee5\nyum  -y localinstall kernel-ml* (\u5305\u542bkernel-ml-5.17.5-1.el7.elrepo.x86_64.rpm\u548ckernel-ml-devel-5.17.5-1.el7.elrepo.x86_64.rpm\u4e24\u4e2a\u5305)\n\u66f4\u6539\u5185\u6838\u542f\u52a8\u987a\u5e8f\uff1a\ngrub2-set-default 0 &amp;&amp; grub2-mkconfig -o \/etc\/grub2.cfg\ngrubby --args=\"user_namespace.enable=1\" --update-kernel=\"$(grubby --default-kernel)\"\ngrubby --default-kernel                       #\u68c0\u67e5\u9ed8\u8ba4\u5185\u6838\nreboot                                        #\u6700\u540e\u91cd\u542f\u5373\u53ef<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u6ce8\uff1a\u5185\u6838\u4e0b\u8f7d\u5730\u5740\uff1a<a rel=\"noreferrer noopener\" href=\"https:\/\/elrepo.org\/linux\/kernel\/el7\/x86_64\/RPMS\/\" target=\"_blank\">\u5185\u6838\u4e0b\u8f7d\u5730\u57401<\/a>  \u3001<a href=\"http:\/\/mirrors.coreix.net\/elrepo-archive-archive\/kernel\/el7\/x86_64\/RPMS\/\" target=\"_blank\" rel=\"noreferrer noopener\">\u5185\u6838\u4e0b\u8f7d\u5730\u57402<\/a><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>11\u3001\u5b89\u88c5ipvsadm\nyum -y install ipvsadm ipset sysstat conntrack libseccomp\n\u5185\u6838\u914d\u7f6eipvs\u6a21\u5757\ncat &gt; \/etc\/modules-load.d\/ipvs.conf &lt;&lt; EOF\nip_vs\nip_vs_lc\nip_vs_wlc\nip_vs_rr\nip_vs_wrr\nip_vs_lblc\nip_vs_lblcr\nip_vs_dh\nip_vs_sh\nip_vs_fo\nip_vs_nq\nip_vs_sed\nip_vs_ftp\nip_vs_sh\nnf_conntrack\nip_tables\nip_set\nxt_set\nipt_set\nipt_rpfilter\nipt_REJECT\nipip\nEOF\n#\u6267\u884c\u547d\u4ee4\u542f\u52a8\u670d\u52a1,\u6ce8\u610f\uff0c\u4e0a\u4e00\u6b65\u5347\u7ea7\u5185\u6838\u5982\u679c\u4e0d\u91cd\u542f\u6b64\u5904\u4f1a\u4e0d\u6210\u529f\nsystemctl enable --now systemd-modules-load.service\n#\u91cd\u542f\u7cfb\u7edf\u540e\u67e5\u770b\u662f\u5426\u52a0\u8f7d\nlsmod | grep -e ip_vs -e nf_conntrack<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\u610f\uff1a\u5982\u679c\u5185\u6838\u7248\u672c\u5c0f\u4e8e4.19\uff0c\u90a3\u4e48\u914d\u7f6e\u4e2d\u7684nf_conntrack\u5e94\u8be5\u4e3anf_conntrack_ipv4<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>12\u3001\u5f00\u542f\u4e00\u4e9bk8s\u96c6\u7fa4\u4e2d\u5fc5\u987b\u7684\u5185\u6838\u53c2\u6570\ncat &gt;&gt; \/etc\/sysctl.d\/k8s.conf &lt;&lt;EOF\nnet.ipv4.ip_forward =1\nfs.may_detach_mounts = 1\nvm.overcommit_memory = 1\nvm.panic_on.oom = 0\nfs.inotify.max_user_watches=89100\nfs.file-max=52706963\nfs.nr_open = 52706963\nnet.netfilter.nf_conntrack_max = 2310720\nnet.ipv4.tcp_keepalive_time = 600\nnet.ipv4.tcp_keepalive_probes = 3\nnet.ipv4.tcp_keepalive_intvl = 15\nnet.ipv4.tcp_max_tw_buckets = 36000\nnet.ipv4.tcp_tw_reuse = 1\nnet.ipv4.tcp_max_orphans = 327680\nnet.ipv4.tcp_orphan_retries = 3\nnet.ipv4.tcp_syncookies = 1\nnet.ipv4.tcp_max_syn_backlog = 16384\nnet.ipv4.ip_contrack_max = 65536\nnet.ipv4.tcp_timestamps = 0\nnet.core.somaxconn = 16384\nEOF\n#\u914d\u7f6e\u5b8c\u6210\u540e\uff0c\u91cd\u542f\u670d\u52a1\u5668\uff0c\u67e5\u770b\u5185\u6838\u52a0\u8f7d\u60c5\u51b5\nlsmod | grep --color=auto -e ip_vs -e nf_conntrack<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-center has-medium-font-size wp-block-paragraph\"><strong>\u4e00\u3001\u90e8\u7f72etcd\u96c6\u7fa4\uff1a<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Etcd \u662f\u4e00\u4e2a\u5206\u5e03\u5f0f\u952e\u503c\u5b58\u50a8\u7cfb\u7edf\uff0cKubernetes \u4f7f\u7528 Etcd \u8fdb\u884c\u6570\u636e\u5b58\u50a8\uff0c\u6240\u4ee5\u5148 \u51c6\u5907\u4e00\u4e2a Etcd \u6570\u636e\u5e93\uff0c\u4e3a\u89e3\u51b3 Etcd \u5355\u70b9\u6545\u969c\uff0c\u5e94\u91c7\u7528\u96c6\u7fa4\u65b9\u5f0f\u90e8\u7f72\uff0c\u8fd9\u91cc\u4f7f\u7528 3 \u53f0\u7ec4\u5efa\u96c6\u7fa4\uff0c\u53ef\u5bb9\u5fcd 1 \u53f0\u673a\u5668\u6545\u969c\uff0c\u5f53\u7136\uff0c\u4f60\u4e5f\u53ef\u4ee5\u4f7f\u7528 5 \u53f0\u7ec4\u5efa\u96c6\u7fa4\uff0c\u53ef\u5bb9\u5fcd 2 \u53f0\u673a\u5668\u6545\u969c\u3002 \u672c\u4f8b\u5b50\u4e2d\u4f7f\u7528\u4e09\u53f0master\u8282\u70b9\u673a\u5668\u7ec4\u6210\u96c6\u7fa4<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td class=\"has-text-align-center\" data-align=\"center\">etcd1<\/td><td class=\"has-text-align-center\" data-align=\"center\">192.168.15.215<\/td><\/tr><tr><td class=\"has-text-align-center\" data-align=\"center\">etcd2<\/td><td class=\"has-text-align-center\" data-align=\"center\">192.168.15.216<\/td><\/tr><tr><td class=\"has-text-align-center\" data-align=\"center\">etcd3<\/td><td class=\"has-text-align-center\" data-align=\"center\">182.168.15.217<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>1\u3001 \u51c6\u5907 cfssl \u8bc1\u4e66\u751f\u6210\u5de5\u5177 \uff1a<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">cfssl \u662f\u4e00\u4e2a\u5f00\u6e90\u7684\u8bc1\u4e66\u7ba1\u7406\u5de5\u5177\uff0c\u4f7f\u7528 json \u6587\u4ef6\u751f\u6210\u8bc1\u4e66\uff0c\u76f8\u6bd4 openssl \u66f4\u65b9\u4fbf\u4f7f\u7528\u3002 <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u627e\u4efb\u610f\u4e00\u53f0\u670d\u52a1\u5668\u64cd\u4f5c\uff0c\u8fd9\u91cc\u7528 Master \u8282\u70b9\uff0c\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>wget https:\/\/pkg.cfssl.org\/R1.2\/cfssl_linux-amd64\nwget https:\/\/pkg.cfssl.org\/R1.2\/cfssljson_linux-amd64\nwget https:\/\/pkg.cfssl.org\/R1.2\/cfssl-certinfo_linux-amd64\nchmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64\nmv cfssl_linux-amd64 \/usr\/bin\/cfssl\nmv cfssljson_linux-amd64 \/usr\/bin\/cfssljson\nmv cfssl-certinfo_linux-amd64 \/usr\/bin\/cfssl-certinfo<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>2\u3001 \u751f\u6210 Etcd \u8bc1\u4e66  \uff1a<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> 2.1 \u81ea\u7b7e\u8bc1\u4e66\u9881\u53d1\u673a\u6784\uff08CA\uff09\uff1a <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u5728\u5f53\u524d\u7528\u6237\u7684\u6839\u76ee\u5f55\u4e0b\u521b\u5efa\u6587\u4ef6\u52a0TLS\uff0c\u91cc\u9762\u521b\u5efa\u76ee\u5f55etcd\u548ck8s \nmkdir -p ~\/TLS\/{etcd,k8s} <\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"> \u81ea\u7b7e CA\uff1a <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u8fdb\u5165\u521b\u5efa\u7684etcd\u76ee\u5f55\u4e2d\ncd ~\/TLS\/etcd\n#\"expiry\": \"87600h\", \u8868\u793a\u8bc1\u4e66\u8fc7\u671f\u65f6\u95f4\ncat &gt; etcd-ca-config.json &lt;&lt;EOF\n{\n    \"signing\": {\n       \"default\": {\n          \"expiry\": \"87600h\"\n       },\n       \"profiles\": {\n           \"www\": {\n              \"expiry\": \"87600h\",\n              \"usages\": &#91;\n                 \"signing\",\n                 \"key encipherment\",\n                 \"server auth\",\n                 \"client auth\"\n               ]\n            }\n        }\n    }\n}\nEOF\n#\u518d\u6b21\u6267\u884c\u547d\u4ee4\u521b\u5efaetcd-ca-csr.json\u8bc1\u4e66\u8bf7\u6c42\u6587\u4ef6\ncat &gt; etcd-ca-csr.json &lt;&lt; EOF\n{\n    \"CN\": \"etcd CA\",\n    \"key\": {\n       \"algo\": \"rsa\",\n       \"size\": 2048\n    },\n    \"names\": &#91;\n      {\n       \"C\": \"CN\",\n       \"L\": \"Beijing\",\n       \"ST\": \"Beijing\"\n      }\n    ]\n}\nEOF<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u53ef\u901a\u8fc7  cat  etcd-ca-config.json | jq .   \u6765\u68c0\u67e5json\u6587\u4ef6\u662f\u5426\u6b63\u786e(\u9700\u8981\u63d0\u524dyum -y install jq )<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"543\" height=\"401\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/10.png\" alt=\"\" class=\"wp-image-4280\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/10.png 543w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/10-300x222.png 300w\" sizes=\"auto, (max-width: 543px) 100vw, 543px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"560\" height=\"324\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/1.png\" alt=\"\" class=\"wp-image-4281\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/1.png 560w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/1-300x174.png 300w\" sizes=\"auto, (max-width: 560px) 100vw, 560px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u63a5\u4e0b\u6765\u6267\u884c\u5982\u4e0b\u547d\u4ee4\u751f\u6210\u8bc1\u4e66\uff0c \u4f1a\u751f\u6210 ca.pem \u548c ca-key.pem \u6587\u4ef6\uff0c\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare etcd-ca -<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"700\" height=\"245\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/2.png\" alt=\"\" class=\"wp-image-4285\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/2.png 700w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/2-300x105.png 300w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/2-600x210.png 600w\" sizes=\"auto, (max-width: 700px) 100vw, 700px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"571\" height=\"172\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/3.png\" alt=\"\" class=\"wp-image-4286\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/3.png 571w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/3-300x90.png 300w\" sizes=\"auto, (max-width: 571px) 100vw, 571px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">2.2  \u4f7f\u7528\u81ea\u7b7e CA \u7b7e\u53d1 Etcd HTTPS \u8bc1\u4e66 \uff1a<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u521b\u5efa\u8bc1\u4e66\u7533\u8bf7\u6587\u4ef6\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#\"192.169.15.*\"  \u8868\u793a\u9884\u7559IP \u53ef\u4ee5\u5199\u591a\u4e2a\ncat &gt; server-csr.json &lt;&lt;EOF\n{\n \"CN\": \"etcd\",\n \"hosts\": &#91;\n \"192.168.15.215\",\n \"192.168.15.216\",\n \"192.168.15.217\",\n \"192.168.15.*\"\n ],\n \"key\": {\n     \"algo\": \"rsa\",\n     \"size\": 2048\n },\n \"names\": &#91;\n    {\n      \"C\": \"CN\",\n      \"L\": \"BeiJing\",\n      \"ST\": \"BeiJing\"\n    }\n ]\n}\nEOF<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\uff1a\u4e0a\u8ff0\u6587\u4ef6 hosts \u5b57\u6bb5\u4e2d IP \u4e3a\u6240\u6709 etcd \u8282\u70b9\u7684\u96c6\u7fa4\u5185\u90e8\u901a\u4fe1 IP\uff0c\u4e00\u4e2a\u90fd\u4e0d\u80fd \u5c11\uff01\u4e3a\u4e86\u65b9\u4fbf\u540e\u671f\u6269\u5bb9\u53ef\u4ee5\u591a\u5199\u51e0\u4e2a\u9884\u7559\u7684 IP\u3002 <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u751f\u6210\u8bc1\u4e66\uff0c \u4f1a\u751f\u6210 server.pem \u548c server-key.pem \u6587\u4ef6\uff0c\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cfssl gencert -ca=etcd-ca.pem -ca-key=etcd-ca-key.pem -config=etcd-ca-config.json -profile=www server-csr.json | cfssljson -bare server<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"565\" height=\"171\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/17-2.png\" alt=\"\" class=\"wp-image-4162\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/17-2.png 565w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/17-2-300x91.png 300w\" sizes=\"auto, (max-width: 565px) 100vw, 565px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">3\u3001\u4e0b\u8f7detcd\uff0c\u5730\u5740\uff1ahttps:\/\/github.com\/etcd-io\/etcd\/releases\/download\/v3.5.23\/etcd-v3.5.23-linux-amd64.tar.gz<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u63a5\u4e0b\u6765\u6b63\u5f0f\u90e8\u7f72etcd\uff0c\u9996\u5148\u73b0\u5728\u8282\u70b91\u4e0a\u64cd\u4f5c\uff0c \u4e3a\u7b80\u5316\u64cd\u4f5c\uff0c\u5f85\u4f1a\u5c06\u8282\u70b9 1 \u751f\u6210\u7684\u6240\u6709\u6587\u4ef6\u62f7\u8d1d\u5230\u8282\u70b9 2 \u548c \u8282\u70b9 3 \u4e0a<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u521b\u5efa\u5de5\u4f5c\u76ee\u5f55\u5e76\u89e3\u538b\u4e8c\u8fdb\u5236\u5305\uff1a<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir \/opt\/etcd\/{bin,cfg,ssl} -p\ntar zxvf etcd-v3.5.0-linux-amd64.tar.gz\nmv etcd-v3.5.0-linux-amd64\/{etcd,etcdctl} \/opt\/etcd\/bin\/<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong> \u521b\u5efa etcd \u914d\u7f6e\u6587\u4ef6 :<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/opt\/etcd\/cfg\/etcd.conf   &lt;&lt;EOF\n#&#91;Member]\nETCD_NAME=\"etcd-1\"\nETCD_DATA_DIR=\"\/opt\/etcd\/data\"\nETCD_LISTEN_PEER_URLS=\"https:\/\/192.168.15.215:2380\"\nETCD_LISTEN_CLIENT_URLS=\"https:\/\/192.168.15.215:2379\"\n#&#91;Clustering]\nETCD_INITIAL_ADVERTISE_PEER_URLS=\"https:\/\/192.168.15.215:2380\"\nETCD_ADVERTISE_CLIENT_URLS=\"https:\/\/192.168.15.215:2379\"\nETCD_INITIAL_CLUSTER=\"etcd-1=https:\/\/192.168.15.215:2380,etcd-2=https:\/\/192.168.15.216:2380,etcd-3=https:\/\/192.168.15.217:2380\"\nETCD_INITIAL_CLUSTER_TOKEN=\"etcd-cluster\"\nETCD_INITIAL_CLUSTER_STATE=\"new\"\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u8bf4\u660e\uff1a <\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li> ETCD_NAME\uff1a\u8282\u70b9\u540d\u79f0\uff0c\u96c6\u7fa4\u4e2d\u552f\u4e00  <\/li>\n\n\n\n<li> ETCD_DATA_DIR\uff1a\u6570\u636e\u76ee\u5f55 <\/li>\n\n\n\n<li> ETCD_LISTEN_PEER_URLS\uff1a\u96c6\u7fa4\u901a\u4fe1\u76d1\u542c\u5730\u5740 <\/li>\n\n\n\n<li> ETCD_LISTEN_CLIENT_URLS\uff1a\u5ba2\u6237\u7aef\u8bbf\u95ee\u76d1\u542c\u5730\u5740 <\/li>\n\n\n\n<li> ETCD_INITIAL_ADVERTISE_PEERURLS\uff1a\u96c6\u7fa4\u901a\u544a\u5730\u5740 <\/li>\n\n\n\n<li> ETCD_ADVERTISE_CLIENT_URLS\uff1a\u5ba2\u6237\u7aef\u901a\u544a\u5730\u5740 <\/li>\n\n\n\n<li> ETCD_INITIAL_CLUSTER\uff1a\u96c6\u7fa4\u8282\u70b9\u5730\u5740 <\/li>\n\n\n\n<li> ETCD_INITIALCLUSTER_TOKEN\uff1a\u96c6\u7fa4 Token <\/li>\n\n\n\n<li> ETCD_INITIALCLUSTER_STATE\uff1a\u52a0\u5165\u96c6\u7fa4\u7684\u5f53\u524d\u72b6\u6001\uff0cnew \u662f\u65b0\u96c6\u7fa4\uff0cexisting \u8868\u793a\u52a0\u5165\u5df2\u6709\u96c6\u7fa4 <\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u901a\u8fc7systemd \u7ba1\u7406 etcd \uff1a<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/usr\/lib\/systemd\/system\/etcd.service &lt;&lt; EOF\n&#91;Unit]\nDescription=Etcd Server\nAfter=network.target\nAfter=network-online.target\nWants=network-online.target\n&#91;Service]\nType=notify\nEnvironmentFile=\/opt\/etcd\/cfg\/etcd.conf\nExecStart=\/opt\/etcd\/bin\/etcd \\\n--cert-file=\/opt\/etcd\/ssl\/server.pem \\\n--key-file=\/opt\/etcd\/ssl\/server-key.pem \\\n--peer-cert-file=\/opt\/etcd\/ssl\/server.pem \\\n--peer-key-file=\/opt\/etcd\/ssl\/server-key.pem \\\n--trusted-ca-file=\/opt\/etcd\/ssl\/etcd-ca.pem \\\n--peer-trusted-ca-file=\/opt\/etcd\/ssl\/etcd-ca.pem \\\n--logger=zap\nRestart=on-failure\nLimitNOFILE=65536\n&#91;Install]\nWantedBy=multi-user.target\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong> <\/strong>\u62f7\u8d1d\u521a\u624d\u751f\u6210\u7684\u8bc1\u4e66 \uff1a \u628a\u521a\u624d\u751f\u6210\u7684\u8bc1\u4e66\u62f7\u8d1d\u5230\u914d\u7f6e\u6587\u4ef6\u4e2d\u7684\u8def\u5f84<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u5c06\u7b2c\u4e00\u6b65\u751f\u6210\u7684\u8bc1\u4e66\u6587\u4ef6\uff0c\u62f7\u8d1d\u5230ssl\u6587\u4ef6\u5939\u4e0b\ncp ~\/TLS\/etcd\/*ca*pem ~\/TLS\/etcd\/server*pem \/opt\/etcd\/ssl\/<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u5c06\u4e0a\u9762\u8282\u70b9 1 \u6240\u6709\u751f\u6210\u7684\u6587\u4ef6\u62f7\u8d1d\u5230master\u8282\u70b9 2 \u548cmaster\u8282\u70b9 3 \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>for i in k8s-master02 k8s-master03 ;do scp -r \/opt\/etcd ${i}:\/opt ;done\nfor i in k8s-master02 k8s-master03 ;do scp -r \/lib\/systemd\/system\/etcd.service  ${i}:\/lib\/systemd\/system ;done<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u7136\u540e\u5728\u8282\u70b9 2 \u548c\u8282\u70b9 3 \u5206\u522b\u4fee\u6539 etcd.conf \u914d\u7f6e\u6587\u4ef6\u4e2d\u7684\u8282\u70b9\u540d\u79f0\u548c\u5f53\u524d\u670d\u52a1\u5668 IP\uff1a \u4f8b\u5982<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ETCD_NAME=\"etcd-2\"                                             #\u4fee\u6539\u96c6\u7fa4\u552f\u4e00\u6807\u8bc6\nETCD_DATA_DIR=\"\/opt\/etcd\/data\"\nETCD_LISTEN_PEER_URLS=\"https:\/\/192.168.15.216:2380\"            #\u6539\u4e3a\u672c\u673aIP\nETCD_LISTEN_CLIENT_URLS=\"https:\/\/192.168.15.216:2379\"          #\u6539\u4e3a\u672c\u673aIP\n#&#91;Clustering]\nETCD_INITIAL_ADVERTISE_PEER_URLS=\"https:\/\/192.168.15.216:2380\" #\u6539\u4e3a\u672c\u673aIP\nETCD_ADVERTISE_CLIENT_URLS=\"https:\/\/192.168.15.216:2379\"       #\u6539\u4e3a\u672c\u673aIP<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u8bbe\u7f6eetcd\u5f00\u542f\u81ea\u542f\u52a8\uff0c\u5982\u4e0b\u547d\u4ee4\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>systemctl daemon-reload\nsystemctl enable etcd  --now    #\u8bbe\u7f6e\u5f00\u673a\u81ea\u542f\uff0c\u5e76\u542f\u52a8<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\u610f\uff1a\u5982\u679cetcd\u96c6\u7fa4\u901a\u8baf\u51fa\u73b0\u95ee\u9898\uff0c\u663e\u793a\u5df2\u7ecf\u6709\u7528\u6237\u5728\u4f7f\u7528\uff0c\u6b64\u65f6\u53ef\u4ee5\u5c06\u96c6\u7fa4\u4e2d\u6bcf\u4e2a\u8282\u70b9\u7684\/opt\/etcd\/data\u4e0b\u7684member\u5220\u9664\uff0c\u7136\u540e\u91cd\u542f\u6240\u6709\u8282\u70b9\u5373\u53ef<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u67e5\u770b\u96c6\u7fa4\u72b6\u6001\uff0c\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ETCDCTL_API=3 \/opt\/etcd\/bin\/etcdctl --cacert=\/opt\/etcd\/ssl\/etcd-ca.pem --cert=\/opt\/etcd\/ssl\/server.pem --key=\/opt\/etcd\/ssl\/server-key.pem --endpoints=\"https:\/\/192.168.15.215:2379,https:\/\/192.168.15.216:2379,https:\/\/192.168.15.217:2379\" endpoint health --write-out=table<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"> \u5982\u679c\u8f93\u51fa\u4e0a\u9762\u4fe1\u606f\uff0c\u5c31\u8bf4\u660e\u96c6\u7fa4\u90e8\u7f72\u6210\u529f:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"561\" height=\"199\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/18-2.png\" alt=\"\" class=\"wp-image-4170\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/18-2.png 561w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/18-2-300x106.png 300w\" sizes=\"auto, (max-width: 561px) 100vw, 561px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"> \u5982\u679c\u6709\u95ee\u9898\u7b2c\u4e00\u6b65\u5148\u770b\u65e5\u5fd7\uff1a\/var\/log\/message \u6216 journalctl -u etcd.service -b &#8211;no-pager -n 50<\/p>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\u610f\uff1aetcd\u96c6\u7fa4\u53ef\u5355\u72ec\u90e8\u7f72\u5728\u4efb\u610f\u673a\u5668\u4e0a\uff0c\u53ea\u8981\u80fd\u548capiserver\u901a\u8baf\u5373\u53ef\uff0cetcd\u5907\u4efd\u53ef\u53c2\u8003\uff1a<a href=\"https:\/\/blog.ywdevops.cn\/index.php\/2023\/02\/14\/etcd\/\">k8s\u4e4betcd\u7684\u5907\u4efd\u4e0e\u56de\u6062\u590d \u2013 IT\u8fd0\u7ef4 (blog.ywdevops.cn)<\/a><\/p>\n\n\n\n<p class=\"has-text-align-center has-medium-font-size wp-block-paragraph\"><strong>\u4e8c\u3001\u90e8\u7f72docker\uff1a<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong> <\/strong>\u4ee5\u4e0b\u5728\u6240\u6709\u8282\u70b9\u64cd\u4f5c\uff0c\u8fd9\u91cc\u91c7\u7528\u4e8c\u8fdb\u5236\u5b89\u88c5\uff0c\u7528 yum \u5b89\u88c5\u4e5f\u4e00\u6837 <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">1\u3001\u89e3\u538b\u4e8c\u8fdb\u5236\u5305\uff0c\u5e76\u5c06\u53ef\u6267\u884c\u6587\u4ef6\u52a0\u5165\u5230\u73af\u5883\u53d8\u91cf\u4e2d\uff0c\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>tar -zxvf docker-19.03.9.tgz       #\u89e3\u538b\u4e8c\u8fdb\u5236\u5305\nmv docker\/*  \/usr\/bin              #\u79fb\u52a8\u5230\u73af\u5883\u53d8\u91cf\/usr\/bin\/\u76ee\u5f55\u4e0b<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">2\u3001\u7531\u4e8e\u65b0\u7248\u7684kubernetes\u4f7f\u7528systemd\u6765\u8fdb\u884c\u8d44\u6e90\u9650\u5236\uff0c\u56e0\u6b64\u4fee\u6539docker\u7684CgoupDriver\u4e3asystemd\uff0c\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir \/etc\/docker\n\ncat &gt; \/etc\/docker\/daemon.json &lt;&lt; EOF\n{\n    \"exec-opts\": &#91;\"native.cgroupdriver=systemd\"],\n    \"max-concurrent-downloads\": 10,\n    \"max-concurrent-uploads\": 5,\n    \"log-driver\": \"json-file\",\n    \"storage-driver\": \"overlay2\",\n    \"data-root\": \"\/data\/docker\",\n    \"storage-opts\": &#91;\n    \"overlay2.override_kernel_check=true\"\n    ],\n    \"log-opts\":{\n        \"max-size\": \"300m\",\n        \"max-file\": \"2\" \n    },\n    \"live-restore\": true\n}\nEOF\n#live-restore:true \u91cd\u542fdocker  \u4e0d\u5f71\u54cd\u8fd0\u884c\u4e2d\u7684\u5bb9\u5668\n#\u8bbe\u7f6e\u5b8c\u6210\u540e\uff0c\u91cd\u542fdocker<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">3\u3001\u901a\u8fc7systemctl  \u7ba1\u7406docker\uff0c\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/usr\/lib\/systemd\/system\/docker.service &lt;&lt; EOF\n&#91;Unit]\nDescription=Docker Application Container Engine\nDocumentation=https:\/\/docs.docker.com\nAfter=network-online.target firewalld.service\nWants=network-online.target\n&#91;Service]\nType=notify\nExecStart=\/usr\/bin\/dockerd\nExecReload=\/bin\/kill -s HUP $MAINPID\nLimitNOFILE=infinity\nLimitNPROC=infinity\nLimitCORE=infinity\nTimeoutStartSec=0\nDelegate=yes\nKillMode=process\nRestart=on-failure\nStartLimitBurst=3\nStartLimitInterval=60s\n&#91;Install]\nWantedBy=multi-user.target\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">3\u3001 \u542f\u52a8\u5e76\u8bbe\u7f6e\u5f00\u673a\u542f\u52a8 \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>systemctl daemon-reload\nsystemctl start docker\nsystemctl enable docker<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\u610f\uff1a\u5982\u679cdocker\u542f\u52a8\u7684\u65f6\u5019\u5361\u6b7b\uff0c\u505c\u6b62\u4e5f\u4e0d\u884c\uff0c\u603b\u4e4b\u65e0\u6cd5\u542f\u52a8\uff0c\u53ef\u80fd\u662fselinux\u6ca1\u6709\u5173\u95ed\uff0ck8s1.24\u4e4b\u540e\u4e0d\u5728\u652f\u6301dockershim\u63d2\u4ef6\uff0c\u6b64\u65f6\u53ef\u4ee5\u901a\u8fc7\u66ff\u4ee3\u54c1Cri-dockerd\u5b9e\u73b0\u7ee7\u7eed\u4f7f\u7528docker<\/p>\n\n\n\n<p class=\"has-text-align-center has-medium-font-size wp-block-paragraph\"><strong>\u4e09\u3001\u90e8\u7f72nginx+Keepalived \u9ad8\u53ef\u7528\u8d1f\u8f7d\u5747\u8861\u5668<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u6ce8\uff1a\u672c\u4f8b\u5b50\u90e8\u7f72\u6240\u6709\u7ec4\u4ef6\u8fde\u63a5api-server\u65f6\u5019\u76f4\u63a5\u6307\u5b9a\u4e86vip\u5730\u5740\uff0c\u8fd8\u53ef\u4ee5\u5148\u6307\u5b9a\u4e00\u53f0\u673a\u5668\u7684api-server\u5730\u5740\u540e\uff0c\u6700\u540e\u7edf\u4e00\u5c06\u914d\u7f6e\u6587\u4ef6\u4e2d\u7684\u673a\u5668IP\u5730\u5740\u548c\u7aef\u53e3\u66ff\u6362\u4e3avip\u7684\u5730\u5740\u548c\u7aef\u53e3<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">1\u3001\u90e8\u7f72keepalived\uff0c\u53ef\u76f4\u63a5\u901a\u8fc7yum\u5b89\u88c5\u6216\u8005\u7f16\u8bd1\u5b89\u88c5\uff0c\u53c2\u8003\u94fe\u63a5\uff1a<a href=\"https:\/\/blog.ywdevops.cn\/index.php\/2020\/04\/02\/keepalived\/\" target=\"_blank\" rel=\"noreferrer noopener\">\u90e8\u7f72keepalived<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u7f16\u8f91\/etc\/keealived\/keepalived.conf\uff0c\u6dfb\u52a0\u5185\u5bb9\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>! Configuration File for keepalived\n\nglobal_defs {\n   notification_email {\n     acassen@firewall.loc\n     failover@firewall.loc\n     sysadmin@firewall.loc\n   }\n   notification_email_from Alexandre.Cassen@firewall.loc\n   smtp_server 192.168.200.1\n   smtp_connect_timeout 30\n   router_id LVS_DEVEL\n   vrrp_skip_check_adv_addr\n   vrrp_strict\n   vrrp_garp_interval 0\n   vrrp_gna_interval 0\n}\n\nvrrp_instance VI_1 {\n    state MASTER                    #\u6ce8\u610f\u5176\u4f59\u6539\u4e3aBACKUP\n    interface ens192\n    virtual_router_id 51\n    priority 100\n    advert_int 1\n    authentication {\n        auth_type PASS\n        auth_pass 1111\n    }\n    virtual_ipaddress {\n        192.168.15.253\/24            #VIP\n    }\n}\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">2\u3001\u7f16\u8bd1\u5b89\u88c5nginx\uff0c\u53ef\u53c2\u8003kubeadm\u90e8\u7f72\u7ae0\u8282\uff0c\u94fe\u63a5\u5730\u5740\uff1a<a href=\"https:\/\/blog.ywdevops.cn\/index.php\/2021\/07\/21\/k8s-3\/\" target=\"_blank\" rel=\"noreferrer noopener\">kubeadm\u90e8\u7f72k8s<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">3\u3001\u6267\u884c\u547d\u4ee4\u542f\u52a8keepalived\u548cnginx\uff0c\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>systemctl enable keepalived --now      #\u6dfb\u52a0\u5f00\u673a\u542f\u52a8\u5e76\u542f\u52a8\nsystemctl enable nginx --now<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-center has-medium-font-size wp-block-paragraph\"><strong>\u56db\u3001\u90e8\u7f72Master Node:<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">1\u3001 \u81ea\u7b7e\u8bc1\u4e66\u9881\u53d1\u673a\u6784\uff08CA\uff09 \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u8fdb\u5165\u76ee\u5f55\ncd ~\/TLS\/k8s \n#\u521b\u5efaca\u6587\u4ef6,\u6ce8\u610fjson\u7684\u6570\u636e\u683c\u5f0f\u8981\u5bf9\u9f50\ncat &gt; ca-config.json &lt;&lt;EOF\n{\n \"signing\": {\n     \"default\": {\n     \"expiry\": \"87600h\"\n     },\n     \"profiles\": {\n       \"kubernetes\": {\n         \"expiry\": \"87600h\",\n         \"usages\": &#91;\n            \"signing\",\n            \"key encipherment\",\n            \"server auth\",\n            \"client auth\"\n          ]\n      }\n    }\n  }\n}\nEOF<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; ca-csr.json &lt;&lt; EOF\n{\n \"CN\": \"kubernetes\",\n \"key\": {\n     \"algo\": \"rsa\",\n     \"size\": 2048\n  },\n \"names\": &#91;\n     {\n         \"C\": \"CN\",\n         \"L\": \"Beijing\",\n         \"ST\": \"Beijing\",\n         \"O\": \"k8s\",\n         \"OU\": \"System\"\n     }\n  ]\n}\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u751f\u6210\u8bc1\u4e66\uff0c \u4f1a\u751f\u6210 ca.pem \u548c ca-key.pem \u6587\u4ef6 \uff0c\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cfssl gencert -initca ca-csr.json | cfssljson -bare ca -<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"556\" height=\"154\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/19-1.png\" alt=\"\" class=\"wp-image-4177\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/19-1.png 556w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/19-1-300x83.png 300w\" sizes=\"auto, (max-width: 556px) 100vw, 556px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"555\" height=\"164\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/20-2.png\" alt=\"\" class=\"wp-image-4178\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/20-2.png 555w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/20-2-300x89.png 300w\" sizes=\"auto, (max-width: 555px) 100vw, 555px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">2\u3001 \u4f7f\u7528\u81ea\u7b7e CA \u7b7e\u53d1 kube-apiserver HTTPS \u8bc1\u4e66 \uff1a<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u521b\u5efa\u8bc1\u4e66\u7533\u8bf7\u6587\u4ef6\uff1a <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; server-csr.json &lt;&lt;EOF\n{\n \"CN\": \"kubernetes\",\n \"hosts\": &#91;\n     \"10.0.0.1\",\n     \"127.0.0.1\",\n     \"192.168.15.215\",\n     \"192.168.15.216\",\n     \"192.168.15.217\",\n     \"192.168.15.222\",\n     \"192.168.15.223\",\n     \"192.168.15.253\",\n     \"kubernetes\",\n     \"kubernetes.default\",\n     \"kubernetes.default.svc\",\n     \"kubernetes.default.svc.cluster\",\n     \"kubernetes.default.svc.cluster.local\"\n  ],\n \"key\": {\n     \"algo\": \"rsa\",\n     \"size\": 2048\n  },\n \"names\": &#91;\n     {\n         \"C\": \"CN\",\n         \"L\": \"BeiJing\",\n         \"ST\": \"BeiJing\",\n         \"O\": \"k8s\",\n         \"OU\": \"System\"\n     }\n  ]\n}\nEOF<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>10.0.0.1\uff1aserverIP \u7684\u9996\u4e2aIP\u5730\u5740<\/li>\n\n\n\n<li>192.168.15.253\uff1aVIP<\/li>\n\n\n\n<li>\u5176\u4f59IP\u4e3aapiserverIP\uff0c\u53ef\u9884\u7559\u591a\u4e2a<\/li>\n<\/ul>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\"> \u6ce8\uff1a\u4e0a\u8ff0\u6587\u4ef6 hosts \u5b57\u6bb5\u4e2d IP \u4e3a\u6240\u6709 Master\/LB\/VIP IP\uff0c\u4e00\u4e2a\u90fd\u4e0d\u80fd\u5c11\uff01\u4e3a\u4e86\u65b9 \u4fbf\u540e\u671f\u6269\u5bb9\u53ef\u4ee5\u591a\u5199\u51e0\u4e2a\u9884\u7559\u7684 IP\u3002 <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u751f\u6210\u8bc1\u4e66\uff0c\u4f1a\u751f\u6210server.pem\u548cserver-key.pem\u6587\u4ef6\uff0c\u5982\u4e0b\u547d\u4ee4\uff0c<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"552\" height=\"199\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/21-1.png\" alt=\"\" class=\"wp-image-4180\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/21-1.png 552w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/21-1-300x108.png 300w\" sizes=\"auto, (max-width: 552px) 100vw, 552px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">3\u3001\u4ecegithub\u4e0a\u4e0b\u8f7dkubeneter-server\u7684\u4e8c\u8fdb\u5236\u6587\u4ef6\uff0c\u4e0b\u8f7d\u5730\u5740\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>https:&#47;&#47;github.com\/kubernetes\/kubernetes\/blob\/master\/CHANGELOG\/CHANGELOG-1.20.md<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u89e3\u538b\u4e8c\u8fdb\u5236\u5305\u5e76\u62f7\u8d1d\u53ef\u6267\u884c\u6587\u4ef6\uff0c\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir -p \/opt\/kubernetes\/{bin,cfg,ssl,logs}       #\u521b\u5efa\u6587\u4ef6\u5939\ntar zxvf kubernetes-server-linux-amd64.tar.gz     #\u89e3\u538b\u7f29\ncd kubernetes\/server\/bin                          #\u62f7\u8d1d\u5305\ncp kube-apiserver kube-scheduler kube-controller-manager \/opt\/kubernetes\/bin\ncp kubectl \/usr\/bin\/\n#\u5c06kubectl\u6587\u4ef6\u590d\u5236\u5230\u5176\u4f59\u4e24\u4e2amaster\u8282\u70b9\nfor i in k8s-master02 k8s-master03; do scp kubectl ${i}:\/usr\/bin\/ ; done\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">4\u3001 <strong>\u90e8\u7f72 kube-apiserver \uff1a<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u521b\u5efa\u914d\u7f6e\u6587\u4ef6\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/opt\/kubernetes\/cfg\/kube-apiserver.conf &lt;&lt; EOF\nKUBE_APISERVER_OPTS=\"--logtostderr=false \\\\\n--v=2 \\\\\n--log-dir=\/opt\/kubernetes\/logs \\\\\n--etcd-servers=https:\/\/192.168.15.215:2379,https:\/\/192.168.15.216:2379,https:\/\/192.168.15.217:2379 \\\\\n--bind-address=192.168.15.215 \\\\\n--secure-port=6443 \\\\\n--advertise-address=192.168.15.215 \\\\\n--allow-privileged=true \\\\\n--service-cluster-ip-range=10.0.0.0\/24 \\\\\n--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \\\\\n--authorization-mode=RBAC,Node \\\\\n--enable-bootstrap-token-auth=true \\\\\n--token-auth-file=\/opt\/kubernetes\/cfg\/token.csv \\\\\n--service-node-port-range=30000-32767 \\\\\n--kubelet-client-certificate=\/opt\/kubernetes\/ssl\/server.pem \\\\\n--kubelet-client-key=\/opt\/kubernetes\/ssl\/server-key.pem \\\\\n--tls-cert-file=\/opt\/kubernetes\/ssl\/server.pem \\\\\n--tls-private-key-file=\/opt\/kubernetes\/ssl\/server-key.pem \\\\\n--client-ca-file=\/opt\/kubernetes\/ssl\/ca.pem \\\\\n--service-account-key-file=\/opt\/kubernetes\/ssl\/ca-key.pem \\\\\n--service-account-issuer=https:\/\/kubernetes.default.svc.cluster.local \\\\\n--service-account-signing-key-file=\/opt\/kubernetes\/ssl\/server-key.pem \\\\\n--etcd-cafile=\/opt\/etcd\/ssl\/etcd-ca.pem \\\\\n--etcd-certfile=\/opt\/etcd\/ssl\/server.pem \\\\\n--etcd-keyfile=\/opt\/etcd\/ssl\/server-key.pem \\\\\n--requestheader-client-ca-file=\/opt\/kubernetes\/ssl\/ca.pem \\\\\n--proxy-client-cert-file=\/opt\/kubernetes\/ssl\/server.pem \\\\\n--proxy-client-key-file=\/opt\/kubernetes\/ssl\/server-key.pem \\\\\n--requestheader-allowed-names=kubernetes \\\\\n--requestheader-extra-headers-prefix=X-Remote-Extra- \\\\\n--requestheader-group-headers=X-Remote-Group \\\\\n--requestheader-username-headers=X-Remote-User \\\\\n--enable-aggregator-routing=true \\\\\n--audit-log-maxage=30 \\\\\n--audit-log-maxbackup=3 \\\\\n--audit-log-maxsize=100 \\\\\n--audit-log-path=\/opt\/kubernetes\/logs\/k8s-audit.log\"\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u6ce8\uff1a\u4e0a\u9762\u4e24\u4e2a\\ \\ \u7b2c\u4e00\u4e2a\u662f\u8f6c\u4e49\u7b26\uff0c\u7b2c\u4e8c\u4e2a\u662f\u6362\u884c\u7b26\uff0c\u4f7f\u7528\u8f6c\u4e49\u7b26\u662f\u4e3a\u4e86\u4f7f\u7528 EOF \u4fdd\u7559\u6362\u884c\u7b26\u3002 <\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li> &#8211;logtostderr\uff1a\u542f\u7528\u65e5\u5fd7 <\/li>\n\n\n\n<li>&#8211;v : \u65e5\u5fd7\u7b49\u7ea7<\/li>\n\n\n\n<li> &#8211;log-dir\uff1a\u65e5\u5fd7\u76ee\u5f55 <\/li>\n\n\n\n<li> &#8211;etcd-servers\uff1aetcd \u96c6\u7fa4\u5730\u5740 <\/li>\n\n\n\n<li> &#8211;bind-address\uff1a\u76d1\u542c\u5730\u5740 \uff0c\u4e5f\u53ef\u4ee5\u51990.0.0.0<\/li>\n\n\n\n<li> &#8211;secure-port\uff1ahttps \u5b89\u5168\u7aef\u53e3 <\/li>\n\n\n\n<li> &#8211;advertise-address\uff1a\u96c6\u7fa4\u901a\u544a\u5730\u5740 <\/li>\n\n\n\n<li> &#8211;allow-privileged\uff1a\u542f\u7528\u6388\u6743 <\/li>\n\n\n\n<li> &#8211;service-cluster-ip-range\uff1aService \u865a\u62df IP \u5730\u5740\u6bb5 <\/li>\n\n\n\n<li> &#8211;enable-admission-plugins\uff1a\u51c6\u5165\u63a7\u5236\u6a21\u5757 <\/li>\n\n\n\n<li> &#8211;authorization-mode\uff1a\u8ba4\u8bc1\u6388\u6743\uff0c\u542f\u7528 RBAC \u6388\u6743\u548c\u8282\u70b9\u81ea\u7ba1\u7406 <\/li>\n\n\n\n<li> &#8211;enable-bootstrap-token-auth\uff1a\u542f\u7528 TLS bootstrap \u673a\u5236 <\/li>\n\n\n\n<li> &#8211;token-auth-file\uff1abootstrap token \u6587\u4ef6 <\/li>\n\n\n\n<li> &#8211;service-node-port-range\uff1aService nodeport \u7c7b\u578b\u9ed8\u8ba4\u5206\u914d\u7aef\u53e3\u8303\u56f4 <\/li>\n\n\n\n<li> &#8211;kubelet-client-xxx\uff1aapiserver \u8bbf\u95ee kubelet \u5ba2\u6237\u7aef\u8bc1\u4e66 <\/li>\n\n\n\n<li> &#8211;tls-xxx-file\uff1aapiserver https \u8bc1\u4e66 <\/li>\n\n\n\n<li> 1.20 \u7248\u672c\u5fc5\u987b\u52a0\u7684\u53c2\u6570\uff1a&#8211;service-account-issuer\uff0c&#8211;service-accountsigning-key-file <\/li>\n\n\n\n<li> &#8211;etcd-xxxfile\uff1a\u8fde\u63a5 Etcd \u96c6\u7fa4\u8bc1\u4e66 <\/li>\n\n\n\n<li> &#8211;audit-log-xxx\uff1a\u5ba1\u8ba1\u65e5\u5fd7 <\/li>\n\n\n\n<li> \u542f\u52a8\u805a\u5408\u5c42\u76f8\u5173\u914d\u7f6e\uff1a&#8211;requestheader-client-ca-file\uff0c&#8211;proxy-clientcert-file\uff0c&#8211;proxy-client-key-file\uff0c&#8211;requestheader-allowed-names\uff0c- -requestheader-extra-headers-prefix\uff0c&#8211;requestheader-group-headers\uff0c &#8211;requestheader-username-headers\uff0c&#8211;enable-aggregator-routing(\u6b64\u53c2\u6570\u8bbe\u7f6e\u4e3atrue\uff0cmetric-server\u624d\u53ef\u91c7\u96c6\u5230\u6570\u636e) <\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"> \u62f7\u8d1d\u521a\u624d\u751f\u6210\u7684\u8bc1\u4e66 \uff1a<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u628a\u521a\u624d\u751f\u6210\u7684\u8bc1\u4e66\u62f7\u8d1d\u5230\u914d\u7f6e\u6587\u4ef6\u4e2d\u7684\u8def\u5f84\uff0c \u6267\u884c\u547d\u4ee4\u5982\u4e0b:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cp ~\/TLS\/k8s\/ca*pem ~\/TLS\/k8s\/server*pem \/opt\/kubernetes\/ssl\/<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"> \u542f\u7528 TLS Bootstrapping \u673a\u5236 \uff1a<\/p>\n\n\n\n<p class=\"has-vivid-purple-color has-text-color wp-block-paragraph\">TLS Bootstraping\uff1aMaster apiserver \u542f\u7528 TLS \u8ba4\u8bc1\u540e\uff0cNode \u8282\u70b9 kubelet \u548c kube-proxy \u8981\u4e0e kube-apiserver \u8fdb\u884c\u901a\u4fe1\uff0c\u5fc5\u987b\u4f7f\u7528 CA \u7b7e\u53d1\u7684\u6709\u6548\u8bc1\u4e66\u624d\u53ef\u4ee5\uff0c \u5f53 Node \u8282\u70b9\u5f88\u591a\u65f6\uff0c\u8fd9\u79cd\u5ba2\u6237\u7aef\u8bc1\u4e66\u9881\u53d1\u9700\u8981\u5927\u91cf\u5de5\u4f5c\uff0c\u540c\u6837\u4e5f\u4f1a\u589e\u52a0\u96c6\u7fa4\u6269\u5c55 \u590d\u6742\u5ea6\u3002\u4e3a\u4e86\u7b80\u5316\u6d41\u7a0b\uff0cKubernetes \u5f15\u5165\u4e86 TLS bootstraping \u673a\u5236\u6765\u81ea\u52a8\u9881\u53d1\u5ba2 \u6237\u7aef\u8bc1\u4e66\uff0ckubelet \u4f1a\u4ee5\u4e00\u4e2a\u4f4e\u6743\u9650\u7528\u6237\u81ea\u52a8\u5411 apiserver \u7533\u8bf7\u8bc1\u4e66\uff0ckubelet \u7684 \u8bc1\u4e66\u7531 apiserver \u52a8\u6001\u7b7e\u7f72\u3002\u6240\u4ee5\u5f3a\u70c8\u5efa\u8bae\u5728 Node \u4e0a\u4f7f\u7528\u8fd9\u79cd\u65b9\u5f0f\uff0c\u76ee\u524d\u4e3b\u8981\u7528 \u4e8e kubelet\uff0ckube-proxy \u8fd8\u662f\u7531\u6211\u4eec\u7edf\u4e00\u9881\u53d1\u4e00\u4e2a\u8bc1\u4e66\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> TLS bootstraping \u5de5\u4f5c\u6d41\u7a0b\uff1a <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"540\" height=\"466\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/22.png\" alt=\"\" class=\"wp-image-4183\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/22.png 540w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/22-300x259.png 300w\" sizes=\"auto, (max-width: 540px) 100vw, 540px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"> \u521b\u5efa\u4e0a\u8ff0\u914d\u7f6e\u6587\u4ef6\u4e2d token \u6587\u4ef6\uff1a <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/opt\/kubernetes\/cfg\/token.csv &lt;&lt; EOF\n$(head -c 16 \/dev\/urandom | od -An -t x | tr -d ' '),kubelet-bootstrap,10001,\"system:kubelet-bootstrap\"\nEOF\n#\u683c\u5f0f\uff1atoken\uff0c\u7528\u6237\u540d\uff0cUID\uff0c\u7528\u6237\u7ec4\n#token \u4e5f\u53ef\u81ea\u884c\u751f\u6210\u66ff\u6362\uff1ahead -c 16 \/dev\/urandom | od -An -t x | tr -d ' '\n#kubelet-bootstrap\u7528\u6237\u5c06\u4f5c\u4e3akubelet\u9996\u6b21\u542f\u52a8\u7684\u9884\u8bbe\u7528\u6237\uff0c\u9ed8\u8ba4\u60c5\u51b5\u4e0b\u6ca1\u6709\u4efb\u4f55\u6743\u9650\uff0c\u9700\u8981\u7ed9\u5b83\u6388\u6743\u4f7f\u7528\u6b64token\u8bf7\u6c42apiserver\uff0c\u5305\u62ec\u521b\u5efaCSR\u8bf7\u6c42<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"> systemd \u7ba1\u7406 apiserver \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/usr\/lib\/systemd\/system\/kube-apiserver.service &lt;&lt; EOF\n&#91;Unit]\nDescription=Kubernetes API Server\nDocumentation=https:\/\/github.com\/kubernetes\/kubernetes\nAfter=etcd.service\nWants=etcd.service\n&#91;Service]\nEnvironmentFile=\/opt\/kubernetes\/cfg\/kube-apiserver.conf\nExecStart=\/opt\/kubernetes\/bin\/kube-apiserver \\$KUBE_APISERVER_OPTS\nRestart=on-failure\nLimitNOFILE=65536\n&#91;Install]\nWantedBy=multi-user.target\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u5c06master\u8282\u70b91\u7684\u6587\u4ef6\u62f7\u8d1d\u5230\u5bf9\u5e94\u7684master\u8282\u70b92\u548cmaster\u8282\u70b93\u4e0a\u9762\uff0c\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>for i in k8s-master02 k8s-master03;do scp -r \/opt\/kubernetes ${i}:\/opt ;done\nfor i in k8s-master02 k8s-master03;do scp \/lib\/systemd\/system\/kube-apiserver.service  ${i}:\/lib\/systemd\/system  ;done<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\uff1a\u62f7\u8d1d\u8fc7\u53bb\u4e4b\u540e\u8bb0\u5f97\u8981\u4fee\u6539\u8282\u70b92\u548c\u8282\u70b9\u7684kube-apiserver.conf\u6587\u4ef6\u4e2dIP\u5730\u5740\u4e3a\u672c\u673a\u7684IP\u5730\u5740\uff0c\u5426\u5219apiserver\u542f\u52a8\u4f1a\u5931\u8d25<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">  \u542f\u52a8\u5e76\u8bbe\u7f6e\u5f00\u673a\u542f\u52a8  <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>systemctl daemon-reload\nsystemctl enable kube-apiserver --now<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"552\" height=\"216\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/23.png\" alt=\"\" class=\"wp-image-4189\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/23.png 552w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/23-300x117.png 300w\" sizes=\"auto, (max-width: 552px) 100vw, 552px\" \/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code>1\u3001\u542f\u52a8\u65e5\u5fd7\u8f93\u51fa\uff1akube-apiserver&#91;7509]: E0715 10:38:57.686241    7509 instance.go:392] Could not construct pre-rendered responses for ServiceAccountIss..., got: api\nHint: Some lines were ellipsized, use -l to show in full.\n\u89e3\u51b3\uff1a\u67e5\u770bkube-apiserver.conf\u4e2d\u7684server-account-issuer\u914d\u7f6e\uff0c\u4fee\u6539\u4e3ahttps:\/\/kubernetes.default.svc.cluster.local\n2\u3001Unable to remove old endpoints from kubernetes service: StorageError: key not found, Code: 1, Key: \/registry\/masterleases\/10.88.15.215, ResourceVersion: 0, AdditionalErrorMsg:  \n\u89e3\u51b3\uff1a\u6b64\u9519\u8bef\u65e0\u5f71\u54cd\n3\u3001etcd\u63d0\u793a\uff1aleader failed to send out heartbeat on time; took too long, leader is overloaded likely from slow disk\uff0c\n\u89e3\u51b3\uff1a\u6b64\u5185\u5bb9\u5e76\u975e\u9519\u8bef\uff0c\u53ea\u662f\u63d0\u793a\u6570\u636e\u5199\u5165etcd\u65f6\u95f4\u957f\uff0c\u4e0d\u5f71\u54cdapiserver\u4f7f\u7528\uff0c\u5c06etcd\u78c1\u76d8\u6362\u6210ssd\u5373\u53ef\n<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\uff1a3\u4e2amaster\u8282\u70b9\u90fd\u90e8\u7f72\u4e86apiserver+controller-manager+scheduler\uff0ckube-controller-manager\u3001kube-scheduler\u662f\u8fde\u63a5\u672c\u673a\u7684kube-apiserver\uff0c\u53ef\u4ee5\u628a\u5b83\u4eec\u770b\u505a\u4e00\u4e2a\u6574\u4f53\uff0c\u53ea\u8981\u6709\u4e00\u4e2a\u8282\u70b9\u7684apiserver\u5b9e\u4f8b\u6b63\u5e38\uff0c\u5c31\u53ef\u4ee5\u4fdd\u8bc1\u96c6\u7fa4apiserver\u53ef\u7528\uff0c\u6700\u5916\u5c42\u901a\u8fc7haproxy\/nginx\u505a\u8d1f\u8f7d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>5\u3001 \u90e8\u7f72 kube-controller-manager(master1\u8282\u70b9\u6267\u884c)  :<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u521b\u5efa\u914d\u7f6e\u6587\u4ef6 :    \uff08\u4e09\u53f0master\u673a\u5668\u90fd\u8981\u6267\u884c\uff0c\u53efcopy\u8fc7\u53bb\uff09<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/opt\/kubernetes\/cfg\/kube-controller-manager.conf &lt;&lt; EOF\nKUBE_CONTROLLER_MANAGER_OPTS=\"--logtostderr=false \\\\\n--v=2 \\\\\n--log-dir=\/opt\/kubernetes\/logs \\\\\n--leader-elect=true \\\\\n--kubeconfig=\/opt\/kubernetes\/cfg\/kube-controller-manager.kubeconfig \\\\\n--bind-address=127.0.0.1 \\\\\n--allocate-node-cidrs=true \\\\\n--cluster-cidr=10.244.0.0\/16 \\\\\n--service-cluster-ip-range=10.0.0.0\/24 \\\\\n--cluster-signing-cert-file=\/opt\/kubernetes\/ssl\/ca.pem \\\\\n--cluster-signing-key-file=\/opt\/kubernetes\/ssl\/ca-key.pem \\\\\n--root-ca-file=\/opt\/kubernetes\/ssl\/ca.pem \\\\\n--service-account-private-key-file=\/opt\/kubernetes\/ssl\/ca-key.pem \\\\\n--cluster-signing-duration=87600h0m0s\"\nEOF<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li> &#8211;kubeconfig\uff1a\u8fde\u63a5 apiserver \u914d\u7f6e\u6587\u4ef6 <\/li>\n\n\n\n<li> &#8211;leader-elect\uff1a\u5f53\u8be5\u7ec4\u4ef6\u542f\u52a8\u591a\u4e2a\u65f6\uff0c\u81ea\u52a8\u9009\u4e3e\uff08HA\uff09 <\/li>\n\n\n\n<li> &#8211;cluster-signing-cert-file\/&#8211;cluster-signing-key-file\uff1a\u81ea\u52a8\u4e3a kubelet \u9881\u53d1\u8bc1\u4e66\u7684 CA\uff0c\u4e0e apiserver \u4fdd\u6301\u4e00\u81f4 <\/li>\n\n\n\n<li>&#8211;cluster-cidr\uff1a pod\u7f51\u6bb5\u5730\u5740<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u5c06\u914d\u7f6e\u6587\u4ef6copy\u5230\u5176\u4f59\u673a\u5668\u4e0a\nfor i in k8s-master02 k8s-master03 ;do scp \/opt\/kubernetes\/cfg\/kube-controller-manager.conf ${i}:\/opt\/kubernetes\/cfg ;done\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"> \u751f\u6210 kube-controller-manager \u8bc1\u4e66\uff1a (<strong>\u53ef\u5c06\u8bc1\u4e66\u62f7\u8d1d\u5230\u5176\u4f59\u8282\u70b9<\/strong>)<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># \u5207\u6362\u5de5\u4f5c\u76ee\u5f55\ncd ~\/TLS\/k8s\n# \u521b\u5efa\u8bc1\u4e66\u8bf7\u6c42\u6587\u4ef6\ncat &gt; kube-controller-manager-csr.json &lt;&lt; EOF\n{\n    \"CN\": \"system:kube-controller-manager\",\n    \"hosts\": &#91;],\n    \"key\": {\n        \"algo\": \"rsa\",\n        \"size\": 2048\n    },\n    \"names\": &#91;\n        {\n            \"C\": \"CN\",\n            \"ST\": \"BeiJing\",\n            \"L\": \"BeiJing\",\n            \"O\": \"system:masters\",\n            \"OU\": \"System\"\n        }\n    ]\n}\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"> \u751f\u6210\u8bc1\u4e66\uff0c\u4f1a\u751f\u6210\u4e24\u4e2apem\u6587\u4ef6 : <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"551\" height=\"171\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/1-9.png\" alt=\"\" class=\"wp-image-4193\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/1-9.png 551w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/1-9-300x93.png 300w\" sizes=\"auto, (max-width: 551px) 100vw, 551px\" \/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u5c06\u751f\u6210\u7684\u8bc1\u4e66\u6587\u4ef6\u590d\u5236\u5230\u672c\u673a\u7684\/opt\/kubernetes\/ssl\ncp kube-controller-manager.pem kube-controller-manager-key.pem \/opt\/kubernetes\/ssl\/\n#\u5c06\u8bc1\u4e66\u6587\u4ef6\u590d\u5236\u5230\u5176\u4f59master\u8282\u70b9\nfor i in k8s-master02 k8s-master03;do scp \/opt\/kubernetes\/ssl\/{kube-controller-manager.pem, kube-controller-manager-key.pem} ${i}:\/opt\/kubernetes\/ssl\/ ;done\n\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u751f\u6210 kubeconfig \u6587\u4ef6\uff0c\u7528\u4e8e\u8fde\u63a5apiserver\uff1a <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>KUBE_CONFIG=\"\/opt\/kubernetes\/cfg\/kube-controller-manager.kubeconfig\"\nKUBE_APISERVER=\"https:\/\/192.168.15.253:16443\"\n#\u6ce8\u610f\u8981\u5728\u8bc1\u4e66\u6240\u5728\u8def\u5f84\u6267\u884c,\u5426\u5219\u63d0\u793a\u627e\u4e0d\u5230\u8bc1\u4e66\u6587\u4ef6\nkubectl config set-cluster kubernetes \\\n --certificate-authority=\/opt\/kubernetes\/ssl\/ca.pem \\\n --embed-certs=true \\\n --server=${KUBE_APISERVER} \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config set-credentials kube-controller-manager \\\n --client-certificate=.\/kube-controller-manager.pem \\\n --client-key=.\/kube-controller-manager-key.pem \\\n --embed-certs=true \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config set-context default \\\n --cluster=kubernetes \\\n --user=kube-controller-manager \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config use-context default --kubeconfig=${KUBE_CONFIG}\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"> systemd \u7ba1\u7406 controller-manager \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/usr\/lib\/systemd\/system\/kube-controller-manager.service &lt;&lt; EOF\n&#91;Unit]\nDescription=Kubernetes Controller Manager\nDocumentation=https:\/\/github.com\/kubernetes\/kubernetes\n&#91;Service]\nEnvironmentFile=\/opt\/kubernetes\/cfg\/kube-controller-manager.conf\nExecStart=\/opt\/kubernetes\/bin\/kube-controller-manager \\$KUBE_CONTROLLER_MANAGER_OPTS\nRestart=on-failure\n&#91;Install]\nWantedBy=multi-user.target\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u5c06\u670d\u52a1\u542f\u52a8\u914d\u7f6e\u6587\u4ef6\u548ckubeconfig\u6587\u4ef6copy\u5230\u5176\u4ed6master\u8282\u70b9\u673a\u5668\u4e0a\uff0c\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>for i in k8s-master02 k8s-master03;do scp \/lib\/systemd\/system\/kube-controller-manager.service ${i}:\/lib\/systemd\/system;done\nfor i in k8s-master02 k8s-master03;do scp \/opt\/kubernetes\/cfg\/kube-controller-manager.kubeconfig ${i}:\/opt\/kubernetes\/cfg\/;done<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"> \u542f\u52a8\u5e76\u8bbe\u7f6e\u5f00\u673a\u542f\u52a8:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>systemctl daemon-reload\nsystemctl enable kube-controller-manager --now<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u542f\u52a8\u540e\u67e5\u770b\u65e5\u5fd7\u662f\u5426\u6709\u62a5\u9519\uff0c\u65e0\u62a5\u9519\u5373\u8868\u793a\u6ca1\u95ee\u9898<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u5982\u4e0b\u9519\u8bef\u53ef\u5ffd\u7565\nFailed to start service controller: WARNING: no cloud provider provided, services of type LoadBalancer will fail<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>6\u3001 \u90e8\u7f72 kube-scheduler  \uff1a<\/strong>(\u4e09\u53f0master\u90fd\u8981\u90e8\u7f72)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u521b\u5efa\u914d\u7f6e\u6587\u4ef6 \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/opt\/kubernetes\/cfg\/kube-scheduler.conf &lt;&lt; EOF\nKUBE_SCHEDULER_OPTS=\"--logtostderr=false \\\\\n--v=2 \\\\\n--log-dir=\/opt\/kubernetes\/logs \\\\\n--leader-elect \\\\\n--kubeconfig=\/opt\/kubernetes\/cfg\/kube-scheduler.kubeconfig \\\\\n--bind-address=127.0.0.1\"\nEOF<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li> &#8211;kubeconfig\uff1a\u8fde\u63a5 apiserver \u914d\u7f6e\u6587\u4ef6 <\/li>\n\n\n\n<li> &#8211;leader-elect\uff1a\u5f53\u8be5\u7ec4\u4ef6\u542f\u52a8\u591a\u4e2a\u65f6\uff0c\u81ea\u52a8\u9009\u4e3e\uff08HA\uff09 <\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u5c06\u914d\u7f6e\u6587\u4ef6\u53d1\u9001\u5230\u5176\u4f59master\u8282\u70b9\nfor i in k8s-master02 k8s-master03 ;do scp \/opt\/kubernetes\/cfg\/kube-scheduler.conf ${i}:\/opt\/kubernetes\/cfg\/ ; done<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">  \u751f\u6210 kube-scheduler \u8bc1\u4e66\u8bf7\u6c42\u6587\u4ef6CSR\uff1a <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># \u5207\u6362\u5de5\u4f5c\u76ee\u5f55\ncd ~\/TLS\/k8s\n#\u521b\u5efa\u8bc1\u4e66\u8bf7\u6c42\u6587\u4ef6\ncat &gt; kube-scheduler-csr.json &lt;&lt; EOF\n{\n \"CN\": \"system:kube-scheduler\",\n \"hosts\": &#91;],\n \"key\": {\n     \"algo\": \"rsa\",\n     \"size\": 2048\n  },\n \"names\": &#91;\n   {\n      \"C\": \"CN\",\n      \"L\": \"BeiJing\",\n      \"ST\": \"BeiJing\",\n      \"O\": \"system:masters\",\n      \"OU\": \"System\"\n   }\n  ]\n}\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u751f\u6210\u8bc1\u4e66\uff0c\u540d\u79f0\u4e3akube-scheduler-key.pem\u548ckube-scheduler.pem\uff0c\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"557\" height=\"160\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/2-5.png\" alt=\"\" class=\"wp-image-4202\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/2-5.png 557w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/2-5-300x86.png 300w\" sizes=\"auto, (max-width: 557px) 100vw, 557px\" \/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u5148\u5c06\u8bc1\u4e66\u6587\u4ef6copy\u5230\u672c\u673a\u7684\/opt\/kubernetes\/ssl\ncp kube-scheduler-key.pem kube-scheduler.pem  \/opt\/kubernetes\/ssl\n#\u518d\u5c06\u8bc1\u4e66copy\u5bfc\u5176\u4f59\u7684master\u8282\u70b9\u4e0a\nfor i in k8s-master02 k8s-master03;do scp \/opt\/kubernetes\/ssl\/{kube-scheduler-key.pem,kube-scheduler.pem} ${i}:\/opt\/kubernetes\/ssl\/ ;done\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u751f\u6210 kubeconfig \u6587\u4ef6\uff0c\u7528\u4e8e\u8fde\u63a5apiserver\uff1a <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>KUBE_CONFIG=\"\/opt\/kubernetes\/cfg\/kube-scheduler.kubeconfig\"\nKUBE_APISERVER=\"https:\/\/192.168.15.253:16443\"\n#\u4e0b\u9762\u6b65\u9aa4\u9700\u8981\u8fdb\u5165\/opt\/kubernetes\/ssl\u76ee\u5f55\u6267\u884c\nkubectl config set-cluster kubernetes \\\n --certificate-authority=\/opt\/kubernetes\/ssl\/ca.pem \\\n --embed-certs=true \\\n --server=${KUBE_APISERVER} \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config set-credentials kube-scheduler \\\n --client-certificate=.\/kube-scheduler.pem \\\n --client-key=.\/kube-scheduler-key.pem \\\n --embed-certs=true \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config set-context default \\\n --cluster=kubernetes \\\n --user=kube-scheduler \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config use-context default --kubeconfig=${KUBE_CONFIG}<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"> systemd \u7ba1\u7406 scheduler \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/usr\/lib\/systemd\/system\/kube-scheduler.service &lt;&lt; EOF\n&#91;Unit]\nDescription=Kubernetes Scheduler\nDocumentation=https:\/\/github.com\/kubernetes\/kubernetes\n&#91;Service]\nEnvironmentFile=\/opt\/kubernetes\/cfg\/kube-scheduler.conf\nExecStart=\/opt\/kubernetes\/bin\/kube-scheduler \\$KUBE_SCHEDULER_OPTS\nRestart=on-failure\n&#91;Install]\nWantedBy=multi-user.target\nEOF<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u670d\u52a1\u914d\u7f6e\u6587\u4ef6\u548ckubeconfig\u6587\u4ef6copy\u5230\u5176\u4f59\u4e24\u4e2amaster\u8282\u70b9\u4e0a\nfor i in k8s-master02 k8s-master03;do scp \/lib\/systemd\/system\/kube-scheduler.service ${i}:\/lib\/systemd\/system ;done\nfor i in k8s-master02 k8s-master03;do scp \/opt\/kubernetes\/cfg\/kube-scheduler.kubeconfig ${i}:\/opt\/kubernetes\/cfg\/ ;done\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"> \u542f\u52a8\u5e76\u8bbe\u7f6e\u5f00\u673a\u542f\u52a8 \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>systemctl daemon-reload\nsystemctl start kube-scheduler\nsystemctl enable kube-scheduler<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>7\u3001 \u67e5\u770b\u96c6\u7fa4\u72b6\u6001 \uff1a<\/strong><\/p>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\u610f\uff1a\u6b64\u6b65\u9aa4\u53ef\u4ee5\u5355\u72ec\u90e8\u7f72\u96c6\u7fa4\u4e4b\u5916\u7684\u673a\u5668\u4e0a<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u751f\u6210 kubectl \u8fde\u63a5\u96c6\u7fa4\u7684\u8bc1\u4e66\uff1a <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># \u5207\u6362\u5de5\u4f5c\u76ee\u5f55\ncd ~\/TLS\/k8s\ncat &gt;  admin-csr.json &lt;&lt; EOF\n{\n \"CN\": \"admin\",\n \"hosts\": &#91;],\n \"key\": {\n     \"algo\": \"rsa\",\n     \"size\": 2048\n },\n \"names\": &#91;\n   {\n     \"C\": \"CN\",\n     \"L\": \"BeiJing\",\n     \"ST\": \"BeiJing\",\n     \"O\": \"system:masters\",\n     \"OU\": \"System\"\n   }\n ]\n}\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u751f\u6210\u8bc1\u4e66\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"551\" height=\"162\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/5-1.png\" alt=\"\" class=\"wp-image-4217\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/5-1.png 551w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/5-1-300x88.png 300w\" sizes=\"auto, (max-width: 551px) 100vw, 551px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u590d\u5236\u8bc1\u4e66\u6587\u4ef6\u5230\/opt\/kubernetes\/ssl\u4e2d<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code> cp admin-key.pem admin.pem \/opt\/kubernetes\/ssl\/<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"> \u751f\u6210 kubeconfig \u6587\u4ef6\uff1a <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u521b\u5efa\u6587\u4ef6\u5939.kube\nmkdir \/root\/.kube\nKUBE_CONFIG=\"\/root\/.kube\/config\"\nKUBE_APISERVER=\"https:\/\/192.168.15.253:16443\"  #\u6ce8\u610f\uff1a\u6b64\u5730\u5740\u4e3aVIP\u5730\u5740\uff0c\u9632\u6b62\u5355\u70b9apiserver\u6302\u6389\ncd \/opt\/kubernetes\/ssl\nkubectl config set-cluster kubernetes \\\n --certificate-authority=\/opt\/kubernetes\/ssl\/ca.pem \\\n --embed-certs=true \\\n --server=${KUBE_APISERVER} \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config set-credentials cluster-admin \\\n --client-certificate=.\/admin.pem \\\n --client-key=.\/admin-key.pem \\\n --embed-certs=true \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config set-context default \\\n --cluster=kubernetes \\\n --user=cluster-admin \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config use-context default --kubeconfig=${KUBE_CONFIG}<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u901a\u8fc7 kubectl \u5de5\u5177\u67e5\u770b\u5f53\u524d\u96c6\u7fa4\u7ec4\u4ef6\u72b6\u6001\uff0c\u8f93\u51fa\u5982\u4e0b\u8868\u793a\u5404\u7ec4\u4ef6\u6b63\u5e38\uff0c\u5982\u56fe\uff1a <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"196\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/6-1.png\" alt=\"\" class=\"wp-image-4221\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/6-1.png 624w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/6-1-300x94.png 300w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/06\/6-1-600x188.png 600w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\u610f\uff1a\u6b64\u6b65\u9aa4\u53ef\u4ee5\u4e0d\u90e8\u7f72\u5728\u4efb\u4f55\u4e00\u4e2a\u8282\u70b9\u4e0a\uff0c\u53ef\u4ee5\u5355\u72ec\u90e8\u7f72\u4e00\u4e2a\u673a\u5668\u7528\u6765\u7ba1\u7406\u96c6\u7fa4\uff0c\u5728\u751f\u6210 kubeconfig \u6587\u4ef6\u65f6kube-apiserver \u5730\u5740\u8981\u5199vip\u5730\u5740(\u9632\u6b62\u5176\u4e2d\u67d0\u4e2a\u8282\u70b9Down) <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u914d\u7f6ekubectl\u547d\u4ee4\u8865\u5168\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>yum install -y bash-completion\nsource \/usr\/share\/bash-completion\/bash_completion\nsource &lt;(kubectl completion bash)\nkubectl completion bash &gt; ~\/.kube\/completion.bash.inc\nsource '\/root\/.kube\/completion.bash.inc'\nsource $HOME\/.bash_profile<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">8\u3001<strong>\u6388\u6743kubelet-bootstrap\u7528\u6237\u5141\u8bb8\u8bf7\u6c42\u8bc1\u4e66<\/strong>:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap\n#\u5728\u9ed8\u8ba4\u60c5\u51b5\u4e0b\uff0ckubelet \u901a\u8fc7 bootstrap.kubeconfig \u4e2d\u7684\u9884\u8bbe\u7528\u6237 Token \u58f0\u660e\u4e86\u81ea\u5df1\u7684\u8eab\u4efd\uff0c\u7136\u540e\u521b\u5efa CSR \u8bf7\u6c42\uff0c\u4f46\u662f\u9ed8\u8ba4\u60c5\u51b5\u4e0b\u6b64\u7528\u6237\u6ca1\u4efb\u4f55\u6743\u9650\u7684\uff0c\u5305\u62ec\u521b\u5efa CSR \u8bf7\u6c42\uff1b\u6240\u4ee5\u9700\u8981\u5982\u4e0b\u547d\u4ee4\u521b\u5efa\u4e00\u4e2a ClusterRoleBinding\uff0c\u5c06\u9884\u8bbe\u7528\u6237 kubelet-bootstrap \u4e0e\u5185\u7f6e\u7684 ClusterRole system:node-bootstrapper \u7ed1\u5b9a\u5230\u4e00\u8d77\uff0c\u4f7f\u5176\u80fd\u591f\u53d1\u8d77 CSR \u8bf7\u6c42<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-center has-medium-font-size wp-block-paragraph\"><strong>\u4e94\u3001 \u90e8\u7f72Node<\/strong>\uff1a<\/p>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background has-normal-font-size wp-block-paragraph\">\u5728master1\u8282\u70b9\u6267\u884c\uff0c\u56e0\u4e3amaster\u8282\u70b9\u4e5f\u90e8\u7f72kubelet\u548cproxy<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u89e3\u538bkubernetes-server-linux-amd64.tar.gz\ntar -zxvf kubernetes-server-linux-amd64.tar.gz\ncd kubernetes\/server\/bin\n#copy\u5230\u672c\u5730\u7684\/opt\/kubernetes\/bin\u8def\u5f84\u4e0b\ncp kubelet kube-proxy \/opt\/kubernetes\/bin\n\u5728node\u8282\u70b9\u521b\u5efa\u8def\u5f84\u548c\u6587\u4ef6\u5939\nfor i in k8s-node01 k8s-node02;do ssh ${i} mkdir -p \/opt\/kubernetes\/{cfg,logs,bin,ssl} ;done\n#\u5c06kubelet\u3001kube-proxy\u62f7\u8d1d\u5230\u5176\u4f59\u7684master\u8282\u70b9\u548cnode\u8282\u70b9\nfor i in k8s-master02 k8s-master03 k8s-node01 k8s-node02;do scp \/opt\/kubernetes\/bin\/{kubelet, kube-proxy} ${i}:\/opt\/kubernetes\/bin\/ ;done\n#\u5c06ca.pem\u3001ca-key.pem\u6587\u4ef6\u62f7\u8d1d\u5230\u5176\u4f59\u7684node\u8282\u70b9\u4e0a\nfor i in k8s-node01 k8s-node02;do scp \/opt\/kubernetes\/ssl\/{ca.pem,ca-key.pem} ${i}:\/opt\/kubernetes\/ssl\/ ;done<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\u610f\uff1akubelet\u542f\u52a8\u7684\u65f6\u5019\u9700\u8981ca.pem\u548cca-key.pem\uff0c\u6ca1\u6709\u8fd9\u4e24\u4e2a\u6587\u4ef6\u542f\u52a8\u4f1a\u62a5\u9519<\/p>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\"><strong>\u90e8\u7f72 kubelet (\u53ea\u5728master1\u6267\u884c):<\/strong> <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">1\u3001  \u521b\u5efa\u914d\u7f6e\u6587\u4ef6 \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u5bfc\u5165pause\u955c\u50cf\uff0c\u6267\u884c\u547d\u4ee4docker load -i pause-amd64:3.3\u6216\u8005\u4ece\u963f\u91cc\u4e91\u4e0b\u8f7d\uff1a\ndocker pull registry.aliyuncs.com\/google_containers\/pause-amd64:3.3\n#\u521b\u5efa\u6587\u4ef6\ncat &gt; \/opt\/kubernetes\/cfg\/kubelet.conf &lt;&lt; EOF\nKUBELET_OPTS=\"--logtostderr=false \\\\\n--v=2 \\\\\n--log-dir=\/opt\/kubernetes\/logs \\\\\n--hostname-override=k8s-master1 \\\\\n--network-plugin=cni \\\\\n--kubeconfig=\/opt\/kubernetes\/cfg\/kubelet.kubeconfig \\\\\n--bootstrap-kubeconfig=\/opt\/kubernetes\/cfg\/bootstrap.kubeconfig \\\\\n--config=\/opt\/kubernetes\/cfg\/kubelet-config.yml \\\\\n--cert-dir=\/opt\/kubernetes\/ssl \\\\\n--pod-infra-container-image=registry.aliyuncs.com\/google_containers\/pause-amd64:3.3\"\nEOF<\/code><\/pre>\n\n\n\n<ul class=\"has-luminous-vivid-amber-background-color has-background wp-block-list\">\n<li>\u6ce8\u610f\u66f4\u6539hostname-override\u540d\u5b57<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/opt\/kubernetes\/cfg\/kubelet-config.yml &lt;&lt; EOF\nkind: KubeletConfiguration\napiVersion: kubelet.config.k8s.io\/v1beta1\naddress: 0.0.0.0\nport: 10250\nreadOnlyPort: 10255\ncgroupDriver: systemd      #\u6ce8\u610f\uff0c\u9a71\u52a8\u8981\u548c\/etc\/docker\/daemon.json\u4e2d\u4e00\u81f4\nclusterDNS:\n- 10.0.0.10\nclusterDomain: cluster.local\nfailSwapOn: false\nauthentication:\n anonymous:\n   enabled: false\n webhook:\n   cacheTTL: 2m0s\n   enabled: true\n x509:\n   clientCAFile: \/opt\/kubernetes\/ssl\/ca.pem\nauthorization:\n mode: Webhook\n webhook:\n   cacheAuthorizedTTL: 5m0s\n   cacheUnauthorizedTTL: 30s\nevictionHard:\n imagefs.available: 15%\n memory.available: 100Mi\n nodefs.available: 10%\n nodefs.inodesFree: 5%\nmaxOpenFiles: 1000000\nmaxPods: 110\nEOF<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>clusterDNS\uff1a\u96c6\u7fa4DNS\u5730\u5740\uff0c\u4e00\u822c\u4e3aserver ip\u7684\u7b2c\u5341\u4e2aIP\u5730\u5740<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">3\u3001 \u751f\u6210 kubelet \u521d\u6b21\u52a0\u5165\u96c6\u7fa4\u5f15\u5bfc kubeconfig \u6587\u4ef6 \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>KUBE_CONFIG=\"\/opt\/kubernetes\/cfg\/bootstrap.kubeconfig\"\n\nKUBE_APISERVER=\"https:\/\/192.168.15.253:16443\"   #\u6ce8\u610f\uff1a\u8fde\u63a5\u7684\u662fVIP\n\nTOKEN=$(awk -F \",\" '{print $1}' \/opt\/kubernetes\/cfg\/token.csv)       # \u4e0e token.csv \u91cc\u4fdd\u6301\u4e00\u81f4\n\n# \u751f\u6210 kubelet bootstrap kubeconfig \u914d\u7f6e\u6587\u4ef6\uff0c\u6ce8\u610f\uff0c\u9700\u8981\u5148\u5c06master\u8282\u70b9\u7684\/opt\/kubernetes\/ssl\u4e0b\u7684\u8bc1\u4e66\u6587\u4ef6\u62f7\u8d1d\u5230worker\u8282\u70b9\u4e2d\n\nkubectl config set-cluster kubernetes \\\n --certificate-authority=\/opt\/kubernetes\/ssl\/ca.pem \\\n --embed-certs=true \\\n --server=${KUBE_APISERVER} \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config set-credentials \"kubelet-bootstrap\" \\\n --token=${TOKEN} \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config set-context default \\\n --cluster=kubernetes \\\n --user=\"kubelet-bootstrap\" \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config use-context default --kubeconfig=${KUBE_CONFIG}<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">4\u3001 systemd \u7ba1\u7406 kubelet \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/usr\/lib\/systemd\/system\/kubelet.service &lt;&lt; EOF\n&#91;Unit]\nDescription=Kubernetes Kubelet\nAfter=docker.service\n&#91;Service]\nEnvironmentFile=\/opt\/kubernetes\/cfg\/kubelet.conf\nExecStart=\/opt\/kubernetes\/bin\/kubelet \\$KUBELET_OPTS\nRestart=on-failure\nLimitNOFILE=65536\n&#91;Install]\nWantedBy=multi-user.target\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">5\u3001\u5c06master1\u8282\u70b9\u7684kubelet\u76f8\u5173\u7ec4\u4ef6\u53d1\u9001\u5230\u5176\u4ed6master\u548cworker\u8282\u70b9\uff0c\u4e3e\u4f8b\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>for i in  k8s-master02 k8s-master03 k8s-node01 k8s-node02;do scp \/opt\/kubernetes\/cfg\/{kubelet.conf,kubelet-config.yml,bootstrap.kubeconfig} ${i}:\/opt\/kubernetes\/cfg\/ ;done\nfor i in  k8s-master02 k8s-master03 k8s-node01 k8s-node02;do scp \/lib\/systemd\/system\/kubelet.service ${i}:\/lib\/systemd\/system ;done\n<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\u610f: \u76ee\u6807\u673a\u5668\u4e0a\u8bb0\u5f97\u4fee\u6539kubelet.conf\u4e2d\u7684hostname-override\u7684<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">6\u3001 \u542f\u52a8\u5e76\u8bbe\u7f6e\u5f00\u673a\u542f\u52a8(\u6240\u6709\u8282\u70b9)  \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>systemctl daemon-reload\nsystemctl start kubelet\nsystemctl enable kubelet<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\"><strong>\u6279\u51c6\u8282\u70b9\u52a0\u5165\u96c6\u7fa4<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">1\u3001\u624b\u52a8\u65b9\u5f0f\u6279\u51c6\uff1a<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5728 kubelet \u9996\u6b21\u542f\u52a8\u540e\uff0c\u5982\u679c\u7528\u6237 Token \u6ca1\u95ee\u9898\uff0c\u5e76\u4e14 RBAC \u4e5f\u505a\u4e86\u76f8\u5e94\u7684\u8bbe\u7f6e\uff0c\u90a3\u4e48\u6b64\u65f6\u5728\u96c6\u7fa4\u5185\u5e94\u8be5\u80fd\u770b\u5230 kubelet \u53d1\u8d77\u7684 CSR \u8bf7\u6c42\uff0c\u5982\u56fe\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"925\" height=\"179\" src=\"https:\/\/www.gycysygg.icu\/wp-content\/uploads\/2021\/12\/12.png\" alt=\"\" class=\"wp-image-5889\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/12\/12.png 925w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/12\/12-300x58.png 300w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/12\/12-768x149.png 768w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/12\/12-600x116.png 600w\" sizes=\"auto, (max-width: 925px) 100vw, 925px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u5728master\u8282\u70b9\u6267\u884c<\/strong>\uff0c\u4e5f\u53ef\u5728\u5355\u72ec\u7684\u7528\u4e8e\u8fde\u63a5\u96c6\u7fa4\u7684\u673a\u5668\u4e0a\u6267\u884c\u5982\u4e0b\u547d\u4ee4<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># \u6279\u51c6\u52a0\u5165\u96c6\u7fa4\nkubectl certificate approve  node-csr-8CrwSP6Rpx7sxwa3T_DNl5AEyLEWuowxbruOasn6nYA<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u67e5\u770b\u8282\u70b9\u4fe1\u606f\uff0c\u5982\u56fe\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"576\" height=\"130\" src=\"https:\/\/www.gycysygg.icu\/wp-content\/uploads\/2021\/12\/13.png\" alt=\"\" class=\"wp-image-5890\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/12\/13.png 576w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/12\/13-300x68.png 300w\" sizes=\"auto, (max-width: 576px) 100vw, 576px\" \/><\/figure>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\"> \u6ce8\uff1a\u7531\u4e8e\u7f51\u7edc\u63d2\u4ef6\u8fd8\u6ca1\u6709\u90e8\u7f72\uff0c\u8282\u70b9\u4f1a\u6ca1\u6709\u51c6\u5907\u5c31\u7eea NotReady <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u6ce8\u610f\uff1a\u5982\u679c\u5220\u9664\u4e86\u8282\u70b9\uff0c\u9700\u8981\u5220\u9664\u4e4b\u524d\u7684csr\u8bf7\u6c42\uff0c\u627e\u5230\u8282\u70b9\u7684\/opt\/kubernetes\/ssl\u4e0e\/opt\/kubernetes\/cfg\u76ee\u5f55\uff0c\u5220\u9664\u5982\u4e0b\u51e0\u4e2a\u6587\u4ef6\u548ckubelet.kubeconfig\u6587\u4ef6\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"711\" height=\"218\" src=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/3-7.png\" alt=\"\" class=\"wp-image-8418\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/3-7.png 711w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/3-7-300x92.png 300w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/3-7-600x184.png 600w\" sizes=\"auto, (max-width: 711px) 100vw, 711px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u5220\u9664\u540e\uff0c\u91cd\u542fkubelet\uff0c\u5373\u53ef\u5728master\u8282\u70b9\u91cd\u65b0\u770b\u5230Pending\u8bf7\u6c42<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u53ef\u901a\u8fc7\u5982\u4e0b\u547d\u4ee4\u67e5\u770bcsr\u5c5e\u4e8e\u54ea\u4e2a\u8282\u70b9\uff0c\u5e76\u6267\u884c\u5220\u9664\u64cd\u4f5c\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl describe csr node-csr-7reKUW9KL3Ec70E6xvE834I_-BgyyH7Glx7wTTUC04s\nkubectl delete csr node-csr-7reKUW9KL3Ec70E6xvE834I_-BgyyH7Glx7wTTUC04s<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">2\u3001\u81ea\u52a8\u65b9\u5f0f\u6279\u51c6\uff1a<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u53c2\u8003\uff1a<a href=\"https:\/\/blog.ywdevops.cn\/index.php\/2022\/05\/24\/kubelet\/\" target=\"_blank\" rel=\"noreferrer noopener\">kubelet\u521d\u59cb\u5316\u8fc7\u7a0b<\/a><\/p>\n\n\n\n<p class=\"has-text-align-center has-black-color has-text-color wp-block-paragraph\"><strong>\u90e8\u7f72 kube-proxy (\u53ea\u5728master1\u6267\u884c)\uff1a<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">1\u3001 \u521b\u5efa\u914d\u7f6e\u6587\u4ef6 \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/opt\/kubernetes\/cfg\/kube-proxy.conf &lt;&lt; EOF\nKUBE_PROXY_OPTS=\"--logtostderr=false \\\\\n--v=2 \\\\\n--log-dir=\/opt\/kubernetes\/logs \\\\\n--config=\/opt\/kubernetes\/cfg\/kube-proxy-config.yml\"\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">2\u3001 \u914d\u7f6e\u53c2\u6570\u6587\u4ef6 \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/opt\/kubernetes\/cfg\/kube-proxy-config.yml &lt;&lt; EOF\nkind: KubeProxyConfiguration\napiVersion: kubeproxy.config.k8s.io\/v1alpha1\nbindAddress: 0.0.0.0\nmetricsBindAddress: 0.0.0.0:10249\nclientConnection:\n  kubeconfig: \/opt\/kubernetes\/cfg\/kube-proxy.kubeconfig\nhostnameOverride: k8s-master1\nclusterCIDR: 10.244.0.0\/16\nmode: \"ipvs\"\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">3\u3001  \u751f\u6210 kube-proxy.kubeconfig \u6587\u4ef6 \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># \u5207\u6362\u5de5\u4f5c\u76ee\u5f55\ncd ~\/TLS\/k8s\n#\u521b\u5efa\u8bc1\u4e66\u8bf7\u6c42\u6587\u4ef6\ncat &gt; kube-proxy-csr.json &lt;&lt; EOF\n{\n \"CN\": \"system:kube-proxy\",\n \"hosts\": &#91;],\n \"key\": {\n     \"algo\": \"rsa\",\n     \"size\": 2048\n },\n \"names\": &#91;\n   {\n     \"C\": \"CN\",\n     \"L\": \"BeiJing\",\n     \"ST\": \"BeiJing\",\n     \"O\": \"k8s\",\n     \"OU\": \"System\"\n   }\n ]\n}\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u751f\u6210\u8bc1\u4e66\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"861\" height=\"272\" src=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/2-5.png\" alt=\"\" class=\"wp-image-8416\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/2-5.png 861w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/2-5-300x95.png 300w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/2-5-768x243.png 768w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/2-5-600x190.png 600w\" sizes=\"auto, (max-width: 861px) 100vw, 861px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u751f\u6210 kubeconfig \u6587\u4ef6\uff1a <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>KUBE_CONFIG=\"\/opt\/kubernetes\/cfg\/kube-proxy.kubeconfig\"\nKUBE_APISERVER=\"https:\/\/192.168.15.253:16443\"       #\u6ce8\u610f\uff1a\u6b64\u4f4d\u7f6e\u4e3aVIP\nkubectl config set-cluster kubernetes \\\n --certificate-authority=\/opt\/kubernetes\/ssl\/ca.pem \\\n --embed-certs=true \\\n --server=${KUBE_APISERVER} \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config set-credentials kube-proxy \\\n --client-certificate=.\/kube-proxy.pem \\\n --client-key=.\/kube-proxy-key.pem \\\n --embed-certs=true \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config set-context default \\\n --cluster=kubernetes \\\n --user=kube-proxy \\\n --kubeconfig=${KUBE_CONFIG}\nkubectl config use-context default --kubeconfig=${KUBE_CONFIG}<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">4\u3001 systemd \u7ba1\u7406 kube-proxy  <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; \/usr\/lib\/systemd\/system\/kube-proxy.service &lt;&lt; EOF\n&#91;Unit]\nDescription=Kubernetes Proxy\nAfter=network.target\n&#91;Service]\nEnvironmentFile=\/opt\/kubernetes\/cfg\/kube-proxy.conf\nExecStart=\/opt\/kubernetes\/bin\/kube-proxy \\$KUBE_PROXY_OPTS\nRestart=on-failure\nLimitNOFILE=65536\n&#91;Install]\nWantedBy=multi-user.target\nEOF<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">5\u3001\u5c06master1\u8282\u70b9\u4e0a\u7684kube-proxy\u76f8\u5173\u6587\u4ef6\u53d1\u9001\u5230master\u548cworker\u8282\u70b9\u4e2d,\u4e3e\u4f8b\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>for i in k8s-master02 k8s-master03 k8s-node01 k8s-node02;do scp kube-proxy.conf kube-proxy-config.yml kube-proxy.kubeconfig ${i}:\/opt\/kubernetes\/cfg\/ ;done\nfor i in k8s-master02 k8s-master03 k8s-node01 k8s-node02;do scp \/lib\/systemd\/system\/kube-proxy.service  ${i}:\/lib\/systemd\/system ;done<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\u610f\uff1a\u8981\u4fee\u6539kube-proxy-config.yml\u4e2d\u7684hostnameOverride\u7684\u503c<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">6\u3001 \u542f\u52a8\u5e76\u8bbe\u7f6e\u5f00\u673a\u542f\u52a8(\u5168\u90e8\u8282\u70b9) \uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>systemctl daemon-reload\nsystemctl start kube-proxy\nsystemctl enable kube-proxy<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\uff1a\u5982\u679cpod\u65e0\u6cd5ping\u901asvc\u7684IP\u5730\u5740\uff0c\u53ef\u80fd\u662fkube-proxy\u6a21\u5f0f\u95ee\u9898\uff0c\u4fee\u6539\u4e3aipvs\u5373\u53ef\uff0c\u67e5\u770b\u5f53\u524d\u6a21\u5f0f\u547d\u4ee4\uff1acurl 127.0.0.1:10249\/proxyMode<\/p>\n\n\n\n<p class=\"has-text-align-center has-medium-font-size wp-block-paragraph\"><strong> \u516d\u3001\u90e8\u7f72\u7f51\u7edc\u7ec4\u4ef6 :<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Calico<\/strong> \u662f\u4e00\u4e2a\u7eaf\u4e09\u5c42\u7684\u6570\u636e\u4e2d\u5fc3\u7f51\u7edc\u65b9\u6848\uff0c\u662f\u76ee\u524d Kubernetes \u4e3b\u6d41\u7684\u7f51\u7edc\u65b9\u6848\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Calico\u4ee5DaemonSet\u65b9\u5f0f\u8fd0\u884c\u5728\u96c6\u7fa4\u7684\u6bcf\u4e2a\u8282\u70b9\u4e0a\uff0c\u5f53\u6269\u5bb9node\u8282\u70b9\u540e\uff0c\u65b0\u8282\u70b9\u4f1a\u81ea\u52a8\u8fd0\u884ccalico\u7684Pod<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u4e0b\u8f7d\u5730\u5740\uff1ahttps:\/\/projectcalico.docs.tigera.io\/getting-started\/kubernetes\/self-managed-onprem\/onpremises\n\u4e0b\u8f7d\u547d\u4ee4\uff1acurl https:\/\/projectcalico.docs.tigera.io\/manifests\/calico.yaml -O<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">1\u3001\u9996\u5148\u4e0b\u8f7dcalico.yaml\u6587\u4ef6\uff0c\u7f16\u8f91\u6587\u4ef6\uff0c\u4fee\u6539\u955c\u50cf\u5730\u5740\u4e3a\u5185\u90e8\u7684harbor\u5730\u5740(\u4e24\u4e2acni\u5730\u5740\u3001\u4e00\u4e2aflexvol\u5730\u5740\u3001\u4e00\u4e2anode\u5730\u5740\u3001\u4e00\u4e2acontrollers\u5730\u5740)\uff0c\u5982\u56fe\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"553\" height=\"157\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/6.png\" alt=\"\" class=\"wp-image-4308\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/6.png 553w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/6-300x85.png 300w\" sizes=\"auto, (max-width: 553px) 100vw, 553px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">2\u3001\u521b\u5efa\u5bc6\u94a5\u8ba4\u8bc1\u6587\u4ef6(<strong>\u96c6\u7fa4master1\u8282\u70b9\u521b\u5efa\u5373\u53ef\uff0c\u5176\u4f59\u8fde\u63a5\u5230\u96c6\u7fa4\u7684\u90fd\u53ef\u4ee5\u4f7f\u7528<\/strong>)<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u9996\u5148\u7f16\u8f91docker\u6587\u4ef6\uff0c\u6dfb\u52a0\u4ed3\u5e93\u5730\u5740\uff0c\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a\nvim   \/etc\/docker\/daemon.json\n#\u6dfb\u52a0\u5185\u5bb9\u5982\u4e0b\uff1a\n\"insecure-registries\":&#91;\"192.168.26.131\"]\n#\u91cd\u542fdocker\nsystemctl restart docker\n#\u767b\u5f55harbor\ndocker login 192.168.26.131\n#\u8fdb\u5165root\/.docker\u76ee\u5f55\u4e0b\uff0c\u6267\u884c\u547d\u4ee4\u751f\u6210\u5bc6\u94a5\ncat config.json | base64     #\u53ef\u4ee5\u5f97\u5230\u4e00\u4e32\u5b57\u7b26\u4e32\n#\u7f16\u8f91\u6587\u4ef6secret.yaml\u6dfb\u52a0\u5185\u5bb9\u5982\u4e0b\uff0c\u5176\u4e2d.dockerconfigjson\u4f4d\u7f6e\u586b\u5199\u4e0a\u4e00\u6b65\u751f\u6210\u7684\u52a0\u5bc6\u5b57\u7b26\u4e32\uff0cnamespace\u6307\u5b9a\u547d\u540d\u7a7a\u95f4\uff0cname\u7528\u4e8e\u6307\u5b9a\u540d\u5b57\uff0c\u53ef\u81ea\u5b9a\u4e49\uff0c\u5176\u4f59\u90fd\u6309\u7167\u4e0b\u56fe\u5199\u5373\u53ef<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"542\" height=\"207\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/7.png\" alt=\"\" class=\"wp-image-4309\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/7.png 542w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/7-300x115.png 300w\" sizes=\"auto, (max-width: 542px) 100vw, 542px\" \/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code>apiVersion: v1\nkind: Secret\nmetadata:\n    name: secret-key\n    namespace: kube-system\ndata:\n    .dockerconfigjson: ewoJImF1dGhzIjogewoJCSIxMC44OC4yNi4xMzEiOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2U0dGeVltOXlNVEl6TkRVPSIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTkuMDMuOSAobGludXgpIgoJfQp9\ntype: kubernetes.io\/dockerconfigjson<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>#\u914d\u7f6e\u5b8c\u6210\u540e\uff0c\u6267\u884c\u547d\u4ee4\u521b\u5efa\u5bc6\u94a5\u8ba4\u8bc1\nkubectl create -f secret.yaml \n#\u6267\u884c\u547d\u4ee4\u67e5\u770b\uff0c\u8981\u6267\u884c\u547d\u540d\u7a7a\u95f4\uff0c\u5982\u4e0b\uff1a\nkubectl get secret -n kube-system<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">3\u3001\u5728calico.yaml\u4e2d\u914d\u7f6e\u5bc6\u94a5\u8ba4\u8bc1\u6587\u4ef6\uff0c\u4e00\u5171\u4e24\u4e2a\u4f4d\u7f6e\u9700\u8981\u914d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u7b2c\u4e00\u4e2a\u4f4d\u7f6e\u662fDaemonSet\u63a7\u5236\u5668\u4f4d\u7f6e\uff0c\u4e5f\u5c31\u662fcni\u5bb9\u5668\u4e0a\u65b9\uff0c\u5982\u56fe\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"551\" height=\"176\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/8.png\" alt=\"\" class=\"wp-image-4312\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/8.png 551w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/8-300x96.png 300w\" sizes=\"auto, (max-width: 551px) 100vw, 551px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u7b2c\u4e8c\u4e2a\u662fDeployment\u63a7\u5236\u5668\u4f4d\u7f6e\uff0c\u4e5f\u5c31\u662fcontrollers\u5bb9\u5668\u4e0a\u65b9\uff0c\u5982\u56fe\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"528\" height=\"216\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/9-1.png\" alt=\"\" class=\"wp-image-4313\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/9-1.png 528w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/9-1-300x123.png 300w\" sizes=\"auto, (max-width: 528px) 100vw, 528px\" \/><\/figure>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\uff1a\u5728\u6267\u884c\u4e0b\u9762\u7684apply\u547d\u4ee4\u4e4b\u524d\u786e\u4fdd\u6240\u6709\u8282\u70b9\u90fd\u53ef\u4ee5\u767b\u5f55\u5230\u4ed3\u5e93\u4e2d\uff0cdocker login \u4ed3\u5e93\u5730\u5740<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">4\u3001\u914d\u7f6e\u5b8c\u6210\u540e\u6267\u884c\u547d\u4ee4\u521b\u5efacalico\u7f51\u7edc\u7ec4\u4ef6\uff0c\u5982\u4e0b\uff1a<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">kubectl apply -f calico.yaml<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">5\u3001\u901a\u8fc7\u5982\u4e0b\u547d\u4ee4\u67e5\u770bcalico\u662f\u5426\u521b\u5efa\u6210\u529f\uff0c\u8bb0\u5f97\u6267\u884c\u547d\u540d\u7a7a\u95f4\uff0c\u5982\u56fe\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"614\" height=\"225\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/1-1.png\" alt=\"\" class=\"wp-image-4425\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/1-1.png 614w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/1-1-300x110.png 300w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/1-1-600x220.png 600w\" sizes=\"auto, (max-width: 614px) 100vw, 614px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u4ece\u4e0a\u56fe\u53ef\u4ee5\u770b\u51fa\uff0c\u72b6\u6001\u90fd\u662frunning\uff0c\u8868\u793a\u6210\u529f\uff0c\u6b64\u65f6\u518d\u6b21\u6267\u884ckubectl get node \u53ef\u4ee5\u770b\u5230\u5df2\u7ecf\u663e\u793aReady\uff0c\u5982\u56fe\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"555\" height=\"196\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/2-1.png\" alt=\"\" class=\"wp-image-4427\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/2-1.png 555w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/2-1-300x106.png 300w\" sizes=\"auto, (max-width: 555px) 100vw, 555px\" \/><\/figure>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background wp-block-paragraph\">\u6ce8\u610f\uff1a\u5982\u679c\u4e0d\u6539\u53d8\u5176\u4ed6\u914d\u7f6e\uff0ccalico\u9ed8\u8ba4\u91c7\u7528IPIP\u7f51\u7edc\u6a21\u5f0f<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">calico\u914d\u7f6e\u6587\u4ef6\uff0c\u9002\u7528\u4e8e1.20.8\u7248\u672c\u7684kubernetes\uff0c\u53c2\u8003\uff1a<a href=\"https:\/\/blog.ywdevops.cn\/index.php\/2022\/05\/26\/calico-2\/\" target=\"_blank\" rel=\"noreferrer noopener\">calico\u6e05\u5355\u6587\u4ef6<\/a><\/p>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\"><strong> \u6388\u6743 apiserver \u8bbf\u95ee kubelet \uff1a<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5728master\u8282\u70b9\u6267\u884c\u547d\u4ee4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat &gt; apiserver-tokkubelet-rbac.yaml &lt;&lt;EOF\napiVersion: rbac.authorization.k8s.io\/v1\nkind: ClusterRole\nmetadata:\n annotations:\n     rbac.authorization.kubernetes.io\/autoupdate: \"true\"\n labels:\n     kubernetes.io\/bootstrapping: rbac-defaults\n name: system:kube-apiserver-to-kubelet\nrules:\n - apiGroups:\n     - \"\"\n   resources:\n     - nodes\/proxy\n     - nodes\/stats\n     - nodes\/log\n     - nodes\/spec\n     - nodes\/metrics\n     - pods\/log\n   verbs:\n     - \"*\"\n---\napiVersion: rbac.authorization.k8s.io\/v1\nkind: ClusterRoleBinding\nmetadata:\n name: system:kube-apiserver\n namespace: \"\"\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: system:kube-apiserver-to-kubelet\nsubjects:\n - apiGroup: rbac.authorization.k8s.io\n   kind: User\n   name: kubernetes\nEOF<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"614\" height=\"700\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/13.png\" alt=\"\" class=\"wp-image-4327\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/13.png 614w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/13-263x300.png 263w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/13-526x600.png 526w\" sizes=\"auto, (max-width: 614px) 100vw, 614px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u6267\u884c\u5982\u4e0b\u547d\u4ee4\u521b\u5efa\u6388\u6743\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl apply -f apiserver-to-kubelet-rbac.yaml<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"759\" height=\"114\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/14.png\" alt=\"\" class=\"wp-image-4330\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/14.png 759w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/14-300x45.png 300w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/14-600x90.png 600w\" sizes=\"auto, (max-width: 759px) 100vw, 759px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>\u6ce8\u610f\uff1a<\/strong>\u5982\u679c\u4e0d\u6388\u6743\uff0c\u90a3\u4e48\u901a\u8fc7kubectl logs\u67e5\u770b\u65e5\u5fd7\u4f1a\u63d0\u793a\u9519\u8bef\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Error from server (Forbidden): Forbidden (user=kubernetes, verb=get, resource=nodes, subresource=proxy) ( pods\/log calico-node-kg9pc)<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\"><strong> \u90e8\u7f72 CoreDNS <\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> CoreDNS \u7528\u4e8e\u96c6\u7fa4\u5185\u90e8 Service \u540d\u79f0\u89e3\u6790 <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u6267\u884c\u5982\u4e0b\u547d\u4ee4\u8fd0\u884ccoredns\uff0c\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl apply -f coredns.yaml <\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">\u67e5\u770b\u662f\u5426\u521b\u5efa\u6210\u529f\uff0c\u5982\u56fe\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"591\" height=\"198\" src=\"http:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/16.png\" alt=\"\" class=\"wp-image-4336\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/16.png 591w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2021\/07\/16-300x101.png 300w\" sizes=\"auto, (max-width: 591px) 100vw, 591px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">coredns\u5185\u5bb9\u53c2\u8003\uff1a<a href=\"https:\/\/blog.ywdevops.cn\/index.php\/2022\/05\/26\/coredns\/\" target=\"_blank\" rel=\"noreferrer noopener\">coredns\u6e05\u5355\u6587\u4ef6<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u8bf4\u660e\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u6ce8\u610fclusterIP\u7684\u5730\u5740\uff0c\u8981\u4e0ekubelet\u914d\u7f6e\u6587\u4ef6\u4e2d\u7684clusterdns\u5730\u5740\u4e00\u81f4<\/li>\n<\/ul>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\"><strong>\u96c6\u7fa4\u53ef\u7528\u6027\u9a8c\u8bc1<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li> pod\u5fc5\u987b\u53ef\u4ee5\u89e3\u6790service<\/li>\n\n\n\n<li>  pod\u5fc5\u987b\u53ef\u4ee5\u89e3\u6790\u8de8namspace\u7684service <\/li>\n\n\n\n<li>\u6bcf\u4e2a\u8282\u70b9\u90fd\u5fc5\u987b\u80fd\u8bbf\u95eekubernetes\u7684service(443\u7aef\u53e3)\u548ckube-dns\u7684service(53\u7aef\u53e3)<\/li>\n\n\n\n<li>Pod\u548cpod\u4e4b\u95f4\u8981\u80fd\u901a(\u540cnamespace\u3001\u8de8namespace\u3001\u8de8\u673a\u5668\u80fd\u901a)<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">1\u3001\u521b\u5efabusybox\u7684pod\uff0c\u5728\u5185\u90e8\u9a8c\u8bc1\u89e3\u6790service\u548c\u8de8namespace\u7684service\uff0c\u5982\u56fe\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"706\" height=\"258\" src=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/1-6.png\" alt=\"\" class=\"wp-image-8537\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/1-6.png 706w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/1-6-300x110.png 300w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/1-6-600x219.png 600w\" sizes=\"auto, (max-width: 706px) 100vw, 706px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">\u6ce8\uff1abusybox\u7248\u672c\u53ef\u75281.28.3\uff0c\u6700\u65b0\u7248\u672c\u53ef\u80fd\u51fa\u73b0\u89e3\u6790\u6709\u95ee\u9898<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">2\u3001\u9a8c\u8bc1\u6240\u6709\u8282\u70b9\u8bbf\u95eekubernetes\u7684service(443\u7aef\u53e3)\u548ckube-dns\u7684service(53\u7aef\u53e3),\u5982\u56fe\uff1a<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"708\" height=\"298\" src=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/2-6.png\" alt=\"\" class=\"wp-image-8539\" srcset=\"https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/2-6.png 708w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/2-6-300x126.png 300w, https:\/\/blog.ywdevops.cn\/wp-content\/uploads\/2022\/05\/2-6-600x253.png 600w\" sizes=\"auto, (max-width: 708px) 100vw, 708px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">3\u3001pod\u4e0epod\u4e4b\u95f4\u901a\u4fe1\u9a8c\u8bc1\uff0c\u767b\u5f55\u8fdb\u5165pod\u5185\u90e8\uff0cping\u5176\u4f59Pod\u7684IP\u5373\u53ef<\/p>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\"><strong>metricsever\u90e8\u7f72<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u53c2\u8003\u94fe\u63a5\uff1a<a href=\"https:\/\/blog.ywdevops.cn\/index.php\/2022\/05\/26\/metrics-server\/\" target=\"_blank\" rel=\"noreferrer noopener\">metric-server\u90e8\u7f72<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u751f\u4ea7\u73af\u5883\u90e8\u7f72 K8s \u96c6\u7fa4\u7684\u4e24\u79cd\u65b9\u5f0f \uff1a \u603b\u7ed3\uff1a Kubeadm \u964d\u4f4e\u90e8\u7f72\u95e8\u69db\uff0c\u4f46\u5c4f\u853d\u4e86\u5f88\u591a\u7ec6\u8282\uff0c\u9047\u5230\u95ee\u9898\u5f88\u96be [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":4207,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[93,23],"tags":[],"class_list":["post-4149","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-kubernetes","category-rong"],"blocksy_meta":{"styles_descriptor":{"styles":{"desktop":"","tablet":"","mobile":""},"google_fonts":[],"version":8}},"_links":{"self":[{"href":"https:\/\/blog.ywdevops.cn\/index.php\/wp-json\/wp\/v2\/posts\/4149","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.ywdevops.cn\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ywdevops.cn\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ywdevops.cn\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ywdevops.cn\/index.php\/wp-json\/wp\/v2\/comments?post=4149"}],"version-history":[{"count":424,"href":"https:\/\/blog.ywdevops.cn\/index.php\/wp-json\/wp\/v2\/posts\/4149\/revisions"}],"predecessor-version":[{"id":20311,"href":"https:\/\/blog.ywdevops.cn\/index.php\/wp-json\/wp\/v2\/posts\/4149\/revisions\/20311"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ywdevops.cn\/index.php\/wp-json\/wp\/v2\/media\/4207"}],"wp:attachment":[{"href":"https:\/\/blog.ywdevops.cn\/index.php\/wp-json\/wp\/v2\/media?parent=4149"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ywdevops.cn\/index.php\/wp-json\/wp\/v2\/categories?post=4149"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ywdevops.cn\/index.php\/wp-json\/wp\/v2\/tags?post=4149"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}