将GitLab Helm Chart安装到AWS EKS
GitLab v11.0.0从Cloud Native GitLab Helm Chart更名为GitLab Helm Chart，并成为Beta版本。
- https://docs.gitlab.com/ee/install/kubernetes/gitlab_chart.html
由于EKS的官方安装步骤不太理想，所以我在这里总结了一份安装指南。
事前准备
我正在使用Mac OS进行工作。您需要提前准备以下内容。
-
- kubectl
-
- Helm
- eksctl
建立EKS集群。
使用之前介绍过的EKSCTL工具创建集群。
> eksctl create cluster \
--cluster-name $NAME \
--nodes 3 \
--nodes-min 3 \
--nodes-max 5 \
--node-type t2.medium \
--region us-west-2
2018-07-18T13:33:01+09:00 [ℹ] importing SSH public key "/Users/jb/.ssh/id_rsa.pub" as "EKS-bakeneco"
2018-07-18T13:33:02+09:00 [ℹ] creating EKS cluster "bakeneco" in "us-west-2" region
2018-07-18T13:33:02+09:00 [ℹ] creating VPC stack "EKS-bakeneco-VPC"
2018-07-18T13:33:02+09:00 [ℹ] creating ServiceRole stack "EKS-bakeneco-ServiceRole"
2018-07-18T13:33:22+09:00 [✔] created ServiceRole stack "EKS-bakeneco-ServiceRole"
2018-07-18T13:34:03+09:00 [✔] created VPC stack "EKS-bakeneco-VPC"
2018-07-18T13:34:03+09:00 [ℹ] creating control plane "bakeneco"
2018-07-18T13:46:05+09:00 [✔] created control plane "bakeneco"
2018-07-18T13:46:05+09:00 [ℹ] creating DefaultNodeGroup stack "EKS-bakeneco-DefaultNodeGroup"
2018-07-18T13:49:47+09:00 [✔] created DefaultNodeGroup stack "EKS-bakeneco-DefaultNodeGroup"
2018-07-18T13:49:47+09:00 [✔] all EKS cluster "bakeneco" resources has been created
2018-07-18T13:49:47+09:00 [ℹ] wrote "kubeconfig"
2018-07-18T13:49:48+09:00 [ℹ] the cluster has 0 nodes
2018-07-18T13:49:48+09:00 [ℹ] waiting for at least 3 nodes to become ready
2018-07-18T13:50:15+09:00 [ℹ] the cluster has 4 nodes
2018-07-18T13:50:15+09:00 [ℹ] node "ip-192-168-100-194.us-west-2.compute.internal" is ready
2018-07-18T13:50:15+09:00 [ℹ] node "ip-192-168-128-38.us-west-2.compute.internal" is ready
2018-07-18T13:50:15+09:00 [ℹ] node "ip-192-168-177-212.us-west-2.compute.internal" is ready
2018-07-18T13:50:15+09:00 [ℹ] node "ip-192-168-209-209.us-west-2.compute.internal" is ready
2018-07-18T13:50:17+09:00 [ℹ] all command should work, try '/usr/local/bin/kubectl --kubeconfig kubeconfig get nodes'
2018-07-18T13:50:17+09:00 [ℹ] EKS cluster "bakeneco" in "us-west-2" region is ready
请设置$KUBECONFIG环境变量，以便通过kubectl进行连接，并进行确认。
> kubectl get nodes
NAME STATUS ROLES AGE VERSION
ip-192-168-100-194.us-west-2.compute.internal Ready <none> 14m v1.10.3
ip-192-168-106-191.us-west-2.compute.internal Ready <none> 13m v1.10.3
ip-192-168-128-38.us-west-2.compute.internal Ready <none> 14m v1.10.3
ip-192-168-177-212.us-west-2.compute.internal Ready <none> 14m v1.10.3
ip-192-168-209-209.us-west-2.compute.internal Ready <none> 14m v1.10.3
> kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 20m
> kubectl get deployments --all-namespaces
NAMESPACE NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kube-system kube-dns 1 1 1 1 24m
> kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system aws-node-d2v4m 1/1 Running 1 14m
kube-system aws-node-fbcr6 1/1 Running 0 13m
kube-system aws-node-ggnps 1/1 Running 1 14m
kube-system aws-node-s2mnr 1/1 Running 0 14m
kube-system aws-node-z274r 1/1 Running 0 14m
kube-system kube-dns-7cc87d595-kdkcv 3/3 Running 0 19m
kube-system kube-proxy-5m9gf 1/1 Running 0 13m
kube-system kube-proxy-8fh9v 1/1 Running 0 14m
kube-system kube-proxy-ggk7j 1/1 Running 0 14m
kube-system kube-proxy-qngg2 1/1 Running 0 14m
kube-system kube-proxy-rxhjn 1/1 Running 0 14m
集群环境的配置
网络设置
亚马逊云容器服务（Amazon EKS）使用弹性负载均衡器，其通过DNS名称进行访问，无法提前预知。请跳过此部分。
好的，我跳过。
存储设置
持久卷采用动态配置，虽然受限于特定的区域，但目前我认为没有其他选择。
每个存储都将被分开，但这次我们将简单地使用一个存储类。
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: gp2
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
zone: us-west-2a
reclaimPolicy: Retain
mountOptions:
- debug
> kubectl create -f gp2-storage-class.yaml
storageclass.storage.k8s.io "gp2" created
> kubectl patch storageclass gp2 -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
storageclass.storage.k8s.io "gp2" patched
> kubectl get storageclass
NAME PROVISIONER AGE
gp2 (default) kubernetes.io/aws-ebs 37s
犁头
对于某些群集，需要使用kubectl进行身份验证才能创建Tiller角色。
对于像亚马逊EKS这样的群集，您可以直接上传RBAC配置。
因此，我们创建了一个用于 Tiller 的 ServiceAccount，并授予了 cluster-admin 角色。
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
> kubectl create -f rbac-config.yaml
serviceaccount "tiller" created
clusterrolebinding.rbac.authorization.k8s.io "tiller" created
然后安装和初始化Tiller。
> helm init --service-account tiller
$HELM_HOME has been configured at /Users/jb/.helm.
Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Happy Helming!
安装GitLab Chart
添加GitLab图表仓库。
> helm repo add gitlab https://charts.gitlab.io/
"gitlab" has been added to your repositories
使用Helm安装GitLab Chart。
> helm upgrade --install gitlab gitlab/gitlab \
--timeout 600 \
--set global.hosts.domain=bakeneco.io \
--set gitlab.migrations.initialRootPassword="XXX" \
--set certmanager-issuer.email=XX@YYY.com
Release "gitlab" does not exist. Installing it now.
NAME: gitlab
LAST DEPLOYED: Wed Jul 18 17:50:08 2018
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/ServiceAccount
NAME SECRETS AGE
gitlab-certmanager-issuer 1 11s
certmanager-gitlab 1 11s
gitlab-gitlab-runner 1 11s
gitlab-nginx-ingress 1 11s
gitlab-prometheus-alertmanager 1 11s
gitlab-prometheus-kube-state-metrics 1 11s
gitlab-prometheus-node-exporter 1 11s
gitlab-prometheus-server 1 11s
==> v1/RoleBinding
NAME AGE
gitlab-certmanager-issuer 3s
gitlab-nginx-ingress 3s
==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
gitlab-unicorn gitlab.bakeneco.io 80, 443 2s
gitlab-minio minio.bakeneco.io 80, 443 2s
gitlab-registry registry.bakeneco.io 80, 443 2s
==> v2beta1/HorizontalPodAutoscaler
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
gitlab-gitlab-shell Deployment/gitlab-gitlab-shell <unknown>/75% 2 10 0 2s
gitlab-sidekiq-all-in-1 Deployment/gitlab-sidekiq-all-in-1 <unknown>/75% 1 10 0 2s
gitlab-unicorn Deployment/gitlab-unicorn <unknown>/75% 2 10 0 2s
gitlab-registry Deployment/gitlab-registry <unknown>/75% 2 10 0 2s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
certmanager-gitlab-54467869c4-c7q6t 0/2 ContainerCreating 0 2s
gitlab-gitlab-runner-845c5b46d8-9vwjm 0/1 Init:0/1 0 2s
gitlab-gitlab-shell-7d8cd44948-lccrx 0/1 Init:0/1 0 2s
gitlab-sidekiq-all-in-1-58c996c9fb-mrcd5 0/1 Init:0/2 0 2s
gitlab-task-runner-866bc87864-tgg4p 0/1 Init:0/1 0 2s
gitlab-unicorn-5c69b9487c-v4pnq 0/1 Init:0/2 0 2s
gitlab-minio-99bff897b-6cplp 0/1 Pending 0 2s
gitlab-nginx-ingress-controller-65d58cbf4d-5zxlm 0/1 ContainerCreating 0 2s
gitlab-nginx-ingress-controller-65d58cbf4d-kp9fg 0/1 ContainerCreating 0 2s
gitlab-nginx-ingress-controller-65d58cbf4d-xttvr 0/1 ContainerCreating 0 2s
gitlab-nginx-ingress-default-backend-699b9476dd-5kjjt 0/1 ContainerCreating 0 1s
gitlab-nginx-ingress-default-backend-699b9476dd-jx86w 0/1 Pending 0 1s
gitlab-postgresql-5578b89f58-4jp9k 0/2 ContainerCreating 0 1s
gitlab-prometheus-server-847c8bb76-9jqm6 0/2 Pending 0 1s
gitlab-redis-6b8b6dbfd9-bs24z 0/2 Init:0/1 0 1s
gitlab-registry-7f4b9ccfc8-k4zsj 0/1 Pending 0 1s
gitlab-gitaly-0 0/1 Pending 0 2s
gitlab-issuer.1-2xqhq 0/1 ContainerCreating 0 2s
gitlab-migrations.1-zs7tj 0/1 Init:0/1 0 2s
gitlab-minio-create-buckets.1-97wvg 0/1 ContainerCreating 0 2s
==> v1beta1/CustomResourceDefinition
NAME AGE
certificates.certmanager.k8s.io 7s
clusterissuers.certmanager.k8s.io 3s
issuers.certmanager.k8s.io 3s
==> v1beta1/ClusterRole
certmanager-gitlab 3s
gitlab-prometheus-kube-state-metrics 3s
gitlab-prometheus-server 3s
==> v1beta1/ClusterRoleBinding
NAME AGE
certmanager-gitlab 3s
gitlab-prometheus-alertmanager 3s
gitlab-prometheus-kube-state-metrics 3s
gitlab-prometheus-node-exporter 3s
gitlab-prometheus-server 3s
==> v1/Role
NAME AGE
gitlab-certmanager-issuer 3s
gitlab-nginx-ingress 3s
==> v1beta2/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
gitlab-gitlab-shell 1 1 1 0 2s
gitlab-sidekiq-all-in-1 1 1 1 0 2s
gitlab-task-runner 1 1 1 0 2s
gitlab-unicorn 1 1 1 0 2s
gitlab-minio 1 1 1 0 2s
gitlab-nginx-ingress-controller 3 0 0 0 2s
gitlab-nginx-ingress-default-backend 2 0 0 0 2s
gitlab-redis 1 0 0 0 2s
gitlab-registry 1 0 0 0 2s
==> v1beta2/StatefulSet
NAME DESIRED CURRENT AGE
gitlab-gitaly 1 1 2s
==> v1beta1/PodDisruptionBudget
NAME MIN AVAILABLE MAX UNAVAILABLE ALLOWED DISRUPTIONS AGE
gitlab-gitaly N/A 1 0 2s
gitlab-gitlab-shell N/A 1 0 2s
gitlab-sidekiq N/A 1 0 2s
gitlab-unicorn N/A 1 0 2s
gitlab-minio-v1 N/A 1 0 2s
gitlab-nginx-ingress-controller 2 N/A 0 2s
gitlab-nginx-ingress-default-backend 1 N/A 0 2s
gitlab-redis-v1 N/A 1 0 2s
gitlab-registry-v1 N/A 1 0 2s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
gitlab-gitaly ClusterIP None <none> 8075/TCP,9236/TCP 3s
gitlab-gitlab-shell ClusterIP 10.100.96.36 <none> 22/TCP 3s
gitlab-unicorn ClusterIP 10.100.170.239 <none> 8080/TCP,8181/TCP 3s
gitlab-minio-svc ClusterIP 10.100.14.232 <none> 9000/TCP 3s
gitlab-nginx-ingress-controller LoadBalancer 10.100.234.221 <pending> 80:30364/TCP,443:30849/TCP,22:31508/TCP 3s
gitlab-nginx-ingress-default-backend ClusterIP 10.100.243.113 <none> 80/TCP 3s
gitlab-postgresql ClusterIP 10.100.10.190 <none> 5432/TCP 3s
gitlab-prometheus-server ClusterIP 10.100.191.234 <none> 80/TCP 3s
gitlab-redis ClusterIP 10.100.110.213 <none> 6379/TCP,9121/TCP 3s
gitlab-registry ClusterIP 10.100.51.212 <none> 5000/TCP 2s
==> v1/Job
NAME DESIRED SUCCESSFUL AGE
gitlab-issuer.1 1 0 2s
gitlab-migrations.1 1 0 2s
gitlab-minio-create-buckets.1 1 0 2s
==> v1/ConfigMap
NAME DATA AGE
gitlab-certmanager-issuer-certmanager 2 11s
gitlab-gitlab-runner 3 11s
gitlab-gitaly 3 11s
gitlab-gitlab-shell 2 11s
gitlab-nginx-ingress-tcp 1 11s
gitlab-migrations 4 11s
gitlab-sidekiq-all-in-1 1 11s
gitlab-sidekiq 6 11s
gitlab-task-runner 4 11s
gitlab-unicorn 8 11s
gitlab-unicorn-tests 1 11s
gitlab-minio-config-cm 3 11s
gitlab-nginx-ingress-controller 7 11s
gitlab-postgresql 0 11s
gitlab-prometheus-server 3 11s
gitlab-redis 2 11s
gitlab-registry 2 11s
==> v1/PersistentVolumeClaim
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
gitlab-minio Pending gp2 11s
gitlab-postgresql Bound pvc-9b3e012b-8a67-11e8-8b92-02b5389a29ae 8Gi RWO gp2 11s
gitlab-prometheus-server Bound pvc-9b3f2af5-8a67-11e8-8b92-02b5389a29ae 8Gi RWO gp2 11s
gitlab-redis Bound pvc-9b400dee-8a67-11e8-8b92-02b5389a29ae 5Gi RWO gp2 11s
==> v1beta1/Role
NAME AGE
gitlab-gitlab-runner 3s
==> v1beta1/RoleBinding
NAME AGE
gitlab-gitlab-runner 3s
==> v1beta1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
certmanager-gitlab 1 1 1 0 2s
gitlab-gitlab-runner 1 1 1 0 2s
gitlab-postgresql 1 1 1 0 2s
gitlab-prometheus-server 1 1 1 0 2s
当我稍微观察一下时，我注意到GitLab Runner无法成功启动。
> kubectl logs gitlab-gitlab-runner-845c5b46d8-9vwjm
WARNING: Running in user-mode.
WARNING: The user-mode requires you to manually start builds processing:
WARNING: $ gitlab-runner run
WARNING: Use sudo for system-mode:
WARNING: $ sudo gitlab-runner...
ERROR: Registering runner... failed runner=3clGLmm7 status=couldn't execute POST against https://gitlab.bakeneco.io/api/v4/runners: Post https://gitlab.bakeneco.io/api/v4/runners: dial tcp: lookup gitlab.bakeneco.io on 10.100.0.10:53: no such host
PANIC: Failed to register this runner. Perhaps you are having network problems
似乎需要在域名的DNS记录中注册Ingress LoadBalancer的地址。
由于只有从Helm install中才能知道这一点，所以我会在这个阶段确认并注册。
> kubectl describe service gitlab-nginx-ingress-controller | grep Ingress
LoadBalancer Ingress: aa03b57e68a6711e88b9202b5389a29a-974622501.us-west-2.elb.amazonaws.com
DNS记录登记后，稍等一会儿就会启动。
> kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default certmanager-gitlab-54467869c4-c7q6t 2/2 Running 0 44m
default gitlab-gitaly-0 1/1 Running 0 44m
default gitlab-gitlab-runner-845c5b46d8-9vwjm 1/1 Running 13 44m
default gitlab-gitlab-shell-7d8cd44948-fxxls 1/1 Running 0 44m
default gitlab-gitlab-shell-7d8cd44948-lccrx 1/1 Running 0 44m
default gitlab-issuer.1-2xqhq 0/1 Completed 0 44m
default gitlab-migrations.1-zs7tj 0/1 Completed 0 44m
default gitlab-minio-99bff897b-6cplp 1/1 Running 0 44m
default gitlab-minio-create-buckets.1-97wvg 0/1 Completed 0 44m
default gitlab-nginx-ingress-controller-65d58cbf4d-5zxlm 1/1 Running 0 44m
default gitlab-nginx-ingress-controller-65d58cbf4d-kp9fg 1/1 Running 0 44m
default gitlab-nginx-ingress-controller-65d58cbf4d-xttvr 1/1 Running 0 44m
default gitlab-nginx-ingress-default-backend-699b9476dd-5kjjt 1/1 Running 0 44m
default gitlab-nginx-ingress-default-backend-699b9476dd-jx86w 1/1 Running 0 44m
default gitlab-postgresql-5578b89f58-4jp9k 2/2 Running 0 44m
default gitlab-prometheus-server-847c8bb76-9jqm6 2/2 Running 0 44m
default gitlab-redis-6b8b6dbfd9-bs24z 2/2 Running 0 44m
default gitlab-registry-7f4b9ccfc8-k4zsj 1/1 Running 0 44m
default gitlab-registry-7f4b9ccfc8-kf6sm 1/1 Running 0 44m
default gitlab-sidekiq-all-in-1-58c996c9fb-mrcd5 1/1 Running 0 44m
default gitlab-task-runner-866bc87864-tgg4p 1/1 Running 0 44m
default gitlab-unicorn-5c69b9487c-j5klw 1/1 Running 0 44m
default gitlab-unicorn-5c69b9487c-v4pnq 1/1 Running 0 44m
kube-system aws-node-d2v4m 1/1 Running 1 4h
kube-system aws-node-fbcr6 1/1 Running 0 4h
kube-system aws-node-ggnps 1/1 Running 1 4h
kube-system aws-node-s2mnr 1/1 Running 0 4h
kube-system aws-node-z274r 1/1 Running 0 4h
kube-system kube-dns-7cc87d595-kdkcv 3/3 Running 0 4h
kube-system kube-proxy-5m9gf 1/1 Running 0 4h
kube-system kube-proxy-8fh9v 1/1 Running 0 4h
kube-system kube-proxy-ggk7j 1/1 Running 0 4h
kube-system kube-proxy-qngg2 1/1 Running 0 4h
kube-system kube-proxy-rxhjn 1/1 Running 0 4h
kube-system tiller-deploy-f5597467b-b2c5c 1/1 Running 0 4h
试着登录到GitLab实例时，似乎使用参数传递的密码不起作用。
我们将确认以下设置的密码并进行登录。
> kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode
nyanyanya
默认情况下，GitLab Shared Runners和Container Registry已启用HTTPS。
