使用Helm Chart在Minikube上部署Prometheus和Grafana
尝试使用Helm在Minikube上安装和运行Prometheus和Grafana。
使用以下的Helm图表进行安装。
-
- https://github.com/helm/charts/tree/master/stable/prometheus
- https://github.com/helm/charts/tree/master/stable/grafana
コンポーネントバージョンMinikubev0.33.1Kubernetesv1.13.2Prometheusチャート8.4.5Prometheus2.6.1Grafanaチャート1.25.1Grafana5.4.3
备好
启动Minikube。
minikube start
启用Minikube的入口附加组件。
minikube addons enable ingress
确认 Minikube 的 IP 地址。
minikube ip
在/etc/hosts文件中添加Minikube的IP地址。为了在通过Ingress访问各个组件时区分路径，分割虚拟主机会更方便，因为要保持路径重写和重定向的一致性比较困难。
192.168.99.106 alertmanager.minikube prometheus.minikube grafana.minikube
创建一个用于部署Prometheus和Grafana的命名空间。
kubectl create ns monitoring
运行 helm init 命令来初始化 Helm，并将 tiller 部署到集群中。
helm init
普罗米修斯
使用helm repo命令查看已注册的仓库。
$ helm repo list
NAME URL
stable https://kubernetes-charts.storage.googleapis.com
local http://127.0.0.1:8879/charts
$
使用helm search命令来确认Prometheus的图表。
$ helm search prometheus
NAME CHART VERSION APP VERSION DESCRIPTION
stable/prometheus 8.4.5 2.6.1 Prometheus is a monitoring system and time seri...
stable/prometheus-adapter v0.4.1 v0.4.1 A Helm chart for k8s prometheus adapter
stable/prometheus-blackbox-exporter 0.2.0 0.12.0 Prometheus Blackbox Exporter
stable/prometheus-cloudwatch-exporter 0.2.1 0.5.0 A Helm chart for prometheus cloudwatch-exporter
stable/prometheus-consul-exporter 0.1.2 0.4.0 A Helm chart for the Prometheus Consul Exporter
stable/prometheus-couchdb-exporter 0.1.0 1.0 A Helm chart to export the metrics from couchdb...
stable/prometheus-mysql-exporter 0.2.1 v0.11.0 A Helm chart for prometheus mysql exporter with...
stable/prometheus-node-exporter 1.1.0 0.17.0 A Helm chart for prometheus node-exporter
stable/prometheus-operator 1.8.0 0.26.0 Provides easy monitoring definitions for Kubern...
stable/prometheus-postgres-exporter 0.6.1 0.4.7 A Helm chart for prometheus postgres-exporter
stable/prometheus-pushgateway 0.3.0 0.6.0 A Helm chart for prometheus pushgateway
stable/prometheus-rabbitmq-exporter 0.3.0 v0.29.0 Rabbitmq metrics exporter for prometheus
stable/prometheus-redis-exporter 1.0.1 0.25.0 Prometheus exporter for Redis metrics
stable/prometheus-snmp-exporter 0.0.1 0.14.0 Prometheus SNMP Exporter
stable/prometheus-to-sd 0.1.1 0.2.2 Scrape metrics stored in prometheus format and ...
stable/elasticsearch-exporter 1.0.0 1.0.2 Elasticsearch stats exporter for Prometheus
stable/helm-exporter 0.1.0 0.1.0 Exports helm release stats to prometheus
stable/karma 1.1.9 v0.21 A Helm chart for Karma - an UI for Prometheus A...
stable/stackdriver-exporter 0.0.6 0.6.0 Stackdriver exporter for Prometheus
stable/weave-cloud 0.3.1 1.1.0 Weave Cloud is a add-on to Kubernetes which pro...
stable/kube-state-metrics 0.13.0 1.4.0 Install kube-state-metrics to generate and expo...
stable/mariadb 5.5.0 10.1.37 Fast, reliable, scalable, and easy to use open-...
$
使用 `helm inspect values` 命令来显示参数的默认值，并将其重定向到文件中保存。
helm inspect values stable/prometheus > prometheus-values.yaml
根据保存的文件和图表的自述文件，自定义设置值。删除使用默认值的部分，只保留自定义部分。本次操作如下所示。
-
- storageClassをstandardと指定
Minikubeではデフォルトでstorage-provisionerとdefault-storageclassのアドオンが有効になっていて、storageClassをstandardと指定することでPersistentVolumeを動的プロビジョニングさせることができる
性能情報の保管期間を12時間に指定
pushgatewayは使わないので無効化
alertmanagerとserver（Prometheus）にIngress経由でアクセスできるようにIngressを有効化し、それぞれの仮想ホストを指定
alertmanager:
ingress:
enabled: true
hosts:
- alertmanager.minikube
persistentVolume:
size: 1Gi
storageClass: "standard"
server:
ingress:
enabled: true
hosts:
- prometheus.minikube
persistentVolume:
size: 1Gi
storageClass: "standard"
retention: "12h"
pushgateway:
enabled: false
发布图表。
$ helm install --name prometheus --namespace monitoring -f prometheus-values.yaml stable/prometheus
NAME: prometheus
LAST DEPLOYED: Fri Feb 1 18:49:48 2019
NAMESPACE: monitoring
STATUS: DEPLOYED
RESOURCES:
==> v1beta1/DaemonSet
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
prometheus-node-exporter 1 1 0 1 0 <none> 1s
==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
prometheus-alertmanager alertmanager.minikube 80 1s
prometheus-server prometheus.minikube 80 1s
==> v1/PersistentVolumeClaim
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
prometheus-alertmanager Bound pvc-b68e41ef-2606-11e9-acbf-080027a3230f 1Gi RWO standard 1s
prometheus-server Bound pvc-b68ea305-2606-11e9-acbf-080027a3230f 1Gi RWO standard 1s
==> v1beta1/ClusterRole
NAME AGE
prometheus-kube-state-metrics 1s
prometheus-server 1s
==> v1beta1/ClusterRoleBinding
NAME AGE
prometheus-kube-state-metrics 1s
prometheus-server 1s
==> v1beta1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
prometheus-alertmanager 1 1 1 0 1s
prometheus-kube-state-metrics 1 1 1 0 1s
prometheus-server 1 1 1 0 1s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
prometheus-node-exporter-tl47v 0/1 ContainerCreating 0 1s
prometheus-alertmanager-77585bb469-9hggb 0/2 Pending 0 1s
prometheus-kube-state-metrics-68cfd7cbd9-4n7vr 0/1 Pending 0 1s
prometheus-server-5d8c6c-wv6fd 0/2 Init:0/1 0 1s
==> v1/ConfigMap
NAME DATA AGE
prometheus-alertmanager 1 1s
prometheus-server 3 1s
==> v1/ServiceAccount
NAME SECRETS AGE
prometheus-alertmanager 1 1s
prometheus-kube-state-metrics 1 1s
prometheus-node-exporter 1 1s
prometheus-server 1 1s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
prometheus-alertmanager ClusterIP 10.106.187.167 <none> 80/TCP 1s
prometheus-kube-state-metrics ClusterIP None <none> 80/TCP 1s
prometheus-node-exporter ClusterIP None <none> 9100/TCP 1s
prometheus-server ClusterIP 10.107.118.222 <none> 80/TCP 1s
NOTES:
The Prometheus server can be accessed via port 80 on the following DNS name from within your cluster:
prometheus-server.monitoring.svc.cluster.local
From outside the cluster, the server URL(s) are:
http://prometheus.minikube
The Prometheus alertmanager can be accessed via port 80 on the following DNS name from within your cluster:
prometheus-alertmanager.monitoring.svc.cluster.local
From outside the cluster, the alertmanager URL(s) are:
http://alertmanager.minikube
For more information on running Prometheus, visit:
https://prometheus.io/
$
确认能够分别访问Prometheus和Alertmanager。
Grafana葛拉芬纳
接下来安装Grafana。
使用helm search命令查看Grafana的图表。
$ helm search grafana
NAME CHART VERSION APP VERSION DESCRIPTION
stable/grafana 1.25.1 5.4.3 The leading tool for querying and visualizing t...
$
与Prometheus时一样，将参数的默认值保存在文件中。
helm inspect values stable/grafana > grafana-values.yaml
根据保存的文件和图表的 README 参考，自定义设置值。这次的设置如下。
-
- Ingress経由でアクセスできるようにIngressを有効化し、仮想ホストを指定
- storageClassNameを指定してPersistentVolumeを動的プロビジョニングさせる
ingress:
enabled: true
hosts:
- grafana.minikube
persistence:
enabled: true
storageClassName: "standard"
size: 1Gi
发布图表。
$ helm install --name grafana --namespace monitoring -f grafana-values.yaml stable/grafana
NAME: grafana
LAST DEPLOYED: Fri Feb 1 18:53:42 2019
NAMESPACE: monitoring
STATUS: DEPLOYED
RESOURCES:
==> v1/Secret
NAME TYPE DATA AGE
grafana Opaque 3 0s
==> v1/ConfigMap
NAME DATA AGE
grafana 1 0s
==> v1/ServiceAccount
NAME SECRETS AGE
grafana 1 0s
==> v1beta1/RoleBinding
NAME AGE
grafana 0s
==> v1beta1/PodSecurityPolicy
NAME DATA CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
grafana false RunAsAny RunAsAny RunAsAny RunAsAny false configMap,emptyDir,projected,secret,downwardAPI,persistentVolumeClaim
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
grafana-65f656d6c4-227tr 0/1 Init:0/1 0 0s
==> v1/PersistentVolumeClaim
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
grafana Bound pvc-41e671a5-2607-11e9-acbf-080027a3230f 1Gi RWO standard 0s
==> v1/ClusterRole
NAME AGE
grafana-clusterrole 0s
==> v1/ClusterRoleBinding
NAME AGE
grafana-clusterrolebinding 0s
==> v1beta1/Role
NAME AGE
grafana 0s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
grafana ClusterIP 10.108.154.68 <none> 80/TCP 0s
==> v1beta2/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
grafana 1 1 1 0 0s
==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
grafana grafana.minikube 80 0s
NOTES:
1. Get your 'admin' user password by running:
kubectl get secret --namespace monitoring grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
2. The Grafana server can be accessed via port 80 on the following DNS name from within your cluster:
grafana.monitoring.svc.cluster.local
From outside the cluster, the server URL(s) are:
http://grafana.minikube
3. Login with the password from step 1 and the username: admin
$
确认能够访问Grafana。
正如发布图表时的消息所指，可以使用以下命令获取登录密码。
kubectl get secret --namespace monitoring grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
Grafana和Prometheus的协作配合
在Grafana中登录并添加数据源。
选择Prometheus类型。
只需指定服务的名称和端口即可访问URL，在集群中访问，指定为http://prometheus-server。
如果可以添加数据源，那么接下来要做的就是添加仪表板。
添加Graph面板。
右键单击面板标题，打开菜单，并点击“编辑”。
在Metrics标签下，输入下面的查询以显示每个Pod在5分钟内的CPU使用率。
sum(rate(container_cpu_usage_seconds_total{image!=""}[5m])) by (namespace, pod_name) * 100
在格式目录中指定以下内容。
{{ namespace }}/{{ pod_name }}
在”一般”标签中指定面板的标题。
在Legend标签中选择将图例显示在右侧的表格中。
点击右侧的“×”按钮以关闭编辑面板，然后拖动面板右下角调整大小。
添加内存面板也要使用以下查询并完成以下仪表板。
sum(container_memory_usage_bytes{image!=""}) by (namespace, pod_name)
