Prometheus(普罗米修斯)——适合k8s和docker的监控系统
<p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><span class="bjh-p" style="line-height: 1.8;">Prometheus(普罗米修斯)是一套开源的监控&报警&时间序列数据库的组合.由SoundCloud公司开发。</span></p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><span class="bjh-p" style="line-height: 1.8;">Prometheus基本原理是通过HTTP协议周期性抓取被监控组件的状态,这样做的好处是任意组件只要提供HTTP接口就可以接入监控系统,不需要任何SDK或者其他的集成过程。这样做非常适合虚拟化环境比如VM或者Docker 。</span></p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><span class="bjh-p" style="line-height: 1.8;">Prometheus应该是为数不多的适合Docker、Mesos、Kubernetes环境的监控系统之一。近几年随着k8s的流行,prometheus成为了一个越来越流行的监控工具。</span></p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><span class="bjh-p" style="line-height: 1.8;">而且Prometheus是开源的,真是我等伸手党的福音。</span></p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><span class="bjh-h3" style="line-height: 1.8;">架构</span></p><div class="img-container" style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><img class="large" src="https://ss1.baidu.com/6ONXsjip0QIZ8tyhnq/it/u=3950099485,2889358013&fm=173&app=49&f=JPEG?w=640&h=384&s=85967533139B41C842C954DB000010B0" alt="" data-loadfunc="0" data-loaded="0" style="border: 0px; max-width: 100%; height: auto;"></div><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><span class="bjh-p" style="line-height: 1.8;">Prometheus可以做什么</span></p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><span class="bjh-p" style="line-height: 1.8;">在业务层用作埋点系统 Prometheus支持各个主流开发语言(Go,java,python,ruby官方提供客户端,其他语言有第三方开源客户端)。我们可以通过客户端方面的对核心业务进行埋点。如下单流程、添加购物车流程。在应用层用作应用监控系统 一些主流应用可以通过官方或第三方的导出器,来对这些应用做核心指标的收集。如redis,mysql。在系统层用作系统监控 除了常用软件, prometheus也有相关系统层和网络层exporter,用以监控服务器或网络。集成其他的监控 prometheus还可以通过各种exporte,集成其他的监控系统,收集监控数据,如AWS CloudWatch,JMX,Pingdom等等。不要用Prometheus做什么</span></p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><span class="bjh-p" style="line-height: 1.8;">prometheus也提供了Grok exporter等工具可以用来读取日志,但是prometheus是监控系统,不是日志系统。应用的日志还是应该走ELK等工具栈。</span></p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><span class="bjh-h3" style="line-height: 1.8;">grafana</span></p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </p><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><span class="bjh-p" style="line-height: 1.8;">一般配合grafana做前端展示</span></p><div class="img-container" style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><img class="large" src="https://ss2.baidu.com/6ONYsjip0QIZ8tyhnq/it/u=474629039,1962848489&fm=173&app=49&f=JPEG?w=640&h=344&s=6552AB665FA883685ED4A41B010080C1" alt="" data-loadfunc="0" data-loaded="0" style="border: 0px; max-width: 100%; height: auto;"></div><p style="margin: 10px auto; color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </p><div class="img-container" style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"><img class="large" src="https://ss0.baidu.com/6ONWsjip0QIZ8tyhnq/it/u=2111410731,1071095943&fm=173&app=49&f=JPEG?w=640&h=494&s=28C2B34ADBADB76C4EE509160300C0C2" alt="" data-loadfunc="0" data-loaded="0" style="border: 0px; max-width: 100%; height: auto;"></div><div class="img-container" style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif;"> </div><div class="img-container" style=""><h1 class="postTitle" style="color: rgb(0, 0, 0); font-family: verdana, Arial, Helvetica, sans-serif; margin-top: 10px; margin-bottom: 10px; border-bottom: 1px solid rgb(221, 221, 221); font-size: 28px; line-height: 1.5;"><a id="cb_post_title_url" class="postTitle2" href="https://www.cnblogs.com/terrycy/p/10058944.html" style="text-decoration-line: none; color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);">Kubernetes使用prometheus+grafana做一个简单的监控方案</a></h1><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">本文介绍在k8s集群中使用node-exporter、prometheus、grafana对集群进行监控。<br>其实现原理有点类似ELK、EFK组合。node-exporter组件负责收集节点上的metrics监控数据,并将数据推送给prometheus, prometheus负责存储这些数据,grafana将这些数据通过网页以图形的形式展现给用户。</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">在开始之前有必要了解下Prometheus是什么?<br>Prometheus (中文名:普罗米修斯)是由 SoundCloud 开发的开源监控报警系统和时序列数据库(TSDB).自2012年起,许多公司及组织已经采用 Prometheus,并且该项目有着非常活跃的开发者和用户社区.现在已经成为一个独立的开源项目。Prometheus 在2016加入 CNCF ( Cloud Native Computing Foundation ), 作为在 kubernetes 之后的第二个由基金会主持的项目。 Prometheus 的实现参考了Google内部的监控实现,与源自Google的Kubernetes结合起来非常合适。另外相比influxdb的方案,性能更加突出,而且还内置了报警功能。它针对大规模的集群环境设计了拉取式的数据采集方式,只需要在应用里面实现一个metrics接口,然后把这个接口告诉Prometheus就可以完成数据采集了,下图为prometheus的架构图。</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><img src="https://img2018.cnblogs.com/blog/1005768/201812/1005768-20181203155201461-1231183520.png" alt="" class="medium-zoom-image" style="border: 0px; max-width: 100%; cursor: zoom-in; height: auto; transition: transform 0.3s cubic-bezier(0.2, 0, 0.2, 1) 0s !important;"></p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">Prometheus的特点:<br>1、多维数据模型(时序列数据由metric名和一组key/value组成)<br>2、在多维度上灵活的查询语言(PromQl)<br>3、不依赖分布式存储,单主节点工作.<br>4、通过基于HTTP的pull方式采集时序数据<br>5、可以通过中间网关进行时序列数据推送(pushing)<br>6、目标服务器可以通过发现服务或者静态配置实现<br>7、多种可视化和仪表盘支持</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">prometheus 相关组件,Prometheus生态系统由多个组件组成,其中许多是可选的:<br>1、Prometheus 主服务,用来抓取和存储时序数据<br>2、client library 用来构造应用或 exporter 代码 (go,java,python,ruby)<br>3、push 网关可用来支持短连接任务<br>4、可视化的dashboard (两种选择,promdash 和 grafana.目前主流选择是 grafana.)<br>4、一些特殊需求的数据出口(用于HAProxy, StatsD, Graphite等服务)<br>5、实验性的报警管理端(alartmanager,单独进行报警汇总,分发,屏蔽等 )</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">promethues 的各个组件基本都是用 golang 编写,对编译和部署十分友好.并且没有特殊依赖.基本都是独立工作。<br><br></p><h3 style="color: rgb(0, 0, 0); font-family: verdana, Arial, Helvetica, sans-serif; margin-top: 10px; margin-bottom: 10px; font-size: 16px; line-height: 1.5;">部署</h3><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">现在我们正式开始部署工作。这里假设你已经为你的K8S集群部署过kube-dns或者coredns了。<br><strong>一、环境介绍</strong><br>操作系统环境:centos linux 7.5 64bit<br>K8S软件版本: 1.12.3<br>Master节点IP: 10.40.0.151/24</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">Node01节点IP: 10.40.0.152/24</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">Node02节点IP: 10.40.0.153/24</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><strong>二、采用daemonset方式部署node-exporter组件</strong></p><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><pre style="overflow: auto; font-family: "Courier New" !important;">cat node-exporter.yamlapiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: node-exporter
namespace: kube-system
labels:
k8s-app: node-exporter
spec:
template:
metadata:
labels:
k8s-app: node-exporter
spec:
containers:
- image: prom/node-exporter
name: node-exporter
ports:
- containerPort: 9100
protocol: TCP
name: http
---
apiVersion: v1
kind: Service
metadata:
labels:
k8s-app: node-exporter
name: node-exporter
namespace: kube-system
spec:
ports:
- name: http
port: 9100
nodePort: 31672
protocol: TCP
type: NodePort
selector:
k8s-app: node-exporter</pre><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div></div><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><strong>三、部署prometheus组件</strong></p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">1、rbac文件</p><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><pre style="overflow: auto; font-family: "Courier New" !important;">cat rbac-setup.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: prometheus
rules:
- apiGroups: [""]
resources:
- nodes
- nodes/proxy
- services
- endpoints
- pods
verbs: ["get", "list", "watch"]
- apiGroups:
- extensions
resources:
- ingresses
verbs: ["get", "list", "watch"]
- nonResourceURLs: ["/metrics"]
verbs: ["get"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: prometheus
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: prometheus
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: prometheus
subjects:
- kind: ServiceAccount
name: prometheus
namespace: kube-system</pre><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div></div><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">2、以configmap的形式管理prometheus组件的配置文件</p><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><pre style="overflow: auto; font-family: "Courier New" !important;">cat configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: prometheus-config
namespace: kube-system
data:
prometheus.yml: |
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: 'kubernetes-apiservers'
kubernetes_sd_configs:
- role: endpoints
scheme: https
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
relabel_configs:
- source_labels:
action: keep
regex: default;kubernetes;https
- job_name: 'kubernetes-nodes'
kubernetes_sd_configs:
- role: node
scheme: https
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels:
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/${1}/proxy/metrics
- job_name: 'kubernetes-cadvisor'
kubernetes_sd_configs:
- role: node
scheme: https
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels:
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/${1}/proxy/metrics/cadvisor
- job_name: 'kubernetes-service-endpoints'
kubernetes_sd_configs:
- role: endpoints
relabel_configs:
- source_labels:
action: keep
regex: true
- source_labels:
action: replace
target_label: __scheme__
regex: (https?)
- source_labels:
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels:
action: replace
target_label: __address__
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
- action: labelmap
regex: __meta_kubernetes_service_label_(.+)
- source_labels:
action: replace
target_label: kubernetes_namespace
- source_labels:
action: replace
target_label: kubernetes_name
- job_name: 'kubernetes-services'
kubernetes_sd_configs:
- role: service
metrics_path: /probe
params:
module:
relabel_configs:
- source_labels:
action: keep
regex: true
- source_labels:
target_label: __param_target
- target_label: __address__
replacement: blackbox-exporter.example.com:9115
- source_labels:
target_label: instance
- action: labelmap
regex: __meta_kubernetes_service_label_(.+)
- source_labels:
target_label: kubernetes_namespace
- source_labels:
target_label: kubernetes_name
- job_name: 'kubernetes-ingresses'
kubernetes_sd_configs:
- role: ingress
relabel_configs:
- source_labels:
action: keep
regex: true
- source_labels:
regex: (.+);(.+);(.+)
replacement: ${1}://${2}${3}
target_label: __param_target
- target_label: __address__
replacement: blackbox-exporter.example.com:9115
- source_labels:
target_label: instance
- action: labelmap
regex: __meta_kubernetes_ingress_label_(.+)
- source_labels:
target_label: kubernetes_namespace
- source_labels:
target_label: kubernetes_name
- job_name: 'kubernetes-pods'
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels:
action: keep
regex: true
- source_labels:
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels:
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
target_label: __address__
- action: labelmap
regex: __meta_kubernetes_pod_label_(.+)
- source_labels:
action: replace
target_label: kubernetes_namespace
- source_labels:
action: replace
target_label: kubernetes_pod_name</pre><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div></div><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">3、Prometheus deployment 文件</p><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><pre style="overflow: auto; font-family: "Courier New" !important;">cat prometheus.yaml
apiVersion: apps/v1beta2
kind: Deployment
metadata:
labels:
name: prometheus-deployment
name: prometheus
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: prometheus
template:
metadata:
labels:
app: prometheus
spec:
containers:
- image: prom/prometheus:v2.0.0
name: prometheus
command:
- "/bin/prometheus"
args:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus"
- "--storage.tsdb.retention=24h"
ports:
- containerPort: 9090
protocol: TCP
volumeMounts:
- mountPath: "/prometheus"
name: data
- mountPath: "/etc/prometheus"
name: config-volume
resources:
requests:
cpu: 100m
memory: 100Mi
limits:
cpu: 500m
memory: 2500Mi
serviceAccountName: prometheus
volumes:
- name: data
emptyDir: {}
- name: config-volume
configMap:
name: prometheus-config
---
kind: Service
apiVersion: v1
metadata:
labels:
app: prometheus
name: prometheus
namespace: kube-system
spec:
type: NodePort
ports:
- port: 9090
targetPort: 9090
nodePort: 30003
selector:
app: prometheus</pre><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div></div><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">4、通过上述yaml文件创建相应的对象</p><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><pre style="overflow: auto; font-family: "Courier New" !important;">kubectl create -f node-exporter.yaml
kubectl create -f rbac-setup.yaml
kubectl create -f configmap.yaml
kubectl create -f promethues.yaml</pre></div><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">5、查看相关pod和service</p><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><pre style="overflow: auto; font-family: "Courier New" !important;"># kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-779dfc4d59-rtpmk 1/1 Running 0 48s
kubernetes-dashboard-b54f75c69-tnn4h 1/1 Running 0 90m
node-exporter-sflqg 1/1 Running 0 9m44s
node-exporter-xfsf8 1/1 Running 0 9m44s
prometheus-58dc44f44c-z86rv 1/1 Running 0 8m44s</pre><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div></div><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div><pre style="overflow: auto; font-family: "Courier New" !important;"># kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.250.0.2 <none> 53/UDP,53/TCP 117s
kubernetes-dashboard NodePort 10.250.1.89 <none> 443:38443/TCP 102m
node-exporter NodePort 10.250.0.165 <none> 9100:31672/TCP 10m
prometheus NodePort 10.250.0.53 <none> 9090:30003/TCP 9m53s</pre><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div></div><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">6、Node-exporter对应的nodeport端口为31672,通过访问http://10.40.0.152:31672/metrics 可以看到对应的metrics</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><img src="https://img2018.cnblogs.com/blog/1005768/201812/1005768-20181203165506410-1781651323.png" alt="" class="medium-zoom-image" style="border: 0px; max-width: 100%; cursor: zoom-in; height: auto; transition: transform 0.3s cubic-bezier(0.2, 0, 0.2, 1) 0s !important;"></p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">7、prometheus对应的nodeport端口为30003,通过访问http://10.40.0.152:30003/targets 可以看到prometheus已经成功连接上了k8s的apiserver</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><img src="https://img2018.cnblogs.com/blog/1005768/201812/1005768-20181203170206302-878216418.png" alt="" class="medium-zoom-image" style="border: 0px; max-width: 100%; cursor: zoom-in; height: auto; transition: transform 0.3s cubic-bezier(0.2, 0, 0.2, 1) 0s !important;"></p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">8、在prometheus的WEB界面上提供了基本的查询K8S集群中每个POD的CPU使用情况,可以使用如下查询条件查询:</p><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><pre style="overflow: auto; font-family: "Courier New" !important;">sum by (pod_name)( rate(container_cpu_usage_seconds_total{image!="", pod_name!=""} ) )</pre></div><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><img src="https://img2018.cnblogs.com/blog/1005768/201812/1005768-20181203184148337-819198277.png" alt="" class="medium-zoom-image" style="border: 0px; max-width: 100%; cursor: zoom-in; height: auto; transition: transform 0.3s cubic-bezier(0.2, 0, 0.2, 1) 0s !important;"></p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">上述的查询有出现数据,说明node-exporter往prometheus中写入数据正常,接下来我们就可以部署grafana组件,实现更友好的webui展示数据了。</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><strong>五、部署grafana组件</strong><br>1、grafana deployment配置文件</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">cat grafana.yaml</p><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><pre style="overflow: auto; font-family: "Courier New" !important;">apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: grafana-core
namespace: kube-system
labels:
app: grafana
component: core
spec:
replicas: 1
template:
metadata:
labels:
app: grafana
component: core
spec:
containers:
- image: grafana/grafana:5.0.0
name: grafana-core
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 100m
memory: 100Mi
requests:
cpu: 100m
memory: 100Mi
env:
- name: GF_AUTH_BASIC_ENABLED
value: "true"
- name: GF_AUTH_ANONYMOUS_ENABLED
value: "false"
readinessProbe:
httpGet:
path: /login
port: 3000
volumeMounts:
- name: grafana-persistent-storage
mountPath: /var
volumes:
- name: grafana-persistent-storage
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: grafana
namespace: kube-system
labels:
app: grafana
component: core
spec:
type: NodePort
ports:
- port: 3000
nodePort: 31000
selector:
app: grafana</pre><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div></div><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">部署grafana</p><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><pre style="overflow: auto; font-family: "Courier New" !important;">kubectl create -f grafana.yaml</pre></div><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">查看grafana pod和service</p><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><pre style="overflow: auto; font-family: "Courier New" !important;"># kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-779dfc4d59-rtpmk 1/1 Running 0 101m
grafana-core-6759c8945-5f4sv 1/1 Running 0 91m
kubernetes-dashboard-b54f75c69-tnn4h 1/1 Running 0 3h11m
node-exporter-sflqg 1/1 Running 0 110m
node-exporter-xfsf8 1/1 Running 0 110m
prometheus-58dc44f44c-z86rv 1/1 Running 0 109m</pre><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div></div><div class="cnblogs_code" style="color: rgb(0, 0, 0); font-family: "Courier New" !important; margin-top: 5px; margin-bottom: 5px; padding: 5px; background-color: rgb(245, 245, 245); border: 1px solid rgb(204, 204, 204); overflow: auto; font-size: 12px !important;"><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><pre style="overflow: auto; font-family: "Courier New" !important;"># kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
grafana NodePort 10.250.1.230 <none> 3000:31000/TCP 93m
kube-dns ClusterIP 10.250.0.2 <none> 53/UDP,53/TCP 103m
kubernetes-dashboard NodePort 10.250.1.89 <none> 443:38443/TCP 3h23m
node-exporter NodePort 10.250.0.165 <none> 9100:31672/TCP 112m
prometheus NodePort 10.250.0.53 <none> 9090:30003/TCP 111m</pre><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border-bottom: 1px dotted rgb(51, 51, 51);"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="border-width: 0px; border-style: initial; max-width: 100%; height: auto;"></a></span></div><div class="cnblogs_code_toolbar" style="margin-top: 5px;"><span class="cnblogs_code_copy" style="padding-right: 5px; line-height: 1.8;"><a title="复制代码" style="color: rgb(0, 0, 0); border: none !important;"><img src="https://common.cnblogs.com/images/copycode.gif" alt="复制代码" style="max-width: 100%; height: auto;"></a></span></div></div><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">可以看到grafana nodeport端口为31000,可使用nodeip+nodeport的方式访问grafana http://10.40.0.152:31000</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><img src="https://img2018.cnblogs.com/blog/1005768/201812/1005768-20181203184436237-418875961.png" alt="" class="medium-zoom-image" style="border: 0px; max-width: 100%; cursor: zoom-in; height: auto; transition: transform 0.3s cubic-bezier(0.2, 0, 0.2, 1) 0s !important;"></p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">默认用户名和密码都是admin</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><img src="https://img2018.cnblogs.com/blog/1005768/201812/1005768-20181203184529615-1390942632.png" alt="" class="medium-zoom-image" style="border: 0px; max-width: 100%; cursor: zoom-in; height: auto; transition: transform 0.3s cubic-bezier(0.2, 0, 0.2, 1) 0s !important;"></p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">配置数据库源为prometheus,导入面板</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> <img src="https://img2018.cnblogs.com/blog/1005768/201812/1005768-20181203184929559-14342691.png" alt="" class="medium-zoom-image" style="border: 0px; max-width: 100%; cursor: zoom-in; height: auto; transition: transform 0.3s cubic-bezier(0.2, 0, 0.2, 1) 0s !important;"></p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">可以直接输入模板编号315在线导入,或者下载好对应的json模板文件本地导入,面板模板下载地址https://grafana.com/dashboards/315</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><img src="https://img2018.cnblogs.com/blog/1005768/201812/1005768-20181203185014385-1684865615.png" alt="" class="medium-zoom-image" style="border: 0px; max-width: 100%; cursor: zoom-in; height: auto; transition: transform 0.3s cubic-bezier(0.2, 0, 0.2, 1) 0s !important;"></p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;">在线加载模板OK,选择prometheus数据库实例</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><img src="https://img2018.cnblogs.com/blog/1005768/201812/1005768-20181203185040024-495459702.png" alt="" class="medium-zoom-image" style="border: 0px; max-width: 100%; cursor: zoom-in; height: auto; transition: transform 0.3s cubic-bezier(0.2, 0, 0.2, 1) 0s !important;"></p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> </p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> 大功告成,可以看到炫酷的监控页面了。</p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"><img src="https://img2018.cnblogs.com/blog/1005768/201812/1005768-20181203185106777-1932893079.png" alt="" class="medium-zoom-image" style="border: 0px; max-width: 100%; cursor: zoom-in; height: auto; transition: transform 0.3s cubic-bezier(0.2, 0, 0.2, 1) 0s !important;"></p><p style="color: rgb(51, 51, 51); font-family: verdana, Arial, Helvetica, sans-serif; margin: 10px auto;"> 来源:https://www.cnblogs.com/zgq123456/articles/10847262.html</p><p style="margin: 10px auto;"><font color="#333333" face="verdana, Arial, Helvetica, sans-serif">最新 git clone https://github.com/ielepro/k8s-prometheus-grafana</font></p></div><p></p>
页:
[1]