Kubernetes 如何使用Istio'；谁让普罗米修斯配置库伯内特斯hpa？

我们有一个Istio集群，我们正在尝试为Kubernetes配置水平吊舱自动缩放。我们希望使用请求计数作为hpa的自定义指标。我们如何利用伊斯蒂奥的普罗米修斯达到同样的目的呢？

这个问题比我想象的要复杂得多，但最后我给出了答案

首先，您需要配置应用程序以提供自定义指标。它位于开发应用程序端。下面是一个示例，如何使用Go语言制作：

其次，您需要定义应用程序（或Pod，或任何您想要的）的部署并部署到Kubernetes，例如：

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: podinfo
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: podinfo
      annotations:
        prometheus.io/scrape: 'true'
    spec:
      containers:
      - name: podinfod
        image: stefanprodan/podinfo:0.0.1
        imagePullPolicy: Always
        command:
          - ./podinfo
          - -port=9898
          - -logtostderr=true
          - -v=2
        volumeMounts:
          - name: metadata
            mountPath: /etc/podinfod/metadata
            readOnly: true
        ports:
        - containerPort: 9898
          protocol: TCP
        readinessProbe:
          httpGet:
            path: /readyz
            port: 9898
          initialDelaySeconds: 1
          periodSeconds: 2
          failureThreshold: 1
        livenessProbe:
          httpGet:
            path: /healthz
            port: 9898
          initialDelaySeconds: 1
          periodSeconds: 3
          failureThreshold: 2
        resources:
          requests:
            memory: "32Mi"
            cpu: "1m"
          limits:
            memory: "256Mi"
            cpu: "100m"
      volumes:
        - name: metadata
          downwardAPI:
            items:
              - path: "labels"
                fieldRef:
                  fieldPath: metadata.labels
              - path: "annotations"
                fieldRef:
                  fieldPath: metadata.annotations
---
apiVersion: v1
kind: Service
metadata:
  name: podinfo
  labels:
    app: podinfo
spec:
  type: NodePort
  ports:
    - port: 9898
      targetPort: 9898
      nodePort: 31198
      protocol: TCP
  selector:
    app: podinfo

注意字段

注释：prometheus.io/scrap:“true”

。需要请求普罗米修斯从资源中读取度量。还要注意的是，还有两个注释具有默认值；但如果在应用程序中更改它们，则需要使用正确的值添加它们：

• prometheus.io/path

  ：如果度量路径不是/metrics，请使用此注释定义它

• prometheus.io/port

  ：在指示的端口而不是pod声明的端口上刮取pod（如果未声明任何端口，则默认为无端口目标）

接下来，Istio中的普罗米修斯使用其自己的修改后的Istio配置，默认情况下，它跳过POD中的自定义度量。因此，您需要稍微修改它。 在我的例子中，我从Istio的Prometheus配置中获取了Pod度量的配置，并仅修改了Pod的配置：

kubectl edit configmap -n istio-system prometheus

我根据前面提到的示例更改了标签的顺序：

# pod's declared ports (default is a port-free target if none are declared).
- job_name: 'kubernetes-pods'
  # if you want to use metrics on jobs, set the below field to
  # true to prevent Prometheus from setting the `job` label
  # automatically.
  honor_labels: false
  kubernetes_sd_configs:
  - role: pod
  # skip verification so you can do HTTPS to pods
  tls_config:
    insecure_skip_verify: true
  # make sure your labels are in order
  relabel_configs:
  # these labels tell Prometheus to automatically attach source
  # pod and namespace information to each collected sample, so
  # that they'll be exposed in the custom metrics API automatically.
  - source_labels: [__meta_kubernetes_namespace]
    action: replace
    target_label: namespace
  - source_labels: [__meta_kubernetes_pod_name]
    action: replace
    target_label: pod
  # these labels tell Prometheus to look for
  # prometheus.io/{scrape,path,port} annotations to configure
  # how to scrape
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
    action: keep
    regex: true
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
    action: replace
    target_label: __metrics_path__
    regex: (.+)
  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
    action: replace
    regex: ([^:]+)(?::\d+)?;(\d+)
    replacement: $1:$2
    target_label: __address__
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme]
    action: replace
    target_label: __scheme__

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: podinfo
spec:
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: podinfo
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Pods
    pods:
      metricName: http_requests
      targetAverageValue: 10

此后，普罗米修斯出现了自定义指标。但是，在更改普罗米修斯配置时要小心，因为Istio所需的某些指标可能会消失，请仔细检查一切

现在是安装的时候了

• 下载存储库
• 在文件

  /deploy/manifests/custom metrics apiserver deployment.yaml

  中更改Prometheus服务器的地址。例如，

  ——普罗米修斯url=http://prometheus.istio-system:9090/

• 运行命令

  kubectl apply-f/deploy/manifests

  一段时间后，

  custom.metrics.k8s.io/v1beta1

  应该出现在命令“kubectl-api-vesions”的输出中

另外，使用命令

kubectl get--raw”/API/custom.metrics.k8s.io/v1beta1“| jq.

和

kubectl get--raw”/API/custom.metrics.k8s.io/v1beta1/namespace/default/pods/*/http|u requests“| jq.

最后一个的输出应如以下示例所示：

{
  "kind": "MetricValueList",
  "apiVersion": "custom.metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/%2A/http_requests"
  },
  "items": [
    {
      "describedObject": {
        "kind": "Pod",
        "namespace": "default",
        "name": "podinfo-6b86c8ccc9-kv5g9",
        "apiVersion": "/__internal"
          },
          "metricName": "http_requests",
          "timestamp": "2018-01-10T16:49:07Z",
          "value": "901m"    },
        {
      "describedObject": {
        "kind": "Pod",
        "namespace": "default",
        "name": "podinfo-6b86c8ccc9-nm7bl",
        "apiVersion": "/__internal"
      },
      "metricName": "http_requests",
      "timestamp": "2018-01-10T16:49:07Z",
      "value": "898m"
    }
  ]
}

如果确实如此，您可以进入下一步。如果没有，请查看CustomMetrics

kubectl get--raw”/api/custom.metrics.k8s.io/v1beta1“| jq.|grep“pods/”

和对于http_请求

kubectl get--raw”/api/custom.metrics.k8s.io/v1beta1“| jq.|grep“http”

。MetricNames是根据普罗米修斯从豆荚中收集的指标生成的，如果它们是空的，你需要朝那个方向看

最后一步是配置HPA并对其进行测试。因此，在我的例子中，我为之前定义的podinfo应用程序创建了HPA：

# pod's declared ports (default is a port-free target if none are declared).
- job_name: 'kubernetes-pods'
  # if you want to use metrics on jobs, set the below field to
  # true to prevent Prometheus from setting the `job` label
  # automatically.
  honor_labels: false
  kubernetes_sd_configs:
  - role: pod
  # skip verification so you can do HTTPS to pods
  tls_config:
    insecure_skip_verify: true
  # make sure your labels are in order
  relabel_configs:
  # these labels tell Prometheus to automatically attach source
  # pod and namespace information to each collected sample, so
  # that they'll be exposed in the custom metrics API automatically.
  - source_labels: [__meta_kubernetes_namespace]
    action: replace
    target_label: namespace
  - source_labels: [__meta_kubernetes_pod_name]
    action: replace
    target_label: pod
  # these labels tell Prometheus to look for
  # prometheus.io/{scrape,path,port} annotations to configure
  # how to scrape
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
    action: keep
    regex: true
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
    action: replace
    target_label: __metrics_path__
    regex: (.+)
  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
    action: replace
    regex: ([^:]+)(?::\d+)?;(\d+)
    replacement: $1:$2
    target_label: __address__
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme]
    action: replace
    target_label: __scheme__

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: podinfo
spec:
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: podinfo
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Pods
    pods:
      metricName: http_requests
      targetAverageValue: 10

并使用简单的Go应用程序测试负载：

#install hey
go get -u github.com/rakyll/hey
#do 10K requests rate limited at 25 QPS
hey -n 10000 -q 5 -c 5 http://<K8S-IP>:31198/healthz

#安装
去获取-u github.com/rakyll/hey
#是否将10K请求速率限制为25 QPS
嘿-n 10000-q 5-c 5 http://:31198/healthz

一段时间后，我通过使用命令

kubectl descripe hpa

和

kubectl get hpa

我使用了文章中关于创建自定义度量的说明

将所有有用的链接放在一个位置：

• -向应用程序添加度量的示例
• -为普罗米修斯创建自定义指标的示例（与上述文章相同）

---

您可以使用Istio的Prometeus为Kubernetes HPA提供的自定义指标。我将尝试复制此案例，并返回给您一个答案。@ArtemGolenyaev您能够复制此案例吗？很抱歉延迟，我将很快回复