Autoscaling
Horizontal Pod Autoscaling (HPA) 可以根据 CPU 使用率或应用自定义 metrics 自动扩展 Pod 数量(支持 replication controller、deployment 和 replica set )。
    控制管理器每隔 15s(可以通过 --horizontal-pod-autoscaler-sync-period 修改)查询 metrics 的资源使用情况
    支持三种 metrics 类型
      预定义 metrics(比如 Pod 的 CPU)以利用率的方式计算
      自定义的 Pod metrics,以原始值(raw value)的方式计算
      自定义的 object metrics
    支持两种 metrics 查询方式:Heapster 和自定义的 REST API
    支持多 metrics
注意:

API 版本对照表

Kubernetes 版本
autoscaling API 版本
支持的 metrics
v1.5+
autoscaling/v1
CPU
v1.6+
autoscaling/v2beta1
Memory及自定义

示例

1
# 创建 pod 和 service
2
$ kubectl run php-apache --image=k8s.gcr.io/hpa-example --requests=cpu=200m --expose --port=80
3
service "php-apache" created
4
deployment "php-apache" created
5
6
# 创建 autoscaler
7
$ kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10
8
deployment "php-apache" autoscaled
9
10
$ kubectl get hpa
11
NAME REFERENCE TARGET MINPODS MAXPODS REPLICAS AGE
12
php-apache Deployment/php-apache/scale 0% / 50% 1 10 1 18s
13
14
# 增加负载
15
$ kubectl run -i --tty load-generator --image=busybox /bin/sh
16
Hit enter for command prompt
17
$ while true; do wget -q -O- http://php-apache.default.svc.cluster.local; done
18
19
# 过一会就可以看到负载升高了
20
$ kubectl get hpa
21
NAME REFERENCE TARGET CURRENT MINPODS MAXPODS REPLICAS AGE
22
php-apache Deployment/php-apache/scale 305% / 50% 305% 1 10 1 3m
23
24
# autoscaler 将这个 deployment 扩展为 7 个 pod
25
$ kubectl get deployment php-apache
26
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
27
php-apache 7 7 7 7 19m
28
29
# 删除刚才创建的负载增加 pod 后会发现负载降低,并且 pod 数量也自动降回 1 个
30
$ kubectl get hpa
31
NAME REFERENCE TARGET MINPODS MAXPODS REPLICAS AGE
32
php-apache Deployment/php-apache/scale 0% / 50% 1 10 1 11m
33
34
$ kubectl get deployment php-apache
35
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
36
php-apache 1 1 1 1 27m
Copied!

自定义 metrics

使用方法
注:可以参考 k8s.io/metics 开发自定义的 metrics API server。
比如 HorizontalPodAutoscaler 保证每个 Pod 占用 50% CPU、1000pps 以及 10000 请求 / s:
HPA 示例
1
apiVersion: autoscaling/v1
2
kind: HorizontalPodAutoscaler
3
metadata:
4
name: php-apache
5
namespace: default
6
spec:
7
scaleTargetRef:
8
apiVersion: apps/v1beta1
9
kind: Deployment
10
name: php-apache
11
minReplicas: 1
12
maxReplicas: 10
13
metrics:
14
- type: Resource
15
resource:
16
name: cpu
17
targetAverageUtilization: 50
18
- type: Pods
19
pods:
20
metricName: packets-per-second
21
targetAverageValue: 1k
22
- type: Object
23
object:
24
metricName: requests-per-second
25
target:
26
apiVersion: extensions/v1beta1
27
kind: Ingress
28
name: main-route
29
targetValue: 10k
30
status:
31
observedGeneration: 1
32
lastScaleTime: <some-time>
33
currentReplicas: 1
34
desiredReplicas: 1
35
currentMetrics:
36
- type: Resource
37
resource:
38
name: cpu
39
currentAverageUtilization: 0
40
currentAverageValue: 0
Copied!

状态条件

v1.7+ 可以在客户端中看到 Kubernetes 为 HorizontalPodAutoscaler 设置的状态条件 status.conditions,用来判断 HorizontalPodAutoscaler 是否可以扩展(AbleToScale)、是否开启扩展(ScalingActive)以及是否受到限制(ScalingLimitted)。
1
$ kubectl describe hpa cm-test
2
Name: cm-test
3
Namespace: prom
4
Labels: <none>
5
Annotations: <none>
6
CreationTimestamp: Fri, 16 Jun 2017 18:09:22 +0000
7
Reference: ReplicationController/cm-test
8
Metrics: (current / target)
9
"http_requests" on pods: 66m / 500m
10
Min replicas: 1
11
Max replicas: 4
12
ReplicationController pods: 1 current / 1 desired
13
Conditions:
14
Type Status Reason Message
15
---- ------ ------ -------
16
AbleToScale True ReadyForNewScale the last scale time was sufficiently old as to warrant a new scale
17
ScalingActive True ValidMetricFound the HPA was able to successfully calculate a replica count from pods metric http_requests
18
ScalingLimited False DesiredWithinRange the desired replica count is within the acceptable range
19
Events:
Copied!

HPA 最佳实践

    为容器配置 CPU Requests
    HPA 目标设置恰当,如设置 70% 给容器和应用预留 30% 的余量
    保持 Pods 和 Nodes 健康(避免 Pod 频繁重建)
    保证用户请求的负载均衡
    使用 kubectl top nodekubectl top pod 查看资源使用情况

参考文档

最近更新 3mo ago