Kubernetes 绕过pod突变钩子提供的资源
我们有气流在kubernetes上运行。以下是我的Kubernetes 绕过pod突变钩子提供的资源,kubernetes,airflow,kubernetespodoperator,Kubernetes,Airflow,Kubernetespodoperator,我们有气流在kubernetes上运行。以下是我的气流定位设置: airflowLocalSettings: | from airflow.contrib.kubernetes.pod import Pod, Resources from airflow.configuration import conf def pod_mutation_hook(pod: Pod): pod.labels.update({"app":
气流定位设置:
airflowLocalSettings: |
from airflow.contrib.kubernetes.pod import Pod, Resources
from airflow.configuration import conf
def pod_mutation_hook(pod: Pod):
pod.labels.update({"app": "airflow-pod"})
pod.annotations.update({"iam.amazonaws.com/role": "role"})
pod.tolerations += [{"key": "spotInstance", "operator": "Exists"}]
pod.resources = Resources(limit_memory = "512Mi", limit_cpu = "300m")
pod.affinity.update({
"nodeAffinity": {
"preferredDuringSchedulingIgnoredDuringExecution": [
{"weight": 100, "preference": {
"matchExpressions": [{
"key": "role.node.kubernetes.io/spot-worker",
"operator": "In",
"values": ["spot-worker"]
}]
}}
]
}
})
我想启动一些选择性任务吊舱,这些吊舱有自己的资源通过KubernetesPodOperator
中的resources
参数传递。
现在的问题是资源被覆盖,它从pod\u-hook
获取默认资源
我们如何绕过pod_变异资源,让这些pod拥有自己的资源设置?我无法从pod\u hook
中删除资源设置,因为其他pod也在使用它。我通过调整pod\u mutaion\u hook代码解决了这个问题,如下所示
resources = pod.resources
is_class = isinstance(resources, Resources)
if is_class:
resource_flag = resources.is_empty_resource_request()
else:
resource_flag = False
if resource_flag:
pod.resources = Resources(limit_memory="512Mi", limit_cpu="300m")
基本上,我正在检查pod中是否已经有可用的资源,然后采取行动