Amazon web services 未授权执行sts：假设身份-403

我一直在尝试使用k8s sig group提供的指南运行外部dns pod。我遵循了指南的每一步，得到了以下错误

time="2021-02-27T13:27:20Z" level=error msg="records retrieval failed: failed to list hosted zones: WebIdentityErr: failed to retrieve credentials\ncaused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity\n\tstatus code: 403, request id: 87a3ca86-ceb0-47be-8f90-25d0c2de9f48"

我已经使用Terraform创建了AWS IAM策略，并成功创建了该策略。除了我使用的服务帐户的IAM角色，其他一切都是通过Terraform旋转的

但后来我发现，使用awscli创建AWS IAM策略可以消除这个错误。因此，我删除了使用Terraform创建的策略，并使用awscli重新创建了它。然而，它却抛出了同样的错误

下面是我的外部dns yaml文件。

apiVersion: v1
kind: ServiceAccount
metadata:
  name: external-dns
  # If you're using Amazon EKS with IAM Roles for Service Accounts, specify the following annotation.
  # Otherwise, you may safely omit it.
  annotations:
    # Substitute your account ID and IAM service role name below.
    eks.amazonaws.com/role-arn: arn:aws:iam::268xxxxxxx:role/eksctl-ats-Eks1-addon-iamserviceaccoun-Role1-WMLL93xxxx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: external-dns
rules:
- apiGroups: [""]
  resources: ["services","endpoints","pods"]
  verbs: ["get","watch","list"]
- apiGroups: ["extensions","networking.k8s.io"]
  resources: ["ingresses"]
  verbs: ["get","watch","list"]
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["list","watch"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: external-dns-viewer
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: external-dns
subjects:
- kind: ServiceAccount
  name: external-dns
  namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-dns
spec:
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: external-dns
  template:
    metadata:
      labels:
        app: external-dns
    spec:
      serviceAccountName: external-dns
      containers:
      - name: external-dns
        image: k8s.gcr.io/external-dns/external-dns:v0.7.6
        args:
        - --source=service
        - --source=ingress
        - --domain-filter=xyz.com # will make ExternalDNS see only the hosted zones matching provided domain, omit to process all available hosted zones
        - --provider=aws
        - --policy=upsert-only # would prevent ExternalDNS from deleting any records, omit to enable full synchronization
        - --aws-zone-type=public # only look at public hosted zones (valid values are public, private or no value for both)
        - --registry=txt
        - --txt-owner-id=Z0471542U7WSPZxxxx
      securityContext:
        fsGroup: 65534 # For ExternalDNS to be able to read Kubernetes and AWS token files

我抓挠我的头，因为没有适当的解决方案，这个错误在任何地方的网络。希望在这个论坛上找到解决这个问题的办法

最终结果必须显示如下内容，并填充托管区域中的记录

time="2020-05-05T02:57:31Z" level=info msg="All records are already up to date"

---

我这里有一些可能性

首先，您的集群是否有与之关联的OIDC提供程序？没有它，IRSA就无法工作

您可以在AWS控制台中检查，或通过CLI检查：

aws eks描述集群——名称{name}——查询“cluster.identity.oidc.issuer”

首先

删除

iamserviceaccount

，重新创建它，从外部DNS管理列表（整个第一部分）中删除

ServiceAccount

定义，然后重新应用它

eksctl delete iamserviceaccount --name {name} --namespace {namespace} --cluster {cluster}
eksctl create iamserviceaccount --name {name} --namespace {namespace} --cluster 
{cluster} --attach-policy-arn {policy-arn} --approve --override-existing-serviceaccounts
kubectl apply -n {namespace} -f {your-externaldns-manifest.yaml}

可能存在一些冲突，因为您通过在ExternalDNS manfiest中指定一个

ServiceAccount

来覆盖您使用

eksctl CreateAmserviceAccount

创建的内容

秒

将群集升级到v1.19（如果还没有）：

eksctl升级集群--name{name}

将向您展示将要执行的操作

eksctl升级集群--name{name}--approve

可以

第三

---

一些文档表明，除了设置

securityContext.fsGroup:65534

，您还需要设置

securityContext.runAsUser:0

，我能够从Kubernetes Slack（向@Rob Del大叫）获得帮助，这就是我们想到的。从文章中可以看出，k8s rbac没有问题，问题在于IAM角色的编写方式。我正在使用Terraform v0.12.24，但我相信类似于以下内容。tf应该适用于Terraform v0.14：

data "aws_caller_identity" "current" {}

resource "aws_iam_role" "external_dns_role" {
  name = "external-dns"

  assume_role_policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
          "Federated": format(
            "arn:aws:iam::${data.aws_caller_identity.current.account_id}:%s", 
            replace(
              "${aws_eks_cluster.<YOUR_CLUSTER_NAME>.identity[0].oidc[0].issuer}", 
              "https://", 
              "oidc-provider/"
            )
          )
        },
        "Action": "sts:AssumeRoleWithWebIdentity",
        "Condition": {
          "StringEquals": {
            format(
              "%s:sub", 
              trimprefix(
                "${aws_eks_cluster.<YOUR_CLUSTER_NAME>.identity[0].oidc[0].issuer}", 
                "https://"
              )
            ) : "system:serviceaccount:default:external-dns"
          }
        }
      }
    ]
  })
}

data“aws\u caller\u identity”当前“{}
资源“aws\u iam\u角色”“外部\u dns\u角色”{
name=“外部dns”
假设角色策略=jsonencode({
“版本”：“2012-10-17”，
“声明”：[
{
“效果”：“允许”，
“委托人”：{
“联邦”：格式(
“arn:aws:iam:：${data.aws\u caller\u identity.current.account\u id}:%s”，
替换(
“${aws_eks_cluster..identity[0].oidc[0].issuer}”，
“https://”，
“oidc提供程序/”
)
)
},
“行动”：“sts:假定身份”，
“条件”：{
“StringEquals”：{
格式(
“%s:sub”，
修剪前缀(
“${aws_eks_cluster..identity[0].oidc[0].issuer}”，
“https://”
)
)：“系统：服务帐户：默认值：外部dns”
}
}
}
]
})
}

---

上面的.tf假设您使用terraform创建了eks集群，并且使用了rbac清单。

我也遇到了这个错误

问题在于信任关系的定义

您可以在一些官方aws教程（如）中看到以下设置：

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:<my-namespace>:<my-service-account>"
        }
      }
    }
  ]
}

当您在AWS web控制台的信任关系选项卡中查看输出时，您可以看到添加了附加条件，后缀为

：aud

，值为

sts.amazonaws.com

：

---

因此，这需要添加到

“${OIDC\U PROVIDER}:sub”

条件之后。

在遵循建议的设置之后，我一直在努力解决类似的问题

我在部署日志中发现了以下异常

time="2021-05-10T06:40:17Z" level=error msg="records retrieval failed: failed to list hosted zones: WebIdentityErr: failed to retrieve credentials\ncaused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity\n\tstatus code: 403, request id: 3fda6c69-2a0a-4bc9-b478-521b5131af9b"
time="2021-05-10T06:41:20Z" level=error msg="records retrieval failed: failed to list hosted zones: WebIdentityErr: failed to retrieve credentials\ncaused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity\n\tstatus code: 403, request id: 7d3e07a2-c514-44fa-8e79-d49314d9adb6"

在我的例子中，这是一个映射到创建的新角色的错误服务帐户名称的问题

这里有一个逐步的方法来完成这项工作，而不需要太多的麻烦

创建IAM策略

为EKS群集创建IAM角色和服务帐户

创建了新的托管区域

aws route53创建托管区域--名称“hosted.domain.com.”--调用方参考“grpc端点外部dns测试-$（日期+%s）”

在创建群集角色并将群集角色绑定到以前创建的服务帐户后，部署ExternalDNS

使用域名更新入口资源并重新应用清单

对于入口对象，ExternalDNS将基于为入口对象指定的主机创建DNS记录

-host:myapp.hosted.domain.com

验证创建的新记录

---

早上好我有一个疑问。在（清单的）服务帐户部分，我指的是使用eksctl（注释）创建的服务帐户。还是我正在覆盖eksctl的创建？@KrisT您正在覆盖您使用

eksctl CreateAmserviceAccount

创建的内容。另外@KrisT，只是为了确认，您确实有一个与此群集关联的OIDC提供程序。我刚刚检查了terraform的eks模块配置，发现irsa已禁用。根据你的评论，看来我必须启用它。我希望这足够了。我将尝试您关于在ext dns yaml manifest中注释SA部分的建议。假设@KrisT为集群提供了terraform，升级应该通过更改terraform模块或资源中的版本值来执行，然后应用，而不是通过eksctl升级，因为我认为后者会

time="2021-05-10T06:40:17Z" level=error msg="records retrieval failed: failed to list hosted zones: WebIdentityErr: failed to retrieve credentials\ncaused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity\n\tstatus code: 403, request id: 3fda6c69-2a0a-4bc9-b478-521b5131af9b"
time="2021-05-10T06:41:20Z" level=error msg="records retrieval failed: failed to list hosted zones: WebIdentityErr: failed to retrieve credentials\ncaused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity\n\tstatus code: 403, request id: 7d3e07a2-c514-44fa-8e79-d49314d9adb6"

{
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "route53:ChangeResourceRecordSets"
          ],
          "Resource": [
            "arn:aws:route53:::hostedzone/*"
          ]
        },
        {
          "Effect": "Allow",
          "Action": [
            "route53:ListHostedZones",
            "route53:ListResourceRecordSets"
          ],
          "Resource": [
            "*"
          ]
        }
      ]
    }

eksctl create iamserviceaccount \
    --name external-dns-sa-eks \
    --namespace default \
    --cluster aecops-grpc-test \
    --attach-policy-arn arn:aws:iam::xxxxxxxx:policy/external-dns-policy-eks  \
    --approve 
    --override-existing-serviceaccounts

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: external-dns
rules:
- apiGroups: [""]
  resources: ["services","endpoints","pods"]
  verbs: ["get","watch","list"]
- apiGroups: ["extensions","networking.k8s.io"]
  resources: ["ingresses"]
  verbs: ["get","watch","list"]
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["list","watch"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: external-dns-viewer
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: external-dns
subjects:
- kind: ServiceAccount
  name: external-dns-sa-eks
  namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-dns
spec:
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: external-dns
  template:
    metadata:
      labels:
        app: external-dns
      # If you're using kiam or kube2iam, specify the following annotation.
      # Otherwise, you may safely omit it.
      annotations:
        iam.amazonaws.com/role: arn:aws:iam::***********:role/eksctl-eks-cluster-name-addon-iamserviceacco-Role1-156KP94SN7D7
    spec:
      serviceAccountName: external-dns-sa-eks
      containers:
      - name: external-dns
        image: k8s.gcr.io/external-dns/external-dns:v0.7.6
        args:
        - --source=service
        - --source=ingress
        - --domain-filter=hosted.domain.com. # will make ExternalDNS see only the hosted zones matching provided domain, omit to process all available hosted zones
        - --provider=aws
        - --policy=upsert-only # would prevent ExternalDNS from deleting any records, omit to enable full synchronization
        - --aws-zone-type=public # only look at public hosted zones (valid values are public, private or no value for both)
        - --registry=txt
        - --txt-owner-id=my-hostedzone-identifier
      securityContext:
        fsGroup: 65534 # For ExternalDNS to be able to read Kubernetes and AWS token files

BASH-3.2$ aws route53 list-resource-record-sets --output json
--hosted-zone-id "/hostedzone/Z065*********" --query "ResourceRecordSets[?Name == 'hosted.domain.com..']|[?Type == 'A']"

[
    {
        "Name": "myapp.hosted.domain.com..",
        "Type": "A",
        "AliasTarget": {
            "HostedZoneId": "ZCT6F*******",
            "DNSName": "****************.elb.ap-southeast-2.amazonaws.com.",
            "EvaluateTargetHealth": true
        }
    } ]