Terraform AWS EKS ALB Kubernetes Ingress赢得'；不要创建侦听器或目标组

我正在尝试使用地形资源创建一个带有ALB入口的AWS EKS集群

指示入口将自动创建具有关联侦听器和目标组的负载平衡器

Kubernetes入口创建ALB负载平衡器、安全组和规则，但不创建目标组或侦听器。我尝试过使用网关或应用程序子网，但没有任何区别。我尝试设置安全组，但ALB设置并使用了自己的自我管理安全组

我一直依靠

一个卷曲的ALB让我

无法连接到 de59ecbf-default-mainingre-8687-1051686593.ap-SOUTHEST-1.elb.amazonaws.com 端口80：连接被拒绝

我分别创建了IAM角色和ACM证书，因为AWS对这些角色有配额限制。EKS群集和节点的我的角色是标准的，节点角色附加了最新的策略

我使用

kubectl

分别应用kubernetes入口，但结果相同。它创建ALB和一个安全组，其中包含端口规则，但没有目标组或侦听器

当我将aws eks descripe cluster中的集群端点——命名我的tf eks cluster——查询“cluster.endpoint”粘贴到浏览器中时，我得到以下结果：

{“种类”：“状态”，“apiVersion”：“v1”，“元数据”：{ }，“状态”：“失败”，“消息”：“禁止：用户”系统：匿名“无法获取路径”/“原因”：“禁止”， “详情”：{ }，“代码”：403}

此外，入口没有ip地址

kubectl describe ingresses 

Name:             main-ingress
Namespace:        default
Address:          
Default backend:  go-hello-world:8080 (<none>)
Rules:
  Host  Path  Backends
  ----  ----  --------
  *     *     go-hello-world:8080 (<none>)


aws eks describe-cluster --name my-tf-eks-cluster --query cluster.endpoint"
"https://88888888B.gr7.ap-southeast-1.eks.amazonaws.com"


curl https://88888888B.gr7.ap-southeast-1.eks.amazonaws.com
curl: (60) SSL certificate problem: unable to get local issuer certificate

这是我的EKS主资源：

data "aws_iam_role" "tf-eks-master" {
  name = "terraform-eks-cluster"
}

resource "aws_eks_cluster" "tf_eks" {
  name     = var.cluster_name
  role_arn = data.aws_iam_role.tf-eks-master.arn
 
  vpc_config {
    security_group_ids      = [aws_security_group.master.id]
    subnet_ids              = var.application_subnet_ids
    endpoint_private_access = true
    endpoint_public_access  = true
  } 
}

ALB入口控制器：

    output "vpc_id" {
      value = data.aws_vpc.selected
    }
    
    data "aws_subnet_ids" "selected" {
      
      vpc_id = data.aws_vpc.selected.id
    
      tags = map(
        "Name", "application",
      )
    }
    
    resource "kubernetes_deployment" "alb-ingress" {
      metadata {
        name = "alb-ingress-controller"
        labels = {
          "app.kubernetes.io/name" = "alb-ingress-controller"
        }
        namespace = "kube-system"
      }
    
      spec {
        selector {
          match_labels = {
            "app.kubernetes.io/name" = "alb-ingress-controller"
          }
        }
    
        template {
          metadata {
            labels = {
              "app.kubernetes.io/name" = "alb-ingress-controller"
            }
          }
          spec {
            volume {
              name = kubernetes_service_account.alb-ingress.default_secret_name
              secret {
                secret_name = kubernetes_service_account.alb-ingress.default_secret_name
              }
            }
            container {
              # This is where you change the version when Amazon comes out with a new version of the ingress controller
              image = "docker.io/amazon/aws-alb-ingress-controller:v1.1.8"
              name  = "alb-ingress-controller"
              args = [
                "--ingress-class=alb",
                "--cluster-name=${var.cluster_name}",
                "--aws-vpc-id=${data.aws_vpc.selected.id}",
                "--aws-region=${var.aws_region}"
              ]
              volume_mount {
                name       = kubernetes_service_account.alb-ingress.default_secret_name
                mount_path = "/var/run/secrets/kubernetes.io/serviceaccount"
                read_only  = true
              }
            }
    
            service_account_name = "alb-ingress-controller"
    
          }
        }
      }
    }
    
    resource "kubernetes_service_account" "alb-ingress" {
      metadata {
        name = "alb-ingress-controller"
        namespace = "kube-system"
        labels = {
          "app.kubernetes.io/name" = "alb-ingress-controller"
        }
      }
    
      automount_service_account_token = true
    }
    

kubernetes_ingress.yml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: main-ingress
  annotations:

    kubernetes.io/ingress.class: "alb"

    alb.ingress.kubernetes.io/scheme: "internet-facing"

    alb.ingress.kubernetes.io/target-type: "ip"

    alb.ingress.kubernetes.io/subnets: 'subnet-0ab65d9cec9451287, subnet-034bf8856ab9157b7, subnet-0c16b1d382fadd0b4'

    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80},{"HTTPS": 443}]'

spec:
  backend:
    serviceName: go-hello-world
    servicePort: 8080

   

角色

来自专有网络的一些代码

data "aws_availability_zones" "available" {}

resource "aws_subnet" "gateway" {
  count = var.subnet_count
  availability_zone = data.aws_availability_zones.available.names[count.index]
  cidr_block        = "10.0.1${count.index}.0/24"
  vpc_id            = aws_vpc.tf_eks.id
  tags = map(
     "Name", "gateway",
  )
}
resource "aws_subnet" "application" {
  count = var.subnet_count
  availability_zone = data.aws_availability_zones.available.names[count.index]
  cidr_block        = "10.0.2${count.index}.0/24"
  vpc_id            = aws_vpc.tf_eks.id
  tags = map(
     "Name", "application",
     "kubernetes.io/cluster/${var.cluster_name}", "shared",
     "kubernetes.io/role/elb", "1",
  )
}

resource "aws_subnet" "database" {
  count = var.subnet_count
  availability_zone = data.aws_availability_zones.available.names[count.index]
  cidr_block        = "10.0.3${count.index}.0/24"
  vpc_id            = aws_vpc.tf_eks.id
  
  tags = map(
     "Name", "database"
  )
}

resource "aws_route_table" "application" {
  count = var.subnet_count
  vpc_id = aws_vpc.tf_eks.id

  route {
    cidr_block = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.tf_eks.*.id[count.index]
  }
  
  tags = {
    Name = "application"
  }
}

resource "aws_route_table" "database" {
  vpc_id = aws_vpc.tf_eks.id

  tags = {
    Name = "database"
  }
}

resource "aws_route_table" "gateway" {
  vpc_id = aws_vpc.tf_eks.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.tf_eks.id
  }

  tags = {
    Name = "gateway"
  }
}

resource "aws_route_table_association" "application" {
  count = var.subnet_count

  subnet_id      = aws_subnet.application.*.id[count.index]
  route_table_id = aws_route_table.application.*.id[count.index]
}

resource "aws_route_table_association" "database" {
  count = var.subnet_count

  subnet_id      = aws_subnet.database.*.id[count.index]
  route_table_id = aws_route_table.database.id
}

resource "aws_route_table_association" "gateway" {
  count = var.subnet_count

  subnet_id      = aws_subnet.gateway.*.id[count.index]
  route_table_id = aws_route_table.gateway.id
}

resource "aws_internet_gateway" "tf_eks" {
  
  vpc_id = aws_vpc.tf_eks.id

  tags = {
    Name = "internet_gateway"
  }
}

resource "aws_eip" "nat_gateway" {
  count = var.subnet_count
  vpc   = true
}

resource "aws_nat_gateway" "tf_eks" {

  count = var.subnet_count
  
  allocation_id = aws_eip.nat_gateway.*.id[count.index]
  
  subnet_id = aws_subnet.gateway.*.id[count.index]
  
  tags = {
    Name = "nat_gateway"
  }
  
  depends_on = [aws_internet_gateway.tf_eks]
}

安全小组

resource "aws_security_group" "eks" {
  name        = "tf-eks-master"
  description = "Cluster communication with worker nodes"
  vpc_id      = var.vpc_id

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}  

resource "aws_security_group" "node" {
  name        = "tf-eks-node"
  description = "Security group for all nodes in the cluster"
  vpc_id      = var.vpc_id

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_security_group_rule" "main-node-ingress-self" {
  type              = "ingress"
  description       = "Allow node to communicate with each other"
  from_port         = 0
  protocol          = "-1"
  security_group_id = aws_security_group.node.id
  to_port           = 65535
  cidr_blocks       = var.subnet_cidrs
}

resource "aws_security_group_rule" "main-node-ingress-cluster" {
  type                     = "ingress"
  description              = "Allow worker Kubelets and pods to receive communication from the cluster control plane"
  from_port                = 1025
  protocol                 = "tcp"
  security_group_id        = aws_security_group.node.id
  source_security_group_id = aws_security_group.eks.id
  to_port                  = 65535
}

kubectl获取所有--所有名称空间

kubectl get all --all-namespaces
NAMESPACE     NAME                                          READY   STATUS    RESTARTS   AGE
default       pod/go-hello-world-68545f84bc-5st4s           1/1     Running   0          35s
default       pod/go-hello-world-68545f84bc-bkwpb           1/1     Running   0          35s
default       pod/go-hello-world-68545f84bc-kmfbq           1/1     Running   0          35s
kube-system   pod/alb-ingress-controller-5f9cb4b7c4-w858g   1/1     Running   0          2m7s
kube-system   pod/aws-node-8jfkf                            1/1     Running   0          67m
kube-system   pod/aws-node-d7s7w                            1/1     Running   0          67m
kube-system   pod/aws-node-termination-handler-g5fmj        1/1     Running   0          67m
kube-system   pod/aws-node-termination-handler-q5tz5        1/1     Running   0          67m
kube-system   pod/aws-node-termination-handler-tmzmr        1/1     Running   0          67m
kube-system   pod/aws-node-vswpf                            1/1     Running   0          67m
kube-system   pod/coredns-5c4dd4cc7-sk474                   1/1     Running   0          71m
kube-system   pod/coredns-5c4dd4cc7-zplwg                   1/1     Running   0          71m
kube-system   pod/kube-proxy-5m9dn                          1/1     Running   0          67m
kube-system   pod/kube-proxy-8tn9l                          1/1     Running   0          67m
kube-system   pod/kube-proxy-qs652                          1/1     Running   0          67m

NAMESPACE     NAME                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)         AGE
default       service/kubernetes   ClusterIP   172.20.0.1    <none>        443/TCP         71m
kube-system   service/kube-dns     ClusterIP   172.20.0.10   <none>        53/UDP,53/TCP   71m

NAMESPACE     NAME                                          DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
kube-system   daemonset.apps/aws-node                       3         3         3       3            3           <none>          71m
kube-system   daemonset.apps/aws-node-termination-handler   3         3         3       3            3           <none>          68m
kube-system   daemonset.apps/kube-proxy                     3         3         3       3            3           <none>          71m

NAMESPACE     NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
default       deployment.apps/go-hello-world           3/3     3            3           37s
kube-system   deployment.apps/alb-ingress-controller   1/1     1            1           2m9s
kube-system   deployment.apps/coredns                  2/2     2            2           71m

NAMESPACE     NAME                                                DESIRED   CURRENT   READY   AGE
default       replicaset.apps/go-hello-world-68545f84bc           3         3         3       37s
kube-system   replicaset.apps/alb-ingress-controller-5f9cb4b7c4   1         1         1       2m9s
kube-system   replicaset.apps/coredns-5c4dd4cc7                   2         2  

kubectl获取所有--所有名称空间
名称空间名称就绪状态重新启动
默认pod/go-hello-world-68545f84bc-5st4s 1/1运行0 35s
默认pod/go-hello-world-68545f84bc-bkwpb 1/1运行0 35s
默认pod/go-hello-world-68545f84bc-kmfbq 1/1运行0 35s
kube系统吊舱/alb-ingress-controller-5f9cb4b7c4-w858g 1/1运行0 2m7秒
kube系统吊舱/aws-node-8jfkf 1/1运行0 67m
kube系统吊舱/aws-node-d7s7w 1/1运行0 67m
kube系统吊舱/aws-node-termination-handler-g5fmj 1/1运行0 67m
kube系统吊舱/aws-node-termination-handler-q5tz5 1/1运行0 67m
kube系统pod/aws节点终止处理程序tmzmr 1/1运行0 67m
kube系统吊舱/aws节点vswpf 1/1运行0 67m
kube系统吊舱/coredns-5C4DD4C7-sk474 1/1运行0 71m
kube系统吊舱/coredns-5C4DD4C7-zplwg 1/1运行0 71m
kube系统pod/kube-proxy-5m9dn 1/1运行0 67m
kube系统吊舱/kube-proxy-8tn9l 1/1运行0 67m
kube系统吊舱/kube-proxy-qs652 1/1运行0 67m
命名空间名称类型CLUSTER-IP EXTERNAL-IP端口年龄
默认服务/kubernetes ClusterIP 172.20.0.1 443/TCP 71m
kube系统服务/kube dns群集IP 172.20.0.10 53/UDP，53/TCP 71m
名称空间名称所需的当前就绪最新可用节点选择器年龄
kube system daemonset.apps/aws-node 3 71m
kube system daemonset.apps/aws-node-termination-handler 3 68m
kube system daemonset.apps/kube-proxy 3 71m
名称空间名称就绪最新可用期限
默认部署.apps/go-hello-world 3/3 37s
kube system deployment.apps/alb-ingress-controller 1/1 2m9s
kube system deployment.apps/coredns 2/2 71m
名称空间名称所需的当前就绪时间
默认replicaset.apps/go-hello-world-68545f84bc 3 37s
kube system replicaset.apps/alb-ingress-controller-5f9cb4b7c4 1 2m9s
kube system replicaset.apps/coredns-5C4DD4C7 2

没有测试它，但我将指出我在您的资源中看到的问题

在alb资源参数中，您具有以下内容：

"--aws-vpc-id=${data.aws_vpc.selected.id}",

然而，您没有任何数据资源来拉这个专有网络。 此外，数据资源运行时没有任何依赖关系，并且不会获取关于尚未创建的专有网络的任何信息

---

如果您将terraform模块化，请从VPC模块输出VPC ID并使用它。如果所有资源都在同一个文件/文件夹中，只需在这一行中直接引用它：aws_vpc.tf_eks.id

能否尝试添加这些行并尝试使用kubectl命令

# ALB's Target Group Configurations
alb.ingress.kubernetes.io/backend-protocol: HTTPS
alb.ingress.kubernetes.io/healthcheck-protocol: HTTPS

仍然没有创建目标组，请检查控制器的日志

kubectl logs -n kube-system   deployment.apps/aws-load-balancer-controller

---

您是否尝试过：资源：aws_lb_target_group_附件提供了向应用程序负载平衡器（ALB）或网络负载平衡器（NLB）目标组注册实例和容器的功能。我不想走这条路，因为它需要额外的地形（或控制台）创建入口后以及每次创建新入口时进行工作。入口类alb

# ALB's Target Group Configurations
alb.ingress.kubernetes.io/backend-protocol: HTTPS
alb.ingress.kubernetes.io/healthcheck-protocol: HTTPS

kubectl logs -n kube-system   deployment.apps/aws-load-balancer-controller