Amazon web services 如何在CodePipeline中处理多个环境？

我正在使用代码管道来部署我的基础设施，我希望能够在不同的环境（开发、登台、产品等）中部署它

我目前有一个buildspec.yml文件，其中包含一些“pip安装”说明和“aws cloudformation package”命令。我还创建了两个管道，一个用于生产，另一个用于开发，指向github上的两个不同分支。我遇到的问题是，由于两个分支中的文件都包含类似的资源，因此我在S3存储桶上遇到了名称冲突

使用AWS CLI和cloudformation创建或更新堆栈时，可以使用--parameters选项传递参数。我想在我创建的两个管道中做一些类似的事情

解决这一问题的最佳解决方案是什么

最终目标是自动化基础架构的部署。我们的基础设施由用户、KMS密钥、LAMDBA（python）、组和存储桶组成

我根据教程创建了两个管道：

第一条管道链接到包含代码的回购协议的主分支，第二条管道链接到暂存分支。我的目标是使用第一条管道在生产环境中自动部署主分支，使用第二条管道在临时环境中自动部署临时分支

我的buildspec.yml文件如下所示：

version: 0.1
phases:
    install:
        commands:
            - pip install requests -t .
            - pip install simplejson -t .
            - pip install Image -t .
            - aws cloudformation package --template-file image_processing_sam.yml --s3-bucket package-bucket --output-template-file new_image_processing_sam.yml
artifacts:
    type: zip
    files:
        - new_image_processing_sam.yml

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Create a thumbnail for an image uploaded to S3
Resources:

  ThumbnailFunction:
    Type: "AWS::Serverless::Function"
    Properties:
      Role: !GetAtt LambdaExecutionRole.Arn
      Handler: create_thumbnail.handler
      Runtime: python2.7
      Timeout: 30
      Description: "A function computing the thumbnail for an image."

  LambdaSecretEncryptionKey:
    Type: "AWS::KMS::Key"
    Properties:
      Description: "A key used to encrypt secrets used in the Lambda functions"
      Enabled: True
      EnableKeyRotation: False
      KeyPolicy:
        Version: "2012-10-17"
        Id: "lambda-secret-encryption-key"
        Statement:
          -
            Sid: "Allow administration of the key"
            Effect: "Allow"
            Principal:
              AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role"
            Action:
              - "kms:Create*"
              - "kms:Describe*"
              - "kms:Enable*"
              - "kms:List*"
              - "kms:Put*"
              - "kms:Update*"
              - "kms:Revoke*"
              - "kms:Disable*"
              - "kms:Get*"
              - "kms:Delete*"
              - "kms:ScheduleKeyDeletion"
              - "kms:CancelKeyDeletion"
            Resource: "*"
          -
            Sid: "Allow use of the key"
            Effect: "Allow"
            Principal:
              AWS:
                - !GetAtt LambdaExecutionRole.Arn
            Action:
              - "kms:Encrypt"
              - "kms:Decrypt"
              - "kms:ReEncrypt*"
              - "kms:GenerateDataKey*"
              - "kms:DescribeKey"
            Resource: "*"

  LambdaExecutionRole:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: "LambdaExecutionRole"
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - "lambda.amazonaws.com"
          Action:
          - "sts:AssumeRole"
      Policies:
        -
          PolicyName: LambdaKMS
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action:
                  - "kms:Decrypt"
                Resource: "*"
              -
                Effect: "Allow"
                Action:
                  - "lambda:InvokeFunction"
                Resource: "*"
      ManagedPolicyArns:
      - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"

  UserGroup:
      Type: "AWS::IAM::Group"

  LambdaTriggerUser:
    Type: "AWS::IAM::User"
    Properties:
      UserName: "LambdaTriggerUser"

  LambdaTriggerUserKeys:
    Type: "AWS::IAM::AccessKey"
    Properties:
      UserName:
        Ref: LambdaTriggerUser

  Users:
    Type: "AWS::IAM::UserToGroupAddition"
    Properties:
      GroupName:
        Ref: UserGroup
      Users:
        - Ref: LambdaTriggerUser

  Policies:
    Type: "AWS::IAM::Policy"
    Properties:
      PolicyName: UserPolicy
      PolicyDocument:
        Statement:
          -
            Effect: "Allow"
            Action:
              - "lambda:InvokeFunction"
            Resource:
              - !GetAtt DispatcherFunction.Arn
      Groups:
        - Ref: UserGroup

  PackageBucket:
    Type: "AWS::S3::Bucket"
    Properties:
      BucketName: "package-bucket"
      VersioningConfiguration:
        Status: "Enabled"

Outputs:
  LambdaTriggerUserAccessKey:
    Value:
      Ref: "LambdaTriggerUserKeys"
    Description: "AWSAccessKeyId of LambdaTriggerUser"

  LambdaTriggerUserSecretKey:
    Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey
    Description: "AWSSecretKey of LambdaTriggerUser"

图像处理\u sam.yml文件如下所示：

version: 0.1
phases:
    install:
        commands:
            - pip install requests -t .
            - pip install simplejson -t .
            - pip install Image -t .
            - aws cloudformation package --template-file image_processing_sam.yml --s3-bucket package-bucket --output-template-file new_image_processing_sam.yml
artifacts:
    type: zip
    files:
        - new_image_processing_sam.yml

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Create a thumbnail for an image uploaded to S3
Resources:

  ThumbnailFunction:
    Type: "AWS::Serverless::Function"
    Properties:
      Role: !GetAtt LambdaExecutionRole.Arn
      Handler: create_thumbnail.handler
      Runtime: python2.7
      Timeout: 30
      Description: "A function computing the thumbnail for an image."

  LambdaSecretEncryptionKey:
    Type: "AWS::KMS::Key"
    Properties:
      Description: "A key used to encrypt secrets used in the Lambda functions"
      Enabled: True
      EnableKeyRotation: False
      KeyPolicy:
        Version: "2012-10-17"
        Id: "lambda-secret-encryption-key"
        Statement:
          -
            Sid: "Allow administration of the key"
            Effect: "Allow"
            Principal:
              AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role"
            Action:
              - "kms:Create*"
              - "kms:Describe*"
              - "kms:Enable*"
              - "kms:List*"
              - "kms:Put*"
              - "kms:Update*"
              - "kms:Revoke*"
              - "kms:Disable*"
              - "kms:Get*"
              - "kms:Delete*"
              - "kms:ScheduleKeyDeletion"
              - "kms:CancelKeyDeletion"
            Resource: "*"
          -
            Sid: "Allow use of the key"
            Effect: "Allow"
            Principal:
              AWS:
                - !GetAtt LambdaExecutionRole.Arn
            Action:
              - "kms:Encrypt"
              - "kms:Decrypt"
              - "kms:ReEncrypt*"
              - "kms:GenerateDataKey*"
              - "kms:DescribeKey"
            Resource: "*"

  LambdaExecutionRole:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: "LambdaExecutionRole"
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - "lambda.amazonaws.com"
          Action:
          - "sts:AssumeRole"
      Policies:
        -
          PolicyName: LambdaKMS
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action:
                  - "kms:Decrypt"
                Resource: "*"
              -
                Effect: "Allow"
                Action:
                  - "lambda:InvokeFunction"
                Resource: "*"
      ManagedPolicyArns:
      - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"

  UserGroup:
      Type: "AWS::IAM::Group"

  LambdaTriggerUser:
    Type: "AWS::IAM::User"
    Properties:
      UserName: "LambdaTriggerUser"

  LambdaTriggerUserKeys:
    Type: "AWS::IAM::AccessKey"
    Properties:
      UserName:
        Ref: LambdaTriggerUser

  Users:
    Type: "AWS::IAM::UserToGroupAddition"
    Properties:
      GroupName:
        Ref: UserGroup
      Users:
        - Ref: LambdaTriggerUser

  Policies:
    Type: "AWS::IAM::Policy"
    Properties:
      PolicyName: UserPolicy
      PolicyDocument:
        Statement:
          -
            Effect: "Allow"
            Action:
              - "lambda:InvokeFunction"
            Resource:
              - !GetAtt DispatcherFunction.Arn
      Groups:
        - Ref: UserGroup

  PackageBucket:
    Type: "AWS::S3::Bucket"
    Properties:
      BucketName: "package-bucket"
      VersioningConfiguration:
        Status: "Enabled"

Outputs:
  LambdaTriggerUserAccessKey:
    Value:
      Ref: "LambdaTriggerUserKeys"
    Description: "AWSAccessKeyId of LambdaTriggerUser"

  LambdaTriggerUserSecretKey:
    Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey
    Description: "AWSSecretKey of LambdaTriggerUser"

我在两个管道中都添加了一个部署操作，以执行beta操作期间计算的更改集

第一条管道就像一个符咒，做了我期望它做的一切。每次我在主分支中推送代码时，它都会被部署

我面临的问题是，当我在登台分支中推送代码时，在到达部署操作之前，所有东西都在管道中工作。deploy操作尝试创建一个新堆栈，但由于它与所处理的buildspec.yml和image_processing_sam.yml完全相同，因此我遇到了如下名称冲突

package-bucket already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx
LambdaTriggerUser already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx
LambdaExecutionRole already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx
...

是否有一种方法可以参数化buildspec.yml，以便能够在image\u processing\u sam.yml中的资源名称中添加后缀？任何其他实现这一目标的想法都是受欢迎的

---

致以最良好的祝愿

检查Eric Nord的答案。就是你要找的那个

---

我也在AWS论坛上问了这个问题

以下是AWS提供的解决方案：

嗨

如果您的目标是为staging和master使用不同的bucket名称，那么另一个选项是使用CloudFormation参数

编辑现有管道时，如果编辑操作，则可以展开“高级”面板并输入参数覆盖，以为每个阶段指定不同的bucket前缀。您还可以在工件中以单独的.json文件的形式输入参数

此处提供了有关执行此操作的更多详细信息：

以下是针对测试和生产的不同堆栈配置的完整演练：

• 蒂姆

您一定要遵循提供的文档。下面是我提出的解决方案

---

这是我自己的解决方案，我并不满意

我添加了一个在构建时运行的脚本，并根据构建项目的CodeBuild代理的ARN修改了我的模板

我添加了“BRANCH\u NAME”，其中可能会发生命名冲突。图像处理sam.yml现在是：

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Create a thumbnail for an image uploaded to S3
Resources:

  ThumbnailFunction:
    Type: "AWS::Serverless::Function"
    Properties:
      Role: !GetAtt LambdaExecutionRole.Arn
      Handler: create_thumbnail.handler
      Runtime: python2.7
      Timeout: 30
      Description: "A function computing the thumbnail for an image."

  LambdaSecretEncryptionKey:
    Type: "AWS::KMS::Key"
    Properties:
      Description: "A key used to encrypt secrets used in the Lambda functions"
      Enabled: True
      EnableKeyRotation: False
      KeyPolicy:
        Version: "2012-10-17"
        Id: "lambda-secret-encryption-keyBRANCH_NAME"
        Statement:
          -
            Sid: "Allow administration of the key"
            Effect: "Allow"
            Principal:
              AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role"
            Action:
              - "kms:Create*"
              - "kms:Describe*"
              - "kms:Enable*"
              - "kms:List*"
              - "kms:Put*"
              - "kms:Update*"
              - "kms:Revoke*"
              - "kms:Disable*"
              - "kms:Get*"
              - "kms:Delete*"
              - "kms:ScheduleKeyDeletion"
              - "kms:CancelKeyDeletion"
            Resource: "*"
          -
            Sid: "Allow use of the key"
            Effect: "Allow"
            Principal:
              AWS:
                - !GetAtt LambdaExecutionRole.Arn
            Action:
              - "kms:Encrypt"
              - "kms:Decrypt"
              - "kms:ReEncrypt*"
              - "kms:GenerateDataKey*"
              - "kms:DescribeKey"
            Resource: "*"

  LambdaExecutionRole:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: "LambdaExecutionRoleBRANCH_NAME"
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - "lambda.amazonaws.com"
          Action:
          - "sts:AssumeRole"
      Policies:
        -
          PolicyName: LambdaKMSBRANCH_NAME
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action:
                  - "kms:Decrypt"
                Resource: "*"
              -
                Effect: "Allow"
                Action:
                  - "lambda:InvokeFunction"
                Resource: "*"
      ManagedPolicyArns:
      - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"

  UserGroup:
      Type: "AWS::IAM::Group"

  LambdaTriggerUser:
    Type: "AWS::IAM::User"
    Properties:
      UserName: "LambdaTriggerUserBRANCH_NAME"

  LambdaTriggerUserKeys:
    Type: "AWS::IAM::AccessKey"
    Properties:
      UserName:
        Ref: LambdaTriggerUser

  Users:
    Type: "AWS::IAM::UserToGroupAddition"
    Properties:
      GroupName:
        Ref: UserGroup
      Users:
        - Ref: LambdaTriggerUser

  Policies:
    Type: "AWS::IAM::Policy"
    Properties:
      PolicyName: UserPolicyBRANCH_NAME
      PolicyDocument:
        Statement:
          -
            Effect: "Allow"
            Action:
              - "lambda:InvokeFunction"
            Resource:
              - !GetAtt DispatcherFunction.Arn
      Groups:
        - Ref: UserGroup

  PackageBucket:
    Type: "AWS::S3::Bucket"
    Properties:
      BucketName: "package-bucketBRANCH_NAME"
      VersioningConfiguration:
        Status: "Enabled"

Outputs:
  LambdaTriggerUserAccessKey:
    Value:
      Ref: "LambdaTriggerUserKeys"
    Description: "AWSAccessKeyId of LambdaTriggerUser"

  LambdaTriggerUserSecretKey:
    Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey
    Description: "AWSSecretKey of LambdaTriggerUser"

version: 0.1
phases:
    install:
        commands:
            # Install required module for python
            - pip install requests -t .
            - pip install simplejson -t .
            - pip install Image -t .
            - bash ./script.sh
            # To be able to see any issue in the generated template
            - cat generated_image_processing_sam.yml
            # Package the generated cloudformation template in order to deploy
            - aws cloudformation package --template-file generated_image_processing_sam.yml --s3-bucket piximate-package-bucket --output-template-file new_image_processing_sam.yml
artifacts:
    type: zip
    files:
        - new_image_processing_sam.yml

替换模板中“分支机构名称”的script.sh为：

#!/bin/bash
echo $CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"master"* ]]; then
    sed "s/BRANCH_NAME//g" image_processing_sam.yml > generated_image_processing_sam.yml;
fi
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"staging"* ]]; then
    sed "s/BRANCH_NAME/staging/g" image_processing_sam.yml > generated_image_processing_sam.yml;
fi

buildspec.yml现在是：

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Create a thumbnail for an image uploaded to S3
Resources:

  ThumbnailFunction:
    Type: "AWS::Serverless::Function"
    Properties:
      Role: !GetAtt LambdaExecutionRole.Arn
      Handler: create_thumbnail.handler
      Runtime: python2.7
      Timeout: 30
      Description: "A function computing the thumbnail for an image."

  LambdaSecretEncryptionKey:
    Type: "AWS::KMS::Key"
    Properties:
      Description: "A key used to encrypt secrets used in the Lambda functions"
      Enabled: True
      EnableKeyRotation: False
      KeyPolicy:
        Version: "2012-10-17"
        Id: "lambda-secret-encryption-keyBRANCH_NAME"
        Statement:
          -
            Sid: "Allow administration of the key"
            Effect: "Allow"
            Principal:
              AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role"
            Action:
              - "kms:Create*"
              - "kms:Describe*"
              - "kms:Enable*"
              - "kms:List*"
              - "kms:Put*"
              - "kms:Update*"
              - "kms:Revoke*"
              - "kms:Disable*"
              - "kms:Get*"
              - "kms:Delete*"
              - "kms:ScheduleKeyDeletion"
              - "kms:CancelKeyDeletion"
            Resource: "*"
          -
            Sid: "Allow use of the key"
            Effect: "Allow"
            Principal:
              AWS:
                - !GetAtt LambdaExecutionRole.Arn
            Action:
              - "kms:Encrypt"
              - "kms:Decrypt"
              - "kms:ReEncrypt*"
              - "kms:GenerateDataKey*"
              - "kms:DescribeKey"
            Resource: "*"

  LambdaExecutionRole:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: "LambdaExecutionRoleBRANCH_NAME"
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - "lambda.amazonaws.com"
          Action:
          - "sts:AssumeRole"
      Policies:
        -
          PolicyName: LambdaKMSBRANCH_NAME
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action:
                  - "kms:Decrypt"
                Resource: "*"
              -
                Effect: "Allow"
                Action:
                  - "lambda:InvokeFunction"
                Resource: "*"
      ManagedPolicyArns:
      - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"

  UserGroup:
      Type: "AWS::IAM::Group"

  LambdaTriggerUser:
    Type: "AWS::IAM::User"
    Properties:
      UserName: "LambdaTriggerUserBRANCH_NAME"

  LambdaTriggerUserKeys:
    Type: "AWS::IAM::AccessKey"
    Properties:
      UserName:
        Ref: LambdaTriggerUser

  Users:
    Type: "AWS::IAM::UserToGroupAddition"
    Properties:
      GroupName:
        Ref: UserGroup
      Users:
        - Ref: LambdaTriggerUser

  Policies:
    Type: "AWS::IAM::Policy"
    Properties:
      PolicyName: UserPolicyBRANCH_NAME
      PolicyDocument:
        Statement:
          -
            Effect: "Allow"
            Action:
              - "lambda:InvokeFunction"
            Resource:
              - !GetAtt DispatcherFunction.Arn
      Groups:
        - Ref: UserGroup

  PackageBucket:
    Type: "AWS::S3::Bucket"
    Properties:
      BucketName: "package-bucketBRANCH_NAME"
      VersioningConfiguration:
        Status: "Enabled"

Outputs:
  LambdaTriggerUserAccessKey:
    Value:
      Ref: "LambdaTriggerUserKeys"
    Description: "AWSAccessKeyId of LambdaTriggerUser"

  LambdaTriggerUserSecretKey:
    Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey
    Description: "AWSSecretKey of LambdaTriggerUser"

version: 0.1
phases:
    install:
        commands:
            # Install required module for python
            - pip install requests -t .
            - pip install simplejson -t .
            - pip install Image -t .
            - bash ./script.sh
            # To be able to see any issue in the generated template
            - cat generated_image_processing_sam.yml
            # Package the generated cloudformation template in order to deploy
            - aws cloudformation package --template-file generated_image_processing_sam.yml --s3-bucket piximate-package-bucket --output-template-file new_image_processing_sam.yml
artifacts:
    type: zip
    files:
        - new_image_processing_sam.yml

我希望它能对你有所帮助。如果有人能提供任何有帮助的改进或文档，我将非常高兴。

模板配置文件通过如下参数文件应用于CodePipeline中的CloudFormation：

{
  "Parameters" : {
    "DBName" : "TestWordPressDB",
    "DBPassword" : "TestDBRootPassword",
    "DBRootPassword" : "TestDBRootPassword",
    "DBUser" : "TestDBuser",    
    "KeyName" : "TestEC2KeyName"
    }
}

将这些文件放在repo的根目录中，可以通过至少两种方式引用它们

在代码管道云信息中：

Configuration:
    ActionMode: REPLACE_ON_FAILURE
    RoleArn: !GetAtt [CFNRole, Arn]
    StackName: !Ref TestStackName
    TemplateConfiguration: !Sub "TemplateSource::${TestStackConfig}"
    TemplatePath: !Sub "TemplateSource::${TemplateFileName}"

或在控制台的模板配置字段中：

值得注意的是，配置文件格式不同于使用cli的CloudFormation

-- parameters

--参数使用以下格式：

[
  {
    "ParameterKey": "team",
    "ParameterValue": "AD-Student Life Applications"
  },
  {
    "ParameterKey": "env",
    "ParameterValue": "dev"
  },
  {
    "ParameterKey": "dataSensitivity",
    "ParameterValue": "public"
  },
  {
    "ParameterKey": "app",
    "ParameterValue": "events-list-test"
  }
]

{
  "Parameters" : {
    "DBName" : "TestWordPressDB",
    "DBPassword" : "TestDBRootPassword",
    "DBRootPassword" : "TestDBRootPassword",
    "DBUser" : "TestDBuser",    
    "KeyName" : "TestEC2KeyName"
  }
}

CodePipeline Cloudformation模板配置文件使用以下格式：

[
  {
    "ParameterKey": "team",
    "ParameterValue": "AD-Student Life Applications"
  },
  {
    "ParameterKey": "env",
    "ParameterValue": "dev"
  },
  {
    "ParameterKey": "dataSensitivity",
    "ParameterValue": "public"
  },
  {
    "ParameterKey": "app",
    "ParameterValue": "events-list-test"
  }
]

{
  "Parameters" : {
    "DBName" : "TestWordPressDB",
    "DBPassword" : "TestDBRootPassword",
    "DBRootPassword" : "TestDBRootPassword",
    "DBUser" : "TestDBuser",    
    "KeyName" : "TestEC2KeyName"
  }
}

---

嗨，这周我没有时间测试，但我会确保尝试并尽快接受你的答案。感谢您的时间和详细的回答。昨天测试完整的代码示例在这里提供第271行：嗨，Eric，我终于花时间尝试验证您的解决方案了。这确实是前进的道路。再次感谢您详细的回答和宝贵的时间。致以最良好的问候。感谢您确认@JonathanGailliez