Amazon s3 使用CloudFormation在S3存储桶中创建Lambda通知

Amazon s3 使用CloudFormation在S3存储桶中创建Lambda通知,amazon-s3,aws-lambda,amazon-cloudformation,Amazon S3,Aws Lambda,Amazon Cloudformation,我试图在CloudFormation模板中为Lambda函数创建一个S3触发器。S3存储桶已经存在,并且正在创建Lambda函数 说不可能用CFT修改预先存在的基础设施(本例中是S3),但似乎说bucket必须是预先存在的 似乎无法使用CFT类型“AWS::Lambda…”创建触发器,源服务需要创建触发器。在我的例子中,这是s3存储桶的NotificationConfiguration LambdaConfiguration。这些都对吗 当我试图用CFT将NotificationConfigur

我试图在CloudFormation模板中为Lambda函数创建一个S3触发器。S3存储桶已经存在,并且正在创建Lambda函数

说不可能用CFT修改预先存在的基础设施(本例中是S3),但似乎说bucket必须是预先存在的

  • 似乎无法使用CFT类型“AWS::Lambda…”创建触发器,源服务需要创建触发器。在我的例子中,这是s3存储桶的NotificationConfiguration LambdaConfiguration。这些都对吗

  • 当我试图用CFT将NotificationConfiguration添加到现有的S3存储桶时,它说我不能。有没有办法做到这一点


  • 不幸的是,官方的
    AWS::CloudFormation
    模板只允许您作为父
    AWS::S3::Bucket
    资源的一部分进行控制,这意味着您不能将此配置附加到任何现有的Bucket,您必须将其应用到CloudFormation管理的Bucket才能工作

    一种解决方法是使用JavaScript API调用将调用直接实现为一个函数。但是,由于修改S3 bucket上的NotificationConfiguration仅限于bucket的创建者,因此还需要添加一个资源,授予Lambda函数对
    S3:PutBucketNotification
    操作的访问权限

    这是一个完整的、自包含的CloudFormation模板,演示了如何在将文件添加到现有S3存储桶时触发Lambda函数,使用2个Lambda支持的自定义资源(
    BucketConfiguration
    设置存储桶通知配置,
    S3Object
    将对象上载到存储桶)以及第三个Lambda函数(
    BucketWatcher
    ,用于在对象上载到bucket时触发等待条件)


    我使用另一种解决方案,因为我个人不喜欢自定义资源。我创建一个云跟踪来捕获相关bucket的S3写事件,然后使用S3事件模式创建一个cloudwatch/eventbridge规则来触发Lambda函数。通过这种方式,我可以将Lambda函数挂接到一个现有的S3 bucket上,而不必触碰它,而且可以通过cloudformation轻松完成。

    我相当确定,虽然bucket必须存在,但在创建模板之前它不必存在。在同一模板中创建通知配置和lambda函数旁边的bucket是否适合您的用例?如果是这样的话,这种方法将比修改现有基础设施更容易帮助您。无论哪种方式,都有一个解决方案,但其中一个要漂亮得多当你说“S3 bucket已经存在”时,你是否也在暗示该bucket是在CloudFormation之外创建的?使用CloudFormation在AWS中设置简单内容所需的代码数量之多令人难以置信使用CFN通用资源提供程序的类似解决方案(基本上是boto3代理):
    Description: Upload an object to an S3 bucket, triggering a Lambda event, returning the object key as a Stack Output.
    Parameters:
      Key:
        Description: S3 Object key
        Type: String
        Default: test
      Body:
        Description: S3 Object body content
        Type: String
        Default: TEST CONTENT
      BucketName:
        Description: S3 Bucket name (must already exist)
        Type: String
    Resources:
      BucketConfiguration:
        Type: Custom::S3BucketConfiguration
        DependsOn:
        - BucketPermission
        - NotificationBucketPolicy
        Properties:
          ServiceToken: !GetAtt S3BucketConfiguration.Arn
          Bucket: !Ref BucketName
          NotificationConfiguration:
            LambdaFunctionConfigurations:
            - Events: ['s3:ObjectCreated:*']
              LambdaFunctionArn: !GetAtt BucketWatcher.Arn
      S3BucketConfiguration:
        Type: AWS::Lambda::Function
        Properties:
          Description: S3 Object Custom Resource
          Handler: index.handler
          Role: !GetAtt LambdaExecutionRole.Arn
          Code:
            ZipFile: !Sub |
              var response = require('cfn-response');
              var AWS = require('aws-sdk');
              var s3 = new AWS.S3();
              exports.handler = function(event, context) {
                var respond = (e) => response.send(event, context, e ? response.FAILED : response.SUCCESS, e ? e : {});
                process.on('uncaughtException', e=>failed(e));
                var params = event.ResourceProperties;
                delete params.ServiceToken;
                if (event.RequestType === 'Delete') {
                  params.NotificationConfiguration = {};
                  s3.putBucketNotificationConfiguration(params).promise()
                    .then((data)=>respond())
                    .catch((e)=>respond());
                } else {
                  s3.putBucketNotificationConfiguration(params).promise()
                    .then((data)=>respond())
                    .catch((e)=>respond(e));
                }
              };
          Timeout: 30
          Runtime: nodejs4.3
      BucketPermission:
        Type: AWS::Lambda::Permission
        Properties:
          Action: 'lambda:InvokeFunction'
          FunctionName: !Ref BucketWatcher
          Principal: s3.amazonaws.com
          SourceAccount: !Ref "AWS::AccountId"
          SourceArn: !Sub "arn:aws:s3:::${BucketName}"
      BucketWatcher:
        Type: AWS::Lambda::Function
        Properties:
          Description: Sends a Wait Condition signal to Handle when invoked
          Handler: index.handler
          Role: !GetAtt LambdaExecutionRole.Arn
          Code:
            ZipFile: !Sub |
              exports.handler = function(event, context) {
                console.log("Request received:\n", JSON.stringify(event));
                var responseBody = JSON.stringify({
                  "Status" : "SUCCESS",
                  "UniqueId" : "Key",
                  "Data" : event.Records[0].s3.object.key,
                  "Reason" : ""
                });
                var https = require("https");
                var url = require("url");
                var parsedUrl = url.parse('${Handle}');
                var options = {
                    hostname: parsedUrl.hostname,
                    port: 443,
                    path: parsedUrl.path,
                    method: "PUT",
                    headers: {
                        "content-type": "",
                        "content-length": responseBody.length
                    }
                };
                var request = https.request(options, function(response) {
                    console.log("Status code: " + response.statusCode);
                    console.log("Status message: " + response.statusMessage);
                    context.done();
                });
                request.on("error", function(error) {
                    console.log("send(..) failed executing https.request(..): " + error);
                    context.done();
                });
                request.write(responseBody);
                request.end();
              };
          Timeout: 30
          Runtime: nodejs4.3
      Handle:
        Type: AWS::CloudFormation::WaitConditionHandle
      Wait:
        Type: AWS::CloudFormation::WaitCondition
        Properties:
          Handle: !Ref Handle
          Timeout: 300
      S3Object:
        Type: Custom::S3Object
        DependsOn: BucketConfiguration
        Properties:
          ServiceToken: !GetAtt S3ObjectFunction.Arn
          Bucket: !Ref BucketName
          Key: !Ref Key
          Body: !Ref Body
      S3ObjectFunction:
        Type: AWS::Lambda::Function
        Properties:
          Description: S3 Object Custom Resource
          Handler: index.handler
          Role: !GetAtt LambdaExecutionRole.Arn
          Code:
            ZipFile: !Sub |
              var response = require('cfn-response');
              var AWS = require('aws-sdk');
              var s3 = new AWS.S3();
              exports.handler = function(event, context) {
                var respond = (e) => response.send(event, context, e ? response.FAILED : response.SUCCESS, e ? e : {});
                var params = event.ResourceProperties;
                delete params.ServiceToken;
                if (event.RequestType == 'Create' || event.RequestType == 'Update') {
                  s3.putObject(params).promise()
                    .then((data)=>respond())
                    .catch((e)=>respond(e));
                } else if (event.RequestType == 'Delete') {
                  delete params.Body;
                  s3.deleteObject(params).promise()
                    .then((data)=>respond())
                    .catch((e)=>respond(e));
                } else {
                  respond({Error: 'Invalid request type'});
                }
              };
          Timeout: 30
          Runtime: nodejs4.3
      LambdaExecutionRole:
        Type: AWS::IAM::Role
        Properties:
          AssumeRolePolicyDocument:
            Version: '2012-10-17'
            Statement:
            - Effect: Allow
              Principal: {Service: [lambda.amazonaws.com]}
              Action: ['sts:AssumeRole']
          Path: /
          ManagedPolicyArns:
          - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
          Policies:
          - PolicyName: S3Policy
            PolicyDocument:
              Version: '2012-10-17'
              Statement:
                - Effect: Allow
                  Action:
                    - 's3:PutObject'
                    - 'S3:DeleteObject'
                  Resource: !Sub "arn:aws:s3:::${BucketName}/${Key}"
      NotificationBucketPolicy:
        Type: AWS::S3::BucketPolicy
        Properties:
          Bucket: !Ref BucketName
          PolicyDocument:
            Statement:
              - Effect: "Allow"
                Action:
                - 's3:PutBucketNotification'
                Resource: !Sub "arn:aws:s3:::${BucketName}"
                Principal:
                  AWS: !GetAtt LambdaExecutionRole.Arn
    Outputs:
      Result:
        Value: !GetAtt Wait.Data