Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/281.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 列出开始日期(包括)和结束日期(不包括)之间的所有S3键_Python_Amazon S3_Boto3_Prefix - Fatal编程技术网

Python 列出开始日期(包括)和结束日期(不包括)之间的所有S3键

Python 列出开始日期(包括)和结束日期(不包括)之间的所有S3键,python,amazon-s3,boto3,prefix,Python,Amazon S3,Boto3,Prefix,是否有办法列出指定日期之间的所有s3文件。开始日期可以作为前缀传递。我对如何通过截止日期感到困惑。请帮忙 import boto3 def get_matching_s3_objects(bucket, prefix=''): """ Generate objects in an S3 bucket. :param bucket: Name of the S3 bucket. :param prefix: Only fetch objects whose k

是否有办法列出指定日期之间的所有s3文件。开始日期可以作为前缀传递。我对如何通过截止日期感到困惑。请帮忙

import boto3


def get_matching_s3_objects(bucket, prefix=''):
    """
    Generate objects in an S3 bucket.

    :param bucket: Name of the S3 bucket.
    :param prefix: Only fetch objects whose key starts with
        this prefix
    """
    s3 = boto3.client('s3')
    kwargs = {'Bucket': bucket}

    if isinstance(prefix, str):
        kwargs['Prefix'] = prefix

    while True:

        # The S3 API response is a large blob of metadata.
        # 'Contents' contains information about the listed objects.
        resp = s3.list_objects_v2(**kwargs)

        try:
            contents = resp['Contents']
        except KeyError:
            return

        for obj in contents:
            key = obj['Key']
            if key.startswith(prefix) and key.endswith(suffix):
                yield obj

        # The S3 API is paginated, returning up to 1000 keys at a time.
        # Pass the continuation token into the next response, until we
        # reach the final page (when this field is missing).
        try:
            kwargs['ContinuationToken'] = resp['NextContinuationToken']
        except KeyError:
            break


def get_matching_s3_keys(bucket, prefix=''):
    """
    Generate the keys in an S3 bucket.

    :param bucket: Name of the S3 bucket.
    :param prefix: Only fetch keys that start with this prefix (optional).
    :param suffix: Only fetch keys that end with this suffix (optional).
    """
    for obj in get_matching_s3_objects(bucket, prefix, suffix):
        yield obj['Key']

如果没有直接的方法使用boto3按日期过滤,那么只有Bucket、分隔符、编码类型、标记、MaxKeys、前缀和RequestPayer

因此,您需要在键/对象上循环,以将开始/结束日期与对象上次修改的日期时间值进行比较,以便在一周前包含到今天排除到某个特定存储桶中的所有对象,我将执行以下操作

从datetime导入datetime,timedelta 进口boto3 从pytz导入UTC作为UTC 注意:我们需要时区感知对象,因为s3对象是一个。 今天=utc.localizedatetime.utcnow 自=今天-timedeltaweeks=1 警告: -调用bot3.resource时,您可能需要提供正确的凭据。。。 -如果bucket不存在,则需要添加错误管理。 钥匙=[ o代表boto3.resource's3.Bucketname='some_bucket'.objects.all中的o 如果o.last_modified<今天和o.last_modified>=自 ]