Python 列出开始日期(包括)和结束日期(不包括)之间的所有S3键
是否有办法列出指定日期之间的所有s3文件。开始日期可以作为前缀传递。我对如何通过截止日期感到困惑。请帮忙Python 列出开始日期(包括)和结束日期(不包括)之间的所有S3键,python,amazon-s3,boto3,prefix,Python,Amazon S3,Boto3,Prefix,是否有办法列出指定日期之间的所有s3文件。开始日期可以作为前缀传递。我对如何通过截止日期感到困惑。请帮忙 import boto3 def get_matching_s3_objects(bucket, prefix=''): """ Generate objects in an S3 bucket. :param bucket: Name of the S3 bucket. :param prefix: Only fetch objects whose k
import boto3
def get_matching_s3_objects(bucket, prefix=''):
"""
Generate objects in an S3 bucket.
:param bucket: Name of the S3 bucket.
:param prefix: Only fetch objects whose key starts with
this prefix
"""
s3 = boto3.client('s3')
kwargs = {'Bucket': bucket}
if isinstance(prefix, str):
kwargs['Prefix'] = prefix
while True:
# The S3 API response is a large blob of metadata.
# 'Contents' contains information about the listed objects.
resp = s3.list_objects_v2(**kwargs)
try:
contents = resp['Contents']
except KeyError:
return
for obj in contents:
key = obj['Key']
if key.startswith(prefix) and key.endswith(suffix):
yield obj
# The S3 API is paginated, returning up to 1000 keys at a time.
# Pass the continuation token into the next response, until we
# reach the final page (when this field is missing).
try:
kwargs['ContinuationToken'] = resp['NextContinuationToken']
except KeyError:
break
def get_matching_s3_keys(bucket, prefix=''):
"""
Generate the keys in an S3 bucket.
:param bucket: Name of the S3 bucket.
:param prefix: Only fetch keys that start with this prefix (optional).
:param suffix: Only fetch keys that end with this suffix (optional).
"""
for obj in get_matching_s3_objects(bucket, prefix, suffix):
yield obj['Key']
如果没有直接的方法使用boto3按日期过滤,那么只有Bucket、分隔符、编码类型、标记、MaxKeys、前缀和RequestPayer 因此,您需要在键/对象上循环,以将开始/结束日期与对象上次修改的日期时间值进行比较,以便在一周前包含到今天排除到某个特定存储桶中的所有对象,我将执行以下操作 从datetime导入datetime,timedelta 进口boto3 从pytz导入UTC作为UTC 注意:我们需要时区感知对象,因为s3对象是一个。 今天=utc.localizedatetime.utcnow 自=今天-timedeltaweeks=1 警告: -调用bot3.resource时,您可能需要提供正确的凭据。。。 -如果bucket不存在,则需要添加错误管理。 钥匙=[ o代表boto3.resource's3.Bucketname='some_bucket'.objects.all中的o 如果o.last_modified<今天和o.last_modified>=自 ]