使用Python删除DynamoDB中的所有项
如何使用python(boto3)从DynamoDB中删除所有项 我正试图做到这一点:使用Python删除DynamoDB中的所有项,python,amazon-dynamodb,boto3,Python,Amazon Dynamodb,Boto3,如何使用python(boto3)从DynamoDB中删除所有项 我正试图做到这一点: scan = table.scan() with table.batch_writer() as batch: for each in scan['Items']: batch.delete_item(Key=each) 但请告诉我这个错误: botocore.exceptions.ClientError: An error occurred (ValidationException) when
scan = table.scan()
with table.batch_writer() as batch:
for each in scan['Items']:
batch.delete_item(Key=each)
但请告诉我这个错误:
botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the BatchWriteItem operation: The provided key element does not match the schema
使用
BatchWriteItem
。美国
BatchWriteItem操作在一个或多个表中放置或删除多个项。对BatchWriteItem的单个调用可以写入高达16MB的数据,其中可以包含多达25个put或delete请求。要写入的单个项可以大到400 KB
我假设Boto3 API也有这个功能,但可能有不同的名称。我找到了一个解决方案!我只是用我的表Id和搜索Id(compId)挂载了这个键,它成功了:)
这里的答案考虑了这样一个事实:如果您试图截断一个大表(或一个包含大项的较小表),那么在第一次调用中可能无法恢复所有记录。它假定您只使用了一个HashKey(称为
id
),因此如果表上还有一个SortKey,则必须向ProjectionExpression
和delete\u item
调用中添加一点
有一些额外的,你可以删掉,只是打印一个计数器到标准输出,以保持我们人类的快乐
虽然我同意删除表并重新创建它更有效,但也可能存在这样的情况:当许多GSI或触发器事件与一个表关联时,您不想重新关联这些事件。下面的脚本将迭代扫描以处理大型表(每个扫描调用将返回1Mb的键),并使用批处理函数删除表中的所有项
import boto3
dynamo = boto3.resource('dynamodb')
def truncateTable(tableName):
table = dynamo.Table(tableName)
#get the table keys
tableKeyNames = [key.get("AttributeName") for key in table.key_schema]
#Only retrieve the keys for each item in the table (minimize data transfer)
projectionExpression = ", ".join('#' + key for key in tableKeyNames)
expressionAttrNames = {'#'+key: key for key in tableKeyNames}
counter = 0
page = table.scan(ProjectionExpression=projectionExpression, ExpressionAttributeNames=expressionAttrNames)
with table.batch_writer() as batch:
while page["Count"] > 0:
counter += page["Count"]
# Delete items in batches
for itemKeys in page["Items"]:
batch.delete_item(Key=itemKeys)
# Fetch the next page
if 'LastEvaluatedKey' in page:
page = table.scan(
ProjectionExpression=projectionExpression, ExpressionAttributeNames=expressionAttrNames,
ExclusiveStartKey=page['LastEvaluatedKey'])
else:
break
print(f"Deleted {counter}")
truncateTable("YOUR_TABLE_NAME")
可能的副本请查看:我已经检查了这些链接,但没有任何帮助:(调用delete_table和create_table如何?这将删除条目;但是,请注意,由于项目的分页,您可能无法删除较长表格中的所有内容。@Matt此表格按天删除。不需要对删除内容进行分页。这是两个文档中的内容:此代码是当前答案中最好的。1)使用批处理操作运行得更快,2)使用ExpressionAttributeNames正确处理具有无效字符的键,3)自动发现键名,nice。此代码在此处工作,但请记住,如果通过向扫描中添加筛选器来更改代码,它可能会停止工作,因为在这种情况下,当第[“计数”]=0页时,可能还有下一页。
import boto3
TABLE = ...
ID = ...
table = boto3.resource('dynamodb').Table(TABLE)
scan = None
with table.batch_writer() as batch:
count = 0
while scan is None or 'LastEvaluatedKey' in scan:
if scan is not None and 'LastEvaluatedKey' in scan:
scan = table.scan(
ProjectionExpression=ID,
ExclusiveStartKey=scan['LastEvaluatedKey'],
)
else:
scan = table.scan(ProjectionExpression=ID)
for item in scan['Items']:
if count % 5000 == 0:
print(count)
batch.delete_item(Key={ID: item[ID]})
count = count + 1
import boto3
dynamo = boto3.resource('dynamodb')
def truncateTable(tableName):
table = dynamo.Table(tableName)
#get the table keys
tableKeyNames = [key.get("AttributeName") for key in table.key_schema]
#Only retrieve the keys for each item in the table (minimize data transfer)
projectionExpression = ", ".join('#' + key for key in tableKeyNames)
expressionAttrNames = {'#'+key: key for key in tableKeyNames}
counter = 0
page = table.scan(ProjectionExpression=projectionExpression, ExpressionAttributeNames=expressionAttrNames)
with table.batch_writer() as batch:
while page["Count"] > 0:
counter += page["Count"]
# Delete items in batches
for itemKeys in page["Items"]:
batch.delete_item(Key=itemKeys)
# Fetch the next page
if 'LastEvaluatedKey' in page:
page = table.scan(
ProjectionExpression=projectionExpression, ExpressionAttributeNames=expressionAttrNames,
ExclusiveStartKey=page['LastEvaluatedKey'])
else:
break
print(f"Deleted {counter}")
truncateTable("YOUR_TABLE_NAME")