Java 无法将所有数据写入dynamodb表?

Java 无法将所有数据写入dynamodb表?,java,amazon-web-services,amazon-s3,amazon-dynamodb,amazon-cloudwatch,Java,Amazon Web Services,Amazon S3,Amazon Dynamodb,Amazon Cloudwatch,我试图将csv文件数据插入dynamodb,但我只能将1537条记录写入dynamodb表 编码 try { S3EventNotificationRecord record = s3event.getRecords().get(0); String srcBucket = record.getS3().getBucket().getName(); String srcKey = record.getS3().getObject().getKe

我试图将csv文件数据插入dynamodb,但我只能将1537条记录写入dynamodb表

编码

 try {

        S3EventNotificationRecord record = s3event.getRecords().get(0);
        String srcBucket = record.getS3().getBucket().getName();
        String srcKey = record.getS3().getObject().getKey().replace('+', ' ');  
        srcKey = URLDecoder.decode(srcKey, "UTF-8");
        AmazonS3 s3Client = new AmazonS3Client();
        S3Object s3Object = s3Client.getObject(new GetObjectRequest(srcBucket, srcKey));
        statusReport.setFileSize(s3Object.getObjectMetadata().getContentLength());


        BufferedReader br = new BufferedReader(new InputStreamReader(s3Object.getObjectContent())); 
        CSVReader reader = new CSVReader(br);

        AmazonDynamoDB dynamoDBClient = new AmazonDynamoDBClient();

        dynamoDBClient.setRegion(AWS_REGION);
        DynamoDB dynamoDB = new DynamoDB(dynamoDBClient);
        TableWriteItems energyDataTableWriteItems = new TableWriteItems(DYNAMO_TABLE_NAME);
        List<Item> itemList = new ArrayList<Item>();
        String[] nextLine;


        while ((nextLine = reader.readNext()) != null) {
            Item newItem = helper.parseIt(nextLine);
            itemList.add(newItem);
        }
for (List<Item> partition : Lists.partition(itemList, 25)) {
            energyDataTableWriteItems.withItemsToPut(partition);
            BatchWriteItemOutcome outcome = dynamoDB.batchWriteItem(energyDataTableWriteItems);

 do {
        Map<String, List<WriteRequest>> unprocessedItems = outcome.getUnprocessedItems();

        if (outcome.getUnprocessedItems().size() > 0) {
                    logger.log("Retrieving the unprocessed " + String.valueOf(outcome.getUnprocessedItems().size())
                            + " items.");
                    outcome = dynamoDB.batchWriteItemUnprocessed(unprocessedItems);
                }

        } while (outcome.getUnprocessedItems().size() > 0);*/
}
 logger.log("Load finish in " + String.valueOf(System.currentTimeMillis() - startTime) + "ms");

        reader.close();
        br.close();
        s3Object.close();

        statusReport.setStatus(true);
    } catch (Exception ex) {
        logger.log(ex.getMessage());
    }

    statusReport.setExecutiongTime(System.currentTimeMillis() - startTime);
    return statusReport;
}
试试看{
S3EventNotificationRecord记录=s3event.getRecords().get(0);
字符串srcBucket=record.getS3().getBucket().getName();
字符串srcKey=record.getS3();
srcKey=urldecker.decode(srcKey,“UTF-8”);
AmazonS3 s3Client=新的AmazonS3客户端();
S3Object S3Object=s3Client.getObject(新的GetObjectRequest(srcBucket,srcKey));
statusReport.setFileSize(s3Object.getObjectMetadata().getContentLength());
BufferedReader br=新的BufferedReader(新的InputStreamReader(s3Object.getObjectContent());
CSVReader reader=新的CSVReader(br);
AmazonDynamoDB dynamoDBClient=新的AmazonDynamoDBClient();
DYNAMODB客户设置区域(AWS_区域);
DynamoDB DynamoDB=新的DynamoDB(DynamoDB客户端);
TableWriteItems energyDataTableWriteItems=新的TableWriteItems(发电机\u表\u名称);
List itemList=new ArrayList();
字符串[]下一行;
而((nextLine=reader.readNext())!=null){
Item newItem=helper.parseIt(nextLine);
itemList.add(newItem);
}
for(列表分区:Lists.partition(itemList,25)){
energyDataTableWriteItems.withItemsToPut(分区);
BatchWriteItemOutcome结果=dynamoDB.batchWriteItem(energyDataTableWriteItems);
做{
Map unprocessedItems=output.getUnprocessedItems();
if(output.getUnprocessedItems().size()>0){
logger.log(“检索未处理的”+String.valueOf(output.getUnprocessedItems().size())
+“项目。”);
结果=dynamoDB.batchWriteItemUnprocessed(未处理);
}
}while(output.getUnprocessedItems().size()>0)*/
}
logger.log(“加载完成于”+String.valueOf(System.currentTimeMillis()-startTime)+“ms”);
reader.close();
br.close();
s3Object.close();
statusReport.setStatus(真);
}捕获(例外情况除外){
logger.log(例如getMessage());
}
statusReport.setExecutiongTime(System.currentTimeMillis()-startTime);
返回状态报告;
}

我能知道它这样做的原因吗。我尝试了5个以上的表,但得到了相同的结果。

您可能遇到了吞吐量限制:

从:

BatchWriteItem操作在一个或多个表中放置或删除多个项。对BatchWriteItem的单个调用可以写入高达16MB的数据,其中可以包含多达25个put或delete请求。要写入的单个项可以大到400 KB

此外:

如果DynamoDB返回任何未处理的项,则应重试这些项的批处理操作

当您将写操作划分为25个项目块时,这并不是唯一的限制。如果查看
BatchWriteItemOutcome
的JavaDoc,您将看到函数
getUnprocessedItems()


最后,您没有显示您的
catch
块,但我们可以假设它确实起了作用吗?

谢谢@guest的快速回复,请检查问题我添加了我的catch块。您能告诉我如何解决此问题吗。