Amazon s3 AmazonS3中的TransferManager与PutRequestObject

Amazon s3 AmazonS3中的TransferManager与PutRequestObject,amazon-s3,Amazon S3,我对对象单上传和多部分上传感到困惑。两者都在同一时间消费。我的代码如下: File file = new File("D:\\AmazonS3\\aws-java-sdk-1.8.3\\lib\\aws-java-sdk-1.8.3-javadoc.jar"); FileInputStream fis = new FileInputStream(file); String keyName = System.currentTimeMillis()+"_aws-java-sdk-1.8.3-jav

我对对象单上传和多部分上传感到困惑。两者都在同一时间消费。我的代码如下:

File file = new File("D:\\AmazonS3\\aws-java-sdk-1.8.3\\lib\\aws-java-sdk-1.8.3-javadoc.jar");
FileInputStream fis = new FileInputStream(file);

String keyName = System.currentTimeMillis()+"_aws-java-sdk-1.8.3-javadoc.jar";

ObjectMetadata metaData = new ObjectMetadata();
metaData.addUserMetadata("test","TEST");
//Object single upload
PutObjectRequest putobejcObjectRequest = new PutObjectRequest(BUCKET_NAME, keyName, fis,metaData);
putobejcObjectRequest.setMetadata(metaData);
s3client.putObject(putobejcObjectRequest);

//Object multipart upload
TransferManagerConfiguration configuration = new TransferManagerConfiguration();
    configuration.setMultipartUploadThreshold(5*com.amazonaws.services.s3.internal.Constants.MB);
TransferManager transferManager = new TransferManager(s3client);
transferManager.setConfiguration(configuration);

Upload upload = transferManager.upload(BUCKET_NAME, keyName, fis,metaData);
upload.waitForCompletion();
transferManager.shutdownNow();

请帮助我,我的代码中是否存在任何问题。

我也遇到了同样的问题,并且(通过检查SDK代码)发现transfer manager仅在您传递文件而不是InputStream时才会使用并行部件上载

查看sdk中的决策代码(版本1.8.9):

其中可上载并行化:

// Each uploaded part in an encrypted upload depends on the encryption context
// from the previous upload, so we cannot parallelize encrypted upload parts.
if (isUsingEncryption) return false;

// Otherwise, if there's a file, we can process the uploads concurrently.
return (getRequestFile(putObjectRequest) != null);
所以,如果您想获得部分并行上传的优势,请将一个文件传递给TransferManager

从流上载选项时,呼叫者必须提供 通过 ObjectMetadata参数。如果未为指定内容长度 输入流,然后TransferManager将尝试缓冲所有 将内容流式存储在内存中,并以传统方式上载选项, 单部分上传。因为整个流的内容必须是 缓冲在内存中,这可能非常昂贵,应该避免 只要可能


来源:

这是否意味着如果我想上传一个>5GB的加密文件,使用SDK是不可能的?由于TransferManager不支持加密,并且对如此大的文件使用
s3client.putObject
将导致堆空间不足…如果您将内容作为文件,则不会耗尽堆空间,因为它不会加载到内存中
// Each uploaded part in an encrypted upload depends on the encryption context
// from the previous upload, so we cannot parallelize encrypted upload parts.
if (isUsingEncryption) return false;

// Otherwise, if there's a file, we can process the uploads concurrently.
return (getRequestFile(putObjectRequest) != null);