Java 带有InputStream长度示例的AmazonS3 putObject

我正在使用Java将一个文件上载到S3-这是我目前得到的：

AmazonS3 s3 = new AmazonS3Client(new BasicAWSCredentials("XX","YY"));

List<Bucket> buckets = s3.listBuckets();

s3.putObject(new PutObjectRequest(buckets.get(0).getName(), fileName, stream, new ObjectMetadata()));

这是我正在上传的文件，

stream

变量是一个

InputStream

，我可以从中获得如下字节数组：

IOUtils.toByteArray（stream）

因此，当我尝试如下设置内容长度和MD5（取自）时：

// get MD5 base64 hash
MessageDigest messageDigest = MessageDigest.getInstance("MD5");
messageDigest.reset();
messageDigest.update(IOUtils.toByteArray(stream));
byte[] resultByte = messageDigest.digest();
String hashtext = new String(Hex.encodeHex(resultByte));

ObjectMetadata meta = new ObjectMetadata();
meta.setContentLength(IOUtils.toByteArray(stream).length);
meta.setContentMD5(hashtext);

它导致从S3返回以下错误：

您指定的Content-MD5无效

我做错了什么

感谢您的帮助

---

p.S.我在Google App Engine上-我无法将文件写入磁盘，或者因为AppEngine不支持FileOutputStream。

在写入S3时，需要指定S3对象的长度，以确保没有内存不足错误

使用

IOUtils.toByteArray（stream）

也容易出现OOM错误，因为这是由ByteArrayOutputStream支持的

---

因此，最好的选择是首先将inputstream写入本地磁盘上的一个临时文件，然后通过指定临时文件的长度使用该文件写入S3。

如果您要做的只是从amazon解决内容长度错误，那么您可以将输入流中的字节读取到Long，然后将其添加到元数据中

/*
 * Obtain the Content length of the Input stream for S3 header
 */
try {
    InputStream is = event.getFile().getInputstream();
    contentBytes = IOUtils.toByteArray(is);
} catch (IOException e) {
    System.err.printf("Failed while reading bytes from %s", e.getMessage());
} 

Long contentLength = Long.valueOf(contentBytes.length);

ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(contentLength);

/*
 * Reobtain the tmp uploaded file as input stream
 */
InputStream inputStream = event.getFile().getInputstream();

/*
 * Put the object in S3
 */
try {

    s3client.putObject(new PutObjectRequest(bucketName, keyName, inputStream, metadata));

} catch (AmazonServiceException ase) {
    System.out.println("Error Message:    " + ase.getMessage());
    System.out.println("HTTP Status Code: " + ase.getStatusCode());
    System.out.println("AWS Error Code:   " + ase.getErrorCode());
    System.out.println("Error Type:       " + ase.getErrorType());
    System.out.println("Request ID:       " + ase.getRequestId());
} catch (AmazonClientException ace) {
    System.out.println("Error Message: " + ace.getMessage());
} finally {
    if (inputStream != null) {
        inputStream.close();
    }
}

---

您需要使用这种精确的方法读取输入流两次，因此，如果您正在上载一个非常大的文件，您可能需要将其读取到数组中一次，然后再从数组中读取。

因为原始问题从未得到回答，我不得不遇到同样的问题，MD5问题的解决方案是S3不需要我们通常所想到的十六进制编码的MD5字符串

相反，我不得不这样做

// content is a passed in InputStream
byte[] resultByte = DigestUtils.md5(content);
String streamMD5 = new String(Base64.encodeBase64(resultByte));
metaData.setContentMD5(streamMD5);

---

本质上，他们想要的MD5值是Base64编码的原始MD5字节数组，而不是十六进制字符串。当我换成这个，它开始为我工作的伟大

我实际上在做一些相同的事情，但在我的AWS S3存储上：-

接收上载文件的servlet的代码：-

import java.io.IOException;
import java.io.PrintWriter;
import java.util.List;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;

import com.src.code.s3.S3FileUploader;

public class FileUploadHandler extends HttpServlet {

    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        doPost(request, response);
    }

    protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        PrintWriter out = response.getWriter();

        try{
            List<FileItem> multipartfiledata = new ServletFileUpload(new DiskFileItemFactory()).parseRequest(request);

            //upload to S3
            S3FileUploader s3 = new S3FileUploader();
            String result = s3.fileUploader(multipartfiledata);

            out.print(result);
        } catch(Exception e){
            System.out.println(e.getMessage());
        }
    }
}

import java.io.IOException；
导入java.io.PrintWriter；
导入java.util.List；
导入javax.servlet.ServletException；
导入javax.servlet.http.HttpServlet；
导入javax.servlet.http.HttpServletRequest；
导入javax.servlet.http.HttpServletResponse；
导入org.apache.commons.fileupload.FileItem；
导入org.apache.commons.fileupload.disk.DiskFileItemFactory；
导入org.apache.commons.fileupload.servlet.ServletFileUpload；
导入com.src.code.s3.S3FileUploader；
公共类FileUploadHandler扩展了HttpServlet{
受保护的void doGet（HttpServletRequest请求，HttpServletResponse响应）抛出ServletException，IOException{
doPost（请求、响应）；
}
受保护的void doPost（HttpServletRequest请求、HttpServletResponse响应）引发ServletException、IOException{
PrintWriter out=response.getWriter（）；
试一试{
List multipartfiledata=new ServletFileUpload（new DiskFileItemFactory（））.parseRequest（request）；
//上传到S3
S3FileUploader s3=新的S3FileUploader（）；
字符串结果=s3.fileUploader（multipartfiledata）；
打印（结果）；
}捕获（例外e）{
System.out.println（e.getMessage（））；
}
}
}

作为AWS对象上载此数据的代码：-

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.util.List;
import java.util.UUID;

import org.apache.commons.fileupload.FileItem;

import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.auth.ClasspathPropertiesFileCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.S3Object;

public class S3FileUploader {


    private static String bucketName     = "***NAME OF YOUR BUCKET***";
    private static String keyName        = "Object-"+UUID.randomUUID();

    public String fileUploader(List<FileItem> fileData) throws IOException {
        AmazonS3 s3 = new AmazonS3Client(new ClasspathPropertiesFileCredentialsProvider());
        String result = "Upload unsuccessfull because ";
        try {

            S3Object s3Object = new S3Object();

            ObjectMetadata omd = new ObjectMetadata();
            omd.setContentType(fileData.get(0).getContentType());
            omd.setContentLength(fileData.get(0).getSize());
            omd.setHeader("filename", fileData.get(0).getName());

            ByteArrayInputStream bis = new ByteArrayInputStream(fileData.get(0).get());

            s3Object.setObjectContent(bis);
            s3.putObject(new PutObjectRequest(bucketName, keyName, bis, omd));
            s3Object.close();

            result = "Uploaded Successfully.";
        } catch (AmazonServiceException ase) {
           System.out.println("Caught an AmazonServiceException, which means your request made it to Amazon S3, but was "
                + "rejected with an error response for some reason.");

           System.out.println("Error Message:    " + ase.getMessage());
           System.out.println("HTTP Status Code: " + ase.getStatusCode());
           System.out.println("AWS Error Code:   " + ase.getErrorCode());
           System.out.println("Error Type:       " + ase.getErrorType());
           System.out.println("Request ID:       " + ase.getRequestId());

           result = result + ase.getMessage();
        } catch (AmazonClientException ace) {
           System.out.println("Caught an AmazonClientException, which means the client encountered an internal error while "
                + "trying to communicate with S3, such as not being able to access the network.");

           result = result + ace.getMessage();
         }catch (Exception e) {
             result = result + e.getMessage();
       }

        return result;
    }
}

import java.io.ByteArrayInputStream；
导入java.io.IOException；
导入java.util.List；
导入java.util.UUID；
导入org.apache.commons.fileupload.FileItem；
导入com.amazonaws.AmazonClientException；
导入com.amazonaws.AmazonServiceException；
导入com.amazonaws.auth.ClasspathPropertiesFileCredentialsProvider；
导入com.amazonaws.services.s3.AmazonS3；
导入com.amazonaws.services.s3.amazon3Client；
导入com.amazonaws.services.s3.model.ObjectMetadata；
导入com.amazonaws.services.s3.model.PutObjectRequest；
导入com.amazonaws.services.s3.model.s3对象；
公共类S3FileUploader{
私有静态字符串bucketName=“***您的BUCKET的名称***”；
私有静态字符串keyName=“Object-”+UUID.randomUUID（）；
公共字符串文件上载程序（列表文件数据）引发IOException{
AmazonS3 s3=新的AmazonS3客户端（新的类路径属性FileCredentialsProvider（））；
String result=“上传失败原因”；
试一试{
S3Object S3Object=新的S3Object（）；
ObjectMetadata omd=新的ObjectMetadata（）；
setContentType（fileData.get（0.getContentType（））；
setContentLength（fileData.get（0.getSize（））；
setHeader（“filename”，fileData.get（0.getName（））；
ByteArrayInputStream bis=新的ByteArrayInputStream（fileData.get（0.get（））；
s3Object.setObjectContent（bis）；
s3.putObject（新的PutObjectRequest（bucketName、keyName、bis、omd））；
s3Object.close（）；
结果=“已成功上载。”；
}捕获（AmazonServiceException ase）{
println（“捕获到一个AmazonServiceException，这意味着您的请求向AmazonS3发出，但被拒绝了”
+“由于某种原因被拒绝，并有错误响应。”）；
System.out.println（“错误消息：+ase.getMessage（））；
System.out.println（“HTTP状态代码：+ase.getStatusCode（））；
System.out.println（“AWS错误代码：+ase.getErrorCode（））；
System.out.println（“错误类型：+ase.getErrorType（））；
System.out.println（“请求ID:+ase.getRequestId（））；
结果=结果+ase.getMessage（）；
}捕获（AmazonClientException ace）{
System.out.println（“捕获到AmazonClientException，这意味着客户端在运行时遇到内部错误”
+“试图与S3通信，例如无法访问网络。”）；
result=result+ace.getMessage（）；
}捕获（例外e）{
结果=结果+e
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.util.List;
import java.util.UUID;

import org.apache.commons.fileupload.FileItem;

import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.auth.ClasspathPropertiesFileCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.S3Object;

public class S3FileUploader {


    private static String bucketName     = "***NAME OF YOUR BUCKET***";
    private static String keyName        = "Object-"+UUID.randomUUID();

    public String fileUploader(List<FileItem> fileData) throws IOException {
        AmazonS3 s3 = new AmazonS3Client(new ClasspathPropertiesFileCredentialsProvider());
        String result = "Upload unsuccessfull because ";
        try {

            S3Object s3Object = new S3Object();

            ObjectMetadata omd = new ObjectMetadata();
            omd.setContentType(fileData.get(0).getContentType());
            omd.setContentLength(fileData.get(0).getSize());
            omd.setHeader("filename", fileData.get(0).getName());

            ByteArrayInputStream bis = new ByteArrayInputStream(fileData.get(0).get());

            s3Object.setObjectContent(bis);
            s3.putObject(new PutObjectRequest(bucketName, keyName, bis, omd));
            s3Object.close();

            result = "Uploaded Successfully.";
        } catch (AmazonServiceException ase) {
           System.out.println("Caught an AmazonServiceException, which means your request made it to Amazon S3, but was "
                + "rejected with an error response for some reason.");

           System.out.println("Error Message:    " + ase.getMessage());
           System.out.println("HTTP Status Code: " + ase.getStatusCode());
           System.out.println("AWS Error Code:   " + ase.getErrorCode());
           System.out.println("Error Type:       " + ase.getErrorType());
           System.out.println("Request ID:       " + ase.getRequestId());

           result = result + ase.getMessage();
        } catch (AmazonClientException ace) {
           System.out.println("Caught an AmazonClientException, which means the client encountered an internal error while "
                + "trying to communicate with S3, such as not being able to access the network.");

           result = result + ace.getMessage();
         }catch (Exception e) {
             result = result + e.getMessage();
       }

        return result;
    }
}

PutObjectRequest(String bucketName, String key, File file)

PutObjectRequest(String bucketName, String key, InputStream input, ObjectMetadata metadata)

AmazonS3 s3Service = new AmazonS3Client(awsCredentials);
File scratchFile = File.createTempFile("prefix", "suffix");
try {
    FileUtils.copyInputStreamToFile(inputStream, scratchFile);    
    PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName, id, scratchFile);
    PutObjectResult putObjectResult = s3Service.putObject(putObjectRequest);

} finally {
    if(scratchFile.exists()) {
        scratchFile.delete();
    }
}

amazonS3.putObject(bucketName, id,fileObject);