使用java api下载大于chunksize的文件时,google云存储意外终止ZLIB输入流
我正在尝试从play store下载评论,这些评论可以使用JavaAPI在bucket中访问(详细信息见消息末尾)。对于压缩后大于块大小的审阅文件,我得到以下异常:使用java api下载大于chunksize的文件时,google云存储意外终止ZLIB输入流,java,google-cloud-storage,Java,Google Cloud Storage,我正在尝试从play store下载评论,这些评论可以使用JavaAPI在bucket中访问(详细信息见消息末尾)。对于压缩后大于块大小的审阅文件,我得到以下异常: java.io.EOFException: Unexpected end of ZLIB input stream at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240) at java.util.zip.InflaterInputStream.
java.io.EOFException: Unexpected end of ZLIB input stream
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:116)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at com.google.api.client.util.ByteStreams.copy(ByteStreams.java:51)
at com.google.api.client.util.IOUtils.copy(IOUtils.java:94)
at com.google.api.client.util.IOUtils.copy(IOUtils.java:63)
at com.google.api.client.googleapis.media.MediaHttpDownloader.executeCurrentRequest(MediaHttpDownloader.java:247)
at com.google.api.client.googleapis.media.MediaHttpDownloader.download(MediaHttpDownloader.java:199)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeMediaAndDownloadTo(AbstractGoogleClientRequest.java:553)
at com.google.api.services.storage.Storage$Objects$Get.executeMediaAndDownloadTo(Storage.java:4540)
at com.google.api.services.storage.Storage$Objects$Get$executeMediaAndDownloadTo.call(Unknown Source)
我使用的代码是:
private void saveObject(StorageObject object) {
try {
System.out.println("Getting object data");
FileOutputStream fos = new FileOutputStream (new File(object.getName()));
Storage.Objects.Get getObject =
storage.objects().get("pubsite_prod_rev_XXXXX", object.getName());
// setting the chunksize to a small value fails pretty much on all files
getObject.getMediaHttpDownloader().setDirectDownloadEnabled(false).setChunkSize(MediaHttpUploader.MINIMUM_CHUNK_SIZE)
// setting the chunksize to larger value allows me to download some of the files but fails on larger ones
//getObject.getMediaHttpDownloader().setDirectDownloadEnabled(false).setChunkSize(MediaHttpDownloader.MAXIMUM_CHUNK_SIZE)
getObject.executeMediaAndDownloadTo(fos);
fos.close();
} catch(Exception e) {
System.err.println("Error downloading " + object.getName());
e.printStackTrace();
}
我可以使用gsutil下载文件,因此MediaHttpDownloader中的分块实现似乎存在问题
我正在使用Java7和下面的libs
compile group: 'com.google.api-client', name: 'google-api-client', version: '1.19.0'
compile group: 'com.google.oauth-client', name: 'google-oauth-client-jetty', version: '1.19.0'
compile group: 'com.google.apis', name: 'google-api-services-storage', version: 'v1beta2-rev56-1.19.0'
我也在使用groovy,但不知道这会如何影响问题,因为错误在google库中
谢谢你的帮助
andre我通过使用refection绕过检查,将chunksize设置为高于最大chunksize的值,从而解决了这个问题。它似乎起作用了<代码>字段字段=getObject.getMediaHttpDownloader().getClass().getDeclaredField(“chunkSize”);字段。setAccessible(true);set(getObject.getMediaHttpDownloader(),MediaHttpDownloader.MAXIMUM_CHUNK_SIZE*10)代码>这是一个有趣的问题。我看到您正在使用Google云存储的JSON API v1beta2。如果使用v1,是否会出现相同的问题?