Java 如何实现非';t字节数组输入流?

Java 如何实现非';t字节数组输入流?,java,asynchttpclient,Java,Asynchttpclient,我经常从网上下载很多(可能很大)文件 在我的特殊情况下,我需要将这些下载URL的字节输入流发送到另一个服务进行解析 一种天真的方法是这样做: AsyncHttpClient asyncHttpClient = Dsl.asyncHttpClient(Dsl.config() .setMaxConnectionsPerHost(-1) .setMaxConnections(-1) .setPooledConnectionIdleTimeout(60 * 10 * 1000)

我经常从网上下载很多(可能很大)文件

在我的特殊情况下,我需要将这些下载URL的字节输入流发送到另一个服务进行解析

一种天真的方法是这样做:

AsyncHttpClient asyncHttpClient = Dsl.asyncHttpClient(Dsl.config()
    .setMaxConnectionsPerHost(-1)
    .setMaxConnections(-1)
    .setPooledConnectionIdleTimeout(60 * 10 * 1000)
    .setConnectionTtl(6 * 60 * 1000)
    .setConnectTimeout(5 * 1000)
    .setRequestTimeout(5 * 60 * 1000)
    .setFollowRedirect(true)
    .setRealm(new Realm.Builder(username, password)
        .setNtlmDomain(domain)
        .setScheme(Realm.AuthScheme.NTLM)
        .build())
Response httpGetResponse = asyncHttpClient.prepareGet(url).execute().get();
return httpGetResponse.getResponseBodyAsStream();
但我们了解到,与HTTP组件HTTP客户端不同,异步HTTP客户端将把整个文件下载到内存中

在我的情况下,这将很快导致OOM

因此,另一种选择是:

Response httpGetResponse = asyncHttpClient.prepareGet(url).execute(new AsyncHandler<Response>() {
    private final Response.ResponseBuilder builder = new Response.ResponseBuilder();

    @Override
    public State onBodyPartReceived(HttpResponseBodyPart bodyPart) throws Exception {
      bodyPart.getBodyByteBuffer(); // Each chunk of bytes will be fed into this method.
                                    // I need to write these bytes to the resuting input stream
                                    // without streaming them all into memory.
      return State.CONTINUE;
    }

    @Override
    public State onHeadersReceived(HttpHeaders headers) throws Exception {
      builder.accumulate(headers);
      return State.CONTINUE;
    }

    @Override
    public State onStatusReceived(HttpResponseStatus responseStatus) throws Exception {
      builder.accumulate(responseStatus);
      return State.CONTINUE;
    }

    @Override
    public Response onCompleted() throws Exception {
      return builder.build();
    }

    @Override
    public void onThrowable(Throwable t) {

    }
  }).get();
Response-httpGetResponse=asyncHttpClient.prepareGet(url).execute(new-asynchHandler()){
private final Response.ResponseBuilder builder=new Response.ResponseBuilder();
@凌驾
onBodyPartReceived的公共状态(HttpResponseBodyPart bodyPart)引发异常{
bodyPart.getBodyByteBuffer();//每个字节块都将被送入此方法。
//我需要将这些字节写入正在恢复的输入流
//不需要把它们全部存入内存。
返回状态。继续;
}
@凌驾
公共状态onHeadersReceived(HttpHeaders)引发异常{
累计(表头);
返回状态。继续;
}
@凌驾
公共状态onStatusReceived(HttpResponseStatus responseStatus)引发异常{
建造商累积(responseStatus);
返回状态。继续;
}
@凌驾
公共响应onCompleted()引发异常{
返回builder.build();
}
@凌驾
可滚动的公共空间(可丢弃的t){
}
}).get();
当这些字节进入输入流时,获取它们的最简单、最干净的方法是什么

我有两个想法:

1) 将输入写入文件,然后对文件进行流式处理 或 2) 立即返回一个管道输入流,接收到的字节将写入管道输入流


有没有人可以与我们分享一个有效的例子

我正确地认为有人已经这样做了。事实上,在搜索“异步http客户端”和“管道输入流”之后,我在项目本身中发现了以下内容:

用法:

  PipedInputStream pipedInputStream = new PipedInputStream();
  PipedOutputStream pipedOutputStream = new PipedOutputStream(pipedInputStream);
  BodyDeferringAsyncHandler bodyDeferringAsyncHandler = new BodyDeferringAsyncHandler(pipedOutputStream);
  Future<Response> futureResponse = asyncHttpClient.prepareGet(url).execute(bodyDeferringAsyncHandler);
  Response response = bodyDeferringAsyncHandler.getResponse();
  if (response.getStatusCode() == 200) {
    return new BodyDeferringAsyncHandler.BodyDeferringInputStream(futureResponse,
        bodyDeferringAsyncHandler,
        pipedInputStream);
  } else {
    return null;
  }
PipedInputStream PipedInputStream=new PipedInputStream();
PipedOutputStream PipedOutputStream=新的PipedOutputStream(pipedInputStream);
BodyDeferringAsyncHandler BodyDeferringAsyncHandler=新的BodyDeferringAsyncHandler(PipedOutStream);
Future futureResponse=asyncHttpClient.prepareGet(url).execute(bodyDeferringAsyncHandler);
Response-Response=bodyDeferringAsyncHandler.getResponse();
if(response.getStatusCode()==200){
返回新的BodyDeferringAsyncHandler.BodyDeferringInputStream(futureResponse,
BodyDeleringAsyncHandler,
pipedInputStream);
}否则{
返回null;
}