DataFileWriter关闭方法上挂起的Avro序列化
我试图使用avro序列化,但当我有多条记录要序列化时,应用程序挂起在DataFileWriter close方法上,但它可以处理少量记录DataFileWriter关闭方法上挂起的Avro序列化,avro,Avro,我试图使用avro序列化,但当我有多条记录要序列化时,应用程序挂起在DataFileWriter close方法上,但它可以处理少量记录 final PipedOutputStream pipedOutputStream = new PipedOutputStream(); PipedInputStream pipedInputStream = new PipedInputStream( pipedOutputStream); DatumWrit
final PipedOutputStream pipedOutputStream = new PipedOutputStream();
PipedInputStream pipedInputStream = new PipedInputStream(
pipedOutputStream);
DatumWriter<DW> userDatumWriter = new SpecificDatumWriter<DW>(DW.class);
DataFileWriter<DW> dataFileWriter = new DataFileWriter<DW>(
userDatumWriter);
dataFileWriter.create(payload.get(0).getSchema(), pipedOutputStream);
for (DW currentRecord : payload) {
dataFileWriter.append(currentRecord);
}
dataFileWriter.close();
return pipedInputStream;
final PipedOutputStream PipedOutputStream=new PipedOutputStream();
PipedInputStream PipedInputStream=新的PipedInputStream(
管道输出流);
DatumWriter用户DatumWriter=新的指定DatumWriter(DW.class);
DataFileWriter DataFileWriter=新的DataFileWriter(
userDatumWriter);
创建(payload.get(0.getSchema(),pipedOutputStream);
用于(DW currentRecord:有效负载){
dataFileWriter.append(currentRecord);
}
dataFileWriter.close();
返回管道输入流;
在一次添加10条记录之后,我尝试刷新,但它挂起在flush方法上
有人能帮我解决这个问题吗?通过返回ByteArrayOutputStream来解决,如下所示:
Schema schema = ReflectData.get().getSchema(DW.class);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
ReflectDatumWriter<Object> reflectDatumWriter = new ReflectDatumWriter<Object>(
schema);
DataFileWriter<Object> writer = new DataFileWriter<Object>(
reflectDatumWriter).create(schema, outputStream);
for (DW currentRecord : payload) {
writer.append(currentRecord);
}
writer.close();
return outputStream.toByteArray();
Schema Schema=ReflectData.get().getSchema(DW.class);
ByteArrayOutputStream outputStream=新建ByteArrayOutputStream();
ReflectDatumWriter ReflectDatumWriter=新的ReflectDatumWriter(
图式);
DataFileWriter=新的DataFileWriter(
reflectDatumWriter)创建(模式、输出流);
用于(DW currentRecord:有效负载){
writer.append(currentRecord);
}
writer.close();
返回outputStream.toByteArray();