Google cloud dataflow apache beam(数据流)的clickhouseIO中的数组类型

Google cloud dataflow apache beam(数据流)的clickhouseIO中的数组类型,google-cloud-dataflow,apache-beam,dataflow,clickhouse,Google Cloud Dataflow,Apache Beam,Dataflow,Clickhouse,我正在使用ApacheBeam使用json并将其插入clickhouse 我当前在数组数据类型方面有问题 在添加数组类型的字段之前,一切都正常 Schema.Field.of(“inputs.value”,Schema.FieldType.array(Schema.FieldType.INT64).withNullable(true)) 转换代码 p.apply(transformNameSuffix+“ReadFromPubSub”, PubsubIO.readStrings().fromS

我正在使用ApacheBeam使用json并将其插入clickhouse

我当前在数组数据类型方面有问题

在添加数组类型的字段之前,一切都正常

Schema.Field.of(“inputs.value”,Schema.FieldType.array(Schema.FieldType.INT64).withNullable(true))
转换代码

p.apply(transformNameSuffix+“ReadFromPubSub”,
PubsubIO.readStrings().fromSubscription(chainConfig.GetPubSubscriptionPrefix()+“transactions”)。带有IDAttribute(PUBSUB\u ID\u属性))
.apply(transformNameSuffix+“ReadFromPubSub”),ParDo.of(new DoFn(){
@过程元素
公共void processElement(ProcessContext c){
String item=c.element();
//系统输出打印(项目);
Transaction=JsonUtils.parseJson(item,Transaction.class);
c、 输出(行和模式(模式.事务)
.addValues(******,
*****
.......
transaction.getInputValues()).build();}
})).setRowSchema(Schemas.TRANSACTIONS).apply(
单击house io.write(
chainConfig.getClickhouseJDBCURI(),
chainConfig.getTransactionsTable())
.withMaxRetries(3)
.withMaxInsertBlockSize(1)
.带初始后退(持续时间.标准秒(5))
.使用InsertDeduplicate(真)
.withInsertDistributedSync(false));
生成输入的方法

public List getInputValues(){
列表值=Lists.newArrayList();
for(事务输入每个输入:输入){
System.out.print(eachInput.getValue());
add(eachInput.getValue());
}
返回值;
}
我得到的错误是:

ru.yandex.clickhouse.except.clickhouse异常:clickhouse异常,代码33,主机35.202.46.77,端口8123;代码:33,e.displayText()=DB::异常:无法读取所有数据。读取字节:6。预期字节数:15。(版本19.17.4.11(正式构建))
在ru.yandex.clickhouse.Exception.clickhouse ExceptionSpecifier.specify(clickhouse ExceptionSpecifier.java:58)
在ru.yandex.clickhouse.Exception.clickhouse ExceptionSpecifier.specify(clickhouse ExceptionSpecifier.java:28)
在ru.yandex.clickhouse.clickhouse语句impl.checkforerror和throw(clickhouse语句impl.java:875)
在ru.yandex.clickhouse.clickhouse语句impl.sendStream(clickhouse语句impl.java:851)
在ru.yandex.clickhouse.Writer.send(Writer.java:106)
在ru.yandex.clickhouse.Writer.send(Writer.java:141)
在ru.yandex.clickhouse.ClickHouseStatementImpl.sendRowBinaryStream(ClickHouseStatementImpl.java:764)
在ru.yandex.clickhouse.ClickHouseStatementImpl.sendRowBinaryStream(ClickHouseStatementImpl.java:758)
在org.apache.beam.sdk.io.clickhouse.ClickHouseIO$WriteFn.flush(ClickHouseIO.java:427)上
在org.apache.beam.sdk.io.clickhouse.ClickHouseIO$WriteFn.processElement上(ClickHouseIO.java:411)
位于org.apache.beam.sdk.io.clickhouse.AutoValue\u clickhouse io\u WriteFn$DoFnInvoker.invokeProcessElement(未知源)
位于org.apache.beam.repackaged.direct_java.runners.core.simpledofnlunner.invokeProcessElement(simpledofnlunner.java:222)
位于org.apache.beam.repackaged.direct_java.runners.core.simpledofnlunner.processElement(simpledofnlunner.java:183)
位于org.apache.beam.repackaged.direct_java.runners.core.SimplePushBacksideInputDofRunner.processElementInReadyWindows(SimplePushBacksideInputDofRunner.java:78)
位于org.apache.beam.runners.direct.ParDoEvaluator.processElement(ParDoEvaluator.java:216)
位于org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.processElement(DoFnLifecycleManagerRemovingTransformEvaluator.java:54)
位于org.apache.beam.runners.DirectTransformExecutor.processElements(DirectTransformExecutor.java:160)
位于org.apache.beam.runners.DirectTransformExecutor.run(DirectTransformExecutor.java:124)
位于java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
在java.util.concurrent.FutureTask.run(FutureTask.java:266)处
位于java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
位于java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
运行(Thread.java:748)
原因:java.lang.Throwable:代码:33,e.displayText()=DB::Exception:无法读取所有数据。读取字节:6。预期字节数:15。(版本19.17.4.11(正式构建))
在ru.yandex.clickhouse.Exception.clickhouse ExceptionSpecifier.specify(clickhouse ExceptionSpecifier.java:53)
... 还有22个
2020年2月6日9:04:38 PM org.apache.beam.sdk.io.clickhouse.clickhouse io$WriteFn flush
警告:写入ClickHouse时出错。重试尝试[1]
ru.yandex.clickhouse.except.clickhouse异常:clickhouse异常,代码33,主机35.202.46.77,端口8123;代码:33,e.displayText()=DB::异常:无法读取所有数据。读取字节:6。预期字节数:93。(版本19.17.4.11(正式构建))
在ru.yandex.clickhouse.Exception.clickhouse ExceptionSpecifier.specify(clickhouse ExceptionSpecifier.java:58)
在ru.yandex.clickhouse.Exception.clickhouse ExceptionSpecifier.specify(clickhouse ExceptionSpecifier.java:28)
在ru.yandex.clickhouse.clickhouse语句impl.checkforerror和throw(clickhouse语句impl.java:875)
在ru.yandex.clickhouse.clickhouse语句impl.sendStream(clickhouse语句impl.java:851)
在ru.yandex.clickhouse.Writer.send(Writer.java:106)
在ru.yandex.clickhouse.Writer.send(Writer.java:141)
在ru.yandex.clickhouse.ClickHouseStatementImpl.sendRowBinaryStream(ClickHouseStatementImpl.java:764)
在ru.yandex.clickhouse.ClickHouseStatementImpl.sendRowBinaryStream(ClickHouseStatementImpl.java:758)
位于org.apache.beam.sdk.io.clickhouse.clickhouse io$WriteFn