Google app engine 使用DatastoreIO将不同类型的实体保存到数据流中的数据存储中
我正在尝试使用DatastoreIO.v1().write()将不同类型的实体写入数据存储Google app engine 使用DatastoreIO将不同类型的实体保存到数据流中的数据存储中,google-app-engine,google-cloud-datastore,google-cloud-dataflow,Google App Engine,Google Cloud Datastore,Google Cloud Dataflow,我正在尝试使用DatastoreIO.v1().write()将不同类型的实体写入数据存储 在我的转换函数中,UpsertStoreItemEntityFn扩展了dofn,感谢Abhishek;我们正在与数据存储客户团队联系,并对此进行调查。@Abhishek应该可以编写多种类型的数据,没有任何限制。您是否在日志中看到任何错误,或者只是数据不存在?您可以尝试将输出拆分为两个pcollection,然后使用不同的数据存储。为每个pcollection编写步骤: @Override publ
在我的转换函数中,UpsertStoreItemEntityFn扩展了dofn,感谢Abhishek;我们正在与数据存储客户团队联系,并对此进行调查。@Abhishek应该可以编写多种类型的数据,没有任何限制。您是否在日志中看到任何错误,或者只是数据不存在?您可以尝试将输出拆分为两个pcollection,然后使用不同的
数据存储。为每个pcollection编写步骤:
@Override
public void processElement(ProcessContext processContext) throws Exception {
IngestionJobRecord jobRecord = convertIntoIngestionJobRecord(processContext.element());
String validRecordMsg = isRecordValid(jobRecord);
if (validRecordMsg.isEmpty()) {
processContext.output(makeSuccessRecordEntity(jobRecord));
} else {
processContext.output(makeErrorRecordEntity(validRecordMsg, jobRecord));
}
}
private Entity makeErrorRecordEntity(String errorMessage, IngestionJobRecord jobRecord) {
Entity.Builder entityBuilder = Entity.newBuilder();
// All created entities have the same ancestor Key.
Key.Builder keyBuilder = makeKey("MyService.ErrorRecordEntity", UUID.randomUUID().toString());
keyBuilder.getPartitionIdBuilder().setNamespaceId(tenantId);
entityBuilder.setKey(keyBuilder.build());
..
..
}
private Entity makeSuccessRecordEntity(StoreItemEntity storeItemEntity) {
Entity.Builder entityBuilder = Entity.newBuilder();
// All created entities have the same ancestor Key.
Key.Builder keyBuilder = makeKey("MyService.SuccessRecordEntity", UUID.randomUUID().toString());
keyBuilder.getPartitionIdBuilder().setNamespaceId(tenantId);
entityBuilder.setKey(keyBuilder.build());
..
..
}
.apply(DatastoreIO.v1().write().withProjectId("myProject"));