Google bigquery 尝试从AVRO文件创建表时BQ load作业失败
我正在尝试从AVRO文件创建一个BQ表。我在运行BQ加载作业时遇到此错误: 读取数据时出错,错误消息:Apache Avro库 无法分析标头,出现以下错误:意外类型 用于默认值。应为长,但为null:null“ AVRO文件的架构为:Google bigquery 尝试从AVRO文件创建表时BQ load作业失败,google-bigquery,schema,avro,bq-load,Google Bigquery,Schema,Avro,Bq Load,我正在尝试从AVRO文件创建一个BQ表。我在运行BQ加载作业时遇到此错误: 读取数据时出错,错误消息:Apache Avro库 无法分析标头,出现以下错误:意外类型 用于默认值。应为长,但为null:null“ AVRO文件的架构为: { "type" : "record", "name" : "Pair", "namespace" : "org.apache.avro.ma
{
"type" : "record",
"name" : "Pair",
"namespace" : "org.apache.avro.mapred",
"fields" : [ {
"name" : "key",
"type" : "int",
"doc" : ""
}, {
"name" : "value",
"type" : {
"type" : "record",
"name" : "CustomerInventoryOrderItems",
"namespace" : "com.test.customer.order",
"fields" : [ {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "inventory_order_items",
"type" : {
"type" : "map",
"values" : {
"type" : "array",
"items" : {
"type" : "record",
"name" : "CustomerInventoryOrderItem",
"fields" : [ {
"name" : "order_item_id",
"type" : "int",
"default" : null
}, {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "created_at",
"type" : "long"
}, {
"name" : "product_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "type_id",
"type" : "int",
"default" : null
}, {
"name" : "event_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "price",
"type" : [ "null", "double" ],
"default" : null
}, {
"name" : "tags",
"type" : [ "null", "string" ],
"default" : null
}, {
"name" : "estimated_ship_date",
"type" : [ "null", "long" ],
"default" : null
} ]
}
}
}
} ]
},
"doc" : "",
"order" : "ignore"
} ]
}
我不确定架构或任何其他方面有什么问题,因此我无法加载数据。问题很可能是类型为
int
但默认值为null
的字段。例如:
"name" : "type_id",
"type" : "int",
"default" : null
应该将默认值更改为整数,或者将类型更改为包含
null
(与许多其他字段一样)的并集。这可能是一个问题,但我获取的错误指向一个长属性,而不是整数。我不知道是哪个库抛出了错误,但在快速搜索时,它似乎可能来自这里:。如果这是正确的库,那么int的检查将使用long类型进行比较:。你有没有试着修复ints?当检查ints时,错误消息可能会说long。