Apache kafka 德鲁伊·卡夫卡摄取的配置
我想设置kafka druid摄取,但即使在common.runtime.properties中配置并添加druid-kafka索引服务后,仍然会出现错误。请帮我解决这个问题。我的数据是csv格式的Apache kafka 德鲁伊·卡夫卡摄取的配置,apache-kafka,druid,Apache Kafka,Druid,我想设置kafka druid摄取,但即使在common.runtime.properties中配置并添加druid-kafka索引服务后,仍然会出现错误。请帮我解决这个问题。我的数据是csv格式的 { "type": "kafka", "spec": { "dataSchema": { "dataSource": "london_crime_by_lsoa", "parser": { "type": "string",
{
"type": "kafka",
"spec": {
"dataSchema": {
"dataSource": "london_crime_by_lsoa",
"parser": {
"type": "string",
"parseSpec": {
"format": "csv",
"dimensionsSpec": {
"dimensions": [
"lsoa_code",
"borough",
"major_category",
"minor_category",
{
"name": "value",
"type": "long"
},
{
"name": "year",
"type": "long"
},
{
"name": "month",
"type": "long"
}
]
},
"timestampSpec": {
"column": "year",
"format": "auto"
},
"columns": [
"lsoa_code",
"borough",
"major_category",
"minor_category",
"value",
"year",
"month"
]
}
},
"metricsSpec": [],
"granularitySpec": {
"type": "uniform",
"segmentGranularity": "year",
"queryGranularity": "NONE",
"rollup": false
}
},
"ioConfig": {
"topic": "london_crime_by_lsoa",
"taskDuration": "PT10M",
"useEarliestOffset": "true",
"consumerProperties": {
"bootstrap.servers": "localhost:9092"
}
},
"tuningConfig": {
"type": "kafka",
"maxRowsPerSegment": 500000
}
}
}
执行此命令后:
curl -XPOST -H'Content-Type: application/json' -d @quickstart/tutorial/crime_supervisor.json http://localhost:8090/druid/indexer/v1/supervisor
我得到这个错误:
{"error":"Instantiation of [simple type, class org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorSpec] value failed: dataSchema"}
我认为这是您在JSON中指定规范的方式的问题。 您必须直接在JSON中指定
dataSchema
,而不是作为spec
的子属性
以下是您应该遵循的格式:
{
"type": "kafka",
"dataSchema": {},
"tuningConfig": {},
"ioConfig": {}
}