elasticsearch Flink with Elasticsearch6.0.0 Sink NoSuchMethodError BulkProcessor.builder

我曾尝试使用elasticsearch（版本为6.0.0）sink构建一个flink流字计数演示。不幸的是，我犯了以下错误。这似乎是相互矛盾的

Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: java.lang.NoSuchMethodError: org.elasticsearch.action.bulk.BulkProcessor.builder(Ljava/util/function/BiConsumer;Lorg/elasticsearch/action/bulk/BulkProcessor$Listener;)Lorg/elasticsearch/action/bulk/BulkProcessor$Builder;
    at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:623)
    at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:123)
    at com.quvideo.xiaoying.flink.elasticsearch.WordCountSinkElasticsearch.main(WordCountSinkElasticsearch.java:68)
Caused by: java.lang.NoSuchMethodError: org.elasticsearch.action.bulk.BulkProcessor.builder(Ljava/util/function/BiConsumer;Lorg/elasticsearch/action/bulk/BulkProcessor$Listener;)Lorg/elasticsearch/action/bulk/BulkProcessor$Builder;
    at org.apache.flink.streaming.connectors.elasticsearch6.Elasticsearch6ApiCallBridge.createBulkProcessorBuilder(Elasticsearch6ApiCallBridge.java:92)
    at org.apache.flink.streaming.connectors.elasticsearch6.Elasticsearch6ApiCallBridge.createBulkProcessorBuilder(Elasticsearch6ApiCallBridge.java:45)
    at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.buildBulkProcessor(ElasticsearchSinkBase.java:353)
    at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.open(ElasticsearchSinkBase.java:297)
    at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
    at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
    at org.apache.flink.streaming.api.operators.StreamSink.open(StreamSink.java:48)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
    at java.lang.Thread.run(Thread.java:748)

Process finished with exit code 1

我的elasticsearch集群是6.0.0，flink依赖项如下

<properties>
    <flink.version>1.6.0</flink.version>
    <elastic>6.0.0</elastic>
</properties>
<dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-streaming-java_2.11</artifactId>
        <version>${flink.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-clients_2.11</artifactId>
        <version>${flink.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-connector-elasticsearch6_2.11</artifactId>
        <version>${flink.version}</version>
    </dependency>

1.6.0
6.0.0
org.apache.flink
flink-streaming-java_2.11
${flink.version}
org.apache.flink
flink-U 2.11
${flink.version}
org.apache.flink
flink-connector-elasticsearch6_2.11
${flink.version}

elasticsearch相关依赖项如下所示：

<dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>transport</artifactId>
        <version>${elastic}</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-client</artifactId>
        <version>${elastic}</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-high-level-client</artifactId>
        <version>${elastic}</version>
    </dependency>

org.elasticsearch.client
运输
${elastic}
org.elasticsearch.client
elasticsearch rest客户端
${elastic}
org.elasticsearch.client
elasticsearch rest高级客户端
${elastic}

与弹性相关的代码：

publicstaticelasticsearchsink getEsSink（）{
List httpHosts=new ArrayList（）；
添加（新的HttpHost（“10.0.35.148”，9200，“http”）；
ElasticsearchSink.Builder esSinkBuilder=新建ElasticsearchSink.Builder(
httpHosts，
新的ElasticsearchSinkFunction（）{
公共索引请求createIndexRequest（WordWithCount元素）{
Map json=newhashmap（）；
put（“word”，element.word）；
put（“count”，element.count）；
返回请求。indexRequest（）
.index（“字数_idx”）
.类型（“测试类型”）
.来源（json）；
}
@凌驾
公共无效进程（WordWithCount元素、RuntimeContext ctx、RequestIndexer indexer）{
add（createIndexRequest（元素））；
}
}
);
esSinkBuilder.setBulkFlushMaxActions（1）；
返回esSinkBuilder.build（）；
}

我存档了。作为解决方法，您只需添加：

<dependency>
  <groupId>org.elasticsearch.client</groupId>
  <artifactId>elasticsearch-rest-high-level-client</artifactId>
  <version>6.3.1</version>
</dependency>

org.elasticsearch.client
elasticsearch rest高级客户端
6.3.1

1.6版似乎没有将Elasticsearch本身打包。我目前正在调查…似乎elasticsearch已经由org.elasticsearch.client:transport:jar:6.0.0导入