elasticsearch,apache-flink,resthighlevelclient,Scala,elasticsearch,Apache Flink,Resthighlevelclient" /> elasticsearch,apache-flink,resthighlevelclient,Scala,elasticsearch,Apache Flink,Resthighlevelclient" />

Scala Flink ES连接未按预期编译

Scala Flink ES连接未按预期编译,scala,elasticsearch,apache-flink,resthighlevelclient,Scala,elasticsearch,Apache Flink,Resthighlevelclient,我的问题有点像描述的那样。 部分代码(实际上取自apache站点)如下所示 val httpHosts = new java.util.ArrayList[HttpHost] httpHosts.add(new HttpHost("127.0.0.1", 9200, "http")) httpHosts.add(new HttpHost("10.2.3.1", 9200, "http")) val esSinkBuilder = new ElasticsearchSink.Builder[St

我的问题有点像描述的那样。 部分代码(实际上取自apache站点)如下所示

val httpHosts = new java.util.ArrayList[HttpHost]
httpHosts.add(new HttpHost("127.0.0.1", 9200, "http"))
httpHosts.add(new HttpHost("10.2.3.1", 9200, "http"))

val esSinkBuilder = new ElasticsearchSink.Builder[String](
  httpHosts,
  new ElasticsearchSinkFunction[String] {
    def createIndexRequest(element: String): IndexRequest = {
      val json = new java.util.HashMap[String, String]
      json.put("data", element)


      return Requests.indexRequest()
              .index("my-index")
              .`type`("my-type")
              .source(json)
如果我加上这三条语句,我得到的错误如下

import org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkFunction
import org.apache.flink.streaming.connectors.elasticsearch.RequestIndexer
import org.apache.flink.streaming.connectors.elasticsearch6.ElasticsearchSink
Compiling 1 Scala source to E:\sar\scala\practice\readstbdata\target\scala-2.11\classes ...
[error] E:\sar\scala\practice\readstbdata\src\main\scala\example\readcsv.scala:35:25: not found: value ElasticsearchSink
[error] val esSinkBuilder = new ElasticsearchSink.Builder[String](
[error]                         ^
[error] E:\sar\scala\practice\readstbdata\src\main\scala\example\readcsv.scala:37:7: not found: type ElasticsearchSinkFunction
[error]   new ElasticsearchSinkFunction[String] {
[error]       ^
[error] two errors found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 1 s, completed 10 Feb, 2020 2:15:04 PM
我犯了一个错误

object elasticsearch is not a member of package org.apache.flink.streaming.connectors
object elasticsearch6 is not a member of package org.apache.flink.streaming.connectors
若我不添加那个些导入语句,我会得到如下错误

import org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkFunction
import org.apache.flink.streaming.connectors.elasticsearch.RequestIndexer
import org.apache.flink.streaming.connectors.elasticsearch6.ElasticsearchSink
Compiling 1 Scala source to E:\sar\scala\practice\readstbdata\target\scala-2.11\classes ...
[error] E:\sar\scala\practice\readstbdata\src\main\scala\example\readcsv.scala:35:25: not found: value ElasticsearchSink
[error] val esSinkBuilder = new ElasticsearchSink.Builder[String](
[error]                         ^
[error] E:\sar\scala\practice\readstbdata\src\main\scala\example\readcsv.scala:37:7: not found: type ElasticsearchSinkFunction
[error]   new ElasticsearchSinkFunction[String] {
[error]       ^
[error] two errors found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 1 s, completed 10 Feb, 2020 2:15:04 PM
上面提到的堆栈流问题,一些函数已经扩展。我的理解是,flink.streaming.connectors.elasticsearch必须扩展到REST库中。1) 我的理解正确吗2)如果是,我可以完整地扩展吗3)如果我的理解错误,请给我一个解决方案

注意:我在build.sbt中添加了以下语句

libraryDependencies += "org.elasticsearch.client" % "elasticsearch-rest-high-level-client" % "7.5.2" ,
    libraryDependencies += "org.elasticsearch" % "elasticsearch" % "7.5.2",

流式连接器不是flink二进制发行版的一部分。您必须将它们与应用程序打包在一起

对于
elasticsearch6
,您需要添加
flink-connector-elasticsearch6_2.11
,您可以按照

libraryDependencies += "org.apache.flink" %% "flink-connector-elasticsearch6" % "1.6.0"
一旦这个jar成为构建的一部分,编译器就会找到缺少的组件。但是,我不知道此ES6客户端是否能与7.5.2版配合使用。

Flink Elasticsearch Connector 7


请看我提供的工作和详细的答案,这是用Scala编写的

我在build.sbt中添加了以下语句。libraryDependencies+=“org.apache.flink”%”flink-connector-elasticsearch6_2.11“%”1.9.0“%”提供了“由于类型的trait-elasticsearchsink函数中的方法进程,我得到以下错误,无法创建对象(x$1:String,x$2:org.apache.flink.api.common.functions.RuntimeContext,x$3:org.apache.flink.streaming.connectors.elasticsearch.RequestIndexer)未定义单元[error]新的ElasticsearchSinkFunction[String]{[error]^我不知道为什么flink docs for ES6中的Scala示例没有定义
过程
方法,但Java版本定义了。听起来可能需要这个方法。