Apache flink 弗林克对尼菲的魔术头球不在场
我正在尝试使用将Nifi连接到Flink:Apache flink 弗林克对尼菲的魔术头球不在场,apache-flink,apache-nifi,Apache Flink,Apache Nifi,我正在尝试使用将Nifi连接到Flink: StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); SiteToSiteClientConfig clientConfig = new SiteToSiteClient.Builder() .url("http://localhost:8090/nifi") .por
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
SiteToSiteClientConfig clientConfig = new SiteToSiteClient.Builder()
.url("http://localhost:8090/nifi")
.portName("Data for Flink")
.requestBatchCount(5)
.buildConfig();
SourceFunction<NiFiDataPacket> nifiSource = new NiFiSource(clientConfig);
DataStream<NiFiDataPacket> streamSource = env.addSource(nifiSource).setParallelism(2);
DataStream<String> dataStream = streamSource.map(new MapFunction<NiFiDataPacket, String>() {
@Override
public String map(NiFiDataPacket value) throws Exception {
return new String(value.getContent(), Charset.defaultCharset());
}
});
dataStream.print();
env.execute();
每次调用都会失败,请登录Nifi:
[Site-to-Site Worker Thread-24] o.a.nifi.remote.SocketRemoteSiteListener
Unable to communicate with remote instance null due to
org.apache.nifi.remote.exception.HandshakeException: Handshake
with nifi://localhost:61680 failed because the Magic Header
was not present; closing connection
Nifi版本:1.7.1,Flink版本:1.7.1在使用
Nifi工具包
后,我删除了Nifi.remote.input.socket.port的自定义值
,然后将传输协议(SiteToSiteTransportProtocol.HTTP)
添加到我的SiteToSiteClientConfig
和http://localhost:8080/nifi
作为URL
我首先更改端口的原因是,没有指定协议HTTP
,默认情况下它将使用RAW
。
当使用Flink端的RAW
协议时,客户端无法创建事务
,并打印以下警告:
Unable to refresh Remote Group's peers due to Remote instance of NiFi
is not configured to allow RAW Socket site-to-site communications
这就是为什么我认为这是一个港口问题
因此,现在使用Nifi的默认配置,它可以按预期工作:
SiteToSiteClientConfig clientConfig = new SiteToSiteClient.Builder()
.url("http://localhost:8080/nifi")
.portName("portNameAsInNifi")
.transportProtocol(SiteToSiteTransportProtocol.HTTP)
.requestBatchCount(1)
.buildConfig();
似乎你试图通过http将flink连接到nifi。。。和nifi侦听原始nifi协议。感谢您的提示,但我尝试了
transportProtocol(SiteToSiteTransportProtocol.raw)
和HTTP
,结果相同
SiteToSiteClientConfig clientConfig = new SiteToSiteClient.Builder()
.url("http://localhost:8080/nifi")
.portName("portNameAsInNifi")
.transportProtocol(SiteToSiteTransportProtocol.HTTP)
.requestBatchCount(1)
.buildConfig();