Scala 如何从齐柏林飞艇加载Flink流媒体数据
我开始在齐柏林飞艇上使用Flink,并尝试运行流媒体中最简单的程序:wordcount。 当我使用终端在本地模式下运行此代码时,它可以工作 我是这样做的: 代码如下:Scala 如何从齐柏林飞艇加载Flink流媒体数据,scala,apache-flink,apache-zeppelin,flink-streaming,Scala,Apache Flink,Apache Zeppelin,Flink Streaming,我开始在齐柏林飞艇上使用Flink,并尝试运行流媒体中最简单的程序:wordcount。 当我使用终端在本地模式下运行此代码时,它可以工作 我是这样做的: 代码如下: object SocketWindowWordCount { /** Main program method */ def main(args: Array[String]) : Unit = { // the host and the port to connect to var hostname:
object SocketWindowWordCount {
/** Main program method */
def main(args: Array[String]) : Unit = {
// the host and the port to connect to
var hostname: String = "localhost"
var port: Int = 0
try {
val params = ParameterTool.fromArgs(args)
hostname = if (params.has("hostname")) params.get("hostname") else "localhost"va
port = params.getInt("port")
} catch {
case e: Exception => {
System.err.println("No port specified. Please run 'SocketWindowWordCount " +
"--hostname <hostname> --port <port>', where hostname (localhost by default) and port " +
"is the address of the text server")
System.err.println("To start a simple text server, run 'netcat -l <port>' " +
"and type the input text into the command line")
return
}
}
// get the execution environment
val env: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment
// get input data by connecting to the socket
val text = env.socketTextStream(hostname, port, '\n')
// parse the data, group it, window it, and aggregate the counts
val windowCounts = text
.flatMap { w => w.split("\\s") }
.map { w => WordWithCount(w, 1) }
.keyBy("word")
.timeWindow(Time.seconds(5))
.sum("count")
// print the results with a single thread, rather than in parallel
windowCounts.print().setParallelism(1)
env.execute("Socket Window WordCount")
}
/** Data type for words with count */
case class WordWithCount(word: String, count: Long)
}
提前感谢您的帮助!:) 在终端中,您可以使用
nc -lk 9999
建立一个绑定到9999端口的服务,该服务将向其客户端发送它在stdin上接收到的任何内容
可能也会有帮助。您好,我不是flink用户,因此不确定如何准确使用flink流媒体,但这可能会对您有所帮助。非常感谢,文档非常有用,解决了我的问题!
nc -lk 9999