Warning: file_get_contents(/data/phpspider/zhask/data//catemap/4/powerbi/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
pyspark从列复制行_Pyspark - Fatal编程技术网
Fatal编程技术网

pyspark从列复制行

pyspark

pyspark从列复制行,pyspark,Pyspark,我怎样才能根据这个复制行 |source_ip |dest_ip |source_port|dest_port| |192.168.1.1|10.0.0.1|5343 |22 | 进入 使用pyspark？尝试使用数组和分解 示例： df.show() #+-----------+--------+-----------+---------+ #| ip| dest_ip|source_port|dest_port| #+-----------+---

我怎样才能根据这个复制行

|source_ip  |dest_ip |source_port|dest_port|
|192.168.1.1|10.0.0.1|5343       |22       |
进入

使用pyspark？

尝试使用
数组
和
分解


示例：


df.show()
#+-----------+--------+-----------+---------+
#|         ip| dest_ip|source_port|dest_port|
#+-----------+--------+-----------+---------+
#|192.168.1.1|10.0.0.1|       5343|       22|
#+-----------+--------+-----------+---------+

df.withColumn("arr",array(col("ip"),col("dest_ip"))).\
selectExpr("explode(arr) as ip","source_port","dest_port").\
show()
#+-----------+-----------+---------+
#|         ip|source_port|dest_port|
#+-----------+-----------+---------+
#|192.168.1.1|       5343|       22|
#|   10.0.0.1|       5343|       22|
#+-----------+-----------+---------+



df.show()
#+-----------+--------+-----------+---------+
#|         ip| dest_ip|source_port|dest_port|
#+-----------+--------+-----------+---------+
#|192.168.1.1|10.0.0.1|       5343|       22|
#+-----------+--------+-----------+---------+

df.withColumn("arr",array(col("ip"),col("dest_ip"))).\
selectExpr("explode(arr) as ip","source_port","dest_port").\
show()
#+-----------+-----------+---------+
#|         ip|source_port|dest_port|
#+-----------+-----------+---------+
#|192.168.1.1|       5343|       22|
#|   10.0.0.1|       5343|       22|
#+-----------+-----------+---------+