Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/300.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/powershell/11.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python:将数据帧作为参数传递给子流程_Python_Pandas_Pyspark_Subprocess_Spark Submit - Fatal编程技术网

Python:将数据帧作为参数传递给子流程

Python:将数据帧作为参数传递给子流程,python,pandas,pyspark,subprocess,spark-submit,Python,Pandas,Pyspark,Subprocess,Spark Submit,如何使用spark submit子流程将数据帧作为参数发送到python脚本。我已经尝试了下面的代码,但没有成功,因为我们无法连接字符串和对象 def spark_submit(self, test_cases, email): command = 'spark-submit TestRunner.py '+test_cases+" "+email print(command) process = subprocess.Popen([command], shell=Tru

如何使用spark submit子流程将数据帧作为参数发送到python脚本。我已经尝试了下面的代码,但没有成功,因为我们无法连接字符串和对象

def spark_submit(self, test_cases, email):
    command = 'spark-submit TestRunner.py '+test_cases+" "+email
    print(command)
    process = subprocess.Popen([command], shell=True,
                               stdout=subprocess.PIPE,
                               stderr=subprocess.PIPE)
    output, error = process.communicate()
   status = process.returncode
   print(status)```

不能连接任何非字符串(或铸造为字符串)的内容。我假设不能直接将数据帧作为命令行参数传递,因此我建议将其转换为文件并传递文件路径,而不是数据帧本身

df.to_csv('mydf.csv')
command = 'spark-submit TestRunner.py mydf.csv ' + email 

不能连接任何非字符串(或铸造为字符串)的内容。我假设不能直接将数据帧作为命令行参数传递,因此我建议将其转换为文件并传递文件路径,而不是数据帧本身

df.to_csv('mydf.csv')
command = 'spark-submit TestRunner.py mydf.csv ' + email 

test\u cases
是您的数据帧吗?
test\u cases
是您的数据帧吗?