Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/333.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 如何访问广播变量的内容_Python_Json_Apache Spark_Pyspark - Fatal编程技术网

Python 如何访问广播变量的内容

Python 如何访问广播变量的内容,python,json,apache-spark,pyspark,Python,Json,Apache Spark,Pyspark,我需要在使用广播值的函数中进行一些计算 json_data = text.map(lambda x: json.loads(x)) .... # code to calculate average and generate tuple with json_data['jsontag'] and avgvalue some rdd filtsubavg with tuples of (jsontag, avgvalue) V = sc.broadcast(filtsubavg.collect()

我需要在使用广播值的函数中进行一些计算

json_data = text.map(lambda x: json.loads(x))
 ....
# code to calculate average and generate tuple with json_data['jsontag'] and avgvalue
some rdd filtsubavg with tuples of (jsontag, avgvalue)
V = sc.broadcast(filtsubavg.collect())
com = json_data.map(lambda l:l['jsontag'],l) 
res = com.map(lambda (cmtag,cm): get_val(cmtag,cm,V))
如果我需要说除以avgvalue,我如何在函数中访问V

def get_val(jsontag,cm,v):
    r1 = cm[jsontag]
    r2 = cm[value]/(get corresponding value for jsontag in v)
    return (r1,r2)

要访问广播变量的内容,可以使用其
属性:

V.value
如果要将其用作查找表,可以将其收集为映射(字典):

然后,您可以简单地使用:

cm[value] / V.value.get(v)
当我从collect()改为collectAsMap()并使用.get功能时,它确实起了作用!谢谢
cm[value] / V.value.get(v)