lection.ReflectionEngine.getConstructor(ReflectionEngine.java:179) 位于py4j.reflection.ReflectionEngine.getConstructor的E(ReflectionEngine.java:196) E位于py4j.Gateway.invoke(Gateway.java:237) E位于py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) E位于py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) E在py4j.GatewayConnection.run处(GatewayConnection.java:238) 运行(Thread.java:748) ../anaconda3/envs/foo/lib/python3.7/site packages/py4j/protocol.py:332:Py4JError 这是怎么回事 当版本为2.3.2时,如何可能只有一个测试通过,但两个测试一起不通过

lection.ReflectionEngine.getConstructor(ReflectionEngine.java:179) 位于py4j.reflection.ReflectionEngine.getConstructor的E(ReflectionEngine.java:196) E位于py4j.Gateway.invoke(Gateway.java:237) E位于py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) E位于py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) E在py4j.GatewayConnection.run处(GatewayConnection.java:238) 运行(Thread.java:748) ../anaconda3/envs/foo/lib/python3.7/site packages/py4j/protocol.py:332:Py4JError 这是怎么回事 当版本为2.3.2时,如何可能只有一个测试通过,但两个测试一起不通过,python,pyspark,pytest,conda,Python,Pyspark,Pytest,Conda,顺便说一句:我正在尝试使用版本2.3.2进行测试,因为我想与我所使用的集群上安装的版本保持一致。您能在测试前后为env添加conda列表吗?创建两个单独的环境是否会产生差异?这两个环境之间唯一的差异是pyparsingpyspark。有趣的是(?)py4j有相同的版本。在PySpark 2.3.2规定的新鲜分离环境中,试验也失败@merv你试过复制吗?不,对不起,我没有安装Spark。我只是查看了这些包及其列出的依赖项,但我想确保我查看的是您正在测试的构建。我确实觉得很奇怪,py4j==0.10

顺便说一句:我正在尝试使用版本2.3.2进行测试,因为我想与我所使用的集群上安装的版本保持一致。

您能在测试前后为env添加
conda列表吗?创建两个单独的环境是否会产生差异?这两个环境之间唯一的差异是
pyparsing
pyspark
。有趣的是(?)
py4j
有相同的版本。在PySpark 2.3.2规定的新鲜分离环境中,试验也失败@merv你试过复制吗?不,对不起,我没有安装Spark。我只是查看了这些包及其列出的依赖项,但我想确保我查看的是您正在测试的构建。我确实觉得很奇怪,
py4j==0.10.7
被用于Python3.7——从技术上讲,但是有可能Conda软件包的构建者对它进行了兼容性审查。你能在之前和之后为env添加你的
Conda列表吗?创建两个单独的环境是否会产生差异?这两个环境之间唯一的差异是
pyparsing
pyspark
。有趣的是(?)
py4j
有相同的版本。在PySpark 2.3.2规定的新鲜分离环境中,试验也失败@merv你试过复制吗?不,对不起,我没有安装Spark。我只是查看了这些包及其列出的依赖项,但我想确保我查看的是您正在测试的构建。我确实觉得很奇怪,
py4j==0.10.7
在Python3.7上被使用——从技术上讲,但康达软件包构建者可能会对其进行兼容性审查。
sqlC = <pyspark.sql.context.SQLContext object at 0x118614990>

    def test_foo(sqlC):
>       df = sqlC.createDataFrame([("charly", 15), ("nick", 40)], ["myCol", "age"])

test_with_import.py:23:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../anaconda3/envs/foo/lib/python3.7/site-packages/pyspark/sql/context.py:302: in createDataFrame
    return self.sparkSession.createDataFrame(data, schema, samplingRatio, verifySchema)
../anaconda3/envs/foo/lib/python3.7/site-packages/pyspark/sql/session.py:692: in createDataFrame
    jrdd = self._jvm.SerDeUtil.toJavaArray(rdd._to_java_object_rdd())
../anaconda3/envs/foo/lib/python3.7/site-packages/pyspark/rdd.py:2296: in _to_java_object_rdd
    return self.ctx._jvm.SerDeUtil.pythonToJava(rdd._jrdd, True)
../anaconda3/envs/foo/lib/python3.7/site-packages/pyspark/rdd.py:2474: in _jrdd
    self.preservesPartitioning)
../anaconda3/envs/foo/lib/python3.7/site-packages/py4j/java_gateway.py:1525: in __call__
    answer, self._gateway_client, None, self._fqn)
../anaconda3/envs/foo/lib/python3.7/site-packages/pyspark/sql/utils.py:63: in deco
    return f(*a, **kw)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

answer = 'xspy4j.Py4JException: Constructor org.apache.spark.api.python.PythonRDD([class org.apache.spark.rdd.ParallelCollectio...java:69)\\n\tat py4j.GatewayConnection.run(GatewayConnection.java:238)\\n\tat java.lang.Thread.run(Thread.java:748)\\n'
gateway_client = <py4j.java_gateway.GatewayClient object at 0x1185e59d0>, target_id = None, name = 'org.apache.spark.api.python.PythonRDD'

    def get_return_value(answer, gateway_client, target_id=None, name=None):
        """Converts an answer received from the Java gateway into a Python object.

        For example, string representation of integers are converted to Python
        integer, string representation of objects are converted to JavaObject
        instances, etc.

        :param answer: the string returned by the Java gateway
        :param gateway_client: the gateway client used to communicate with the Java
            Gateway. Only necessary if the answer is a reference (e.g., object,
            list, map)
        :param target_id: the name of the object from which the answer comes from
            (e.g., *object1* in `object1.hello()`). Optional.
        :param name: the name of the member from which the answer comes from
            (e.g., *hello* in `object1.hello()`). Optional.
        """
        if is_error(answer)[0]:
            if len(answer) > 1:
                type = answer[1]
                value = OUTPUT_CONVERTER[type](answer[2:], gateway_client)
                if answer[1] == REFERENCE_TYPE:
                    raise Py4JJavaError(
                        "An error occurred while calling {0}{1}{2}.\n".
                        format(target_id, ".", name), value)
                else:
                    raise Py4JError(
                        "An error occurred while calling {0}{1}{2}. Trace:\n{3}\n".
>                       format(target_id, ".", name, value))
E                   py4j.protocol.Py4JError: An error occurred while calling None.org.apache.spark.api.python.PythonRDD. Trace:
E                   py4j.Py4JException: Constructor org.apache.spark.api.python.PythonRDD([class org.apache.spark.rdd.ParallelCollectionRDD, class org.apache.spark.api.python.PythonFunction, class java.lang.Boolean]) does not exist
E                       at py4j.reflection.ReflectionEngine.getConstructor(ReflectionEngine.java:179)
E                       at py4j.reflection.ReflectionEngine.getConstructor(ReflectionEngine.java:196)
E                       at py4j.Gateway.invoke(Gateway.java:237)
E                       at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80)
E                       at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69)
E                       at py4j.GatewayConnection.run(GatewayConnection.java:238)
E                       at java.lang.Thread.run(Thread.java:748)

../anaconda3/envs/foo/lib/python3.7/site-packages/py4j/protocol.py:332: Py4JError