Pyspark AnalysisException:无法解析给定的输入列:

Pyspark AnalysisException:无法解析给定的输入列:,pyspark,apache-spark-sql,azure-databricks,Pyspark,Apache Spark Sql,Azure Databricks,当我试图从临时表中选择几列时,我遇到了这个错误 pd_df = pd.read_sql('select * from abc.cars limit 10', conn) df = spark.createDataFrame(pd_df) df.createOrReplaceTempView("cars_tmp") df.show() print('***************') print("Reading from tmp table") dat

当我试图从临时表中选择几列时,我遇到了这个错误

pd_df = pd.read_sql('select * from abc.cars limit 10', conn)

df = spark.createDataFrame(pd_df)
df.createOrReplaceTempView("cars_tmp")
df.show()

print('***************')
print("Reading from tmp table")
data = spark.sql('select location_id from cars_tmp')
data.show()


AnalysisException: cannot resolve '`location_id`' given input columns: [cars_tmp.abc.product_id, cars_tmp.abc.location_id ...]
当我选择所有列时,我会得到结果。所以这是成功的:

data = spark.sql('select * from cars_tmp')
data.show()
我尝试了以下查询,但也失败了,出现了相同的错误:

data = spark.sql('select cars_tmp.abc.location_id from cars_tmp')
data.show()

data = spark.sql('select cars_tmp.location_id from cars_tmp')
data.show()

data = spark.sql('select abc.location_id from cars_tmp')
data.show()
我正在用砖做这些。 Databricks运行时版本:7.0 Apache Spark版本:3.0 斯卡拉:2.12

或“spark_版本”:“7.0.x-scala2.12”

我们将非常感谢您的帮助


谢谢,表中不存在列名
select*from cars\u tmp
有效,因为您没有指定列名


请以相同的错误处理方式查看此答案。

我通过在panda select查询中添加每一列来解决此问题。比如说:

pd_df = pd.read_sql('select id, location_id, product_id from abc.cars limit 10', conn)

你能试试“cars\u tmp.abc.location\u id”吗。用反勾号来转义名称。您还可以尝试打印(data.columns)以获取名称。