Apache spark 如何查看访问MariaDB时Spark生成的SQL表达式?
我试图用以下代码将流写入RDMSApache spark 如何查看访问MariaDB时Spark生成的SQL表达式?,apache-spark,mariadb,Apache Spark,Mariadb,我试图用以下代码将流写入RDMS def foreach_batch_function(df1, epoch_id): df1.write.format("jdbc") \ .option("createTableColumnTypes", "cnt int, minBestLapTime double, trackName varchar(64), trackVersion varchar(64), carClass v
def foreach_batch_function(df1, epoch_id):
df1.write.format("jdbc") \
.option("createTableColumnTypes", "cnt int, minBestLapTime double, trackName varchar(64), trackVersion varchar(64), carClass varchar(64), carName varchar(64), carVersion varchar(64)") \
.option("url", "jdbc:mariadb://IPADDRESS") \
.option("dbtable", "mydatabase.pysparktest") \
.option("user", config["jdbc"]["user"]) \
.option("password", config["jdbc"]["password"]) \
.option("driver", "org.mariadb.jdbc.Driver") \
.save()
并得到以下错误
java.sql.SQLException: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '"cnt" int NOT NULL, "minBestLapTime" double , "trackName" varchar(64) , "trac...' at line 1
因为我在这里没有看到任何语法错误,所以我假设Spark生成错误的SQL。但是怎么看呢
我设定
但是这没有帮助您的完整查询应该记录在服务器上
log4j.properties:
log4j.rootCategory=DEBUG, console