Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/amazon-s3/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Amazon s3 使用SQLite jdbc连接字符串-pyspark从s3位置加载SQLite DB_Amazon S3_Pyspark Sql - Fatal编程技术网

Amazon s3 使用SQLite jdbc连接字符串-pyspark从s3位置加载SQLite DB

Amazon s3 使用SQLite jdbc连接字符串-pyspark从s3位置加载SQLite DB,amazon-s3,pyspark-sql,Amazon S3,Pyspark Sql,我在s3 bucket上存储了一个SQLite DB文件,希望加载该文件在pyspark dataframe中的特定表。我是这样做的,但最终得到了一个错误:“调用o662.load时出错:java.sql.SQLException:s3://brexit bucket/fake_db_081119.sqlite':”/s3:“不存在” 这是我的代码。 df = sqlContext.read.format('jdbc') \ .options(url='jdbc:sqli

我在s3 bucket上存储了一个SQLite DB文件,希望加载该文件在pyspark dataframe中的特定表。我是这样做的,但最终得到了一个错误:“调用o662.load时出错:java.sql.SQLException:s3://brexit bucket/fake_db_081119.sqlite':”/s3:“不存在”

这是我的代码。

df = sqlContext.read.format('jdbc') \
            .options(url='jdbc:sqlite:s3://brexit-bucket/fake_db_081119.sqlite', \
                     dbtable='user',driver='org.sqlite.JDBC').load()
这里是错误:

An error occurred while calling o662.load.: java.sql.SQLException: path to 's3://brexit-bucket/fake_db_081119.sqlite': '/s3:' does not exist
at org.sqlite.core.CoreConnection.open(CoreConnection.java:117)
at org.sqlite.core.CoreConnection.<init>(CoreConnection.java:66)
at org.sqlite.jdbc3.JDBC3Connection.<init>(JDBC3Connection.java:21)
at org.sqlite.jdbc4.JDBC4Connection.<init>(JDBC4Connection.java:23)
at org.sqlite.SQLiteConnection.<init>(SQLiteConnection.java:45)
at org.sqlite.JDBC.createConnection(JDBC.java:114)
at org.sqlite.JDBC.connect(JDBC.java:88)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:63)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:54)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:56)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:748)
调用o662.load时出错。:java.sql.SQLException:指向“s3://brexit bucket/fake_db_081119.sqlite”的路径:“/s3:”不存在
位于org.sqlite.core.CoreConnection.open(CoreConnection.java:117)
在org.sqlite.core.CoreConnection上。(CoreConnection.java:66)
位于org.sqlite.jdbc3.JDBC3Connection(JDBC3Connection.java:21)
位于org.sqlite.jdbc4.JDBC4Connection(JDBC4Connection.java:23)
在org.sqlite.SQLiteConnection.(SQLiteConnection.java:45)
位于org.sqlite.JDBC.createConnection(JDBC.java:114)
位于org.sqlite.JDBC.connect(JDBC.java:88)
位于org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:63)
位于org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:54)
位于org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:56)
位于org.apache.spark.sql.execution.datasources.jdbc.jdbcrations$.getSchema(jdbcrations.scala:210)
位于org.apache.spark.sql.execution.datasources.jdbc.jdbrelationprovider.createRelation(jdbrelationprovider.scala:35)
位于org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
位于org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
位于org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
位于org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
在sun.reflect.GeneratedMethodAccessor70.invoke(未知源)
在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)中
位于java.lang.reflect.Method.invoke(Method.java:498)
位于py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
位于py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
在py4j.Gateway.invoke处(Gateway.java:282)
位于py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
在py4j.commands.CallCommand.execute(CallCommand.java:79)
在py4j.GatewayConnection.run处(GatewayConnection.java:238)
运行(Thread.java:748)