SQL数据库未使用execute/addBatch通过Java填充
我目前有一个非常大的文件,其中包含几百万行条目,并希望将它们插入到数据库中。从java到SQL建立的连接可以正常工作,因为我已经尝试过奇异地插入数据,它可以正常工作,但是,当我切换到使用SQL数据库未使用execute/addBatch通过Java填充,java,jdbc,batch-processing,Java,Jdbc,Batch Processing,我目前有一个非常大的文件,其中包含几百万行条目,并希望将它们插入到数据库中。从java到SQL建立的连接可以正常工作,因为我已经尝试过奇异地插入数据,它可以正常工作,但是,当我切换到使用executeBatch和addBatch时,它似乎会循环,但不会将任何内容填充到我的数据库中 代码如下: import java.io.BufferedReader; import java.io.FileReader; import java.sql.Connection; import java.sql.D
executeBatch
和addBatch
时,它似乎会循环,但不会将任何内容填充到我的数据库中
代码如下:
import java.io.BufferedReader;
import java.io.FileReader;
import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.DriverManager;
import java.sql.SQLException;
public class fedOrganiser6 {
private static String directory = "C:\\Users\\x\\Desktop\\Files\\";
private static String file = "combined.fed";
private static String mapperValue = "";
public static void main(String[] args) throws Exception {
Connection conn = null;
try {
BufferedReader mapper = new BufferedReader(new FileReader(directory + file));
String dbURL = "jdbc:sqlserver://localhost\\SQLExpress;database=TIMESTAMP_ORGANISER;integratedSecurity=true";
String user = "sa";
String pass = "password";
conn = DriverManager.getConnection(dbURL, user, pass);
if (conn != null) {
DatabaseMetaData dm = (DatabaseMetaData) conn.getMetaData();
System.out.println("Driver name: " + dm.getDriverName());
System.out.println("Driver version: " + dm.getDriverVersion());
System.out.println("Product name: " + dm.getDatabaseProductName());
System.out.println("Product version: " + dm.getDatabaseProductVersion());
System.out.println("clearing database");
conn.createStatement().executeUpdate("truncate table TimestampsStorage");
System.out.println("bulk insert into database");
System.out.println("complete");
int i = 0;
int records = 0;
String query = "INSERT INTO TimestampsStorage " + "values(" + "'" + mapperValue.toString() + "'"+ ")";
conn.prepareStatement(query);
for (mapperValue = mapper.readLine(); mapperValue != null; mapperValue = mapper.readLine()) {
i++;
records++;
System.out.println("Batching " + records + " records...");
conn.createStatement().addBatch(query);
if (i == 100000) {
conn.createStatement().executeBatch();
i = 0;
}
}
}
conn.createStatement().executeBatch();
conn.createStatement().close();
System.out.print("Done");
} catch (SQLException ex) {
ex.printStackTrace();
} finally {
try {
if (conn != null && !conn.isClosed()) {
conn.close();
}
} catch (SQLException ex) {
ex.printStackTrace();
}
}
}
}
你把准备好的声明扔掉了
String query = "INSERT INTO TimestampsStorage VALUES (?)";
PreparedStatement statement = conn.prepareStatement(query);
for (mapperValue = mapper.readLine(); mapperValue != null; mapperValue = mapper.readLine()) {
i++;
records++;
System.out.println("Batching " + records + " records...");
statement.setString(1,mapperValue);
statement.addBatch();
if (i == 100000) {
statement.executeBatch();
i = 0;
}
创建一个新的语句对象,以便执行与批处理语句不同的语句。您应该创建一次PreparedStatement
,向其中添加几个批,然后在同一对象上执行:
String query = "INSERT INTO TimestampsStorage VALUES (?)";
PreparedStatement ps = conn.prepareStatement(query);
for (mapperValue = mapper.readLine();
mapperValue != null;
mapperValue = mapper.readLine()) {
i++;
records++;
System.out.println("Batching " + records + " records...");
ps.setString(1, mapperValue);
ps.addBatch();
if (i == 100000) {
ps.executeBatch();
i = 0;
}
}
我认为您在JDBC批处理的工作原理上有点错误 每次调用
conn.createStatement()
时,您都在创建一个新的语句
相反,您需要使用PreparedStatement
。首先,将您的查询更改为包含一个?
,您希望将值放在哪里
String query = "INSERT INTO TimestampsStorage VALUES(?)";
然后,当调用conn.prepareStatement(query)
时,存储返回的PreparedStatement
PreparedStatement ps = conn.prepareStatement(query);
这个PreparedStatement
将“记住”您的查询,您只需在循环的每次迭代中更改?
所在位置所需的值
ps.setString(1, mapperValue);
setString
方法将获取您的mapperValue
并使用它,而不是在查询中找到的第一个?
(因为您传递了索引1)
然后,您将调用ps.addBatch()
,而不是调用conn.createStatement().addBatch()
然后,在循环之外,可以调用ps.executeBatch()
。(无需在循环中调用此函数,因此如果(i==100000)
条件满足,则可以删除)
最后,如果您使用的是Java 7+,则可以使用try with resources,这样您就不必担心在Finally块中关闭PreparedStatement
或连接
以下是您的最终结果
String query = "INSERT INTO TimestampsStorage VALUES (?)";
try (Connection con = DriverManager.getConnection(dbURL, user, pass); PreparedStatement ps = con.prepareStatement(query);) {
for (mapperValue = mapper.readLine(); mapperValue != null; mapperValue = mapper.readLine()) {
records++;
ps.setString(1, mapperValue);
ps.addBatch();
}
System.out.println("Executing batch of " + records + " records...");
ps.executeBatch();
} catch (SQLException ex) {
//handle exception
}
对于preparedstation
,没有addBatch(字符串查询)
,只有addBatch()
。仍然错误。您需要将索引传递给setString()
@“穆雷尼克有正确的方法来做这件事。我撒谎了,”蒂姆说PreparedStatement
从语句继承addBatch(字符串查询)
。但是,当您得到的查询与您刚刚更改参数的查询相同时,@Mureinik的解决方案就是要使用的。@AndrewMairoseaddBatch(String)
对PreparedStatement
(和CallableStatement
)必须抛出SQLException
(如果没有,则驱动程序不符合JDBC)