如何在Snowflake JDBC中执行批处理更新
我正在尝试对PreparedStatement执行snowflake批处理更新,但运气不佳。我能够执行每个单独的查询(preparedStatement.executeQuery()),以便查询工作。但一旦我切换到batchUpdate(),我就会得到一个异常,它似乎没有设置我的变量(被抛出insertEventStatement.executeBatch()) 有人看到我遗漏了什么吗如何在Snowflake JDBC中执行批处理更新,jdbc,snowflake-cloud-data-platform,Jdbc,Snowflake Cloud Data Platform,我正在尝试对PreparedStatement执行snowflake批处理更新,但运气不佳。我能够执行每个单独的查询(preparedStatement.executeQuery()),以便查询工作。但一旦我切换到batchUpdate(),我就会得到一个异常,它似乎没有设置我的变量(被抛出insertEventStatement.executeBatch()) 有人看到我遗漏了什么吗 public void insertBotEvents(List<SnowflakeRecord>
public void insertBotEvents(List<SnowflakeRecord> snowflakeRecords) throws SQLException
{
try
{
PreparedStatement insertEventStatement = connection.prepareStatement(INSERT_EVENT);
PreparedStatement insertMetaEventStatement = connection.prepareStatement(INSERT_META_EVENT);
for (SnowflakeRecord snowflakeRecord : snowflakeRecords)
{
insertEventStatement.setString(1, snowflakeRecord.getUserId());
insertEventStatement.setString(2, snowflakeRecord.getSessionId());
insertEventStatement.setString(3, snowflakeRecord.getEventTime().toString());
insertEventStatement.setString(4, snowflakeRecord.getEventType());
insertEventStatement.setString(5, snowflakeRecord.getEventValue());
insertEventStatement.setString(6, snowflakeRecord.getAdditionalInfo());
insertEventStatement.setString(7, snowflakeRecord.getAffiliateName());
System.out.println("\nINSERTING NEW RECORD: \n" + snowflakeRecord);
insertEventStatement.addBatch();
if (snowflakeRecord.getSmVuiEventDto() != null && snowflakeRecord.getSmVuiEventDto().getParameters() != null)
{
for (ParametersDto parametersDto : snowflakeRecord.getSmVuiEventDto().getParameters())
{
insertMetaEventStatement.setString(1, snowflakeRecord.getSmVuiEventDto().getName());
insertMetaEventStatement.setString(2, parametersDto.getName());
insertMetaEventStatement.setString(3, parametersDto.getValue());
insertMetaEventStatement.setString(4, snowflakeRecord.getUserId());
insertMetaEventStatement.setString(5, snowflakeRecord.getSessionId());
insertMetaEventStatement.setString(6, snowflakeRecord.getEventTime().toString());
insertMetaEventStatement.setString(7, snowflakeRecord.getEventType());
insertMetaEventStatement.setString(8, snowflakeRecord.getEventValue());
insertMetaEventStatement.setString(9, snowflakeRecord.getAdditionalInfo());
insertMetaEventStatement.addBatch();
}
}
}
insertEventStatement.executeBatch();
insertMetaEventStatement.executeBatch();
System.out.println("\nCommitting changes to snowflake.");
connection.commit();
}
catch (Exception e)
{
connection.rollback();
System.err.println("Unable to execute SQL. Rolling back snowflake import");
e.printStackTrace();
throw e;
}
}
编辑:显然,在我使用的JDBC版本(3.0.18)中没有它,但在最近的一个版本中添加了它()。如果它不起作用,我将进一步更新。快速观察,我没有发现任何错误。您能否联系Snowflake支持部门并在跟踪中提供您的JDBC日志 如果不知道如何启用日志记录,请参阅
另外,如果您可以提供查询id,我可以快速查看 刚刚确认Snowflake目前不支持在包含子查询的INSERT语句中提供绑定数组,但我们正在研究它。谢谢您提供的信息。你知道什么时候发布吗?我目前没有太多的数据被发送。但是,我预计很快会有大量数据,根据我运行的测试,单独执行查询是不可行的。还是有更好的方式上传这些数据?应该在下周发布。但是,实现仍然是客户端循环,并单独执行查询。如果要加载大量数据,则应考虑复制命令。
public void insertBotEvents(List<SnowflakeRecord> snowflakeRecords) throws SQLException
{
try
{
PreparedStatement insertEventStatement = connection.prepareStatement(INSERT_EVENT);
PreparedStatement insertMetaEventStatement = connection.prepareStatement(INSERT_META_EVENT);
for (SnowflakeRecord snowflakeRecord : snowflakeRecords)
{
insertEventStatement.setString(1, snowflakeRecord.getUserId());
insertEventStatement.setString(2, snowflakeRecord.getSessionId());
insertEventStatement.setString(3, snowflakeRecord.getEventTime().toString());
insertEventStatement.setString(4, snowflakeRecord.getEventType());
insertEventStatement.setString(5, snowflakeRecord.getEventValue());
insertEventStatement.setString(6, snowflakeRecord.getAdditionalInfo());
insertEventStatement.setString(7, snowflakeRecord.getAffiliateName());
System.out.println("\nINSERTING NEW RECORD: \n" + snowflakeRecord);
insertEventStatement.addBatch();
if (snowflakeRecord.getSmVuiEventDto() != null && snowflakeRecord.getSmVuiEventDto().getParameters() != null)
{
for (ParametersDto parametersDto : snowflakeRecord.getSmVuiEventDto().getParameters())
{
insertMetaEventStatement.setString(1, snowflakeRecord.getSmVuiEventDto().getName());
insertMetaEventStatement.setString(2, parametersDto.getName());
insertMetaEventStatement.setString(3, parametersDto.getValue());
insertMetaEventStatement.setString(4, snowflakeRecord.getUserId());
insertMetaEventStatement.setString(5, snowflakeRecord.getSessionId());
insertMetaEventStatement.setString(6, snowflakeRecord.getEventTime().toString());
insertMetaEventStatement.setString(7, snowflakeRecord.getEventType());
insertMetaEventStatement.setString(8, snowflakeRecord.getEventValue());
insertMetaEventStatement.setString(9, snowflakeRecord.getAdditionalInfo());
insertMetaEventStatement.addBatch();
}
}
}
insertEventStatement.executeBatch();
insertMetaEventStatement.executeBatch();
System.out.println("\nCommitting changes to snowflake.");
connection.commit();
}
catch (Exception e)
{
connection.rollback();
System.err.println("Unable to execute SQL. Rolling back snowflake import");
e.printStackTrace();
throw e;
}
}
private static final String INSERT_EVENT =
"INSERT INTO EVENT " +
"(USER_ID, " +
" SESSION_ID, " +
" EVENT_DATETIME, " +
" EVENT_TYPE, " +
" EVENT_VALUE, " +
" ADDITIONAL_INFO, " +
" AFFILIATE_ID " +
") " +
"SELECT " +
" ?, " +
" ?, " +
" ?, " +
" ?, " +
" ?, " +
" ?, " +
" AFFILIATE_ID " +
"FROM AFFILIATE_LOOKUP " +
"WHERE AFFILIATE_NAME = ?";
private static final String INSERT_META_EVENT =
"INSERT INTO EVENT_META " +
"(EVENT_ID, " +
" META_TYPE, " +
" META_NAME, " +
" META_VALUE " +
") " +
"SELECT EVENT_ID, ?, ?, ? " +
"FROM SM_VUI_EVENT " +
"WHERE USER_ID=? AND " +
" SESSION_ID=? AND " +
" EVENT_DATETIME=? AND " +
" EVENT_TYPE=? AND " +
" EVENT_VALUE=? AND " +
" ADDITIONAL_INFO=?";