Question

我目前有一个非常大的文件，其中包含几百万行条目，并希望将它们插入到数据库中。从java到SQL建立的连接起作用，因为我尝试单独插入数据并且它可以工作，但是，当我切换到使用executeBatch和addBatch时，它似乎循环但不填充任何东西到我的数据库中。

代码如下：

import java.io.BufferedReader;
import java.io.FileReader;
import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.DriverManager;
import java.sql.SQLException;

public class fedOrganiser6 {
    private static String directory = "C:\\Users\\x\\Desktop\\Files\\";
    private static String file = "combined.fed";
    private static String mapperValue = "";

    public static void main(String[] args) throws Exception {

        Connection conn = null;

        try {
            BufferedReader mapper = new BufferedReader(new FileReader(directory + file));
            String dbURL = "jdbc:sqlserver://localhost\\SQLExpress;database=TIMESTAMP_ORGANISER;integratedSecurity=true";
            String user = "sa";
            String pass = "password";
            conn = DriverManager.getConnection(dbURL, user, pass);
            if (conn != null) {
                DatabaseMetaData dm = (DatabaseMetaData) conn.getMetaData();
                System.out.println("Driver name: " + dm.getDriverName());
                System.out.println("Driver version: " + dm.getDriverVersion());
                System.out.println("Product name: " + dm.getDatabaseProductName());
                System.out.println("Product version: " + dm.getDatabaseProductVersion());
                System.out.println("clearing database");
                conn.createStatement().executeUpdate("truncate table TimestampsStorage");
                System.out.println("bulk insert into database");
                System.out.println("complete");
                int i = 0;
                int records = 0;
                String query = "INSERT INTO TimestampsStorage " + "values(" + "'" + mapperValue.toString() + "'"+ ")";
                conn.prepareStatement(query);
                for (mapperValue = mapper.readLine(); mapperValue != null; mapperValue = mapper.readLine()) {
                    i++;
                    records++;
                    System.out.println("Batching " + records + " records...");

                    conn.createStatement().addBatch(query);
                    if (i == 100000) {
                        conn.createStatement().executeBatch();
                        i = 0;
                    }
                }
            }
            conn.createStatement().executeBatch();
            conn.createStatement().close();
            System.out.print("Done");

        } catch (SQLException ex) {
            ex.printStackTrace();
        } finally {
            try {
                if (conn != null && !conn.isClosed()) {
                    conn.close();
                }
            } catch (SQLException ex) {
                ex.printStackTrace();
            }
        }

    }
}

Answer 1

createStatement()创建一个新的语句对象，因此您执行的语句与您正在批处理的语句不同。您应该创建PreparedStatement一次，向其添加几个批次，然后在同一个对象上执行：

String query = "INSERT INTO TimestampsStorage VALUES (?)";
PreparedStatement ps = conn.prepareStatement(query);
for (mapperValue = mapper.readLine(); 
     mapperValue != null; 
     mapperValue = mapper.readLine()) {

    i++;
    records++;
    System.out.println("Batching " + records + " records...");

    ps.setString(1, mapperValue);
    ps.addBatch();

    if (i == 100000) {
        ps.executeBatch();
        i = 0;
    }
}

Answer 2

你正扔掉准备好的陈述

String query = "INSERT INTO TimestampsStorage VALUES (?)";
                PreparedStatement statement = conn.prepareStatement(query);
                for (mapperValue = mapper.readLine(); mapperValue != null; mapperValue = mapper.readLine()) {
                    i++;
                    records++;
                    System.out.println("Batching " + records + " records...");
                    statement.setString(1,mapperValue);
                    statement.addBatch();
                    if (i == 100000) {
                        statement.executeBatch();
                        i = 0;
                    }

Answer 3

我认为你对JDBC的批处理工作方式有点误解。

每次致电Statement时，您都会创建新的conn.createStatement()。

相反，您需要使用PreparedStatement。首先，更改您的查询以包含您想要值的?。

String query = "INSERT INTO TimestampsStorage VALUES(?)";

然后，当您致电conn.prepareStatement(query)时，请存储已退回的PreparedStatement。

PreparedStatement ps = conn.prepareStatement(query);

此PreparedStatement将“记住”您的查询，您只需在循环的每次迭代中更改?所需的值。

ps.setString(1, mapperValue);

setString方法将使用您的mapperValue并使用它而不是您在查询中找到的第一个?（因为您传入了索引1）。

然后，您可以拨打conn.createStatement().addBatch()。

，而不是致电ps.addBatch()

然后，在循环之外，您可以致电ps.executeBatch()。（无需在循环中调用此方法，因此您可以删除if (i == 100000)条件。

最后，如果您使用的是Java 7+，则可以尝试使用资源，这样您就不必担心在finally块中关闭PreparedStatement或Connection。 / p>

以下是您最终结果的样子。

String query = "INSERT INTO TimestampsStorage VALUES (?)";

try (Connection con = DriverManager.getConnection(dbURL, user, pass); PreparedStatement ps = con.prepareStatement(query);) {
    for (mapperValue = mapper.readLine(); mapperValue != null; mapperValue = mapper.readLine()) {            
        records++;

        ps.setString(1, mapperValue);
        ps.addBatch();
    }
    System.out.println("Executing batch of " + records + " records...");
    ps.executeBatch();
} catch (SQLException ex) {
    //handle exception
}

SQL数据库不使用execute / addBatch通过Java填充

3 个答案: