Question

每次使用concat调用凝聚时，

1.db的大小将比输入大小大一倍。插入20 MB数据后，1.db的大小变为30MB。

输入：行现在有10MB的数据

File file = new File("input.txt");
BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(file), StandardCharsets.UTF_8));
char[] buffer = new char[10 * 1000 * 1000];    
StringBuilder lines = new StringBuilder();    
lines.append(String.valueOf(buffer, 0, reader.read(buffer)));

第1次插入-

Connection connection = DriverManager.getConnection("jdbc:sqlite:target/1.db");
Settings settings = new Settings();
settings.setExecuteLogging(false);
File reader2 = new File("target/1.db");

try (DSLContext dsl = DSL.using(connection,settings)) {

    dsl.createTable("TABLE1")
            .column("COL1", SQLiteDataType.CLOB)
            .column("COL2", SQLiteDataType.CLOB)
            .execute();


    dsl.insertInto(TABLE1)
            .columns(TABLE1.COL1, TABLE1.COL2)
            .values("ABB", null)
            .execute();

    Field<String> coalesce = DSL.coalesce(TABLE1.COL2, "");
    dsl.update(TABLE1).set(TABLE1.COL2, DSL.concat(coalesce, DSL.val(lines.toString())))
            .where(TABLE1.COL1.eq("ABB"))
            .execute();

    Result<Record2<String, String>> result = dsl.select(TABLE1.COL1, TABLE1.COL2)
            .from(TABLE1)
            .where(TABLE1.COL1.eq("ABB"))
            .fetch();

    System.out.println("input length: " + lines.length());
    System.out.println("File size:" + reader2.length());
    if (result.getValue(0, TABLE1.COL2) != null) {
        System.out.println("Col1 length: " + result.getValue(0, TABLE1.COL1).length());
        System.out.println("Col2 length: " + result.getValue(0, TABLE1.COL2).length());
    }

输出：

input length: 10000000 //10 mb payload
File size:10113024  // file size increased by 10 MB
Col1 length: 3
Col2 length:10000000 // col2 length

再插入10MB，然后使用“ coalesce”和“ concat”将文件大小扩大到30MB。无法确定在此处添加了额外的10 MB。

第2次插入-

dsl.update(TABLE1).set(TABLE1.COL2, DSL.concat(coalesce, DSL.val(lines.toString())))
    .where(TABLE1.COL1.eq("ABB"))
    .execute();


result = dsl.select(TABLE1.COL1, TABLE1.COL2)
    .from(TABLE1)
    .where(TABLE1.COL1.eq("ABB"))
    .fetch();

System.out.println("input length: " + lines.length());
System.out.println("File size:" + reader2.length());
if (result.getValue(0, TABLE1.COL2) != null) {
    System.out.println("Col1 length: " + result.getValue(0, TABLE1.COL1).length());
    System.out.println("Col2 length: " + result.getValue(0, TABLE1.COL2).length());
}

输出：

input length: 10000000 //input size
File size:30322688  // increased from 10MB to 30MB for additional 10MB input 
Col1 length: 3
Col2 length:20000000 //col 2 length

它与Unicode编码有关吗？

Answer 1

来了解一下1个sqlite workaroud-

dsl.execute("VACUUM; ");

在插入和更新后将文件放回20MB以上即可运行。
我们可以使用jooq为sqlite启用自动吸尘吗？

编辑-使用事务和VACUUM

try (DSLContext dsl = DSL.using(dbUrl)) {
    dsl.transaction((ctx) -> {
        dsl.createTable("TABLE1")
            .column("COL1", SQLiteDataType.CLOB)
            .column("COL2", SQLiteDataType.CLOB)
            .execute();

        dsl.insertInto(TABLE1)
            .columns(TABLE1.COL1, TABLE1.COL2)
            .values("ABB", null)
            .execute();

        Field<String> coalesce = DSL.coalesce(TABLE1.COL2, "");
        dsl.update(TABLE1).set(TABLE1.COL2, DSL.concat(coalesce, DSL.val(lines.toString())))
            .where(TABLE1.COL1.eq("ABB"))
            .execute();         

    });
    dsl.execute("VACUUM; ");
}

目标数据库大小在合并时加倍

1 个答案: