Question

我正在尝试在Neo4j的图形中加载16807个节点，其中有17.210.368个关系。至于这样做，我加载一个包含vincinity表的文件，我得到一个列表，其中包含必须与关系连接的节点。

在下面找到我的代码：

    String inputFile = "Desktop\kron7cd_unix.t01";
    FileInputStream in = new FileInputStream(inputFile);
    FileChannel ch = in.getChannel();
    ByteBuffer buf = ByteBuffer.allocate(1024);

    ArrayList<Integer> list = new ArrayList<Integer>();
    int NumOfOnes = 0;
    int column=-1;
    int row=0;
    int rd;
    while ((rd = ch.read( buf )) != -1){
        buf.flip();
        while (buf.hasRemaining()){
            byte byteVal = buf.get();
            if((byteVal == 48) || (byteVal == 49)){// when finds 1 or 0
                column++;
            }
            if (byteVal == 92){//when finds '/'
                    row++;
                    column=-1;
            }
            if(byteVal == 49){//when finds 1
                NumOfOnes++;
                list.add(column);
                list.add(row);
            }
        }
    buf.clear();
    }
    ch.close();


    GraphDatabaseFactory dbFactory = new GraphDatabaseFactory();
    GraphDatabaseService graphDb = dbFactory.newEmbeddedDatabase("C:\Neo4j\default.graphdb");

    Transaction tx = graphDb.beginTx();
    try {

        Label myLabel = DynamicLabel.label("Data");
        ArrayList<Node> nodelist = new ArrayList<Node>();

        for (int k = 0; k < row; k++) {
            nodelist.add(graphDb.createNode());
        }

        for (int k = 0; k < row; k++) {
            nodelist.get(k).setProperty("ID", k);
            nodelist.get(k).setProperty("Group","Random");
            nodelist.get(k).addLabel(myLabel);
        }

        Relationship rel;
        final RelationshipType type2 = DynamicRelationshipType.withName("Rel");

        for (int j = 0; j < list.size()-1 ; j += 2) { //list.size()=34420736

            rel = nodelist.get(list.get(j)).createRelationshipTo(nodelist.get(list.get(j+1)), type2);
            rel.setProperty("Team", "Common");

            if (j > 0 && j % 10000 == 0) {// as to commit transaction every now and then and dont throw heap space
                tx.success();
                tx.close();
                tx = graphDb.beginTx();
            }                

        }

        tx.success();
    }

    finally {
        tx.close();
    }
    graphDb.shutdown();

当我运行此代码时，它会抛出以下错误。我使用Neo4j 2.3.3和Netbeans 8.1和Java 8. 我想了解问题是堆空间还是尝试提交事务。我还添加了命令行选项 - 我的项目中的Xmx1g用于增加堆空间。

有什么想法吗？

查找以下错误消息：

Exception in thread "main" org.neo4j.graphdb.TransactionFailureException: Transaction was marked as successful, but unable to commit transaction so rolled back.
at org.neo4j.kernel.TopLevelTransaction.close(TopLevelTransaction.java:121)
at com.mycompany.traverse_test.traverse_main.main(traverse_main.java:232)

Caused by: org.neo4j.kernel.api.exceptions.TransactionFailureException: Could not apply the transaction to the store after written to log
at org.neo4j.kernel.impl.api.TransactionRepresentationCommitProcess.applyToStore(TransactionRepresentationCommitProcess.java:105)
at org.neo4j.kernel.impl.api.TransactionRepresentationCommitProcess.commit(TransactionRepresentationCommitProcess.java:58)
at org.neo4j.kernel.impl.api.KernelTransactionImplementation.commit(KernelTransactionImplementation.java:565)
at org.neo4j.kernel.impl.api.KernelTransactionImplementation.close(KernelTransactionImplementation.java:458)
at org.neo4j.kernel.TopLevelTransaction.close(TopLevelTransaction.java:97)
... 1 more

Caused by: java.lang.OutOfMemoryError
at sun.misc.Unsafe.allocateMemory(Native Method)
at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:386)
at org.neo4j.unsafe.impl.internal.dragons.MemoryManager$Slab.<init>(MemoryManager.java:111)
at org.neo4j.unsafe.impl.internal.dragons.MemoryManager.allocateAligned(MemoryManager.java:82)
at org.neo4j.io.pagecache.impl.muninn.MuninnPage.initBuffer(MuninnPage.java:417)
at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.pageFault(MuninnPageCursor.java:230)
at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.pin(MuninnPageCursor.java:157)
at org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor.next(MuninnWritePageCursor.java:58)
at org.neo4j.kernel.impl.store.PropertyStore.updateRecord(PropertyStore.java:144)
at org.neo4j.kernel.impl.transaction.command.NeoStoreTransactionApplier.visitPropertyCommand(NeoStoreTransactionApplier.java:99)
at org.neo4j.kernel.impl.api.CommandApplierFacade.visitPropertyCommand(CommandApplierFacade.java:120)
at org.neo4j.kernel.impl.transaction.command.Command$PropertyCommand.handle(Command.java:288)
at org.neo4j.kernel.impl.api.CommandApplierFacade.visit(CommandApplierFacade.java:82)
at org.neo4j.kernel.impl.api.CommandApplierFacade.visit(CommandApplierFacade.java:45)
at org.neo4j.kernel.impl.transaction.log.PhysicalTransactionRepresentation.accept(PhysicalTransactionRepresentation.java:69)
at org.neo4j.kernel.impl.api.TransactionRepresentationStoreApplier.apply(TransactionRepresentationStoreApplier.java:111)
at org.neo4j.kernel.impl.api.TransactionRepresentationCommitProcess.applyToStore(TransactionRepresentationCommitProcess.java:100)
... 5 more

Answer 1

您应该每1k - 10k个元素提交一次事务，以避免将所有数据保存在堆上。例如，请参阅https://github.com/graphaware/neo4j-framework/tree/master/tx-executor#batch-transactional-operations

Answer 2

正如我所知，通过尝试在另一个系统中运行相同的代码，我明白问题是堆空间错误。我在我的系统中使用3GB RAM运行代码并且弹出上述错误，但是我在12GB RAM的系统中运行它正常运行。（我想甚至8GB内存也不会有问题）

Neo4j嵌入式Java - 事务被标记为成功，但无法提交事务以便回滚

2 个答案: