Question

我从经典的Spring 4.3.2 + Hibernate（HQL）迁移到Spring Boot 1.4.1 + Hibernate（JPA）+ HikariCP连接池，并且在批量插入时遇到问题。我需要保存1000到527 000个元素的大型列表。

当我只保存一个实体的批处理，没有任何依赖关系（例如：Category类的5000个元素）时，一切正常。
但是，如果我尝试同时保存子实体和父实体（Website并为其设置Category），则会出现例外情况：org.hibernate.PersistentObjectException: detached entity passed to persist: com.example.entities.Category

但当列表大小小于 hibernate.jdbc.batch_size 时，一切正常！两个实体都正确保存，父母和孩子。这看起来很奇怪！我已经google了足够的搜索，在StackOverflow搜索，但无法找到特定问题的解决方案。我已经尝试将连接池替换为Apache DBCP，甚至根本没有包含一个），更改了Hibernate性能设置，将PostgreSQL数据库替换为H2 - 但没有结果。

哪里可能是错误？
是否有可能在JPA中做我想要的事情？或者我应该单独保存实体（首先是父母，然后是孩子）？另一种方法是从我的程序中排除flush()和clear()，但它对性能不利！
希望得到帮助。这是详细信息。这就是我试图保存实体的方式：

    @Repository("experimentDao")
    @Transactional
    public class Dao {
        @PersistenceContext
        private EntityManager entityManager;

        @Value("${spring.jpa.properties.hibernate.jdbc.batch_size}")
        private int batchSize;

        public <T extends Entity> void bulkSave(Collection<T> entities) {
            int i = 0;
            for (Entity entity : entities) {
                persistOrMerge(entity);
                i++;
                if (i % batchSize == 0) {
                    // Flush a batch of inserts and release memory.
                    entityManager.flush();
                    entityManager.clear();
                }
            }
        }

    private <T extends Entity> void persistOrMerge(T entity) {
        if (entity.getId() == null) {
            entityManager.persist(entity);
        } else {
            entityManager.merge(entity);
        }
    }
 }

（示例见于：http://frightanic.com/software-development/jpa-batch-inserts/）。而且我正在跟踪堆栈跟踪：

java.lang.IllegalStateException: Failed to execute CommandLineRunner
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:803) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:784) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
    at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:771) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1186) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1175) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
    at com.example.DemoApplication.main(DemoApplication.java:24) [classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_101]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_101]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_101]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_101]
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) [idea_rt.jar:na]
Caused by: org.springframework.dao.InvalidDataAccessApiUsageException: detached entity passed to persist: com.example.entities.Category; nested exception is org.hibernate.PersistentObjectException: detached entity passed to persist: com.example.entities.Category
    at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:299) ~[spring-orm-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:244) ~[spring-orm-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:491) ~[spring-orm-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:59) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:213) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:147) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at com.example.Dao$$EnhancerBySpringCGLIB$$79470367.bulkSave(<generated>) ~[classes/:na]
    at com.example.DemoApplication.insertWebsites(DemoApplication.java:44) [classes/:na]
    at com.example.DemoApplication.run(DemoApplication.java:30) [classes/:na]
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) [spring-boot-1.4.1.RELEASE.jar:1.4.1.RELEASE]
    ... 11 common frames omitted
Caused by: org.hibernate.PersistentObjectException: detached entity passed to persist: com.example.entities.Category
    at org.hibernate.event.internal.DefaultPersistEventListener.onPersist(DefaultPersistEventListener.java:124) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.internal.SessionImpl.firePersist(SessionImpl.java:765) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.internal.SessionImpl.persist(SessionImpl.java:758) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.jpa.event.internal.core.JpaPersistEventListener$1.cascade(JpaPersistEventListener.java:80) ~[hibernate-entitymanager-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.engine.internal.Cascade.cascadeToOne(Cascade.java:398) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.engine.internal.Cascade.cascadeAssociation(Cascade.java:323) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.engine.internal.Cascade.cascadeProperty(Cascade.java:162) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.engine.internal.Cascade.cascade(Cascade.java:111) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.event.internal.AbstractSaveEventListener.cascadeBeforeSave(AbstractSaveEventListener.java:425) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.event.internal.AbstractSaveEventListener.performSaveOrReplicate(AbstractSaveEventListener.java:249) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.event.internal.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:178) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.event.internal.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:121) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.jpa.event.internal.core.JpaPersistEventListener.saveWithGeneratedId(JpaPersistEventListener.java:67) ~[hibernate-entitymanager-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.event.internal.DefaultPersistEventListener.entityIsTransient(DefaultPersistEventListener.java:189) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.event.internal.DefaultPersistEventListener.onPersist(DefaultPersistEventListener.java:132) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.event.internal.DefaultPersistEventListener.onPersist(DefaultPersistEventListener.java:58) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.internal.SessionImpl.firePersist(SessionImpl.java:775) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.internal.SessionImpl.persist(SessionImpl.java:748) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.internal.SessionImpl.persist(SessionImpl.java:753) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
    at org.hibernate.jpa.spi.AbstractEntityManagerImpl.persist(AbstractEntityManagerImpl.java:1146) ~[hibernate-entitymanager-5.0.11.Final.jar:5.0.11.Final]
    at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) ~[na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_101]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_101]
    at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:298) ~[spring-orm-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at com.sun.proxy.$Proxy62.persist(Unknown Source) ~[na:na]
    at com.example.Dao.persistOrMerge(Dao.java:36) ~[classes/:na]
    at com.example.Dao.bulkSave(Dao.java:24) ~[classes/:na]
    at com.example.Dao$$FastClassBySpringCGLIB$$2ada4001.invoke(<generated>) ~[classes/:na]
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136) ~[spring-tx-4.3.3.RELEASE.jar:4.3.3.RELEASE]
    ... 21 common frames omitted

Website和Category个实体具有一对一关系，Category是父级。而且我也有抽象的超类，它应该坚持下去：

public abstract class Entity {
    public abstract Long getId();
}

@javax.persistence.Entity
@Table(name = "websites")
public class Website extends Entity implements Serializable {
    @Id
    @GeneratedValue(strategy = SEQUENCE)
    @Column(name = "website_id")
    private Long id;

    @Column(name = "url", unique = true)
    private String url;

    @OneToOne(cascade = CascadeType.ALL)
    @JoinColumn(name="category_id")
    private Category category;
    // getters and setters, toString
}

@javax.persistence.Entity
@Table(name = "categories")
public class Category extends Entity implements Serializable {
    @Id
    @GeneratedValue(strategy = SEQUENCE)
    @Column(name = "category_id")
    private Long id;

    @Column(name = "category_name", unique = true)
    private String categoryName;
    // getters, setters, toString
}

现在我要保存属于同一Website的5000 Category个。这是我如何批量插入：

@SpringBootApplication
public class DemoApplication implements CommandLineRunner {

    private Dao dao;

    @Autowired
    public void setDao(Dao dao) {
        this.dao = dao;
    }

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    @Override
    public void run(String... strings) throws Exception {
        insertWebsites();
    }

    private void insertWebsites() {
        List<Website> entities = new ArrayList<>(5000);
        Category category = new Category();
        category.setCategoryName("my category");

        for (int i = 0; i < 5000; i++) {
            Website website = new Website();
            website.setUrl("http://example.com" + i);
            website.setCategory(category);
            entities.add(website);
        }
        dao.bulkSave(entities);
    }
}

设置和参数：
（application.properties）

spring.datasource.driver-class-name = org.postgresql.Driver
spring.datasource.url = jdbc:postgresql://localhost:5432/spring_web
spring.datasource.username = postgres
spring.datasource.password = postgresql
spring.datasource.type = com.zaxxer.hikari.HikariDataSource

# Hibernate
spring.jpa.hibernate.ddl-auto = update
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.PostgreSQL94Dialect
spring.jpa.properties.hibernate.max_fetch_depth = 3
spring.jpa.properties.hibernate.jdbc.fetch_size = 30
spring.jpa.properties.hibernate.jdbc.batch_size = 30
spring.jpa.properties.hibernate.order_inserts = true
spring.jpa.properties.hibernate.order_updates = true
spring.jpa.properties.hibernate.cache.use_second_level_cache = false

（pom.xml中）

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>demo</name>
    <description>Demo project for Spring Boot</description>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.4.1.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>

        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <!--<scope>runtime</scope>-->
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>com.zaxxer</groupId>
            <artifactId>HikariCP</artifactId>
            <version>2.5.1</version>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

很抱歉，我试图提供所有细节。期待得到任何帮助！

Answer 1

执行后，问题就会明白：

创建单个类别
batch_size网站已创建并保留;他们都指向上面创建的单一类别
- 一旦第一个网站被保留，由于关系中的级联，该类别也会持续存在
由于所有batch_size个网站和类别都已保留，因此它们位于持久性上下文中。但是一旦你调用entityManager.clear()，它们就会被抛弃持久化上下文（释放内存）。 因此单个类别变得分离（即从持久化上下文中删除）并且您获得异常。您提到的站点中的示例不会通过关系持续存在，这就是它工作的原因。

鉴于这种关系，我不相信bulkSave()能以这种通用的方式工作。如果您想批量插入网站，请执行以下操作：

@Value("${spring.jpa.properties.hibernate.jdbc.batch_size}")
private int batchSize;

@PersistenceContext
private EntityManager em; // you will need the EntityManager

private void insertWebsites() {
    Category category = new Category();
    category.setCategoryName("my category");
    // persist it separately to keep the id for later
    em.persist(category);
    Long categoryId = category.getId();

    for (int i = 1; i <= 5000; i++) {
        Website website = new Website();
        website.setUrl("http://example.com" + i);
        website.setCategory(category);
        if (i % batchSize == 0) {
            em.flush();
            em.clear();
            // get an attached reference to the category (you know
            // that it exists) without querying the database again
            category = em.getReference(Category.class, categoryId);
        }
    }
}

JPA：同时批量保存子实体和父实体

1 个答案: