Question

目前我正在开发一款使用Neo4j的网络应用程序。我们的应用程序需要部署在Tomcat环境中（客户需求）。我们决定嵌入Neo4j，因为我们可以使用Neo4j提供的Java API，它更易于部署，并且我们获得了更高的性能。但是，我们还需要访问REST API，因为我们有一个用Angular编写的单页webapp，目前正在使用这个界面。然而，嵌入式Neo4j数据库不公开REST api。 Neo4j-server工件包含可以使用嵌入式图形数据库引导jetty服务器的代码。所以我们的Tomcat webapp正在启动一个Jetty服务器。我们可以通过在一个端口（8080）上部署在Tomcat中的webapp以及另一个端口（7474）上的Neo4j REST接口和Neo4j浏览器来访问graphDb。虽然这有点奇怪但它工作正常，除了我们试图停止我们的webapp（例如重新部署）。关闭我们的webapp时，我们从tomcat收到这些错误：

SEVERE: The web application [/chainmonitor] appears to have started a thread named [GC-Monitor] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [RRD4J Sync Pool [Thread-1]] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [Statistics Gatherer[primitives]] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [pool-1-thread-1] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [pool-1-thread-2] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [DateCache] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-31-selector-3] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-32-selector-0] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-33-selector-1] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-34-selector-2] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-35-selector-4] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-36-selector-5] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-37-acceptor-0-ServerConnector@1425c689{HTTP/1.1}{localhost:7474}] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-38-acceptor-1-ServerConnector@1425c689{HTTP/1.1}{localhost:7474}] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [HashSessionScavenger-0] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/chainmonitor] appears to have started a thread named [qtp835579386-54] but has failed to stop it. This is very likely to create a memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [org.eclipse.jetty.http.HttpFields$1] (value [org.eclipse.jetty.http.HttpFields$1@4a3d3a5e]) and a value of type [org.eclipse.jetty.http.HttpFields.DateGenerator] (value [org.eclipse.jetty.http.HttpFields$DateGenerator@24f0137b]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:17 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [/chainmonitor] created a ThreadLocal with key of type [scala.util.DynamicVariable$$anon$1] (value [scala.util.DynamicVariable$$anon$1@281a099d]) and a value of type [java.lang.Integer] (value [0]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
jun 10, 2014 6:43:27 PM org.apache.catalina.startup.HostConfig undeploy
INFO: Undeploying context [/chainmonitor]
jun 10, 2014 6:43:27 PM org.apache.catalina.startup.ExpandWar deleteDir
SEVERE: [C:\dev\apache-tomcat-7.0.54\webapps\chainmonitor\WEB-INF\lib] could not be completely deleted. The presence of the remaining files may cause problems
jun 10, 2014 6:43:27 PM org.apache.catalina.startup.ExpandWar deleteDir
SEVERE: [C:\dev\apache-tomcat-7.0.54\webapps\chainmonitor\WEB-INF] could not be completely deleted. The presence of the remaining files may cause problems
jun 10, 2014 6:43:27 PM org.apache.catalina.startup.ExpandWar deleteDir
SEVERE: [C:\dev\apache-tomcat-7.0.54\webapps\chainmonitor] could not be completely deleted. The presence of the remaining files may cause problems
jun 10, 2014 6:43:27 PM org.apache.catalina.startup.ExpandWar delete
SEVERE: [C:\dev\apache-tomcat-7.0.54\webapps\chainmonitor] could not be completely deleted. The presence of the remaining files may cause problems

我们的pom.xml看起来像这样：

    <dependency>
        <groupId>org.neo4j</groupId>
        <artifactId>neo4j</artifactId>
        <version>2.0.3</version>
    </dependency>
    <dependency>
        <groupId>org.neo4j.app</groupId>
        <artifactId>neo4j-server</artifactId>
        <version>2.0.3</version>
    </dependency>

这就是我们启动嵌入式数据库并启动jetty服务器的方法：

graphDb = (EmbeddedGraphDatabase) new GraphDatabaseFactory().
      newEmbeddedDatabaseBuilder(dataDir).loadPropertiesFromURL(Neo4jPropertiesUrl).newGraphDatabase();
bootstrapper = new WrappingNeoServerBootstrapper(graphDb);

目前neo4j-server.properties为空。我没有尝试过很多不同的配置，但这样做似乎不是问题。

这就是我们停止嵌入式数据库和jetty服务器的方式：

    @Override
    public void contextDestroyed(ServletContextEvent event) {
            graphDb.shutdown();
            bootstrapper.stop();
    }

这显然是由Tomcat的内存泄漏保护（http://wiki.apache.org/tomcat/MemoryLeakProtection）引起的。

结果是我们的webapp只能通过完全杀死Tomcat进程（kill -9）来重新部署，这是非常不受欢迎的。

我们尝试过很多东西：

不关闭嵌入式grapbDb。没效果。
没有明确关闭引导程序。没有效果。（引导程序有自己的关闭挂钩）
首先关闭引导程序，然后关闭嵌入式数据库。
线程杀害者：https://github.com/Neo4j/Neo4j/issues/1070。代码运行并删除59个线程本地值，但没有明显的效果。有趣的是，当我在关机期间在一个循环中运行它一分钟时，它一直说它已经死了6个值。对我来说，这表明这个过程根本不起作用，因为Neo4j在关闭时需要一些时间。
关机后延迟。我们尝试这个的原因是因为在tomcat收到关闭信号之后我们需要等待几秒才能实际杀死进程，否则我们在下次启动时会遇到错误，我们的图形数据已经损坏（有一条消息没有被正确关闭了）。延迟5秒以上似乎也有效果，因为之后我们只得到6个严重错误而不是16个。这也是我认为献祭过程根本不起作用的原因。
我们尝试将嵌入式Neo4j添加到集群中，因此可以同时使用REST API和嵌入式Neo4j。然而，经过大约四个小时试图让这个工作，我们放弃了。在嵌入式Neo4j数据库启动期间，我们的web应用程序就冻结了。经过一些调试后，它似乎在Neo4j的代码中陷入僵局。我们尝试将所有HA超时配置设置为较小的值，但没有任何效果。

问题是由Neo4j jetty服务器引起的。没有它，就不会出现这个问题。我们需要REST API。你知道我们怎么解决这个问题吗？或者您是否有另一种方法在Tomcat环境中使用嵌入式数据库和REST API？

我们正在使用：

Java：1.7.0_60
Tomcat 7.0.54
Neo4j：2.0.3（我们已经尝试过2.1.1，但得到了完全相同的结果）
在Windows 7和Linux Redhat上观察到同样的行为

Answer 1

你可能有泄漏的交易尚未关闭，然后Neo4j在关机时等待最多20秒以关闭这些交易。

如果等待5秒以上，那些报告的问题仍然存在？

Answer 2

我们有非常相似的设置和需求。以下是我们针对此问题提供的解决方案。 JVM运行时shutdownHook在Tomcat中不起作用，因此我们保存对包含GraphDatabaseService的包装器对象的引用，并将该包装器放在存储库管理器对象中。对存储库管理器对象中的方法使用@PreDestroy批注，该方法将在销毁存储库管理器类之前关闭存储库。

import javax.annotation.PreDestroy;

...

@PreDestroy
public void destroy() {
    log.info("Destroying Neo4jRepositoryManager...");
    shutdown(Neo4jPropertyKeys.SANDBOX_DATABASE.getUrlKey());
    shutdown(Neo4jPropertyKeys.PRODUCTION_DATABASE.getUrlKey());
    shutdown(Neo4jPropertyKeys.TEST_DATABASE.getUrlKey());
}

private void shutdown(String urlKey) {
    if(repositories.containsKey(urlKey)) {
        LabeledNodeGraphRepository graphdb = repositories.get(urlKey);
        graphdb.getGraphDatabaseService().shutdown();
        log.info(urlKey + " shutdown...");
    }
}

<bean id="repositoryManager" class="mypath.neo4j.repository.Neo4jRepositoryManager"/>

我们的环境是：

Java 1.7.0_72
Tomcat 7.0.56
Neo4j 2.1.6
Spring 4.0.5
泽西岛2.1.4

无法在Tomcat webapp中关闭Neo4j Jetty服务器

2 个答案: