Java Hibernate搜索:如何通过两个Spring(JVM)进程更新索引?

Java Hibernate搜索:如何通过两个Spring(JVM)进程更新索引?,java,spring,hibernate,lucene,hibernate-search,Java,Spring,Hibernate,Lucene,Hibernate Search,我的应用程序有两个组件运行在两个不同的JVM进程中 1) 弹簧靴 2) Spring引导批处理(处理API提交的作业) 我在Spring中使用“Hibernate搜索”,这两个组件都需要更新搜索索引 然而,JVM进程似乎首先启动了一个锁,当另一个组件试图更新索引时,它抛出以下异常 如何使这两个JVM进程都更新索引而不出现锁问题 2017-05-22 02:33:56.795 ERROR 14701 --- [del.FeatureMeta] o.h.s.exception.impl.LogErr

我的应用程序有两个组件运行在两个不同的JVM进程中

1) 弹簧靴

2) Spring引导批处理(处理API提交的作业)

我在Spring中使用“Hibernate搜索”,这两个组件都需要更新搜索索引

然而,JVM进程似乎首先启动了一个锁,当另一个组件试图更新索引时,它抛出以下异常

如何使这两个JVM进程都更新索引而不出现锁问题

2017-05-22 02:33:56.795 ERROR 14701 --- [del.FeatureMeta] o.h.s.exception.impl.LogErrorHandler     : HSEARCH000058: Exception occurred org.apache.lucene.store.LockObtainFailedException: Lock held by another program: /home/bisuser/cdna-meta-index/default/com.company.dsd.cdna.repository.model.FeatureMeta/write.lock
Primary Failure:
    Entity com.company.dsd.cdna.repository.model.FeatureMeta  Id 169  Work Type  org.hibernate.search.backend.UpdateLuceneWork


org.apache.lucene.store.LockObtainFailedException: Lock held by another program: /home/bisuser/cdna-meta-index/default/com.company.dsd.cdna.repository.model.FeatureMeta/write.lock
    at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:118) ~[lucene-core-5.5.4.jar!/:5.5.4 31012120ebbd93744753eb37f1dbc5e654628291 - jpountz - 2017-02-08 19:08:03]
    at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-5.5.4.jar!/:5.5.4 31012120ebbd93744753eb37f1dbc5e654628291 - jpountz - 2017-02-08 19:08:03]
    at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-5.5.4.jar!/:5.5.4 31012120ebbd93744753eb37f1dbc5e654628291 - jpountz - 2017-02-08 19:08:03]
    at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:776) ~[lucene-core-5.5.4.jar!/:5.5.4 31012120ebbd93744753eb37f1dbc5e654628291 - jpountz - 2017-02-08 19:08:03]
    at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:126) ~[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
    at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:92) ~[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
    at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117) ~[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
    at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriterDelegate(AbstractWorkspaceImpl.java:203) ~[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
    at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:81) [hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
    at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:46) [hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
    at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.applyChangesets(SyncWorkProcessor.java:165) [hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
    at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.run(SyncWorkProcessor.java:151) [hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
2017-05-22 02:33:56.795错误14701-[del.FeatureMeta]o.h.s.exception.impl.LogErrorHandler:HSEARCH000058:异常发生org.apache.lucene.store.lockOcctainFailedException:另一个程序持有的锁:/home/biuser/cdna元索引/default/com.company.dsd.cdna.repository.model.FeatureMeta/write.Lock
主要故障:
实体com.company.dsd.cdna.repository.model.FeatureMeta Id 169工作类型org.hibernate.search.backend.updateLuceWork
org.apache.lucene.store.lockOcctainFailedException:另一个程序持有的锁:/home/bisuser/cdna meta index/default/com.company.dsd.cdna.repository.model.FeatureMeta/write.Lock
在org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:118)~[lucene-core-5.5.4.jar!/:5.5.4 31012120ebbd93744753eb37f1dbc554628291-jpountz-2017-02-08 19:08:03]
在org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)~[lucene-core-5.5.4.jar!/:5.5.4 31012120ebbd93744753eb37f1dbc554628291-jpountz-2017-02-08 19:08:03]
在org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)~[lucene-core-5.5.4.jar!/:5.5.4 31012120ebbd93744753eb37f1dbc5e65462891-jpountz-2017-02-08 19:08:03]
在org.apache.lucene.index.IndexWriter.(IndexWriter.java:776)~[lucene-core-5.5.4.jar!/:5.5.4 31012120ebbd93744753eb37f1dbc5e654628291-jpountz-2017-02-08 19:08:03]
在org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:126)~[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
在org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:92)~[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
在org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)~[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
在org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriterDelegate(AbstractWorkspaceImpl.java:203)~[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
在org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdate(LuceneBackendQueueTask.java:81)[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
在org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:46)[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
在org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.applyChangesets(SyncWorkProcessor.java:165)[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
在org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.run(SyncWorkProcessor.java:151)[hibernate-search-engine-5.6.1.Final.jar!/:5.6.1.Final]
在java.lang.Thread.run(Thread.java:745)[na:1.8.0_112]

为了Lucene提供良好的性能,独占锁是必需的。你可以禁用它,但要付出代价

原因是仍然会有一个锁定机制,尽管锁只会在很短的时间内使用。这意味着索引不会被并行写入,而另一个JVM进程可能会等待尝试获取锁(性能可能很差)。更糟糕的是,这些获取锁的尝试可能会失败:它们可能会超时。考虑到锁队列是不公平的(请求锁的最后一个JVM进程可能是第一个获得锁的进程),并且超时处理相当粗糙(一次尝试,然后是另一次尝试,比如2秒后,然后超时),这里有很高的失败风险

因此,如果您确信不会有任何关于锁的激烈争论(例如,仅在夜间或周末运行的大规模索引作业),禁用独占锁定可能是一种选择;否则,由于锁定超时,您很可能会出现显著的延迟和/或失败。 请参阅
hibernate.search.[default |].独占索引_使用
in

或者:

  • 您可以考虑是否真的需要在不同的JVM中发生这两件事(RESTAPI和批处理)(但我想您确实需要)
  • 您可以看看用于Hibernate搜索的/alternative架构。然而,这些都很难配置,您应该意识到这一点
  • 如果您需要JVM在单独的服务器上运行,并且在您的特定情况下,水平扩展(添加更多应用程序服务器)很重要,那么您可以看看
看看是否有帮助。