Java 在Spring Data MongoDB中返回流时指定游标选项?
我正在使用Spring Data MongoDB(Spring boot中的Spring boot starter Data MongoDBJava 在Spring Data MongoDB中返回流时指定游标选项?,java,spring,mongodb,spring-boot,spring-data-mongodb,Java,Spring,Mongodb,Spring Boot,Spring Data Mongodb,我正在使用Spring Data MongoDB(Spring boot中的Spring boot starter Data MongoDB1.5.2.RELEASE)和MongoDB3.4.9,并定义了一个如下所示的存储库: interface MyMongoDBRepository extends CrudRepository<MyDTO, String> { Stream<MyDTO> findAllByCategory(String category);
1.5.2.RELEASE
)和MongoDB3.4.9
,并定义了一个如下所示的存储库:
interface MyMongoDBRepository extends CrudRepository<MyDTO, String> {
Stream<MyDTO> findAllByCategory(String category);
}
数据库中有大量数据,有时会发生以下错误:
2018-01-01 18:16:56.631 ERROR 1 --- [ask-scheduler-6] o.s.integration.handler.LoggingHandler : org.springframework.dao.DataAccessResourceFailureException:
Query failed with error code -5 and error message 'Cursor 73973161000 not found on server <mongodb-server>' on server <mongodb-server>;
nested exception is com.mongodb.MongoCursorNotFoundException:
Query failed with error code -5 and error message 'Cursor 73973161000 not found on server <mongodb-server>' on server <mongodb-server>
at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:77)
at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:2135)
at org.springframework.data.mongodb.core.MongoTemplate.access$1100(MongoTemplate.java:147)
at org.springframework.data.mongodb.core.MongoTemplate$CloseableIterableCursorAdapter.hasNext(MongoTemplate.java:2506)
at java.util.Iterator.forEachRemaining(Iterator.java:115)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at com.mycompany.MyService.doStuff(MyService.java:108)
at com.mycompany.AnotherService.doStuff(AnotherService.java:42)
at sun.reflect.GeneratedMethodAccessor2026.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748) Caused by: com.mongodb.MongoCursorNotFoundException: Query failed with error code -5 and error message 'Cursor 73973161000 not found on server <mongodb-server>' on server <mongodb-server>
at com.mongodb.operation.QueryHelper.translateCommandException(QueryHelper.java:27)
at com.mongodb.operation.QueryBatchCursor.getMore(QueryBatchCursor.java:213)
at com.mongodb.operation.QueryBatchCursor.hasNext(QueryBatchCursor.java:103)
at com.mongodb.MongoBatchCursorAdapter.hasNext(MongoBatchCursorAdapter.java:46)
at com.mongodb.DBCursor.hasNext(DBCursor.java:145)
at org.springframework.data.mongodb.core.MongoTemplate$CloseableIterableCursorAdapter.hasNext(MongoTemplate.java:2504) ... 24 more
2018-01-01 18:16:56.631错误1---[ask-scheduler-6]o.s.integration.handler.LoggingHandler:org.springframework.dao.DataAccessResourceFailureException:
查询失败,错误代码为-5,服务器上出现错误消息“服务器上未找到游标73973161000”;
嵌套异常为com.mongodb.MongoCursorNotFoundException:
查询失败,错误代码为-5,服务器上出现错误消息“未在服务器上找到游标73973161000”
位于org.springframework.data.mongodb.core.MongoExceptionTranslator.TranslateExceptionIfAbsible(MongoExceptionTranslator.java:77)
位于org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:2135)
位于org.springframework.data.mongodb.core.MongoTemplate.access$1100(MongoTemplate.java:147)
位于org.springframework.data.mongodb.core.MongoTemplate$CloseableIterableCursorAdapter.hasNext(MongoTemplate.java:2506)
位于java.util.Iterator.ForEachLeving(Iterator.java:115)
位于java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
位于java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
位于java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
位于java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
位于java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
位于java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
位于java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
在com.mycompany.MyService.doStuff(MyService.java:108)
在com.mycompany.AnotherService.doStuff(AnotherService.java:42)
位于sun.reflect.GeneratedMethodAccessor2026.invoke(未知源)
在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)中
位于java.lang.reflect.Method.invoke(Method.java:498)
位于org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
位于org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
位于org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
在java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)在java.util.concurrent.FutureTask.run(FutureTask.java:266)处
位于java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
位于java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
位于java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
位于java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
在java.lang.Thread.run(Thread.java:748)处,由以下原因引起:com.mongodb.mongocursorsornotfoundException:查询失败,错误代码为-5,服务器上出现错误消息“游标73973161000未在服务器上找到”
位于com.mongodb.operation.QueryHelper.translateCommandException(QueryHelper.java:27)
在com.mongodb.operation.QueryBatchCursor.getMore(QueryBatchCursor.java:213)
位于com.mongodb.operation.QueryBatchCursor.hasNext(QueryBatchCursor.java:103)
位于com.mongodb.MongoBatchCursorAdapter.hasNext(MongoBatchCursorAdapter.java:46)
位于com.mongodb.DBCursor.hasNext(DBCursor.java:145)
位于org.springframework.data.mongodb.core.MongoTemplate$CloseableIterableCursorAdapter.hasNext(MongoTemplate.java:2504)。。。还有24个
我在不同的地方读到过,当使用vanilla MongoDB Java客户机时,您可以将MongoDB游标配置为没有超时,或者设置一个批处理大小来缓解这一问题
如果是这样的话,那么在从Spring Data MongoDB返回
流时如何提供光标选项?关于您提到的两个选项
批大小,不能使用存储库类设置批大小。您可以使用MongoTemplate
来完成此操作。像这样的
final DBCursor cursor = mongoTemplate
.getCollection(collectionName)
.find(queryBuilder.get(), projection)
.batchSize(readBatchSize);
while (cursor.hasNext()) {
......
......
}
但要使用MongoTemplate,您需要创建一个自定义存储库
关于游标超时。你可以这样做
@Configuration
public class MongoDbSettings {
@Bean
public MongoClientOptions setmongoOptions() {
return MongoClientOptions.builder().socketTimeout(5000).build();
}
}
您可以为Mongo设置许多其他选项(heartbeat
,connectiontimeout
)。您可以在application.properties文件中设置这些属性,然后使用上述类中的@Value
将其绑定并设置(而不是硬编码)。
不幸的是,spring boot没有提供任何方法在应用程序.properties
文件中指定这些属性从spring Data MongoDB返回流时,不需要提供游标选项。此异常的可能原因是您的服务如何从Mongo读取数据。可能的原因:
您正在跨多个线程共享一个光标
一次请求的元素太多
Mongo服务器之前的负载均衡器
有关适用于您的应用程序的一些想法和方向,请参阅Jira topic的评论 发生错误是因为处理流太慢,因此光标在进入下一批之前超时
批大小可以在上设置,也可以使用注释在存储库上设置。例如:
Query query = query(where("firstname").is("luke"))
.batchSize(100);
或者在使用存储库时:
@Meta(batchSize = 100)
List<Person> findByFirstname(String firstname);
@Meta(batchSize=100)
列出findByFirstname(字符串名);
有关更多详细信息,请参阅
还可以使用相同的配置在每个查询的基础上禁用游标超时。e、 g.@Meta(flags={CursorOption.NO\u TIMEOUT})
不能基于每个查询更改游标超时。这是一种服务器配置。您需要使用服务器参数来更改服务器范围。1)
@Meta(batchSize = 100)
List<Person> findByFirstname(String firstname);