Neo4J引发IOException，然后定期停止

因此，我们在rancher中有一个Neo4J实例，它使用3.3版的docker映像。它有1GB的heapspace和页面缓存，但由于这个问题，我们将其增加到2GB

在我们开始出现此错误之前，一切都会平稳运行一段时间：

2018-08-02 06:03:30.310+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [1942]:  Starting check pointing...
2018-08-02 06:03:30.311+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [1942]:  Starting store flush...
2018-08-02 06:03:30.313+0000 ERROR [o.n.k.i.t.l.c.CheckPointerImpl] Error performing check point java.io.IOException: I/O error
org.neo4j.kernel.impl.store.UnderlyingStorageException: java.io.IOException: I/O error
        at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:263)
        at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce(RecordStorageEngine.java:460)
        at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint(CheckPointerImpl.java:160)
        at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded(CheckPointerImpl.java:134)
        at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:64)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        at org.neo4j.helpers.NamedThreadFactory$2.run(NamedThreadFactory.java:109)
Caused by: java.io.IOException: I/O error
        at sun.nio.ch.FileDispatcherImpl.force0(Native Method)
        at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:76)
        at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:388)
        at org.neo4j.io.fs.StoreFileChannel.force(StoreFileChannel.java:111)
        at org.neo4j.io.pagecache.impl.SingleFilePageSwapper.force(SingleFilePageSwapper.java:711)
        at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForceInternal(MuninnPagedFile.java:425)
        at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce(MuninnPagedFile.java:274)
        at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:901)
        at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:894)
        at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:259)
        ... 12 more

此错误发生多次，然后我们得到此错误：

2018-08-02 06:03:30.310+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [1942]:  Starting check pointing...
2018-08-02 06:03:30.311+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Check Pointing triggered by scheduler for time threshold [1942]:  Starting store flush...
2018-08-02 06:03:30.313+0000 ERROR [o.n.k.i.t.l.c.CheckPointerImpl] Error performing check point java.io.IOException: I/O error
org.neo4j.kernel.impl.store.UnderlyingStorageException: java.io.IOException: I/O error
        at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:263)
        at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce(RecordStorageEngine.java:460)
        at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint(CheckPointerImpl.java:160)
        at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded(CheckPointerImpl.java:134)
        at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:64)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        at org.neo4j.helpers.NamedThreadFactory$2.run(NamedThreadFactory.java:109)
Caused by: java.io.IOException: I/O error
        at sun.nio.ch.FileDispatcherImpl.force0(Native Method)
        at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:76)
        at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:388)
        at org.neo4j.io.fs.StoreFileChannel.force(StoreFileChannel.java:111)
        at org.neo4j.io.pagecache.impl.SingleFilePageSwapper.force(SingleFilePageSwapper.java:711)
        at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForceInternal(MuninnPagedFile.java:425)
        at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce(MuninnPagedFile.java:274)
        at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:901)
        at org.neo4j.index.internal.gbptree.GBPTree.checkpoint(GBPTree.java:894)
        at org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force(NativeLabelScanStore.java:259)
        ... 12 more

2018-08-02 06:05:00.364+0000错误[o.n.k.i.DatabaseHealth]数据库死机：数据库遇到严重错误，需要重新启动。有关详细信息，请参阅数据库日志。执行检查点时出错
org.neo4j.kernel.impl.store.underlyngStorageException:执行检查点时出错
位于org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.constructCombinedFailure（CheckPointScheduler.java:100）
位于org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run（CheckPointScheduler.java:81）
位于java.util.concurrent.Executors$RunnableAdapter.call（Executors.java:511）
在java.util.concurrent.FutureTask.run（FutureTask.java:266）处
位于java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201（ScheduledThreadPoolExecutor.java:180）
位于java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run（ScheduledThreadPoolExecutor.java:293）
位于java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1149）
位于java.util.concurrent.ThreadPoolExecutor$Worker.run（ThreadPoolExecutor.java:624）
运行（Thread.java:748）
位于org.neo4j.helpers.NamedThreadFactory$2.run（NamedThreadFactory.java:109）
抑制：org.neo4j.kernel.impl.store.underlyngStorageException:java.io.IOException:I/O错误
位于org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force（NativeLabelScanStore.java:263）
位于org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce（RecordStorageEngine.java:460）
位于org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint（CheckPointerImpl.java:160）
位于org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded（CheckPointerImpl.java:134）
位于org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run（CheckPointScheduler.java:64）
... 8个以上
原因：java.io.IOException:I/O错误
位于sun.nio.ch.FileDispatcherImpl.force0（本机方法）
位于sun.nio.ch.FileDispatcherImpl.force（FileDispatcherImpl.java:76）
位于sun.nio.ch.FileChannelImpl.force（FileChannelImpl.java:388）
位于org.neo4j.io.fs.StoreFileChannel.force（StoreFileChannel.java:111）
位于org.neo4j.io.pagecache.impl.SingleFilePageSwapper.force（SingleFilePageSwapper.java:711）
位于org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForceInternal（MuninnPagedFile.java:425）
位于org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce（MuninnPagedFile.java:274）
位于org.neo4j.index.internal.gbptree.gbptree.checkpoint（gbptree.java:901）
位于org.neo4j.index.internal.gbptree.gbptree.checkpoint（gbptree.java:894）
位于org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force（NativeLabelScanStore.java:259）
... 还有12个
抑制：org.neo4j.kernel.impl.store.underlyngStorageException:java.io.IOException:I/O错误
位于org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force（NativeLabelScanStore.java:263）
位于org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce（RecordStorageEngine.java:460）
位于org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint（CheckPointerImpl.java:160）
位于org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded（CheckPointerImpl.java:134）
位于org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run（CheckPointScheduler.java:64）
... 8个以上
原因：java.io.IOException:I/O错误
位于sun.nio.ch.FileDispatcherImpl.force0（本机方法）
位于sun.nio.ch.FileDispatcherImpl.force（FileDispatcherImpl.java:76）
位于sun.nio.ch.FileChannelImpl.force（FileChannelImpl.java:388）
位于org.neo4j.io.fs.StoreFileChannel.force（StoreFileChannel.java:111）
位于org.neo4j.io.pagecache.impl.SingleFilePageSwapper.force（SingleFilePageSwapper.java:711）
位于org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForceInternal（MuninnPagedFile.java:425）
位于org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce（MuninnPagedFile.java:274）
位于org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.flushAndForce（MuninnPagedFile.java:262）
位于org.neo4j.index.internal.gbptree.gbptree.checkpoint（gbptree.java:921）
位于org.neo4j.index.internal.gbptree.gbptree.checkpoint（gbptree.java:894）
位于org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force（NativeLabelScanStore.java:259）
... 还有12个
抑制：org.neo4j.kernel.impl.store.underlyngStorageException:java.io.IOException:I/O错误
位于org.neo4j.kernel.impl.index.labelscan.NativeLabelScanStore.force（NativeLabelScanStore.java:263）
位于org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce（RecordStorageEngine.java:460）
位于org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint（CheckPointerImpl.java:160）
位于org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded（CheckPointerImpl.java:134）
位于org.neo4j.kernel.impl