Serialization Hazelcast-巨大的反序列化开销

我在分布式缓存中保存了很多小对象。大多数操作都是本地的，这意味着它们在同一个节点上访问数据（我正在使用分区感知功能）。即使数据在同一个节点上，Hazelcast仍然需要对项目进行反序列化，这会消耗大量内存，并使应用程序崩溃，同时会导致OutOfMemoryError导致更多顶点和并行执行更多操作。我知道有一个近缓存，我可以将其与缓存本地条目和等于OBJECT的数据格式一起使用，但这并不适合，因为我对该数据执行了一些变异操作。我还使用IdentifiedDataSerializable来表示存储的数据和正在序列化的所有内容，所以我想没有办法从中获得任何东西

下面是示例OOM错误stacktrace（它在节点端抛出，而不是在客户端抛出）：

这就是Hazelcast mancenter的情况

这是它几乎崩溃的时候。任何更大的问题都会失败

有趣的是，我要使用多少节点并不重要。它们仍然会消耗大量内存空间，并且仍然会失败。所以这不是关于存储数据，而是关于检索数据的能力

---

有没有其他方法可以使反序列化消耗更少的内存？

避免反序列化开销的最佳方法是将实际数据存储在每个节点内的静态映射中，并使用Hazelcast执行器作为路由器将请求路由到包含该数据的节点。

IExecutorService.submitToKeyOwner（）

将请求路由到拥有段密钥的节点，在该节点内，您可以在静态映射中维护实际值。

---

请注意，在这种方法中，您需要自己清理本地静态映射。尽管您可以使用MapEvent侦听器来处理清理工作。

我想我可能会使用WeakHashMap来完成这项工作-仅为本地读取激活“我自己的缓存”。让我知道它是如何进行的。

java.lang.OutOfMemoryError: Java heap space
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.ensureAvailable(ByteArrayObjectDataOutput.java:359)
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.write(ByteArrayObjectDataOutput.java:70)
    at java.io.OutputStream.write(OutputStream.java:75)
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.writeByteArray(ByteArrayObjectDataOutput.java:263)
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.writeData(ByteArrayObjectDataOutput.java:376)
    at com.hazelcast.map.impl.MapEntries.writeData(MapEntries.java:141)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:197)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:49)
    at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.write(StreamSerializerAdapter.java:43)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.writeObject(AbstractSerializationService.java:249)
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.writeObject(ByteArrayObjectDataOutput.java:370)
    at com.hazelcast.spi.impl.operationservice.impl.operations.PartitionIteratingOperation$PartitionResponse.writeData(PartitionIteratingOperation.java:289)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:197)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:49)
    at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.write(StreamSerializerAdapter.java:43)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.writeObject(AbstractSerializationService.java:249)
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.writeObject(ByteArrayObjectDataOutput.java:370)
    at com.hazelcast.spi.impl.operationservice.impl.responses.NormalResponse.writeData(NormalResponse.java:91)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:197)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:49)
    at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.write(StreamSerializerAdapter.java:43)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:151)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:125)
    at com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.send(OutboundResponseHandler.java:88)
    at com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.sendResponse(OutboundResponseHandler.java:64)
    at com.hazelcast.spi.Operation.sendResponse(Operation.java:338)
    at com.hazelcast.spi.impl.operationservice.impl.operations.PartitionIteratingOperation$OperationResponseHandlerImpl.sendResponse(PartitionIteratingOperation.java:219)
    at com.hazelcast.spi.impl.operationservice.impl.operations.PartitionIteratingOperation$OperationResponseHandlerImpl.sendResponse(PartitionIteratingOperation.java:207)
    at com.hazelcast.spi.Operation.sendResponse(Operation.java:338)
    at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.sendResponse(OperationRunnerImpl.java:279)
    at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.handleResponse(OperationRunnerImpl.java:270)
    at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:187)
Jun 11, 2017 9:57:51 PM com.hazelcast.map.impl.operation.PartitionWideEntryWithPredicateOperation
SEVERE: [10.6.3.57]:5701 [kpts-cluster] [3.8] Java heap space
java.lang.OutOfMemoryError: Java heap space
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.ensureAvailable(ByteArrayObjectDataOutput.java:359)
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.write(ByteArrayObjectDataOutput.java:70)
    at java.io.OutputStream.write(OutputStream.java:75)
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.writeByteArray(ByteArrayObjectDataOutput.java:263)
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.writeData(ByteArrayObjectDataOutput.java:376)
    at com.hazelcast.map.impl.MapEntries.writeData(MapEntries.java:141)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:197)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:49)
    at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.write(StreamSerializerAdapter.java:43)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.writeObject(AbstractSerializationService.java:249)
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.writeObject(ByteArrayObjectDataOutput.java:370)
    at com.hazelcast.spi.impl.operationservice.impl.operations.PartitionIteratingOperation$PartitionResponse.writeData(PartitionIteratingOperation.java:289)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:197)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:49)
    at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.write(StreamSerializerAdapter.java:43)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.writeObject(AbstractSerializationService.java:249)
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.writeObject(ByteArrayObjectDataOutput.java:370)
    at com.hazelcast.spi.impl.operationservice.impl.responses.NormalResponse.writeData(NormalResponse.java:91)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:197)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:49)
    at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.write(StreamSerializerAdapter.java:43)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:151)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:125)
    at com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.send(OutboundResponseHandler.java:88)
    at com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.sendResponse(OutboundResponseHandler.java:64)
    at com.hazelcast.spi.Operation.sendResponse(Operation.java:338)
    at com.hazelcast.spi.impl.operationservice.impl.operations.PartitionIteratingOperation$OperationResponseHandlerImpl.sendResponse(PartitionIteratingOperation.java:219)
    at com.hazelcast.spi.impl.operationservice.impl.operations.PartitionIteratingOperation$OperationResponseHandlerImpl.sendResponse(PartitionIteratingOperation.java:207)
    at com.hazelcast.spi.Operation.sendResponse(Operation.java:338)
    at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.sendResponse(OperationRunnerImpl.java:279)
    at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.handleResponse(OperationRunnerImpl.java:270)
    at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:187)