Java OpenHFT/历史记录映射失败
我使用一个VanillaChroncile临时存储和检索条目,除了有巨大的负载外,它工作得非常好。我得到地图失败异常。虽然我有处理此异常的恢复逻辑,但我想知道为什么首先会出现此异常。任何帮助都将不胜感激Java OpenHFT/历史记录映射失败,java,chronicle,chronicle-map,Java,Chronicle,Chronicle Map,我使用一个VanillaChroncile临时存储和检索条目,除了有巨大的负载外,它工作得非常好。我得到地图失败异常。虽然我有处理此异常的恢复逻辑,但我想知道为什么首先会出现此异常。任何帮助都将不胜感激 public enum PreAuthEventChronicle { INSTANCE; private String basePath = PropertyUtilsEnum.INSTANCE.getChronicleDirPath(); private final Logger
public enum PreAuthEventChronicle {
INSTANCE;
private String basePath = PropertyUtilsEnum.INSTANCE.getChronicleDirPath();
private final Logger logger = Logger.getLogger(PreAuthEventChronicle.class);
private long indexBlockSize = 64L;
// Length of the excerpt.
private final int excerptLength = 1024;
private final int cycleLength = "yyyyMMddhhmmss";
// Number of entries that are stored in the chronicle queue.
private final long entriesPerCycle = 1L << 20;
// Format for the folder. Chronicle writes in GMT time zone.
private final String format = ApplicationConstants.CHRONICLE_FORMAT;
private Chronicle chronicle = null;
private List<PreAuthEventListener> listeners = new ArrayList<PreAuthEventListener>();
public Chronicle getChr() {
return chronicle;
}
/**
* There can only be one chronicle built.
*/
private PreAuthEventChronicle() {
if (basePath != null) {
if (!basePath.endsWith(ApplicationConstants.FILE_SEP)) {
basePath = basePath + ApplicationConstants.FILE_SEP + ApplicationConstants.CHRONICLE_DATA_PATH;
}
logger.debug("Starting from a clean state");
cleanUp();
logger.debug("Building a Vanilla Chronicle instance with path: " + basePath);
buildChronicle();
} else {
throw new RuntimeException("No directory specified for chronicle to be built.");
}
}
private void buildChronicle() {
logger.debug("Begin-Starting to build a vanilla chronicle");
try {
if (chronicle != null) {
chronicle.clear();
chronicle = null;
}
chronicle = ChronicleQueueBuilder.vanilla(basePath).cycleLength(cycleLength, false).cycleFormat(format)
.indexBlockSize(indexBlockSize).entriesPerCycle(entriesPerCycle).build();
} catch (IOException e) {
logger.error("Error building chronicle" + e.getMessage());
}
logger.debug("End-Finished building the vanilla chronicle");
}
/**
* Clean up the resources
*/
public void cleanUp() {
logger.debug("Begin-Cleaning up chronicle resources");
File f = new File(basePath);
if (f.exists() && f.isDirectory()) {
File[] dirs = f.listFiles();
for (File dir : dirs) {
if (dir.isDirectory()) {
try {
FileUtils.deleteDirectory(dir);
} catch (IOException ignore) {
}
}
}
}
buildChronicle();
logger.debug("End-Done cleaning up chronicle resources");
}
/**
* Write the object to the chronicle queue, and notify the listeners
*
* @param event
* @throws IOException
*/
public synchronized void writeObject(Object event) throws IOException {
ExcerptAppender appender = INSTANCE.getChr().createAppender();
if (appender != null && event != null) {
logger.debug("Begin-Writing event to the chronicle queue");
appender.startExcerpt(excerptLength);
appender.writeObject(event);
appender.finish();
appender.clear();
appender.close();
notifyListeners();
logger.debug("End-Done writing event to the chronicle queue.");
}
}
/**
* Read the object from the queue
*
* @return
* @throws IOException
*/
public synchronized Object readObject() throws IOException {
ExcerptTailer reader = INSTANCE.getChr().createTailer().toStart();
Object evt = null;
while (reader != null && reader.nextIndex()) {
logger.debug("Begin-Reading event from the chronicle queue");
evt = reader.readObject();
reader.finish();
reader.clear();
reader.close();
logger.debug("End-Done reading the event from the chronicle queue.");
}
return evt;
}
/**
* Attach a listener
*
* @param listen
*/
public void attachListener(PreAuthEventListener listen) {
listeners.add(listen);
}
/**
* Notify the listeners that an event has been written.
*/
private void notifyListeners() {
for (PreAuthEventListener listener : listeners) {
logger.debug("Notification received from the chronicle queue. Performing action.");
listener.perform();
}
}
公共枚举预授权事件记录{
实例;
私有字符串basePath=PropertyUtilsEnum.INSTANCE.GetChroniceDirPath();
私有最终记录器=Logger.getLogger(PreAuthEventChronicle.class);
专用长索引块大小=64L;
//摘录的长度。
私有最终整型摘录长度=1024;
专用最终整数循环长度=“yyyyMMddhhmmss”;
//存储在历史记录队列中的条目数。
private final long-entriesPerCycle=1L了解您得到的异常情况会很有用,但我打赌原因是OutOfMemoryError
关于您的代码:
- 您希望每秒滚动一次,这是可能的,但不是真正有效的。对于每一次滚动,历史记录队列需要mmap至少两个区域
- readObject/writeObject是不需要作为Va的同步方法
nillaChronicle是线程安全的
- 在readObject中,关闭每个循环中的尾部,这可能会导致映射区域在每次迭代中被取消映射/映射,最好在循环外调用close()
- 在writeObject中,您可以显式地关闭摘录,如果您无法控制调用该方法的线程数,则可以关闭摘录,但这可能不是最有效的方法,因为对于每个新线程,都必须映射一个新区域
- 无需调用[tailer | appender].clear()
了解您得到的异常情况会很有用,但我敢打赌原因是OutOfMemoryError
关于您的代码:
- 您希望每秒滚动一次,这是可能的,但不是真正有效的。对于每一次滚动,历史记录队列需要mmap至少两个区域
- readObject/writeObject是不需要作为Va的同步方法
nillaChronicle是线程安全的
- 在readObject中,关闭每个循环中的尾部,这可能会导致映射区域在每次迭代中被取消映射/映射,最好在循环外调用close()
- 在writeObject中,您可以显式地关闭摘录,如果您无法控制调用该方法的线程数,则可以关闭摘录,但这可能不是最有效的方法,因为对于每个新线程,都必须映射一个新区域
- 无需调用[tailer | appender].clear()