如何在Java中对直接缓冲区进行垃圾收集
我有一个内存泄漏,我已经隔离到错误配置的直接字节缓冲区 ByteBuffer buff = ByteBuffer.allocateDirect(7777777); ByteBuffer buff=ByteBuffer.allocateDirect(777); GC收集包含这些缓冲区的对象,但不处理缓冲区本身。如果我实例化了足够多的包含缓冲区的瞬态对象,我会得到以下令人鼓舞的消息: java.lang.OutOfMemoryError: Direct buffer memory java.lang.OutOfMemoryError:直接缓冲内存 我一直在寻找这个问题,显然 buff.clear(); buff.clear(); 及 gc();如何在Java中对直接缓冲区进行垃圾收集,java,memory-leaks,buffer,bytebuffer,Java,Memory Leaks,Buffer,Bytebuffer,我有一个内存泄漏,我已经隔离到错误配置的直接字节缓冲区 ByteBuffer buff = ByteBuffer.allocateDirect(7777777); ByteBuffer buff=ByteBuffer.allocateDirect(777); GC收集包含这些缓冲区的对象,但不处理缓冲区本身。如果我实例化了足够多的包含缓冲区的瞬态对象,我会得到以下令人鼓舞的消息: java.lang.OutOfMemoryError: Direct buffer memory java.lang
不工作。我怀疑您的应用程序在某处引用了ByteBuffer实例,这会阻止它被垃圾收集 直接ByteBuffer的缓冲内存分配在正常堆之外(这样GC就不会移动它!!)。但是,ByteBufferAPI没有提供显式处理/释放缓冲区的方法。所以我假设垃圾收集器会这样做。。。一旦确定ByteBuffer对象不再被引用。文档中说: 可以通过调用此类的工厂方法来创建直接字节缓冲区。此方法返回的缓冲区通常比非直接缓冲区具有更高的分配和释放成本。直接缓冲区的内容可能位于正常垃圾收集堆之外,因此它们对应用程序内存占用的影响可能并不明显。因此,建议直接缓冲区主要分配给受底层系统本机I/O操作影响的大型长寿命缓冲区。通常,只有当直接缓冲区在程序性能上产生可测量的增益时,才最好分配直接缓冲区
特别是,语句“可能位于正常垃圾收集堆之外”似乎与您的示例相关。分配的内存是通过本机库实现的。当调用ByteBuffer#finalize方法时,该内存将被释放,这与缓冲区为gc'd时的情况相同。请查看的allocate()和finalize()实现
buff.clear()
是不必要的,System.gc()
只有在像前面提到的其他对象一样,不再有对ByteBuffer对象的引用时才有帮助。DBB一旦进入引用队列并运行终结器,它就会被释放。然而,由于我们不能依赖终结器来运行,我们可以使用反射来手动调用它的“清理器”
使用反射:
/**
* DirectByteBuffers are garbage collected by using a phantom reference and a
* reference queue. Every once a while, the JVM checks the reference queue and
* cleans the DirectByteBuffers. However, as this doesn't happen
* immediately after discarding all references to a DirectByteBuffer, it's
* easy to OutOfMemoryError yourself using DirectByteBuffers. This function
* explicitly calls the Cleaner method of a DirectByteBuffer.
*
* @param toBeDestroyed
* The DirectByteBuffer that will be "cleaned". Utilizes reflection.
*
*/
public static void destroyDirectByteBuffer(ByteBuffer toBeDestroyed)
throws IllegalArgumentException, IllegalAccessException,
InvocationTargetException, SecurityException, NoSuchMethodException {
Preconditions.checkArgument(toBeDestroyed.isDirect(),
"toBeDestroyed isn't direct!");
Method cleanerMethod = toBeDestroyed.getClass().getMethod("cleaner");
cleanerMethod.setAccessible(true);
Object cleaner = cleanerMethod.invoke(toBeDestroyed);
Method cleanMethod = cleaner.getClass().getMethod("clean");
cleanMethod.setAccessible(true);
cleanMethod.invoke(cleaner);
}
下面是一个适用于任何直接缓冲区的优化实现:
public static void destroyBuffer(Buffer buffer) {
if(buffer.isDirect()) {
try {
if(!buffer.getClass().getName().equals("java.nio.DirectByteBuffer")) {
Field attField = buffer.getClass().getDeclaredField("att");
attField.setAccessible(true);
buffer = (Buffer) attField.get(buffer);
}
Method cleanerMethod = buffer.getClass().getMethod("cleaner");
cleanerMethod.setAccessible(true);
Object cleaner = cleanerMethod.invoke(buffer);
Method cleanMethod = cleaner.getClass().getMethod("clean");
cleanMethod.setAccessible(true);
cleanMethod.invoke(cleaner);
} catch(Exception e) {
throw new QuartetRuntimeException("Could not destroy direct buffer " + buffer, e);
}
}
}
只要您依赖于特定于sun(oracle)的实现,比尝试更改java.nio.DirectByteBuffer的可见性更好的选择是通过反射使用sun.nio.ch.DirectBuffer接口
/**
* Sun specific mechanisms to clean up resources associated with direct byte buffers.
*/
@SuppressWarnings("unchecked")
private static final Class<? extends ByteBuffer> SUN_DIRECT_BUFFER = (Class<? extends ByteBuffer>) lookupClassQuietly("sun.nio.ch.DirectBuffer");
private static final Method SUN_BUFFER_CLEANER;
private static final Method SUN_CLEANER_CLEAN;
static
{
Method bufferCleaner = null;
Method cleanerClean = null;
try
{
// operate under the assumption that if the sun direct buffer class exists,
// all of the sun classes exist
if (SUN_DIRECT_BUFFER != null)
{
bufferCleaner = SUN_DIRECT_BUFFER.getMethod("cleaner", (Class[]) null);
Class<?> cleanClazz = lookupClassQuietly("sun.misc.Cleaner");
cleanerClean = cleanClazz.getMethod("clean", (Class[]) null);
}
}
catch (Throwable t)
{
t.printStackTrace();
}
SUN_BUFFER_CLEANER = bufferCleaner;
SUN_CLEANER_CLEAN = cleanerClean;
}
public static void releaseDirectByteBuffer(ByteBuffer buffer)
{
if (SUN_DIRECT_BUFFER != null && SUN_DIRECT_BUFFER.isAssignableFrom(buffer.getClass()))
{
try
{
Object cleaner = SUN_BUFFER_CLEANER.invoke(buffer, (Object[]) null);
SUN_CLEANER_CLEAN.invoke(cleaner, (Object[]) null);
}
catch (Throwable t)
{
logger.trace("Exception occurred attempting to clean up Sun specific DirectByteBuffer.", t);
}
}
}
/**
*特定于Sun的机制,用于清理与直接字节缓冲区关联的资源。
*/
@抑制警告(“未选中”)
私有静态最终类cleanClazz=lookupclassquiely(“sun.misc.Cleaner”);
cleanerClean=cleanClazz.getMethod(“clean”,(Class[])null;
}
}
捕获(可丢弃的t)
{
t、 printStackTrace();
}
SUN\u BUFFER\u CLEANER=缓冲清洁剂;
阳光清洁剂=清洁剂清洁剂;
}
公共静态无效释放DirectByTeBuffer(ByteBuffer缓冲区)
{
if(SUN_DIRECT_BUFFER!=null&&SUN_DIRECT_BUFFER.isAssignableFrom(BUFFER.getClass()))
{
尝试
{
objectcleaner=SUN\u BUFFER\u cleaner.invoke(BUFFER,(Object[])null;
SUN\u CLEANER\u CLEAN.invoke(CLEANER,(Object[])null;
}
捕获(可丢弃的t)
{
trace(“试图清除特定于Sun的DirectByteBuffer时发生异常。”,t);
}
}
}
你确定没有其他东西保存对这个ByteBuffer的引用吗?是的,我非常确定,我实例化了方法级类,这些类保存的缓冲区在方法完成调用后立即超出范围。相关:我猜System.gc在任何情况下都不会有帮助。我希望没有足够缓冲区内存的“事件”会触发GC,试图释放旧的缓冲区。只有当缓冲区内存不足时才会引发OOM异常。。。在GC之后。很抱歉,显示GNU类路径的源代码不是一个好主意,因为每个JVM都有不同的直接字节缓冲区实现,OpenJDK比GNU类路径使用得更多。看看我的代码,你会看到这些区别:可以将ByteBuffer强制转换为DirectBuffer,然后调用.cleaner().clean()不可以。由于DirectByteBuffer是java.nio包中的包级别类,因此您的客户端将看不到它。这是(尝试)取消分配直接字节缓冲区的正确方法。值得注意的是,这种方法可能会在JRE版本之间中断(尽管不太可能)。Oracle不保证内部结构的API向后兼容性。Luke是对的,您的代码与Java 1.9不同:如果不是,您将不得不使用某种黑客自己完成,正如我在这里解释的那样:IBM题为“感谢内存”的文章或多或少地告诉了我这里指出的一样:不,它没有,它工作得更好一些,因为它支持Java1.7到1.9中的已查看缓冲区,但它与Java断开了关系—Java1.9不再支持它,请参见JEP260。看看我上面的评论。。。
/**
* Sun specific mechanisms to clean up resources associated with direct byte buffers.
*/
@SuppressWarnings("unchecked")
private static final Class<? extends ByteBuffer> SUN_DIRECT_BUFFER = (Class<? extends ByteBuffer>) lookupClassQuietly("sun.nio.ch.DirectBuffer");
private static final Method SUN_BUFFER_CLEANER;
private static final Method SUN_CLEANER_CLEAN;
static
{
Method bufferCleaner = null;
Method cleanerClean = null;
try
{
// operate under the assumption that if the sun direct buffer class exists,
// all of the sun classes exist
if (SUN_DIRECT_BUFFER != null)
{
bufferCleaner = SUN_DIRECT_BUFFER.getMethod("cleaner", (Class[]) null);
Class<?> cleanClazz = lookupClassQuietly("sun.misc.Cleaner");
cleanerClean = cleanClazz.getMethod("clean", (Class[]) null);
}
}
catch (Throwable t)
{
t.printStackTrace();
}
SUN_BUFFER_CLEANER = bufferCleaner;
SUN_CLEANER_CLEAN = cleanerClean;
}
public static void releaseDirectByteBuffer(ByteBuffer buffer)
{
if (SUN_DIRECT_BUFFER != null && SUN_DIRECT_BUFFER.isAssignableFrom(buffer.getClass()))
{
try
{
Object cleaner = SUN_BUFFER_CLEANER.invoke(buffer, (Object[]) null);
SUN_CLEANER_CLEAN.invoke(cleaner, (Object[]) null);
}
catch (Throwable t)
{
logger.trace("Exception occurred attempting to clean up Sun specific DirectByteBuffer.", t);
}
}
}