Java 泄漏:未调用ByteBuf.release()-我们如何解决此问题?
我们有一个基于netty的网络流量密集型Java应用程序/服务器 旁注:我主要支持这个应用程序,我没有构建它,所以我不完全了解它 我们有时会出现如下所示的错误 以前我们经常在服务器运行3-4天后出现此错误。现在我注意到,即使在重新启动服务器/应用程序10-15分钟后,我们也会遇到这个错误 我不明白这怎么可能。这个错误令人担忧吗?我们如何解决它?我记得过去曾对同一个错误进行过广泛的研究,当时我甚至尝试升级和修补netty,但都无助于完全解决问题 操作系统:LinuxJava 泄漏:未调用ByteBuf.release()-我们如何解决此问题?,java,netty,Java,Netty,我们有一个基于netty的网络流量密集型Java应用程序/服务器 旁注:我主要支持这个应用程序,我没有构建它,所以我不完全了解它 我们有时会出现如下所示的错误 以前我们经常在服务器运行3-4天后出现此错误。现在我注意到,即使在重新启动服务器/应用程序10-15分钟后,我们也会遇到这个错误 我不明白这怎么可能。这个错误令人担忧吗?我们如何解决它?我记得过去曾对同一个错误进行过广泛的研究,当时我甚至尝试升级和修补netty,但都无助于完全解决问题 操作系统:Linux Java版本:1.8 Nett
Java版本:1.8
Netty版本:Netty-all-4.1.30.Final.jar 这是唯一一行特定于应用程序的代码,其他一切都发生在Netty中
com.company.japp.protocol.http.decoders.ConditionalHttpChunkAggregator.channelRead
这是Netty自身的某种缺陷吗?Netty upgrade或任何其他配置调优是否有帮助
[2020-09-04 08:33:53,072]
ERROR
io.netty.util.ResourceLeakDetector
LEAK: ByteBuf.release() was not called before it's garbage-collected.
See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records:
Created at:
io.netty.buffer.AbstractByteBufAllocator.compositeDirectBuffer(AbstractByteBufAllocator.java:221)
io.netty.buffer.AbstractByteBufAllocator.compositeBuffer(AbstractByteBufAllocator.java:199)
io.netty.handler.codec.MessageAggregator.decode(MessageAggregator.java:255)
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
com.company.japp.protocol.http.decoders.ConditionalHttpChunkAggregator.channelRead(ConditionalHttpChunkAggregator.java:112)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:579)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:496)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
以下是
条件HttpChunkAggregator
的代码
package com.company.japp.protocol.http.decoders;
import com.company.japp.IHttpProxyServer;
import io.netty.channel.ChannelDuplexHandler;
import io.netty.channel.ChannelHandler;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.http.HttpHeaders;
import io.netty.handler.codec.http.HttpMessage;
import io.netty.handler.codec.http.HttpObjectAggregator;
import io.netty.handler.codec.http.HttpResponse;
import io.netty.util.internal.logging.InternalLogger;
import io.netty.util.internal.logging.InternalLoggerFactory;
import java.util.HashSet;
@ChannelHandler.Sharable
public class ConditionalHttpChunkAggregator extends HttpObjectAggregator {
private static final InternalLogger logger = InternalLoggerFactory.getInstance(ConditionalHttpChunkAggregator.class);
private volatile boolean sendaschunked;
private volatile int maxContentLength;
private static IHttpProxyServer iHttpProxyServer;
public static void initialize(IHttpProxyServer iHttpProxyServer) {
ConditionalHttpChunkAggregator.iHttpProxyServer = iHttpProxyServer;
}
public ConditionalHttpChunkAggregator(int maxContentLength) {
super(maxContentLength);
this.maxContentLength = maxContentLength;
sendaschunked = false;
}
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg){
if ((msg instanceof HttpResponse)) {
HttpResponse response = (HttpResponse)msg;
if ((msg instanceof HttpMessage)) {
HttpMessage httpmessage= (HttpMessage)msg;
try {
// If the content length exceeds the threshhold, then send it as chunked
// It's too large to process substitutions
Long contentlength =
httpmessage.headers().get(HttpHeaders.Names.CONTENT_LENGTH) != null ?
Long.valueOf(httpmessage.headers().get(HttpHeaders.Names.CONTENT_LENGTH)) : -1;
if (contentlength >= maxContentLength) {
sendaschunked = true;
} else {
// Check content types
HashSet<String> chunkabletypes = iHttpProxyServer.getConfig().getProperty("chunkabletypes");
if (!chunkabletypes.isEmpty() && response.headers().contains(HttpHeaders.Names.CONTENT_TYPE)) {
String contentType = response.headers().get(HttpHeaders.Names.CONTENT_TYPE).toLowerCase().trim();
if (contentType.length()>0) {
sendaschunked = chunkabletypes.contains(contentType);
if (!sendaschunked) {
for (String chunkabletype: chunkabletypes) {
// Begins with
if (contentType.indexOf(chunkabletype)==0) {
sendaschunked = true;
break;
}
}
}
}
}
}
if (sendaschunked) {
ctx.fireChannelRead(msg);
return;
}
}
catch(Exception ex) {
logger.error("error determining chunkable viability", ex);
}
}
}
if (sendaschunked) {
ctx.fireChannelRead(msg);
return;
}
try {
super.channelRead(ctx, msg);
} catch (Exception e) {
logger.error("error determining chunkable viability", e);
e.printStackTrace();
}
}
}
您需要释放您分配的
Bytebuf
。这不是一只讨厌的虫子
ByteBuf位于
com.company.japp.protocol.http.decoders.ConditionalHttpChunkAggregator.channelRead(ConditionalHttpChunkAggregator.java:112)
这是什么意思?我想我在那里没有任何参考资料。我在超类中调用了一个方法。你的代码没有第112行。介意发布完整的代码吗?代码是完整的,我刚刚删除了一些注释,因此行号不匹配。这是第112行super.channelRead(ctx,msg)代码>我的意思是这是我们自己的类ConditionalHttpChunkAggregator中的第112行。但问题可能就在内蒂的内心深处。。。也可能是我们这边的坏习惯,我不知道。我只是支持这个网络应用,我没有构建它。我查看了一些有争议的消息来源,并对其进行了编辑,为问题添加了更多信息。super.channelRead(ctx,msg)
您需要在下一个处理程序中释放ByteBuf。ConditionalHttpChunkAggregator
用@ChannelHandler.Sharable注释,表示将不存在竞争条件,但是在channelRead()
中多次读取和写入字段sendaschunked
。更糟糕的是,它被声明为
volatile,这保证了写操作在线程之间是可见的。所以我认为sendaschunked应该是一个方法变量。
// A streamed message - initialize the cumulative buffer, and wait for incoming chunks.
CompositeByteBuf content = ctx.alloc().compositeBuffer(maxCumulationBufferComponents); // LINE 255
if (m instanceof ByteBufHolder) {
appendPartialContent(content, ((ByteBufHolder) m).content());
}
currentMessage = beginAggregation(m, content);