Multithreading 阿克卡的线程数量不断增加。有什么不对劲吗?
为什么线程数一直在增加 查看此图像中的右下角 整个流程如下所示:Multithreading 阿克卡的线程数量不断增加。有什么不对劲吗?,multithreading,mongodb,scala,
elasticsearch,akka,Multithreading,Mongodb,Scala,
elasticsearch,Akka,为什么线程数一直在增加 查看此图像中的右下角 整个流程如下所示: Akka HTTP Server API -> on http request, sendMessageTo DataProcessingActor -> sendMessageTo StorageActor -> sendMessageTo DataBaseActor -> sendMessageTo IndexActor 这是Akka
Akka HTTP Server API
-> on http request, sendMessageTo DataProcessingActor
-> sendMessageTo StorageActor
-> sendMessageTo DataBaseActor
-> sendMessageTo IndexActor
这是Akka HTTP API(伪代码)的定义:
以下是参与者定义(伪代码):
运行此安装程序后,在向“input/”HTTP端点发送多个HTTP请求时,会出现以下错误:
for( i <- 0 until 1000000) {
post("input/", someMessage+i)
}
编辑1
下面是正在使用的application.conf
文件:
akka {
loglevel = "INFO"
stdout-loglevel = "INFO"
logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
actor {
default-dispatcher {
throughput = 10
}
}
actor {
provider = "akka.remote.RemoteActorRefProvider"
}
remote {
enabled-transports = ["akka.remote.netty.tcp"]
netty.tcp {
hostname = "127.0.0.1"
port = 2558
}
}
}
我发现ElasticSearch是个问题。我正在使用JavaAPI进行ElasticSearch,这是由于JavaAPI的使用方式导致套接字泄漏。现在按照这里的描述解决 下面是使用JavaAPI的弹性搜索客户端服务
trait ESClient { def getClient(): Client }
case class ElasticSearchService() extends ESClient {
def getClient(): Client = {
val client = new TransportClient().addTransportAddress(
new InetSocketTransportAddress(Config.ES_HOST, Config.ES_PORT)
)
client
}
}
这就是导致泄漏的因素:
class IndexerActor() extends Actor {
val elasticSearchSvc = new ElasticSearchService()
lazy val client = elasticSearchSvc.getClient()
override def preStart = {
// initialize index, and mappings etc.
}
def receive() = {
case message =>
// do indexing here
indexMessage(ES.client, message)
}
}
注意:每次创建actor实例时,都会建立一个新连接
每次调用new ElasticSearchService()
都会创建到ElasticSearch的新连接。我将其移动到一个单独的对象中,如下所示,演员也使用此对象:
object ES {
val elasticSearchSvc = new ElasticSearchService()
lazy val client = elasticSearchSvc.getClient()
}
class IndexerActor() extends Actor {
override def preStart = {
// initialize index, and mappings etc.
}
def receive() = {
case message =>
// do indexing here
indexMessage(ES.client, message)
}
}
可能是mongo驱动程序只使用了一个mongo数据库连接,它位于scala对象
对象数据库{lazy val DB=mongodb.connect()}
中。基本上,在一个演员里面,一个新的联系并没有建立起来。新连接仅在DB对象第一次初始化时建立,并且引用了DB
。它认为它不可能是MongoDB连接。我遗漏了什么吗?我会附加一个调试器/探查器,看看那数千个线程在做什么。我刚刚发现ElasticSearch是个问题。我正在使用JavaAPI进行ElasticSearch,这就是泄漏套接字。让我修复并重新检查。是的,这是ElasticSearch的问题。现在解决了。
trait ESClient { def getClient(): Client }
case class ElasticSearchService() extends ESClient {
def getClient(): Client = {
val client = new TransportClient().addTransportAddress(
new InetSocketTransportAddress(Config.ES_HOST, Config.ES_PORT)
)
client
}
}
class IndexerActor() extends Actor {
val elasticSearchSvc = new ElasticSearchService()
lazy val client = elasticSearchSvc.getClient()
override def preStart = {
// initialize index, and mappings etc.
}
def receive() = {
case message =>
// do indexing here
indexMessage(ES.client, message)
}
}
object ES {
val elasticSearchSvc = new ElasticSearchService()
lazy val client = elasticSearchSvc.getClient()
}
class IndexerActor() extends Actor {
override def preStart = {
// initialize index, and mappings etc.
}
def receive() = {
case message =>
// do indexing here
indexMessage(ES.client, message)
}
}