Python 来自不同虚拟IP的1024个连接后超时
我有一个问题,我已经挣扎了一段时间,但没有解决它 我正在实现一个流量生成器,它有一个客户端和一个服务器端。客户端模拟具有唯一IP地址的设备。这些IP作为虚拟接口添加到客户端,然后模拟设备将绑定到该客户端。然后,这些设备将连接到服务器端,并生成与服务器端的通信量 问题是我只能连接1023台设备,然后以下设备在连接时超时。我已经在服务器端签入wireshark,我可以看到连接的SYN,但应用程序中从未收到它。当我重新使用IPs时,使用的IPs少于2014个,我可以建立任意数量的连接 我创建了一个python程序,该程序更易于运行,并且存在相同的问题: client.pyPython 来自不同虚拟IP的1024个连接后超时,python,linux,sockets,tcp,network-programming,Python,Linux,Sockets,Tcp,Network Programming,我有一个问题,我已经挣扎了一段时间,但没有解决它 我正在实现一个流量生成器,它有一个客户端和一个服务器端。客户端模拟具有唯一IP地址的设备。这些IP作为虚拟接口添加到客户端,然后模拟设备将绑定到该客户端。然后,这些设备将连接到服务器端,并生成与服务器端的通信量 问题是我只能连接1023台设备,然后以下设备在连接时超时。我已经在服务器端签入wireshark,我可以看到连接的SYN,但应用程序中从未收到它。当我重新使用IPs时,使用的IPs少于2014个,我可以建立任意数量的连接 我创建了一个py
import socket
import thread
import time
from subprocess import call
TCP_IP = '192.168.169.218'
TCP_PORT = 9999
BUFFER_SIZE = 1024
MESSAGE = "-" * 1000
if __name__=='__main__':
sockets = []
for i in range(0, 10020):
ip = "13.1."+ str(((i/254)%254) + 1) + "." + str((i % 254) + 1)
cmd = "ip addr add " + ip + " dev eth1;"
call(cmd, shell=True)
s = socket.create_connection((TCP_IP, TCP_PORT), 10, (ip, 0))
sockets.append(s)
while 1:
for s in sockets:
s.send(MESSAGE)
data = s.recv(BUFFER_SIZE)
for s in sockets:
s.close()
from socket import *
import thread
BUFF = 1024
HOST = '192.168.169.218'
PORT = 9999
def handler(clientsock,addr):
while 1:
data = clientsock.recv(BUFF)
if not data: break
clientsock.send(data)
# type 'close' on client console to close connection from the server side
if "close" == data.rstrip(): break
clientsock.close()
print addr, "- closed connection" #log on console
if __name__=='__main__':
count = 0
ADDR = (HOST, PORT)
serversock = socket(AF_INET, SOCK_STREAM)
serversock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
serversock.bind(ADDR)
serversock.listen(5)
while 1:
print 'waiting for connection... listening on port', PORT
clientsock, addr = serversock.accept()
count += 1
print count
thread.start_new_thread(handler, (clientsock, addr))
server.py
import socket
import thread
import time
from subprocess import call
TCP_IP = '192.168.169.218'
TCP_PORT = 9999
BUFFER_SIZE = 1024
MESSAGE = "-" * 1000
if __name__=='__main__':
sockets = []
for i in range(0, 10020):
ip = "13.1."+ str(((i/254)%254) + 1) + "." + str((i % 254) + 1)
cmd = "ip addr add " + ip + " dev eth1;"
call(cmd, shell=True)
s = socket.create_connection((TCP_IP, TCP_PORT), 10, (ip, 0))
sockets.append(s)
while 1:
for s in sockets:
s.send(MESSAGE)
data = s.recv(BUFFER_SIZE)
for s in sockets:
s.close()
from socket import *
import thread
BUFF = 1024
HOST = '192.168.169.218'
PORT = 9999
def handler(clientsock,addr):
while 1:
data = clientsock.recv(BUFF)
if not data: break
clientsock.send(data)
# type 'close' on client console to close connection from the server side
if "close" == data.rstrip(): break
clientsock.close()
print addr, "- closed connection" #log on console
if __name__=='__main__':
count = 0
ADDR = (HOST, PORT)
serversock = socket(AF_INET, SOCK_STREAM)
serversock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
serversock.bind(ADDR)
serversock.listen(5)
while 1:
print 'waiting for connection... listening on port', PORT
clientsock, addr = serversock.accept()
count += 1
print count
thread.start_new_thread(handler, (clientsock, addr))
我运行的是CentOS 7.1 64位,我测试的Python版本是2.7.5
到目前为止我所做的:-将打开的文件数限制(nofile)增加到1040000
-将net.core.somaxconn增加到65535
-将net.ipv4.tcp_max_syn_积压和net.core.netdev_max_积压增加到30000
-增加了核心和TCP缓冲区
-禁用防火墙并清除所有iptables规则
我测试了让python客户端在每次连接后休眠一秒钟,然后就没有问题了,所以我猜是有一些洪水防护措施。有人有什么想法吗?有趣的问题,所以我用我的虚拟机做了一个测试。我发现,您的ARP邻居条目达到了极限
# sysctl -a|grep net.ipv4.neigh.default.gc_thresh
net.ipv4.neigh.default.gc_thresh1 = 128
net.ipv4.neigh.default.gc_thresh2 = 512
net.ipv4.neigh.default.gc_thresh3 = 1024
以上是默认值,当您的1024次连接填满此表时,垃圾收集器将再次开始运行arp,这会减慢速度并导致客户端超时
我可以如下设置这些值
net.ipv4.neigh.default.gc_thresh1 = 16384
net.ipv4.neigh.default.gc_thresh2 = 32768
net.ipv4.neigh.default.gc_thresh3 = 65536
瞧!!不再有1024个限制..HTH这听起来像是tcp/ip堆栈的某种队列限制。也许这个链接可以给我们一些启示,但除此之外,我真的很无知。