Python在多线程处理时节省执行时间
在Python2.7中使用多线程和队列时,我遇到了一个问题。我希望有线程的代码要比没有线程的代码花费大约一半的时间,但我认为我做错了什么。我正在使用一种简单的循环技术来处理斐波那契序列,以最好地说明这个问题 这是没有线程和队列的代码。它打印了19.9190001488秒作为执行时间Python在多线程处理时节省执行时间,python,multithreading,python-2.7,time,queue,Python,Multithreading,Python 2.7,Time,Queue,在Python2.7中使用多线程和队列时,我遇到了一个问题。我希望有线程的代码要比没有线程的代码花费大约一半的时间,但我认为我做错了什么。我正在使用一种简单的循环技术来处理斐波那契序列,以最好地说明这个问题 这是没有线程和队列的代码。它打印了19.9190001488秒作为执行时间 import time start_time = time.time() def fibonacci(priority, num): if num == 1 or num == 2: re
import time
start_time = time.time()
def fibonacci(priority, num):
if num == 1 or num == 2:
return 1
a = 1
b = 1
for i in range(num-2):
c = a + b
b = a
a = c
return c
print fibonacci(0, 200000)
print fibonacci(1, 100)
print fibonacci(2, 200000)
print fibonacci(3, 2)
print("%s seconds" % (time.time() - start_time))
import time
start_time = time.time()
from Queue import *
from threading import *
numbers = [200000,100,200000,2]
q = PriorityQueue()
threads = []
def fibonacci(priority, num):
if num == 1 or num == 2:
q.put((priority, 1))
return
a = 1
b = 1
for i in range(num-2):
c = a + b
b = a
a = c
q.put((priority, c))
return
for i in range(4):
priority = i
num = numbers[i]
t = Thread(target = fibonacci, args = (priority, num))
threads.append(t)
#print threads
for t in threads:
t.start()
for t in threads:
t.join()
while not q.empty():
ans = q.get()
q.task_done()
print ans[1]
print("%s seconds" % (time.time() - start_time))
下面是包含线程和队列的代码。它打印了21.726999981秒作为执行时间
import time
start_time = time.time()
def fibonacci(priority, num):
if num == 1 or num == 2:
return 1
a = 1
b = 1
for i in range(num-2):
c = a + b
b = a
a = c
return c
print fibonacci(0, 200000)
print fibonacci(1, 100)
print fibonacci(2, 200000)
print fibonacci(3, 2)
print("%s seconds" % (time.time() - start_time))
import time
start_time = time.time()
from Queue import *
from threading import *
numbers = [200000,100,200000,2]
q = PriorityQueue()
threads = []
def fibonacci(priority, num):
if num == 1 or num == 2:
q.put((priority, 1))
return
a = 1
b = 1
for i in range(num-2):
c = a + b
b = a
a = c
q.put((priority, c))
return
for i in range(4):
priority = i
num = numbers[i]
t = Thread(target = fibonacci, args = (priority, num))
threads.append(t)
#print threads
for t in threads:
t.start()
for t in threads:
t.join()
while not q.empty():
ans = q.get()
q.task_done()
print ans[1]
print("%s seconds" % (time.time() - start_time))
我认为多线程代码所用的时间是没有线程的代码的一半。本质上,我认为所有线程同时工作,因此计算20万斐波那契数的两个线程将同时完成,因此执行速度大约是没有线程的代码的两倍。显然事情不是这样的。我做错什么了吗?我只想同时执行所有线程,按照它们开始的顺序打印,占用时间最长的线程几乎就是执行时间
import time
start_time = time.time()
def fibonacci(priority, num):
if num == 1 or num == 2:
return 1
a = 1
b = 1
for i in range(num-2):
c = a + b
b = a
a = c
return c
print fibonacci(0, 200000)
print fibonacci(1, 100)
print fibonacci(2, 200000)
print fibonacci(3, 2)
print("%s seconds" % (time.time() - start_time))
import time
start_time = time.time()
from Queue import *
from threading import *
numbers = [200000,100,200000,2]
q = PriorityQueue()
threads = []
def fibonacci(priority, num):
if num == 1 or num == 2:
q.put((priority, 1))
return
a = 1
b = 1
for i in range(num-2):
c = a + b
b = a
a = c
q.put((priority, c))
return
for i in range(4):
priority = i
num = numbers[i]
t = Thread(target = fibonacci, args = (priority, num))
threads.append(t)
#print threads
for t in threads:
t.start()
for t in threads:
t.join()
while not q.empty():
ans = q.get()
q.task_done()
print ans[1]
print("%s seconds" % (time.time() - start_time))
编辑:
我更新了代码以使用进程,但现在结果没有打印出来。仅显示0.163000106812秒的执行时间。以下是新代码:
import time
start_time = time.time()
from Queue import *
from multiprocessing import *
numbers = [200000,100,200000,2]
q = PriorityQueue()
processes = []
def fibonacci(priority, num):
if num == 1 or num == 2:
q.put((priority, 1))
return
a = 1
b = 1
for i in range(num-2):
c = a + b
b = a
a = c
q.put((priority, c))
return
for i in range(4):
priority = i
num = numbers[i]
p = Process(target = fibonacci, args = (priority, num))
processes.append(p)
#print processes
for p in processes:
p.start()
for p in processes:
p.join()
while not q.empty():
ans = q.get()
q.task_done()
print ans[1]
print("%s seconds" % (time.time() - start_time))
您已经遇到了CPython实现的一个基本限制因素,or GIL。这样可以有效地序列化程序,线程将轮流执行。一个线程将拥有GIL,而其他线程将等待GIL释放
一种解决方案是使用单独的流程。每个进程都有自己的GIL,因此可以并行执行。可能最简单的方法是使用Python的模块替换线程模块。您已经在CPython实现的一个基本限制因素或GIL中运行。这样可以有效地序列化程序,线程将轮流执行。一个线程将拥有GIL,而其他线程将等待GIL释放
一种解决方案是使用单独的流程。每个进程都有自己的GIL,因此可以并行执行。最简单的方法可能是使用Python模块代替线程模块。您使用的是哪种操作系统?Windows?你在使用哪个操作系统?窗户?