为什么Python对于简单的for循环如此缓慢？
我们正在用Python实现一些为什么Python对于简单的for循环如此缓慢？,python,performance,jit,Python,Performance,Jit,我们正在用Python实现一些kNN和SVD。其他人选择了Java。我们的执行时间非常不同。我用cProfile查看我犯错误的地方，但实际上一切都很正常。是的，我也使用numpy。但我想问一个简单的问题 total = 0.0 for i in range(9999): # xrange is slower according for j in range(1, 9999): #to my test but more memory-friendly.
kNNSVDnumpytotal = 0.0
for i in range(9999): # xrange is slower according
for j in range(1, 9999): #to my test but more memory-friendly.
total += (i / j)
print total
Problem= (1+2+...+(num-1)) * (1/1+1/2+...+1/(num-1))
1+2+...+(num-1)=np.sum(np.arange(1,num))=num*(num-1)/2
1/1+1/2+...+1/(num-1)=np.true_divide (1,y)=np.reciprocal(y.astype(np.float64))
Psyco另一个问题是，如果我正确地使用它，我能用
numpyscipy- 较小的代码库
- 减少冗余（更干燥）
- 清洁代码
但是，如果您使用C语言编写的库（来自python），性能可能会更好（比较：
picklecpicklenumpyi=np.arange（9999，dtype=np.float）In [30]: %timeit f1(9999)
1 loops, best of 3: 27.2 s per loop
In [31]: %timeit f2(9999)
1 loops, best of 3: 1.46 s per loop
In [32]: %timeit f3(9999)
1 loops, best of 3: 915 ms per loop
您会发现列表理解或生成器表达式要快得多。例如：
import numpy as np
def f1(num):
total = 0.0
for i in range(num):
for j in range(1, num):
total += (float(i) / j)
return total
def f2(num):
i = np.arange(num, dtype=np.float)
j = np.arange(1, num, dtype=np.float)
return np.divide.outer(i, j).sum()
def f3(num):
"""Less memory-hungry (and faster) version of f2."""
total = 0.0
j = np.arange(1, num, dtype=np.float)
for i in xrange(num):
total += (i / j).sum()
return total
total = sum(i / j for j in xrange(1, 9999) for i in xrange(9999))
total00.0在我的机器上，Psyco实际上会将生成器表达式的速度降低到与原始循环相同的速度（它根本不会加速）。为什么在这个示例循环中Java比Python快？ 新手解释：想象一个像货运列车一样的程序，它在前进时铺设自己的列车轨道。火车开动前必须铺设轨道。爪哇货运列车可以在列车前方发送数千个轨道层，所有轨道层都可以提前数英里平行铺设，wheras python一次只能发送一名工人，并且只能在列车前方10英尺铺设轨道 Java具有强大的类型，这使编译器能够使用JIT特性：（），这使CPU能够在将来需要指令之前获取内存并并行执行指令。Java可以在某种程度上并行运行for循环中的指令。Python没有具体的类型，因此必须在每一条指令中确定要完成的工作的性质。这会导致整个计算机停止并等待所有变量中的所有内存被重新扫描。python中的循环是多项式的
O（n^2）rangexrangerangetotal = sum(i / j for j in xrange(1, 9999) for i in xrange(9999))
if __name__ =='__main__':
total = 0.0
i=1
while i<=9999:
j=1
while j<=9999:
total=1
j+=1
i+=1
print total
public class Main{
public static void main(String args[]){
float total = 0f;
long start_time = System.nanoTime();
int i=1;
while (i<=9999){
int j=1;
while(j<=9999){
total+=1;
j+=1;
}
i+=1;
}
long end_time = System.nanoTime();
System.out.println("total: " + total);
System.out.println("total milliseconds: " +
(end_time - start_time)/1000000);
}
}
from __future__ import division
cdef double total = 0.00
cdef int i, j
for i in range(9999):
for j in range(1, 10000+i):
total += (i / j)
from time import time
t = time()
print("total = %d" % total)
print("time = %f[s]" % (time() - t))
$ cython loops.pyx
$ gcc -I/usr/include/python2.7 -shared -pthread -fPIC -fwrapv -Wall -fno-strict-aliasing -O3 -o loops.so loops.c
$ python -c "import loops"
total = 514219068
time = 0.000047[s]
%timeit f3(9999)
704 ms ± 2.33 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
def f4(num):
x=np.ones(num-1)
y=np.arange(1,num)
return np.sum(np.true_divide(x,y))*np.sum(y)
155 µs ± 284 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
Problem= (1+2+...+(num-1)) * (1/1+1/2+...+1/(num-1))
1+2+...+(num-1)=np.sum(np.arange(1,num))=num*(num-1)/2
1/1+1/2+...+1/(num-1)=np.true_divide (1,y)=np.reciprocal(y.astype(np.float64))
def f5(num):
return np.sum(np.reciprocal(np.arange(1, num).astype(np.float64))) * num*(num-1)/2
%timeit f5(9999)
106 µs ± 615 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
1/1+1/2+...+1/(num-1)=np.log(num-1)+1/(2*num-2)+np.euler_gamma
(n>2)
def f6(num):
return (np.log(num-1)+1/(2*num-2)+np.euler_gamma)* num*(num-1)/2
%timeit f6(9999)
4.82 µs ± 29.1 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
%timeit f3(99999)
56.7 s ± 590 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
%timeit f5(99999)
534 µs ± 86.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit f5(99999999)
1.42 s ± 15.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
9.498947911958**416**e+16
%timeit f6(99999999)
4.88 µs ± 26.7 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
9.498947911958**506**e+16
%timeit f6(9999999999999999999)
17.9 µs ± 921 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
from numba import jit
@jit
def f7(num):
return (np.log(num-1)+1/(2*num-2)+np.euler_gamma)* num*(num-1)/2
# same code with f6(num)
%timeit f6(999999999999999)
5.63 µs ± 29.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
f7(123) # compile f7(num)
%timeit f7(999999999999999)
331 ns ± 1.9 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
%timeit f7(9999)
286 ns ± 3.09 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
import time
N=100
i=0
j=0
StartTime=time.time()
while j<N:
j=j+1
while i<1000000:
a=float(i)/float(j)
i=i+1
EndTime=time.time()
DeltaTime=(EndTime-StartTime) # time in seconds
MIPS=(1/DeltaTime)*N
print("This program estimates the MIPS that your computational unit can perform")
print("------------------------------------------")
print("Execution Time in Seconds=",DeltaTime)
print("MIPS=",MIPS)
print("------------------------------------------")
#include <stdio.h>
#include <time.h>
int main(){
int i,j;
int N=100;
float a, DeltaTime, MIPS;
clock_t StartTime, EndTime;
StartTime=clock();
// This calculates n-time one million divisions
for (j=1;j<N; j++)
{
for(i=1;i<1000000;i=i+1)
{
a=(float)(i)/(float)(j);
}
}
EndTime=clock(); // measures time in microseconds
DeltaTime=(float)(EndTime - StartTime)/1000000;
MIPS=(1/DeltaTime)*N;
printf("------------------------------------------\n");
printf("Execution Time in Seconds=%f \n", DeltaTime);
printf("MIPS=%f \n", MIPS);
printf("------------------------------------------\n");
return 0;
}