为什么python哈希环如此缓慢？

嗯，我找到了hash_环。MemcacheRing太慢了，速率测试无法直接击败db访问 同时，我用memcache.Client替换它，速率测试恢复正常 rps在多线程环境下进行了测试

ncalls tottime percall cumtime percall filename:lineno(function) 10000 0.066 0.000 8.297 0.001 Memcached.py:16(__getitem__) 10000 0.262 0.000 6.544 0.001 build/bdist.linux-x86_64/egg/memcache.py:818(_unsafe_get) 1 0.000 0.000 0.000 0.000 /usr/local/Python-2.6.4/lib/python2.6/socket.py:180(__init__) 20000 0.064 0.000 0.064 0.000 build/bdist.linux-x86_64/egg/memcache.py:310(_statlog) 80000 0.118 0.000 0.118 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:148() 10000 0.148 0.000 8.168 0.001 build/bdist.linux-x86_64/egg/memcache.py:812(_get) 20000 0.086 0.000 0.157 0.000 build/bdist.linux-x86_64/egg/memcache.py:1086(_get_socket) 30000 10.539 0.000 10.539 0.000 build/bdist.linux-x86_64/egg/memcache.py:1118(readline) 10000 0.102 0.000 0.102 0.000 build/bdist.linux-x86_64/egg/memcache.py:1142(recv) 20000 0.243 0.000 0.243 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:156(_hash_digest) 20000 0.103 0.000 0.103 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:124(distinct_filter) 10000 0.167 0.000 0.269 0.000 build/bdist.linux-x86_64/egg/memcache.py:965(_recv_value) 1 0.148 0.148 15.701 15.701 Memcached.py:25(foo) 10000 0.236 0.000 5.409 0.001 build/bdist.linux-x86_64/egg/memcache.py:953(_expectvalue) 20002 0.336 0.000 0.336 0.000 :1(settimeout) 10000 0.061 0.000 7.207 0.001 build/bdist.linux-x86_64/egg/memcache.py:541(set) 20000 0.116 0.000 0.606 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:141(gen_key) 20000 0.279 0.000 1.680 0.000 build/bdist.linux-x86_64/egg/hash_ring/memcache_ring.py:20(_get_server) 40000 0.346 0.000 1.236 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:111(iterate_nodes) 10000 0.144 0.000 7.146 0.001 build/bdist.linux-x86_64/egg/memcache.py:771(_set) 20000 0.129 0.000 0.247 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:150(_hash_val) 20000 0.098 0.000 0.433 0.000 build/bdist.linux-x86_64/egg/memcache.py:1111(send_cmd) 10000 0.063 0.000 8.231 0.001 build/bdist.linux-x86_64/egg/memcache.py:857(get) 10000 0.049 0.000 7.256 0.001 Memcached.py:13(__setitem__) 20000 0.144 0.000 5.510 0.000 build/bdist.linux-x86_64/egg/memcache.py:1135(expect) 20000 0.181 0.000 0.787 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:92(get_node_pos) 20000 0.070 0.000 0.070 0.000 build/bdist.linux-x86_64/egg/memcache.py:1070(_check_dead) 10000 0.084 0.000 0.084 0.000 build/bdist.linux-x86_64/egg/memcache.py:725(_val_to_store_info) 20000 0.068 0.000 0.225 0.000 build/bdist.linux-x86_64/egg/memcache.py:1076(connect) 20000 1.036 0.000 1.036 0.000 build/bdist.linux-x86_64/egg/memcache.py:1000(check_key) 10000 0.216 0.000 5.702 0.001 build/bdist.linux-x86_64/egg/memcache.py:777(_unsafe_set) 0 0.000 0.000 profile:0(profiler) memcache.Client rps2500 hash_ring.MemcacheRing rps 600 直接数据库访问：rps960

我使用hotshot跟踪hash_环。MemcacheRing，我发现hash_环肯定有问题，你可以发现hash_环花费了太多的cpu，如下所示 收集的个人资料数据 在单线程环境中收集的热快照配置文件数据

ncalls tottime percall cumtime percall filename:lineno(function) 10000 0.066 0.000 8.297 0.001 Memcached.py:16(__getitem__) 10000 0.262 0.000 6.544 0.001 build/bdist.linux-x86_64/egg/memcache.py:818(_unsafe_get) 1 0.000 0.000 0.000 0.000 /usr/local/Python-2.6.4/lib/python2.6/socket.py:180(__init__) 20000 0.064 0.000 0.064 0.000 build/bdist.linux-x86_64/egg/memcache.py:310(_statlog) 80000 0.118 0.000 0.118 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:148() 10000 0.148 0.000 8.168 0.001 build/bdist.linux-x86_64/egg/memcache.py:812(_get) 20000 0.086 0.000 0.157 0.000 build/bdist.linux-x86_64/egg/memcache.py:1086(_get_socket) 30000 10.539 0.000 10.539 0.000 build/bdist.linux-x86_64/egg/memcache.py:1118(readline) 10000 0.102 0.000 0.102 0.000 build/bdist.linux-x86_64/egg/memcache.py:1142(recv) 20000 0.243 0.000 0.243 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:156(_hash_digest) 20000 0.103 0.000 0.103 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:124(distinct_filter) 10000 0.167 0.000 0.269 0.000 build/bdist.linux-x86_64/egg/memcache.py:965(_recv_value) 1 0.148 0.148 15.701 15.701 Memcached.py:25(foo) 10000 0.236 0.000 5.409 0.001 build/bdist.linux-x86_64/egg/memcache.py:953(_expectvalue) 20002 0.336 0.000 0.336 0.000 :1(settimeout) 10000 0.061 0.000 7.207 0.001 build/bdist.linux-x86_64/egg/memcache.py:541(set) 20000 0.116 0.000 0.606 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:141(gen_key) 20000 0.279 0.000 1.680 0.000 build/bdist.linux-x86_64/egg/hash_ring/memcache_ring.py:20(_get_server) 40000 0.346 0.000 1.236 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:111(iterate_nodes) 10000 0.144 0.000 7.146 0.001 build/bdist.linux-x86_64/egg/memcache.py:771(_set) 20000 0.129 0.000 0.247 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:150(_hash_val) 20000 0.098 0.000 0.433 0.000 build/bdist.linux-x86_64/egg/memcache.py:1111(send_cmd) 10000 0.063 0.000 8.231 0.001 build/bdist.linux-x86_64/egg/memcache.py:857(get) 10000 0.049 0.000 7.256 0.001 Memcached.py:13(__setitem__) 20000 0.144 0.000 5.510 0.000 build/bdist.linux-x86_64/egg/memcache.py:1135(expect) 20000 0.181 0.000 0.787 0.000 build/bdist.linux-x86_64/egg/hash_ring/hash_ring.py:92(get_node_pos) 20000 0.070 0.000 0.070 0.000 build/bdist.linux-x86_64/egg/memcache.py:1070(_check_dead) 10000 0.084 0.000 0.084 0.000 build/bdist.linux-x86_64/egg/memcache.py:725(_val_to_store_info) 20000 0.068 0.000 0.225 0.000 build/bdist.linux-x86_64/egg/memcache.py:1076(connect) 20000 1.036 0.000 1.036 0.000 build/bdist.linux-x86_64/egg/memcache.py:1000(check_key) 10000 0.216 0.000 5.702 0.001 build/bdist.linux-x86_64/egg/memcache.py:777(_unsafe_set) 0 0.000 0.000 profile:0(profiler) 下面是memcache.Client的配置文件数据

ncalls tottime percall cumtime percall filename:lineno(function) 10000 0.064 0.000 8.022 0.001 Memcached.py:16(__getitem__) 10000 0.263 0.000 6.885 0.001 build/bdist.linux-x86_64/egg/memcache.py:818(_unsafe_get) 1 0.000 0.000 0.000 0.000 /usr/local/Python-2.6.4/lib/python2.6/socket.py:180(__init__) 20000 0.069 0.000 0.069 0.000 build/bdist.linux-x86_64/egg/memcache.py:310(_statlog) 10000 0.127 0.000 7.897 0.001 build/bdist.linux-x86_64/egg/memcache.py:812(_get) 20000 0.091 0.000 0.164 0.000 build/bdist.linux-x86_64/egg/memcache.py:1086(_get_socket) 30000 11.074 0.000 11.074 0.000 build/bdist.linux-x86_64/egg/memcache.py:1118(readline) 10000 0.098 0.000 0.098 0.000 build/bdist.linux-x86_64/egg/memcache.py:1142(recv) 20000 0.259 0.000 0.568 0.000 build/bdist.linux-x86_64/egg/memcache.py:329(_get_server) 1 0.149 0.149 15.036 15.036 Memcached.py:25(foo) 10000 0.236 0.000 5.645 0.001 build/bdist.linux-x86_64/egg/memcache.py:953(_expectvalue) 20002 0.351 0.000 0.351 0.000 :1(settimeout) 10000 0.064 0.000 6.814 0.001 build/bdist.linux-x86_64/egg/memcache.py:541(set) 10000 0.124 0.000 6.751 0.001 build/bdist.linux-x86_64/egg/memcache.py:771(_set) 20000 1.039 0.000 1.039 0.000 build/bdist.linux-x86_64/egg/memcache.py:1000(check_key) 20000 0.092 0.000 0.442 0.000 build/bdist.linux-x86_64/egg/memcache.py:1111(send_cmd) 10000 0.061 0.000 7.958 0.001 build/bdist.linux-x86_64/egg/memcache.py:857(get) 10000 0.050 0.000 6.864 0.001 Memcached.py:13(__setitem__) 20000 0.148 0.000 5.812 0.000 build/bdist.linux-x86_64/egg/memcache.py:1135(expect) 20000 0.073 0.000 0.073 0.000 build/bdist.linux-x86_64/egg/memcache.py:1070(_check_dead) 10000 0.087 0.000 0.087 0.000 build/bdist.linux-x86_64/egg/memcache.py:725(_val_to_store_info) 20000 0.072 0.000 0.236 0.000 build/bdist.linux-x86_64/egg/memcache.py:1076(connect) 20000 0.072 0.000 0.072 0.000 build/bdist.linux-x86_64/egg/memcache.py:57(cmemcache_hash) 10000 0.218 0.000 5.905 0.001 build/bdist.linux-x86_64/egg/memcache.py:777(_unsafe_set) 0 0.000 0.000 profile:0(profiler) 10000 0.156 0.000 0.255 0.000 build/bdist.linux-x86_64/egg/memcache.py:965(_recv_value) 和配置文件数据的以下内容

ncalls tottime percall cumtime percall filename:lineno(function) 20000 2.278 0.000 4.260 0.000 build/bdist.linux-x86_64/egg/memcache.py:1000(check_key) 640000 1.624 0.000 1.624 0.000 :0(ord) 20000 0.658 0.000 1.445 0.000 build/bdist.linux-x86_64/egg/memcache.py:329(_get_server) 100010 0.477 0.000 0.477 0.000 :0(len) 30000 0.460 0.000 0.827 0.000 build/bdist.linux-x86_64/egg/memcache.py:1118(readline) 110000 0.414 0.000 0.414 0.000 :0(isinstance) 20000 0.412 0.000 0.412 0.000 :0(sendall) 10000 0.349 0.000 1.974 0.000 build/bdist.linux-x86_64/egg/memcache.py:818(_unsafe_get) 10000 0.344 0.000 0.633 0.000 build/bdist.linux-x86_64/egg/memcache.py:725(_val_to_store_info) 10000 0.344 0.000 2.006 0.000 build/bdist.linux-x86_64/egg/memcache.py:777(_unsafe_set) 10000 0.325 0.000 0.624 0.000 build/bdist.linux-x86_64/egg/memcache.py:965(_recv_value) 50000 0.233 0.000 0.233 0.000 :0(find) 20000 0.200 0.000 0.808 0.000 build/bdist.linux-x86_64/egg/memcache.py:1135(expect) 10000 0.194 0.000 5.292 0.001 build/bdist.linux-x86_64/egg/memcache.py:812(_get) 20000 0.190 0.000 0.357 0.000 build/bdist.linux-x86_64/egg/memcache.py:1076(connect) 10000 0.185 0.000 4.771 0.000 build/bdist.linux-x86_64/egg/memcache.py:771(_set) 10000 0.173 0.000 0.187 0.000 build/bdist.linux-x86_64/egg/memcache.py:1142(recv) 10000 0.163 0.000 0.400 0.000 build/bdist.linux-x86_64/egg/memcache.py:953(_expectvalue) 20000 0.152 0.000 0.564 0.000 :1(sendall) 20000 0.147 0.000 0.711 0.000 build/bdist.linux-x86_64/egg/memcache.py:1111(send_cmd) 20000 0.142 0.000 0.232 0.000 build/bdist.linux-x86_64/egg/memcache.py:57(cmemcache_hash)