在新的linux内核中，上下文切换要慢得多_Linux_Multithreading_Linux Kernel_Kernel_Ubuntu 12.04

在新的linux内核中，上下文切换要慢得多

linux multithreading linux-kernel kernel

在新的linux内核中，上下文切换要慢得多,linux,multithreading,linux-kernel,kernel,ubuntu-12.04,Linux,Multithreading,Linux Kernel,Kernel,Ubuntu 12.04,我们希望将服务器上的操作系统从Ubuntu 10.04 LTS升级到Ubuntu 12.04 LTS。不幸的是，从2.6内核到3.2内核，运行已成为可运行线程的延迟似乎显著增加。事实上，我们得到的延迟数字令人难以置信让我更具体地谈谈测试。我们有一个运行两个线程的程序。第一个线程获取当前时间（使用RDTSC以滴答为单位），然后每秒向条件变量发送一次信号。第二个线程等待条件变量，并在发出信号时将其唤醒。然后获取当前时间（使用RDTSC以滴答为单位）。第二个线程中的时间与第一个线程中的时间之间的差值

我们希望将服务器上的操作系统从Ubuntu 10.04 LTS升级到Ubuntu 12.04 LTS。不幸的是，从2.6内核到3.2内核，运行已成为可运行线程的延迟似乎显著增加。事实上，我们得到的延迟数字令人难以置信

让我更具体地谈谈测试。我们有一个运行两个线程的程序。第一个线程获取当前时间（使用RDTSC以滴答为单位），然后每秒向条件变量发送一次信号。第二个线程等待条件变量，并在发出信号时将其唤醒。然后获取当前时间（使用RDTSC以滴答为单位）。第二个线程中的时间与第一个线程中的时间之间的差值将计算并显示在控制台上。在此之后，第二个线程再次等待条件变量。大约一秒钟后，第一个线程将再次发出信号

因此，简而言之，我们通过每秒一次的条件变量延迟测量来实现线程到线程的通信

在内核2.6.32中，这个延迟大约为2.8-3.5us，这是合理的。在内核3.2.0中，这个延迟已经增加到大约40-100us。我已经排除了两台主机在硬件上的任何差异。它们运行在相同的硬件上（双插槽X5687{Westmile EP}处理器运行在3.6 GHz，超读、speedstep和所有C状态均关闭）。测试应用程序更改线程的关联性，使其在同一套接字的独立物理内核上运行（即，第一个线程在内核0上运行，第二个线程在内核1上运行），因此内核上的线程不会反弹，套接字之间也不会反弹/通信

这两台主机之间的唯一区别在于，一台运行的是内核为2.6.32-28的Ubuntu 10.04 LTS（快速上下文切换框），另一台运行的是内核为3.2.0-23的最新Ubuntu 12.04 LTS（慢速上下文切换框）。所有BIOS设置和硬件都相同

内核中是否有任何变化可以解释线程调度运行所需时间的这种荒谬的减慢

更新： 如果你想在你的主机和linux版本上运行这个测试，我已经准备好了供你阅读。编译时使用：

g++ -O3 -o test_latency test_latency.cpp -lpthread

运行（假设您至少有一个双核机箱）：

更新2：

在对内核参数、内核更改帖子和个人研究进行了大量搜索之后，我找到了问题所在，并发布了解决方案作为对这个问题的回答。

可能速度较慢的是futex，条件变量的构建块。这将有助于了解：

strace -r ./test_latency 0 1 &> test_latency_strace & sleep 8 && killall test_latency

然后

它将显示感兴趣的系统调用所花费的微秒数，按时间排序

关于内核2.6.32

$ for i in futex nanosleep rt_sig;do echo $i;grep $i test_latency_strace | sort -rn;done
futex
 1.000140 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000129 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000124 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000119 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000106 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000103 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000102 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 0.000125 futex(0x7f98ce4c0b88, FUTEX_WAKE_PRIVATE, 2147483647) = 0
 0.000042 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000038 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000037 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000030 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000029 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 0
 0.000028 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000027 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000018 futex(0x7fff82f0ec3c, FUTEX_WAKE_PRIVATE, 1) = 0
nanosleep
 0.000027 nanosleep({1, 0}, {1, 0}) = 0
 0.000019 nanosleep({1, 0}, {1, 0}) = 0
 0.000019 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, 0x7fff82f0eb40) = ? ERESTART_RESTARTBLOCK (To be restarted)
 0.000017 nanosleep({1, 0}, {1, 0}) = 0
rt_sig
 0.000045 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000040 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000038 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000034 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000033 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000032 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000032 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000031 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000031 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000028 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000028 rt_sigaction(SIGRT_1, {0x37f8c052b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x37f8c0e4c0}, NULL, 8) = 0
 0.000027 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000027 rt_sigaction(SIGRTMIN, {0x37f8c05370, [], SA_RESTORER|SA_SIGINFO, 0x37f8c0e4c0}, NULL, 8) = 0
 0.000027 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000025 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000025 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000023 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000023 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000022 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
 0.000022 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000021 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000021 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000019 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    29085 us; per iteration:   29 ns / 9.4e-05 context switches.
c.v. ping-pong test   elapsed:  4771993 us; per iteration: 4771 ns / 4.03 context switches.
signal ping-pong test elapsed:  8685423 us; per iteration: 8685 ns / 4.05 context switches.

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    29629 us; per iteration:   29 ns / 0.000418 context switches.
c.v. ping-pong test   elapsed:  6225637 us; per iteration: 6225 ns / 4.1 context switches.
signal ping-pong test elapsed:  5602248 us; per iteration: 5602 ns / 4.09 context switches.
$ chrt -f 1 ./condvar-perf 1000000
NPTL
mutex                 elapsed:    29049 us; per iteration:   29 ns / 0.000407 context switches.
c.v. ping-pong test   elapsed: 16131360 us; per iteration: 16131 ns / 4.29 context switches.
signal ping-pong test elapsed: 11817819 us; per iteration: 11817 ns / 4.16 context switches.
$

关于内核3.1.9

$ for i in futex nanosleep rt_sig;do echo $i;grep $i test_latency_strace | sort -rn;done
futex
 1.000129 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000126 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000122 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000115 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000114 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000112 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000109 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 0.000139 futex(0x3f8b8f2fb0, FUTEX_WAKE_PRIVATE, 2147483647) = 0
 0.000043 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000041 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000037 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000036 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000034 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000034 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
nanosleep
 0.000025 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000022 nanosleep({1, 0}, {0, 3925413}) = ? ERESTART_RESTARTBLOCK (Interrupted by signal)
 0.000021 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
rt_sig
 0.000045 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000044 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000043 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000040 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000038 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000037 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000036 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000036 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000035 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000034 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000031 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigaction(SIGRT_1, {0x3f892067b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x3f8920f500}, NULL, 8) = 0
 0.000026 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000026 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000025 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000024 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000023 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
 0.000023 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000022 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000019 rt_sigaction(SIGRTMIN, {0x3f89206720, [], SA_RESTORER|SA_SIGINFO, 0x3f8920f500}, NULL, 8) = 0

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    26811 us; per iteration:   26 ns / 8e-06 context switches.
c.v. ping-pong test   elapsed: 10930794 us; per iteration: 10930 ns / 4.01 context switches.
signal ping-pong test elapsed: 10949670 us; per iteration: 10949 ns / 4.01 context switches.

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    26830 us; per iteration:   26 ns / 5.7e-05 context switches.
c.v. ping-pong test   elapsed: 12812788 us; per iteration: 12812 ns / 4.01 context switches.
signal ping-pong test elapsed: 13126865 us; per iteration: 13126 ns / 4.01 context switches.
$ chrt -f 1 ./condvar-perf 1000000
NPTL
mutex                 elapsed:    27025 us; per iteration:   27 ns / 3.7e-05 context switches.
c.v. ping-pong test   elapsed:  5099885 us; per iteration: 5099 ns / 4 context switches.
signal ping-pong test elapsed:  5508227 us; per iteration: 5508 ns / 4 context switches.
$

我发现它包含一个“乒乓球”性能测试，比较

单线程libpthread互斥体

libpthread条件变量

普通旧Unix信号

我必须补充一点

#include <stdint.h>

关于内核2.6.32

$ for i in futex nanosleep rt_sig;do echo $i;grep $i test_latency_strace | sort -rn;done
futex
 1.000140 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000129 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000124 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000119 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000106 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000103 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000102 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 0.000125 futex(0x7f98ce4c0b88, FUTEX_WAKE_PRIVATE, 2147483647) = 0
 0.000042 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000038 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000037 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000030 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000029 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 0
 0.000028 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000027 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000018 futex(0x7fff82f0ec3c, FUTEX_WAKE_PRIVATE, 1) = 0
nanosleep
 0.000027 nanosleep({1, 0}, {1, 0}) = 0
 0.000019 nanosleep({1, 0}, {1, 0}) = 0
 0.000019 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, 0x7fff82f0eb40) = ? ERESTART_RESTARTBLOCK (To be restarted)
 0.000017 nanosleep({1, 0}, {1, 0}) = 0
rt_sig
 0.000045 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000040 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000038 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000034 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000033 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000032 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000032 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000031 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000031 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000028 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000028 rt_sigaction(SIGRT_1, {0x37f8c052b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x37f8c0e4c0}, NULL, 8) = 0
 0.000027 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000027 rt_sigaction(SIGRTMIN, {0x37f8c05370, [], SA_RESTORER|SA_SIGINFO, 0x37f8c0e4c0}, NULL, 8) = 0
 0.000027 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000025 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000025 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000023 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000023 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000022 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
 0.000022 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000021 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000021 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000019 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    29085 us; per iteration:   29 ns / 9.4e-05 context switches.
c.v. ping-pong test   elapsed:  4771993 us; per iteration: 4771 ns / 4.03 context switches.
signal ping-pong test elapsed:  8685423 us; per iteration: 8685 ns / 4.05 context switches.

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    29629 us; per iteration:   29 ns / 0.000418 context switches.
c.v. ping-pong test   elapsed:  6225637 us; per iteration: 6225 ns / 4.1 context switches.
signal ping-pong test elapsed:  5602248 us; per iteration: 5602 ns / 4.09 context switches.
$ chrt -f 1 ./condvar-perf 1000000
NPTL
mutex                 elapsed:    29049 us; per iteration:   29 ns / 0.000407 context switches.
c.v. ping-pong test   elapsed: 16131360 us; per iteration: 16131 ns / 4.29 context switches.
signal ping-pong test elapsed: 11817819 us; per iteration: 11817 ns / 4.16 context switches.
$

关于内核3.1.9

$ for i in futex nanosleep rt_sig;do echo $i;grep $i test_latency_strace | sort -rn;done
futex
 1.000129 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000126 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000122 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000115 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000114 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000112 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000109 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 0.000139 futex(0x3f8b8f2fb0, FUTEX_WAKE_PRIVATE, 2147483647) = 0
 0.000043 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000041 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000037 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000036 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000034 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000034 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
nanosleep
 0.000025 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000022 nanosleep({1, 0}, {0, 3925413}) = ? ERESTART_RESTARTBLOCK (Interrupted by signal)
 0.000021 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
rt_sig
 0.000045 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000044 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000043 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000040 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000038 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000037 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000036 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000036 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000035 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000034 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000031 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigaction(SIGRT_1, {0x3f892067b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x3f8920f500}, NULL, 8) = 0
 0.000026 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000026 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000025 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000024 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000023 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
 0.000023 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000022 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000019 rt_sigaction(SIGRTMIN, {0x3f89206720, [], SA_RESTORER|SA_SIGINFO, 0x3f8920f500}, NULL, 8) = 0

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    26811 us; per iteration:   26 ns / 8e-06 context switches.
c.v. ping-pong test   elapsed: 10930794 us; per iteration: 10930 ns / 4.01 context switches.
signal ping-pong test elapsed: 10949670 us; per iteration: 10949 ns / 4.01 context switches.

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    26830 us; per iteration:   26 ns / 5.7e-05 context switches.
c.v. ping-pong test   elapsed: 12812788 us; per iteration: 12812 ns / 4.01 context switches.
signal ping-pong test elapsed: 13126865 us; per iteration: 13126 ns / 4.01 context switches.
$ chrt -f 1 ./condvar-perf 1000000
NPTL
mutex                 elapsed:    27025 us; per iteration:   27 ns / 3.7e-05 context switches.
c.v. ping-pong test   elapsed:  5099885 us; per iteration: 5099 ns / 4 context switches.
signal ping-pong test elapsed:  5508227 us; per iteration: 5508 ns / 4 context switches.
$

我的结论是，内核2.6.32和3.1.9之间的上下文切换确实变慢了，尽管不像您在内核3.2中观察到的那么多。我知道这还不能回答你的问题，我会继续挖掘

编辑：我发现更改进程的实时优先级（两个线程）可以提高3.1.9版本的性能，以匹配2.6.32版本。但是，在2.6.32上设置相同的优先级会使速度减慢。。。算算吧，我会更仔细地调查的

下面是我现在的结果：

关于内核2.6.32

$ for i in futex nanosleep rt_sig;do echo $i;grep $i test_latency_strace | sort -rn;done
futex
 1.000140 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000129 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000124 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000119 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000106 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000103 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000102 futex(0x601ac4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601ac0, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 0.000125 futex(0x7f98ce4c0b88, FUTEX_WAKE_PRIVATE, 2147483647) = 0
 0.000042 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000038 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000037 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000030 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000029 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 0
 0.000028 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000027 futex(0x601b00, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000018 futex(0x7fff82f0ec3c, FUTEX_WAKE_PRIVATE, 1) = 0
nanosleep
 0.000027 nanosleep({1, 0}, {1, 0}) = 0
 0.000019 nanosleep({1, 0}, {1, 0}) = 0
 0.000019 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, {1, 0}) = 0
 0.000018 nanosleep({1, 0}, 0x7fff82f0eb40) = ? ERESTART_RESTARTBLOCK (To be restarted)
 0.000017 nanosleep({1, 0}, {1, 0}) = 0
rt_sig
 0.000045 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000040 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000038 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000034 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000033 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000032 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000032 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000031 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000031 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000028 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000028 rt_sigaction(SIGRT_1, {0x37f8c052b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x37f8c0e4c0}, NULL, 8) = 0
 0.000027 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000027 rt_sigaction(SIGRTMIN, {0x37f8c05370, [], SA_RESTORER|SA_SIGINFO, 0x37f8c0e4c0}, NULL, 8) = 0
 0.000027 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000025 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000025 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000023 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000023 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000022 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
 0.000022 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000021 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000021 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000019 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    29085 us; per iteration:   29 ns / 9.4e-05 context switches.
c.v. ping-pong test   elapsed:  4771993 us; per iteration: 4771 ns / 4.03 context switches.
signal ping-pong test elapsed:  8685423 us; per iteration: 8685 ns / 4.05 context switches.

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    29629 us; per iteration:   29 ns / 0.000418 context switches.
c.v. ping-pong test   elapsed:  6225637 us; per iteration: 6225 ns / 4.1 context switches.
signal ping-pong test elapsed:  5602248 us; per iteration: 5602 ns / 4.09 context switches.
$ chrt -f 1 ./condvar-perf 1000000
NPTL
mutex                 elapsed:    29049 us; per iteration:   29 ns / 0.000407 context switches.
c.v. ping-pong test   elapsed: 16131360 us; per iteration: 16131 ns / 4.29 context switches.
signal ping-pong test elapsed: 11817819 us; per iteration: 11817 ns / 4.16 context switches.
$

关于内核3.1.9

$ for i in futex nanosleep rt_sig;do echo $i;grep $i test_latency_strace | sort -rn;done
futex
 1.000129 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000126 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000122 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000115 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000114 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000112 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 1.000109 futex(0x601764, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x601760, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
 0.000139 futex(0x3f8b8f2fb0, FUTEX_WAKE_PRIVATE, 2147483647) = 0
 0.000043 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000041 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000037 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000036 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000034 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
 0.000034 futex(0x601720, FUTEX_WAKE_PRIVATE, 1) = 1
nanosleep
 0.000025 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000022 nanosleep({1, 0}, {0, 3925413}) = ? ERESTART_RESTARTBLOCK (Interrupted by signal)
 0.000021 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
 0.000017 nanosleep({1, 0}, 0x7fff70091d00) = 0
rt_sig
 0.000045 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000044 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000043 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000040 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000038 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000037 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000036 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000036 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000035 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000035 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000034 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000031 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000027 rt_sigaction(SIGRT_1, {0x3f892067b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x3f8920f500}, NULL, 8) = 0
 0.000026 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000026 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000025 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000024 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000023 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
 0.000023 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
 0.000022 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
 0.000021 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
 0.000019 rt_sigaction(SIGRTMIN, {0x3f89206720, [], SA_RESTORER|SA_SIGINFO, 0x3f8920f500}, NULL, 8) = 0

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    26811 us; per iteration:   26 ns / 8e-06 context switches.
c.v. ping-pong test   elapsed: 10930794 us; per iteration: 10930 ns / 4.01 context switches.
signal ping-pong test elapsed: 10949670 us; per iteration: 10949 ns / 4.01 context switches.

$ ./condvar-perf 1000000
NPTL
mutex                 elapsed:    26830 us; per iteration:   26 ns / 5.7e-05 context switches.
c.v. ping-pong test   elapsed: 12812788 us; per iteration: 12812 ns / 4.01 context switches.
signal ping-pong test elapsed: 13126865 us; per iteration: 13126 ns / 4.01 context switches.
$ chrt -f 1 ./condvar-perf 1000000
NPTL
mutex                 elapsed:    27025 us; per iteration:   27 ns / 3.7e-05 context switches.
c.v. ping-pong test   elapsed:  5099885 us; per iteration: 5099 ns / 4 context switches.
signal ping-pong test elapsed:  5508227 us; per iteration: 5508 ns / 4 context switches.
$

最近的内核中出现的线程唤醒性能问题的解决方案与从

acpi\u idle

切换到

intel\u idle

cpuidle驱动程序有关，后者是旧内核中使用的驱动程序。遗憾的是，

intel\u idle

驱动程序忽略了用户对C状态的BIOS配置，并按照自己的方式运行。换句话说，即使您完全禁用PC（或服务器）BIOS中的所有C状态，此驱动程序仍会在短暂的不活动期间强制它们打开，除非运行全核心消耗合成基准测试（例如，压力），否则几乎总是发生这种情况。您可以在最兼容的硬件上使用精彩的Google监控C状态转换，以及与处理器频率相关的其他有用信息

要查看哪个cpuidle驱动程序当前在您的设置中处于活动状态，只需在

/sys/devices/system/cpu

的

cpuidle

部分中对

current\u驱动程序

文件进行如下分类：

cat /sys/devices/system/cpu/cpuidle/current_driver

如果希望现代Linux操作系统具有尽可能低的上下文切换延迟，请添加以下内核引导参数以禁用所有这些节能功能：

在Ubuntu 12.04上，您可以通过将它们添加到

/etc/DEFAULT/GRUB

中的

GRUB\u CMDLINE\u LINUX\u DEFAULT

条目中，然后运行

update GRUB

。要添加的启动参数包括：

intel_idle.max_cstate=0 processor.max_cstate=0 idle=poll

以下是关于三个启动选项的详细信息：

将

intel_idle.max_cstate

设置为零将使您的cpuidle驱动程序恢复为

acpi_idle

（至少根据选项文档），或者完全禁用它。在“我的盒子”上，它被完全禁用（即，在

/sys/devices/system/cpu/cpuidle

中显示

当前\u驱动程序

文件会产生

无

的输出）。在这种情况下，不需要第二个引导选项，

processor.max_cstate=0

。但是，文档中指出，将

intel\u idle

驱动程序的max\u cstate设置为零应将操作系统还原为

acpi\u idle

驱动程序。因此，为了以防万一，我加入了第二个启动选项

processor.max_cstate

选项将

acpi\u idle

驱动程序的最大C状态设置为零，跃点