Performance golang profile runtime.usleep的70%
我正在用golang编写一个小工具,它读取CSV文件并将每一行传递给一个通道。每行都有一个接收器go例行程序。 不知何故,这是非常缓慢的-使用评测,我发现该程序在runtime.usleep上占68%。我的代码中没有“睡眠” 有人能告诉我这是从哪里来的,它意味着什么,以及如何摆脱它吗Performance golang profile runtime.usleep的70%,performance,csv,go,profiling,usleep,Performance,Csv,Go,Profiling,Usleep,我正在用golang编写一个小工具,它读取CSV文件并将每一行传递给一个通道。每行都有一个接收器go例行程序。 不知何故,这是非常缓慢的-使用评测,我发现该程序在runtime.usleep上占68%。我的代码中没有“睡眠” 有人能告诉我这是从哪里来的,它意味着什么,以及如何摆脱它吗 (pprof) top10 28.98s of 30.12s total (96.22%) Dropped 64 nodes (cum <= 0.15s) Showing top 10 nodes out o
(pprof) top10
28.98s of 30.12s total (96.22%)
Dropped 64 nodes (cum <= 0.15s)
Showing top 10 nodes out of 44 (cum >= 19.38s)
flat flat% sum% cum cum%
20.53s 68.16% 68.16% 20.53s 68.16% runtime.usleep
6.86s 22.78% 90.94% 6.86s 22.78% syscall.Syscall
0.30s 1% 91.93% 0.30s 1% runtime/internal/atomic.Cas64
0.28s 0.93% 92.86% 0.28s 0.93% runtime.mach_semaphore_signal
0.26s 0.86% 93.73% 19.15s 63.58% runtime.findrunnable
0.24s 0.8% 94.52% 0.24s 0.8% runtime.mach_semaphore_wait
0.21s 0.7% 95.22% 17.11s 56.81% runtime.runqgrab
0.13s 0.43% 95.65% 0.23s 0.76% runtime.runqempty
0.12s 0.4% 96.05% 4.17s 13.84% runtime.lock
0.05s 0.17% 96.22% 19.38s 64.34% runtime.schedule
(pprof)
(pprof)top10
28.98秒,共30.12秒(96.22%)
丢弃64个节点(cum=19.38s)
单位百分比总和百分比总和百分比
20.53S68.16%68.16%20.53S68.16%runtime.usleep
6.86s 22.78%90.94%6.86s 22.78%syscall.syscall
0.30s1%91.93%0.30s1%runtime/internal/atomic.Cas64
0.28s 0.93%92.86%0.28s 0.93%runtime.mach\u信号量
0.26s 0.86%93.73%19.15s 63.58%runtime.findrunnable
0.24s0.8%94.52%0.24s0.8%runtime.mach\u信号量\u等待
0.21s 0.7%95.22%17.11s 56.81%runtime.runqgrab
0.13s 0.43%95.65%0.23s 0.76%runtime.runqempty
0.12s 0.4%96.05%4.17s 13.84%runtime.lock
0.05s 0.17%96.22%19.38s 64.34%runtime.schedule
(警察公共关系科)
相关源代码:
[...snip from main...]
var wg_p sync.WaitGroup
var wg_w sync.WaitGroup
for i := 0; i < workercount; i++ {
wg_p.Add(1)
go func(i int) {
processor(i, chn_read, chn_write)
wg_p.Done()
}(i)
}
wg_w.Add(1)
go func() {
writer(chn_write)
wg_w.Done()
}()
// read headers/1st line from csv
headers, _ := csvf.Read()
g_fieldnames = headers
chn_write <- g_fieldnames
// read 'n feed linewise
for {
chunk, _ := csvf.Read()
if chunk == nil {
close(chn_read)
break
}
chn_read <- chunk
}
[...snip end from main...]
[…从主…剪断]
var wg_p sync.WaitGroup
var wg_w sync.WaitGroup
对于i:=0;我<工人计数;i++{
工作组附录(1)
go func(i int){
处理器(i,chn_读取,chn_写入)
工作小组已完成()
}(一)
}
工作组增编(1)
go func(){
作家(中国作家)
wg_w.Done()
}()
//从csv读取标题/第1行
标题,vf:=csvf.Read()
g_fieldnames=标题
你有太多的争论。您无法并行读取文件,因此goroutines可能没有并发运行。如果没有完整的示例,我们无法确定发生了什么。听起来您的程序是I/O绑定的,这意味着它在等待下一个I/O事件时休眠。你不睡觉,但是I/O例程可以。我已经修改了代码,使其没有go例程。usleep现在已经消失,性能仍然很差:读取一个300MB的csv文件需要8-9秒。。。这仅为33mb/s-从nvme/ssd读取。。。
func processor(num_p int, chn_read chan []string, chn_write chan []string) {
for msg := range chn_read {
// snipped working on 'msg'
chn_write <- msg
}
}
func writer(chn_write chan []string) {
fout, _ := os.Create("./out.csv")
defer fout.Close()
w := csv.NewWriter(fout)
counter := 0
for msg := range chn_write {
w.Write(msg)
counter++
}
w.Flush()
}