Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/performance/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Performance golang profile runtime.usleep的70%_Performance_Csv_Go_Profiling_Usleep - Fatal编程技术网

Performance golang profile runtime.usleep的70%

Performance golang profile runtime.usleep的70%,performance,csv,go,profiling,usleep,Performance,Csv,Go,Profiling,Usleep,我正在用golang编写一个小工具,它读取CSV文件并将每一行传递给一个通道。每行都有一个接收器go例行程序。 不知何故,这是非常缓慢的-使用评测,我发现该程序在runtime.usleep上占68%。我的代码中没有“睡眠” 有人能告诉我这是从哪里来的,它意味着什么,以及如何摆脱它吗 (pprof) top10 28.98s of 30.12s total (96.22%) Dropped 64 nodes (cum <= 0.15s) Showing top 10 nodes out o

我正在用golang编写一个小工具,它读取CSV文件并将每一行传递给一个通道。每行都有一个接收器go例行程序。 不知何故,这是非常缓慢的-使用评测,我发现该程序在runtime.usleep上占68%。我的代码中没有“睡眠”

有人能告诉我这是从哪里来的,它意味着什么,以及如何摆脱它吗

(pprof) top10
28.98s of 30.12s total (96.22%)
Dropped 64 nodes (cum <= 0.15s)
Showing top 10 nodes out of 44 (cum >= 19.38s)
  flat  flat%   sum%        cum   cum%
20.53s 68.16% 68.16%     20.53s 68.16%  runtime.usleep
 6.86s 22.78% 90.94%      6.86s 22.78%  syscall.Syscall
 0.30s     1% 91.93%      0.30s     1%  runtime/internal/atomic.Cas64
 0.28s  0.93% 92.86%      0.28s  0.93%  runtime.mach_semaphore_signal
 0.26s  0.86% 93.73%     19.15s 63.58%  runtime.findrunnable
 0.24s   0.8% 94.52%      0.24s   0.8%  runtime.mach_semaphore_wait
 0.21s   0.7% 95.22%     17.11s 56.81%  runtime.runqgrab
 0.13s  0.43% 95.65%      0.23s  0.76%  runtime.runqempty
 0.12s   0.4% 96.05%      4.17s 13.84%  runtime.lock
 0.05s  0.17% 96.22%     19.38s 64.34%  runtime.schedule
(pprof)
(pprof)top10
28.98秒,共30.12秒(96.22%)
丢弃64个节点(cum=19.38s)
单位百分比总和百分比总和百分比
20.53S68.16%68.16%20.53S68.16%runtime.usleep
6.86s 22.78%90.94%6.86s 22.78%syscall.syscall
0.30s1%91.93%0.30s1%runtime/internal/atomic.Cas64
0.28s 0.93%92.86%0.28s 0.93%runtime.mach\u信号量
0.26s 0.86%93.73%19.15s 63.58%runtime.findrunnable
0.24s0.8%94.52%0.24s0.8%runtime.mach\u信号量\u等待
0.21s 0.7%95.22%17.11s 56.81%runtime.runqgrab
0.13s 0.43%95.65%0.23s 0.76%runtime.runqempty
0.12s 0.4%96.05%4.17s 13.84%runtime.lock
0.05s 0.17%96.22%19.38s 64.34%runtime.schedule
(警察公共关系科)
相关源代码:

[...snip from main...]
var wg_p sync.WaitGroup
var wg_w sync.WaitGroup

for i := 0; i < workercount; i++ {
    wg_p.Add(1)
    go func(i int) {
        processor(i, chn_read, chn_write)
        wg_p.Done()
    }(i)
}

wg_w.Add(1)
go func() {
    writer(chn_write)
    wg_w.Done()
}()

// read headers/1st line from csv
headers, _ := csvf.Read()
g_fieldnames = headers
chn_write <- g_fieldnames

// read 'n feed linewise
for {
    chunk, _ := csvf.Read()
    if chunk == nil {
        close(chn_read)
        break
    }
    chn_read <- chunk
}
[...snip end from main...]
[…从主…剪断]
var wg_p sync.WaitGroup
var wg_w sync.WaitGroup
对于i:=0;我<工人计数;i++{
工作组附录(1)
go func(i int){
处理器(i,chn_读取,chn_写入)
工作小组已完成()
}(一)
}
工作组增编(1)
go func(){
作家(中国作家)
wg_w.Done()
}()
//从csv读取标题/第1行
标题,vf:=csvf.Read()
g_fieldnames=标题

你有太多的争论。您无法并行读取文件,因此goroutines可能没有并发运行。如果没有完整的示例,我们无法确定发生了什么。听起来您的程序是I/O绑定的,这意味着它在等待下一个I/O事件时休眠。你不睡觉,但是I/O例程可以。我已经修改了代码,使其没有go例程。usleep现在已经消失,性能仍然很差:读取一个300MB的csv文件需要8-9秒。。。这仅为33mb/s-从nvme/ssd读取。。。
func processor(num_p int, chn_read chan []string, chn_write chan []string) {
    for msg := range chn_read {
        // snipped working on 'msg'
        chn_write <- msg
    }
}

func writer(chn_write chan []string) {
    fout, _ := os.Create("./out.csv")
    defer fout.Close()
    w := csv.NewWriter(fout)
    counter := 0
    for msg := range chn_write {
        w.Write(msg)
        counter++
    }
    w.Flush()
}