Http golang:防止对等错误导致连接重置的策略
该程序同时生成了许多goroutine(getStock),我相信这会导致远程服务器立即断开连接。我不想创建DOS,但我仍然希望在不出现“连接重置”错误的情况下积极获取数据 哪些策略最多只能有N个(例如20个)同时连接?golang Http客户端中是否有用于GET请求的内置队列?我还在学习go,如果能了解这种类型的代码是否有更好的设计模式,那就太好了 输出Http golang:防止对等错误导致连接重置的策略,http,go,http-get,task-queue,flooding,Http,Go,Http Get,Task Queue,Flooding,该程序同时生成了许多goroutine(getStock),我相信这会导致远程服务器立即断开连接。我不想创建DOS,但我仍然希望在不出现“连接重置”错误的情况下积极获取数据 哪些策略最多只能有N个(例如20个)同时连接?golang Http客户端中是否有用于GET请求的内置队列?我还在学习go,如果能了解这种类型的代码是否有更好的设计模式,那就太好了 输出 $ go run s1w.go sl(size): 1280 body: "AAPL",17.92 body: "GOOG",32.13
$ go run s1w.go
sl(size): 1280
body: "AAPL",17.92
body: "GOOG",32.13
body: "FB",42.02
body: "AMZN",195.83
body: "GOOG",32.13
body: "AMZN",195.83
body: "GOOG",32.13
body: "FB",42.02
body: "AAPL",17.92
2017/07/26 00:01:23 NFLX: Get http://goanuj.freeshell.org/go/NFLX.txt: read tcp 192.168.86.28:56674->205.166.94.30:80: read: connection reset by peer
2017/07/26 00:01:23 AAPL: Get http://goanuj.freeshell.org/go/AAPL.txt: read tcp 192.168.86.28:56574->205.166.94.30:80: read: connection reset by peer
2017/07/26 00:01:23 NFLX: Get http://goanuj.freeshell.org/go/NFLX.txt: read tcp 192.168.86.28:56760->205.166.94.30:80: read: connection reset by peer
2017/07/26 00:01:23 FB: Get http://goanuj.freeshell.org/go/FB.txt: read tcp 192.168.86.28:56688->205.166.94.30:80: read: connection reset by peer
2017/07/26 00:01:23 AMZN: Get http://goanuj.freeshell.org/go/AMZN.txt: read tcp 192.168.86.28:56689->205.166.94.30:80: read: connection reset by peer
2017/07/26 00:01:23 AAPL: Get http://goanuj.freeshell.org/go/AAPL.txt: read tcp 192.168.86.28:56702->205.166.94.30:80: read: connection reset by peer
s1.开始
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"time"
)
// https://www.youtube.com/watch?v=f6kdp27TYZs (15m)
// Generator: function that returns a channel
func getStocks(sl []string) <-chan string {
c := make(chan string)
for _, s := range sl {
go getStock(s, c)
}
return c
}
func getStock(s string, c chan string) {
resp, err := http.Get("http://goanuj.freeshell.org/go/" + s + ".txt")
if err != nil {
log.Printf(s + ": " + err.Error())
c <- err.Error() // channel send
return
}
body, _ := ioutil.ReadAll(resp.Body)
resp.Body.Close() // close ASAP to prevent too many open file desriptors
val := string(body)
//fmt.Printf("body: %s", val)
c <- val // channel send
return
}
func main() {
start := time.Now()
var sl = []string{"AAPL", "AMZN", "GOOG", "FB", "NFLX"}
// creates slice of 1280 elements
for i := 0; i < 8; i++ {
sl = append(sl, sl...)
}
fmt.Printf("sl(size): %d\n", len(sl))
// get channel that returns only strings
c := getStocks(sl)
for i := 0; i < len(sl); i++ {
fmt.Printf("%s", <-c) // channel recv
}
fmt.Printf("main: %.2fs elapsed.\n", time.Since(start).Seconds())
}
主程序包
进口(
“fmt”
“io/ioutil”
“日志”
“net/http”
“时间”
)
// https://www.youtube.com/watch?v=f6kdp27TYZs (15米)
//生成器:返回通道的函数
func getStocks(sl[]string)不要为每个请求旋转新的goroutine,而是在程序启动时创建一个固定池,并通过共享通道传递订单。每个订单都是一个结构,对应于当前传递给getStock
的参数。如果你需要杀死游泳池,事情会变得更复杂,但这仍然没有那么难
基本上,您的新处理程序将是一个循环,从所有处理程序共享的通道读取订单,执行它,然后在订单的响应通道上发送结果。您需要使用缓冲通道来限制并行操作的数量。在循环中启动一个新的goroutine之前,您需要发送到这个通道,并在调用完成后从中接收,这样它将释放一个位置,新的请求将能够启动。查看您修改的代码清除池很简单:关闭您将作业传递到的频道。您是否有任何示例代码或链接来演示如何执行此操作?请查看它,它以比您需要的更详细的方式解释了一切:)