Python 下面的curl和pycurl语句之间有什么区别？

Edit:我在下面添加了

curl

和

pycurl

的详细输出。除了

http200

与

500

的响应之外，我仍然看不到任何区别

我想下载一个Python脚本文件。我在

bash

中编写了

curl

语句来完成这项工作：

t=$(echo `date +%s`' * 1000 - 3600000' | bc)

token=`curl 'https://www.intomics.com/inbio/api/login_guest?ref=&_='"$t"\
    | python -c "import json,sys;obj=json.load(sys.stdin);print(obj['token']);"`

curl 'https://www.intomics.com/inbio/map/api/get_data?file=InBio_Map_core_2016_09_12.tar.gz'\
    -H 'Cookie: access_token='"$token"  -o result.tar.gz

[ bash ] * TCP_NODELAY set
[ bash ] * Connected to www.intomics.com (77.72.50.69) port 443 (#0)
[ bash ] * ALPN, offering http/1.1
[ bash ] * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
[ bash ] * successfully set certificate verify locations:
[ bash ] *   CAfile: /etc/ssl/certs/ca-certificates.crt
[ bash ]   CApath: none
[ bash ] * TLSv1.2 (OUT), TLS header, Certificate Status (22):
[ bash ] } [5 bytes data]
[ bash ] * TLSv1.2 (OUT), TLS handshake, Client hello (1):
[ bash ] } [512 bytes data]
[ bash ] * TLSv1.2 (IN), TLS handshake, Server hello (2):
[ bash ] { [98 bytes data]
[ bash ] * TLSv1.2 (IN), TLS handshake, Certificate (11):
[ bash ] { [2279 bytes data]
[ bash ] * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
[ bash ] { [333 bytes data]
[ bash ] * TLSv1.2 (IN), TLS handshake, Server finished (14):
[ bash ] { [4 bytes data]
[ bash ] * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
[ bash ] } [70 bytes data]
[ bash ] * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
[ bash ] } [1 bytes data]
[ bash ] * TLSv1.2 (OUT), TLS handshake, Finished (20):
[ bash ] } [16 bytes data]
[ bash ] * TLSv1.2 (IN), TLS change cipher, Client hello (1):
[ bash ] { [1 bytes data]
[ bash ] * TLSv1.2 (IN), TLS handshake, Finished (20):
[ bash ] { [16 bytes data]
[ bash ] * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
[ bash ] * ALPN, server did not agree to a protocol
[ bash ] * Server certificate:
[ bash ] *  subject: OU=GT95967783; OU=See www.rapidssl.com/resources/cps (c)15; OU=Domain Control Validated - RapidSSL(R); CN=*.intomics.com
[ bash ] *  start date: Mar 24 02:21:50 2015 GMT
[ bash ] *  expire date: Mar 26 04:01:35 2018 GMT
[ bash ] *  subjectAltName: host "www.intomics.com" matched cert's "*.intomics.com"
[ bash ] *  issuer: C=US; O=GeoTrust Inc.; CN=RapidSSL SHA256 CA - G3
[ bash ] *  SSL certificate verify ok.
[ bash ] } [5 bytes data]
[ bash ] > GET /inbio/map/api/get_data?file=InBio_Map_core_2016_09_12.tar.gz HTTP/1.1
[ bash ] > Host: www.intomics.com
[ bash ] > User-Agent: curl/7.53.1
[ bash ] > Accept: */*
[ bash ] > Cookie: access_token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJ2IjoyLCJsb2dpbiI6bnVsbCwiZXhwIjoxNDkxODIzMjAxLCJpYXQiOjE0OTE4MjMwODEsInNpZCI6MTA4MDcsInNleHAiOjE0OTE4MjMzODEwNzcsImFtcyI6IiJ9.jMh6NUvy-Vd4YGiNSzxrbcE6c8VaO20ryvHOwx-C385GkSmaNWFZz4TSsQm2n-gf6jQvBYWC289HibRHx_tktg
[ bash ] > 
[ bash ] { [5 bytes data]
[ bash ] < HTTP/1.1 200 OK
[ bash ] < Date: Mon, 10 Apr 2017 11:18:02 GMT
[ bash ] < Server: Apache/2.4.18 (Ubuntu)
[ bash ] < Content-Disposition: attachment; filename="InBio_Map_core_2016_09_12.tar.gz"
[ bash ] < Content-Type: application/x-compressed
[ bash ] < X-Frame-Options: sameorigin
[ bash ] < Transfer-Encoding: chunked
[ bash ] < 
[ bash ] { [5 bytes data]

但是，我的

pycurl

实现相同结果时出现

HTTP 500

错误：

import pycurl
import time
import json

t = int(time.time() * 1000) - 3600000

url   = 'https://www.intomics.com/inbio/map/api/'\
        'get_data?file=InBio_Map_core_2016_09_12.tar.gz'
login = 'https://www.intomics.com/inbio/api/login_guest?ref=&_=%u' % t

fp_login = open('imweb.login.tmp', 'wb')
fp_imweb = open('imweb.tmp.tar.gz', 'wb')

c0 = pycurl.Curl()
c0.setopt(pycurl.URL, login)
c0.setopt(pycurl.WRITEFUNCTION, fp_login.write)

c0.perform()

fp_login.close()

with open('imweb.login.tmp', 'r') as fp:
    token = json.loads(fp.read())['token']

print('Token: %s' % token)

hdrs = ['Cookie: access-token=%s' % token]

c1 = pycurl.Curl()
c1.setopt(pycurl.URL, url)
c1.setopt(pycurl.WRITEFUNCTION, fp_imweb.write)
c1.setopt(pycurl.HTTPHEADER, [h.encode('ascii') for h in hdrs])
c1.setopt(pycurl.ENCODING, 'gzip, deflate')

c1.perform()

fp_imweb.close()

我看不出两者有什么不同，是吗

编辑：详细输出

curl

从

bash

调用：

t=$(echo `date +%s`' * 1000 - 3600000' | bc)

token=`curl 'https://www.intomics.com/inbio/api/login_guest?ref=&_='"$t"\
    | python -c "import json,sys;obj=json.load(sys.stdin);print(obj['token']);"`

curl 'https://www.intomics.com/inbio/map/api/get_data?file=InBio_Map_core_2016_09_12.tar.gz'\
    -H 'Cookie: access_token='"$token"  -o result.tar.gz

[ bash ] * TCP_NODELAY set
[ bash ] * Connected to www.intomics.com (77.72.50.69) port 443 (#0)
[ bash ] * ALPN, offering http/1.1
[ bash ] * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
[ bash ] * successfully set certificate verify locations:
[ bash ] *   CAfile: /etc/ssl/certs/ca-certificates.crt
[ bash ]   CApath: none
[ bash ] * TLSv1.2 (OUT), TLS header, Certificate Status (22):
[ bash ] } [5 bytes data]
[ bash ] * TLSv1.2 (OUT), TLS handshake, Client hello (1):
[ bash ] } [512 bytes data]
[ bash ] * TLSv1.2 (IN), TLS handshake, Server hello (2):
[ bash ] { [98 bytes data]
[ bash ] * TLSv1.2 (IN), TLS handshake, Certificate (11):
[ bash ] { [2279 bytes data]
[ bash ] * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
[ bash ] { [333 bytes data]
[ bash ] * TLSv1.2 (IN), TLS handshake, Server finished (14):
[ bash ] { [4 bytes data]
[ bash ] * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
[ bash ] } [70 bytes data]
[ bash ] * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
[ bash ] } [1 bytes data]
[ bash ] * TLSv1.2 (OUT), TLS handshake, Finished (20):
[ bash ] } [16 bytes data]
[ bash ] * TLSv1.2 (IN), TLS change cipher, Client hello (1):
[ bash ] { [1 bytes data]
[ bash ] * TLSv1.2 (IN), TLS handshake, Finished (20):
[ bash ] { [16 bytes data]
[ bash ] * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
[ bash ] * ALPN, server did not agree to a protocol
[ bash ] * Server certificate:
[ bash ] *  subject: OU=GT95967783; OU=See www.rapidssl.com/resources/cps (c)15; OU=Domain Control Validated - RapidSSL(R); CN=*.intomics.com
[ bash ] *  start date: Mar 24 02:21:50 2015 GMT
[ bash ] *  expire date: Mar 26 04:01:35 2018 GMT
[ bash ] *  subjectAltName: host "www.intomics.com" matched cert's "*.intomics.com"
[ bash ] *  issuer: C=US; O=GeoTrust Inc.; CN=RapidSSL SHA256 CA - G3
[ bash ] *  SSL certificate verify ok.
[ bash ] } [5 bytes data]
[ bash ] > GET /inbio/map/api/get_data?file=InBio_Map_core_2016_09_12.tar.gz HTTP/1.1
[ bash ] > Host: www.intomics.com
[ bash ] > User-Agent: curl/7.53.1
[ bash ] > Accept: */*
[ bash ] > Cookie: access_token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJ2IjoyLCJsb2dpbiI6bnVsbCwiZXhwIjoxNDkxODIzMjAxLCJpYXQiOjE0OTE4MjMwODEsInNpZCI6MTA4MDcsInNleHAiOjE0OTE4MjMzODEwNzcsImFtcyI6IiJ9.jMh6NUvy-Vd4YGiNSzxrbcE6c8VaO20ryvHOwx-C385GkSmaNWFZz4TSsQm2n-gf6jQvBYWC289HibRHx_tktg
[ bash ] > 
[ bash ] { [5 bytes data]
[ bash ] < HTTP/1.1 200 OK
[ bash ] < Date: Mon, 10 Apr 2017 11:18:02 GMT
[ bash ] < Server: Apache/2.4.18 (Ubuntu)
[ bash ] < Content-Disposition: attachment; filename="InBio_Map_core_2016_09_12.tar.gz"
[ bash ] < Content-Type: application/x-compressed
[ bash ] < X-Frame-Options: sameorigin
[ bash ] < Transfer-Encoding: chunked
[ bash ] < 
[ bash ] { [5 bytes data]

编辑2:带有

请求的示例

：

import requests
import time
import json

t = int(time.time() * 1000) - 3600000

url   = 'https://www.intomics.com/inbio/map/api/'\
        'get_data?file=InBio_Map_core_2016_09_12.tar.gz'
login = 'https://www.intomics.com/inbio/api/login_guest?ref=&_=%u' % t

r0 = requests.get(login)
token = json.loads(r0.text)['token']
hdrs = {'Cookie': 'access-token=%s' % token}

with open('imweb.tmp.tar.gz', 'wb') as fp:

    r1 = requests.get(url, headers = hdrs, stream = True)

    for block in r1.iter_content(4096):

        fp.write(block)

---

奇怪的是，这也会导致HTTP 500出现错误。因此，它在

pycurl

中不是一个bug，但是

bash

或

firefox

版本和

pycurl

或

请求

示例之间肯定存在一些差异。

其他标题——用户代理，例如？@pbuck：首先，我从firefox的检查器获得了curl语句，然后，我删除了不必要的头测试，如果它仍然有效。在Python中，我也使用与Firefox完全相同的标题进行了测试（

User-Agent:Mozilla…

etc），结果是Python请求的相同工作？（不知道pycurl中是否有bug）谢谢@pbuck，我尝试了

请求

，它给了我

HTTP500

错误。但问题又来了，有什么不同呢？