Scrapy.FormRequest timed out using proxy request, but requests request is normal

it is normal for the same proxy ip, to request with requests, but the request with scrapy.FormRequest will time out

.

related codes

In [11]: r = requests.post("http://httpbin.org/post", proxies={"http": proxy_server, "https": proxy_server})
2018-07-10 14:10:04 [urllib3.connectionpool] DEBUG: Starting new HTTP connection (1): 113.128.26.187
2018-07-10 14:10:04 [urllib3.connectionpool] DEBUG: http://113.128.26.187:4352 "POST http://httpbin.org/post HTTP/1.1" 200 275

In [12]: r.text
Out[12]: "{"args":{},"data":"","files":{},"form":{},"headers":{"Accept":"*/*","Accept-Encoding":"gzip, deflate","Connection":"close","Content-Length":"0","Host":"httpbin.org","User-Agent":"python-requests/2.18.4"},"json":null,"origin":"113.128.26.187","url":"http://httpbin.org/post"}\n"

In [13]: proxy_server
Out[13]: "http://113.128.26.187:4352"

In [14]: req = scrapy.FormRequest("http://httpbin.org/post",formdata={"foo":"bar"}, meta={"proxy": proxy_server})

In [15]: fetch(req)
2018-07-10 14:13:58 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <POST http://httpbin.org/post> (failed 1 times): User timeout caused connection failure: Getting http://httpbin.org/post took longer than 180.0 seconds..

as shown in figure
2018-07-10 14:27:39 use FormRequest request
clipboard.png
2018, 7, 10, 14, 28, 19Requests,ip
clipboard.png

Menu