Crawling a web page, being blocked

description:

20


Code:

def handle(self):
    request = urllib.request.Request(self.url,headers=self.headers)
    response = urllib.request.urlopen(request)
    print(response)
    html=response.read()

error report:

urllib.error.HTTPError: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop.

distress

15+1
Nov.01,2021

try to save url into an array, and then when validation occurs, when you use try and expect to avoid verification, and
skip validation at the same time, read the last value of the array, then + 1, and then continue to access

.
Menu