When scrapy grabs data, cookie passes differently.

    self.co_headers = {"Referer": "https://rong.36kr.com/list/detail&?sortField=HOT_SCORE",
                               "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
                , "Host": "rong.36kr.com", "cookie": self.cookie_str, ":authorit": "rong.36kr.com"}
            
 headers cookie 

self.headers = {"Referer": "https://rong.36kr.com/list/detail&?sortField=HOT_SCORE",
                        "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"
            , "Host": "rong.36kr.com"}
            
 header
cookie 
cookies=cookieheaders=headers
scrapy cookie cookie co_header  headers cookie 
Jan.25,2022

No difference. Mainly look at the cookies format. Scrapy also sends requests through the requests library. The format of coookies=' {} = {}; {} = {}'in headers. If you pick it out alone, cookies= {} is just a dictionary, which will eventually be converted into a format accepted by cookiejar.

Menu