Python is very simple crawler problem

"""
parse  post

1:
2:girlletterrequest
3::http://fanyi.baidu.com/sug
4:formdata kw:girl
5:return  json==> need package json
"""


from urllib import parse,request
-sharpmanage json moudel

import json

"""
1:data urlopen
2:json style result
3:encode "girl"
"""
baseurl = "http://fanyi.baidu.com/sug"

-sharp form ==> dict style


data ={
    "ke":"girl"
}


-sharpenode

data =parse.urlencode(data).encode("utf-8")-sharptype -bytes
-sharp-sharpencode string in "utf-8"style== change style to bytes ,not change content


-sharprequest headed (include data_lenth)

headers={
    -sharp post need content-lenth
    "Content-Lengh":len(data)

}

-sharp we have request header ,try request
req=request.Request(url=baseurl,data=data,headers=headers)

rsp=request.urlopen(req)
json_data= rsp.read().decode("utf-8")

print(type(json_data))-sharpstr

-sharpchange style str to dict

json_data=json.loads(json_data)

print(type(json_data))-sharpdict
print(json_data)

Front-end web-crawler python

Mar.24,2021

import requests
import json

url = 'http://fanyi.baidu.com/sug'
data = {'kw':'girl'}
res = requests.post(url, data=data)
content = json.loads(res.content.decode())
print(content)

Previous: How to get the index value of the largest three numbers in an array

Next: How to define a byte string in python