Why is there a lack of content in the China Weather Network returned by requests?

I downloaded the page of China Weather Network using Python and requests, but the content of "Typhoon Dynamics" on the left was missing. That"s strange. How to solve it?

this is what happens when I download the web page and open it.

clipboard.png

clipboard.png

the following is my source code, please help me solve it.

problem description

the environmental background of the problems and what methods you have tried

related codes

/ / Please paste the code text below (do not replace the code with pictures)

import requests, os
from bs4 import BeautifulSoup

url =" http://typhoon.weather.com.cn/"
page = requests.get (url)
with open ("page.html", "walled, encoding="utf-8") as f:

f.write(page.text)

what result do you expect? What is the error message actually seen?

Jul.01,2022

this part of the content is rendered through the data provided by typhoonindex.js
request API, of course, the source code cannot be caught. You assemble the rendered page by yourself according to the logic of js or use selenium Chrome to get the rendered page directly.


it is possible to use selenium


the data you climb may be loaded asynchronously, and you may not request this part of the data after you get the page.

Menu