Node crawler problem

if you use superagent to request a web page, not all ssr page data can be obtained through the interface (I know it"s good to climb the interface directly, but I have a special need to do so).
I hope to get the data in the form of cheerio analysis pages. I use the html, requested by cheerio load, but sometimes there is no data on the html after load. What"s going on?


turned out to be anti-crawler


as you said, the web page is not server side render. Most likely, the browser uses js to continue to pull resources and render dynamically.

in this case, look directly at chrome's network to see what resources have been requested, sort through, find what you want, and then grab the core data for that url.

Menu