Pyspider crawler page contains lazy load lazy-load, to get no data

use pyspider to get Mango TV page popular variety column content ("div.mg-main ul > li.v-item"), because the page uses a lazy loading mode, so can not get specific information, how to let the page to load this part of the content, and then get the data ah?

Mar.15,2021

can be solved by setting the js_viewport_height attribute value in self.crawl or by using js_script


js_script can be solved, window.scrollTo (document.body.scrollWidth, document.body.scrollHeight);

Menu