Use xpath to get the value of the node as None

want to crawl http://47.99.86.238/portal/li. the data of this website, use scrapy, to set everything else, only one value to get is None, please take a look at it.

I can match the value using the plug-in myself, but the result printed by storyMale in the program is None.

related codes

/ / Please paste the code text below (do not replace the code with pictures)

-sharp 
name = "sister_story_spider"

-sharp 
allowed_domains = ["http://47.99.86.238"]

-sharp 
start_urls = ["http://47.99.86.238/portal/list/index/id/11.html"]

def parse(self, response):
    -sharp 
    data_list = response.xpath("//div[@id="List"]")

    for data_item in data_list:
        -sharp 
        sisterStoryDataItem = SisterStoryDataItem()

        -sharp 
        sisterStoryDataItem["storyFeMale"] = data_item.xpath(".//span//span//text()").extract_first()
        print("storyFeMale is: ", sisterStoryDataItem["storyFeMale"])

        sisterStoryDataItem["storyMale"] = data_item.xpath("normalize-space(.//div[@class="H-flex-item H-margin-vertical-both-10 H-margin-10"]//div[@class="wz1wz H-theme-font-color-333 H-font-size-14 H-text-horizontal-left H-text-show-row-2"]//text())").extract_first()
        print("storyMale is: ".format(sisterStoryDataItem["storyMale"]))

        sisterStoryDataItem["storySort"] = data_item.xpath(".//span[@class="H-display-block H-flex-item H-text-align-right H-theme-font-color-999 H-font-size-12"]//text()").extract_first()
        print("storySort is: ", sisterStoryDataItem["storySort"])
        

what result do you expect? What is the error message actually seen?

The expected result of

can be matched to a specific value, but it cannot be matched at the moment.
error log is as follows:

Aug.20,2021

.//span//span/text()

the previous one of text () will be fine

  • Problem parsed by xpath

    crawls the movie of Douban, saying that the tag of each movie is parsed into list, but after traversing the list, it is found that every element in the list is the same . movies = selector.xpath( *[@id="content"] div div[1] ol li ) -sharp...

    Mar.04,2021
  • How does Xpath match multiple times in one range?

    The code is a little messy, but that s what the original page looks like. You can change it . <li class="list__item"><div class="list__title">The world this week< div><a itemProp="url" class="link...

    Sep.16,2021
Menu