It takes too long for PHP to get the xml (sitemap) file and write it to the database.

1. The requirements are as follows:

-sitemap-:http://xxx.xxx/xx.xml
:http://xxx/xxx/xx ()
-xml- title

2. My code looks like this:

set_time_limit(0);

function get_title($url) {

    $output = file_get_contents($url);
    $page = [];
    $page["title"] = "";
    preg_match("/<TITLE>([\w\W]*?)<\/TITLE>/si", $output, $m);

    $page["title"] = $m[1];

    return $page["title"];
}

$xml_str = file_get_contents($xml_url);

// $data = simplexml_load_file($xml_str);
$xml_obj = simplexml_load_string($xml_str);

$data = [];
$current = time();

foreach ($xml_obj as $k => $v) {
        
    $data[$k]["title"] = get_title($v->loc);
    $data[$k]["url"] = $v->loc;
    $data[$k]["created_at"] = $current;
}

$ret = Sitemap::insert($data);

but this is easy to collapse, and the local service cannot run directly when the data is submitted.
? Is there a better solution

Dec.17,2021

php cli mode. Do not use curl in the browser


to replace file_get_contents. It is much more efficient and stable


what do you mean by collapse? Is there a memory overflow? Or just slow


I read, your writing is not a big problem, at most, curl instead of file_get_contents will be faster, but it is not a big problem.
at first I thought file_get_contents burst memory, but after thinking about it, no matter how big the html, is, it should not be this problem, so in the end, I locate the problem here: Sitemap::insert ($data);
how is it implemented in your insert? There is a problem with foreach insert

Menu