Use node.js to crawl pictures on the website and save them locally

1. Description problem:
I made a small crawler, want to crawl some pictures of a website, now climb down, the image path can be printed out.
however, I need to download these pictures to a local images folder, write the code is no problem, I have tried, if the site crawled to the image path is normal, that is, the form of http://.img, then there is no problem to save.
however, the problem is that the image addresses on many websites are base64 . The code I write will always report errors, and I don"t know how to solve it.
2. Code map

clipboard.png
3.
clipboard.png

Nodejs crawler
Mar.18,2021

for base64 images, you can write binary data to a file alone. Do not use stream 

request(url, function (err, res, body) {
    if (!err && res.statusCode == 200) {
        var $ = cheerio.load(res.body.toString());
        $(".codelist").eq(0).children("a").each(function () {
            var listImgUrl = $(this).find("img").attr("src");
            links.push(listImgUrl)
        })
        
        async.mapSeries(links, function (item, callback) {
            let base64 = item.replace(/^data:image\/\w+;base64,/, ""); //base64data:image/png;base64
            let dataBuffer = new Buffer(base64, 'base64'); //base64buffer
            //download(item, dir, Math.floor(Math.random() * 100000) + item.substr(-4, 4));
            fs.writeFile('./images/'+Math.floor(Math.random() * 100000) + item.substr(-4, 4)+'.jpg',dataBuffer,function(err){
                if(err){
                    console.log(err);
                }else{
                    console.log('');
                }
            })
            callback(null, item)

        }, function (err, results) {});
    }

});

solved it on its own, using fs's writeFile method. Is the personal test effective?
Happy?

Previous: How to dynamically add attribute values to pug templates in veu? why didn't I render this?

Next: Phpmailer reported an error The following From address failed under windows

css mysql arrays josn react html typescript webpack npm sass R objective-c .net sql-server jquery python-3.x angularjs django angular excel regex iphone ajax linux xml pandas vba spring database wordpress string wpf xcode windows bash postgresql oracle multithreading eclipse list firebase algorithm macos forms image scala visual-studio azure bootstrap spring-boot react-native python-2.7 docker performance function winforms matlab powershell apache dataframe api sqlite numpy rest shell selenium flutter dart maven loops qt swing android-studio csv express file class tensorflow sorting codeigniter perl
© 2021 CodesHelper
Advertising Contact us About us
Menu