Tuesday 30 May 2017

Get links with cheerio issue - NodeJS

In order to get all links from a webpage with Node JS using cheerio, I use these lines that work 90% of the time:

const request = require('request');
const cheerio = require('cheerio');

var url = 'an URL';
request(url, function(err, resp, body){
  $ = cheerio.load(body);
  links = $('a');
  $(links).each(function(i, link){
    console.log($(link).text());
  });
});

But for some websites, it doesn't work properly, for example: http://www.sylire.com/ http://www.bernieshoot.fr/

And I can't figure it out. Did someone could give me hints to solve this issue?

Note that I can normaly get all links for these website in browser console using :

var link = document.querySelectorAll("a");
for (var i of link){
  console.log(i.text);
}

Regards,



via Blq56

No comments:

Post a Comment