I'm using casperjs and I had an array of links and I was opening each page. the cycle of the array stopped at http://finishline.com/ all the time. It seems like I get no response. I isolated the situation I just cant get info from that link at all.
it just stops:
... [info] [phantom] Running suite: 2 steps [debug] [phantom] opening url: http://finishline.com/, HTTP GET [debug] [phantom] Navigation requested: url=http://finishline.com/, type=Other, willNavigate=true, isMainFrame=true [debug] [phantom] Navigation requested: url=http://www.finishline.com/, type=Oth er, willNavigate=true, isMainFrame=true
Here is the Isolation code:
var casper = require("casper").create({
verbose : true,
logLevel : "debug",
pageSettings: {
userAgent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36",
customHeaders: {
"Server": "Apache/2.4.1 (Unix)"
}
},
clientScripts: ["jquery.js"] ,
viewportSize: {
width: 1920,
height: 1080
}
});
casper.start().thenOpen("http://finishline.com")
.then(function(){
console.log("title : ", this.getTitle())
})
casper.run();
Why can't I scrape or get a response from finishline.
how to handle if I cant get a response to continue with the code when there is no response.
via jack blank
No comments:
Post a Comment