Sunday 11 June 2017

Navigates to URL but gets no response

I'm using casperjs and I had an array of links and I was opening each page. the cycle of the array stopped at http://finishline.com/ all the time. It seems like I get no response. I isolated the situation I just cant get info from that link at all.

it just stops:

... [info] [phantom] Running suite: 2 steps [debug] [phantom] opening url: http://finishline.com/, HTTP GET [debug] [phantom] Navigation requested: url=http://finishline.com/, type=Other, willNavigate=true, isMainFrame=true [debug] [phantom] Navigation requested: url=http://www.finishline.com/, type=Oth er, willNavigate=true, isMainFrame=true

Here is the Isolation code:

var casper = require("casper").create({
    verbose : true,
    logLevel : "debug",
    pageSettings: {
        userAgent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36",
         customHeaders: {
            "Server": "Apache/2.4.1 (Unix)"
        }
     },
    clientScripts: ["jquery.js"] ,
    viewportSize: {
        width: 1920,
        height: 1080
    }
});

casper.start().thenOpen("http://finishline.com")
.then(function(){
    console.log("title : ", this.getTitle())
})
casper.run();

Why can't I scrape or get a response from finishline.

how to handle if I cant get a response to continue with the code when there is no response.



via jack blank

No comments:

Post a Comment