Tuesday, 6 June 2017

Page have reloaded all time when I try to get her via cURL

So, I need to parse some content from http://israelbar.org.il, and using for this cURL, but when I run script - tab in browser all time reload, and nothing showed.

    $browser = curl_init();
    curl_setopt($browser, CURLOPT_URL, $url);
    curl_setopt($browser, CURLOPT_REFERER, $referer);
    curl_setopt($browser, CURLOPT_USERAGENT, $agent);
    curl_setopt($browser, CURLOPT_SSL_VERIFYPEER, FALSE);
    curl_setopt($browser, CURLOPT_SSL_VERIFYHOST, FALSE);
    curl_setopt($browser, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($browser, CURLOPT_CONNECTTIMEOUT, 10); //times out after 11s
    curl_setopt($browser, CURLOPT_TIMEOUT, 50); //times out after 51s
    curl_setopt($browser, CURLOPT_COOKIEJAR, $cookie_file_path);
    curl_setopt($browser, CURLOPT_COOKIEFILE, $cookie_file_path);
    $retVal = curl_exec ($browser);
    curl_close ($browser);
    unset($browser);
    return $retVal;

Also, I try NodeJS, and get some, unknown for me listing of JS code in console. I think, that main problem - it's different headers, and I must send the same headers via cURL, like a browser.

Thank you for any information!



via Сашко Лисий

No comments:

Post a Comment