Monday, 13 March 2017

Pagination with Selenium and WebdriverIO

I'm scraping a website (https://techtransfer.universityofcalifornia.edu/Default.aspx?RunSearch=true), using WebdriverIO and Selenium to handle the pagination. I need to be able to grab all the links off a page, push them to an array, then navigate to the next page and do the same thing until I reach the end of the results.

The next button is structured like this.

<li class="next">
    <a id="ctl00_ContentPlaceHolder1_ucNCDList_ucPagination_nextPage" name="1" title="Next" onclick="javascript:showSpin(); return true;" href="javascript:WebForm_DoPostBackWithOptions(new WebForm_PostBackOptions(&quot;ctl00$ContentPlaceHolder1$ucNCDList$ucPagination$nextPage&quot;, &quot;&quot;, true, &quot;&quot;, &quot;&quot;, false, true))">
        <i class="demo-icon icon-right-open-1">&#xe83c;</i>                
    </a>            
 </li>

I then call

await client.click('#ctl00_ContentPlaceHolder1_ucNCDList_ucPagination_nextPage i')

When I do so, I get this error.

    { Error
  message: 'unknown error: Element <i class="demo-icon icon-right-open-1">...</i> is not clickable at point (442, 586). Other element would receive the click: <div class="middle-content">...</div>',
  type: 'RuntimeError',
  seleniumStack: 
   { status: 13,
     type: 'UnknownError',
     message: 'An unknown server-side error occurred while processing the command.',
     orgStatusMessage: 'unknown error: Element <i class="demo-icon icon-right-open-1">...</i> is not clickable at point (442, 586). Other element would receive the click: <div class="middle-content">...</div>\n  (Session info: chrome=56.0.2924.87)\n  (Driver info: chromedriver=2.28.455517 (2c6d2707d8ea850c862f04ac066724273981e88f),platform=Mac OS X 10.12.3 x86_64) (WARNING: The server did not provide any stacktrace information)\nCommand duration or timeout: 155 milliseconds\nBuild info: version: \'3.2.0\', revision: \'8c03df6\', time: \'2017-03-02 09:34:51 -0800\'\nSystem info: host: \'Robbys-MacBook-Pro.local\', ip: \'fe80:0:0:0:86:153c:7bdd:53f4%en0\', os.name: \'Mac OS X\', os.arch: \'x86_64\', os.version: \'10.12.3\', java.version: \'1.8.0_121\'\nDriver info: org.openqa.selenium.chrome.ChromeDriver\nCapabilities [{applicationCacheEnabled=false, rotatable=false, mobileEmulationEnabled=false, networkConnectionEnabled=false, chrome={chromedriverVersion=2.28.455517 (2c6d2707d8ea850c862f04ac066724273981e88f), userDataDir=/var/folders/89/rzglzpkn1x32wt29pff31zd40000gn/T/.org.chromium.Chromium.7xCPwY}, takesHeapSnapshot=true, pageLoadStrategy=normal, databaseEnabled=false, handlesAlerts=true, hasTouchScreen=false, version=56.0.2924.87, platform=MAC, browserConnectionEnabled=false, nativeEvents=true, acceptSslCerts=true, locationContextEnabled=true, webStorageEnabled=true, browserName=chrome, takesScreenshot=true, javascriptEnabled=true, cssSelectorsEnabled=true, unexpectedAlertBehaviour=}]\nSession ID: 717540fedaad0982a731fe8e2c012549' } }

This website uses very complicated encrypted requests with the pagination, and I wonder if this has something to do with the problem. How can I use WebdriverIO to sucessfully navigate to the next page?



via Robby Schlesinger

No comments:

Post a Comment