I am trying to create web scrapper service with node.js that does the following for a site say amazon.com/mens/section/item1
- get the name of the item (I can do this with cherrio)
- get the price for the item (this also depends on the color of the item)
- select the color and create json array for each color and its price.
I tried to use nighmarejs for this to select the color. As this event is dynamic in nature w.r.t selecting drop-down list. The site uses dojo.js Few Question here
-
Is nighmarejs the correct way to scrape the site for dynamic content (static content is straightforward with cherrio)
-
I was able to get say an h1.class text with nighmarejs but having trouble with select element
-
What is the right library to use for web scraping dynamic content and create a service out of it so I can make a curl request t it with query url=amazon.com/mens/section/item1 and response is
-
{url:myUrl, name:itemName, prices:{blue:10, black:15}}
via Mani
No comments:
Post a Comment