So I am having quite a strange issue with a couple side questions and would really appreciate any help I can get on this.
So I'm trying to create a application (using node.js for no particular reason other than it's the language i'm comfortable with) that makes a post request and parses the response. On this webpage there is a search box and upon searching it makes a post request and returns the html of the results page.
For some reason, when I make the request using the node.js request library, it returns a page with 403 Forbidden as the page title and nginx within the page content. I am 100% sure the post request url and path are correct.
When intercepting the request using postman interceptor, I am able to repost the request and it successfully returns the result, however it has two additional headers that do not appear in the request under the network tab of chrome.
X-DevTools-Emulate-Network-Conditions-Client-Id
X-DevTools-Request-Id
When these two headers are unchecked within postman, the request returns the site html with no results (but does not return a 403 forbidden like it does using the node.js request library).
What are these headers? I searched for x-devtools and can't find exactly what these headers are. I suspect they have something to do with chrome rather than the actual request.
Furthermore, when I use the built in postman function that allows me to generate a code snippet that recreates this post request in my language of choice, running that code results in a 403 forbidden response (in both python and node.js).
What could be causing this? What pieces of information are not being sent when creating code that mimics the request? Why am i receiving 403 forbidden rather than just the html of the page without results which is typical on an un-authenticated request. I know there is a lot to this question but i'm happy to respond with any additional information. It is not an ajax request, and there is nothing under the authorization tab in the intercepted post request in postman. Here are the headers for the request that appear in postman (edited out some parts)
X-DevTools-Request-Id:xxx
Origin:xxx
X-DevTools-Emulate-Network-Conditions-Client-Id:xxx
Upgrade-Insecure-Requests:1
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36
Content-Type:application/x-www-form-urlencoded
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Referer:xxx
Accept-Encoding:gzip, deflate, br
Accept-Language:en-US,en;q=0.8,af;q=0.6
Cookie:xxx
So to summarize:
What could be causing a 403 response despite having the url and path correct and all the headers seemingly correct? Esepcially considering that the code i'm using to make the request is directly outputted from postman that is suppposed to mimic the exact actions of the real post request, so why would it give me different results from within postman to the postman-generated node.js code?
This was working at some point until the site administrator seemed to make some change that ended up blocking my request
via Mike B
No comments:
Post a Comment