I am not able to capture POST request payload which is sent by chrome browser for forms.
I have tried Browser mob proxy but it only captures in har file the request url, response code but not request payload.
I want to validate the request payload via java or selenium or any automation script and not via manual verification.
Any suggestion will be helpful.
As browser mob only captures request url and response status but not payload data. I stopped using browsermob.
I used Fiddler instead which worked perfectly for me. The only problem was unlike browser mob I had to install Fiddler application in my system for which I had to approached to my superiors for permissions for fiddler installation.
Related
I am trying to upload a sound to myinstants.com using java and OkHttp.
I used the Chrome dev tools to look at what requests get made and try to recreate them using OkHttp but I'm failing at the login part.
Chrome dev tools tells me that this is the form post, with a content-type of application/x-www-form-urlencoded.
And I try to replicate the post using the following code:
RequestBody loginBody = new FormBody.Builder()
.add("csrfmiddlewaretoken", token) //this token is comes from inside the <input> tag that is retrieved in the HTML of a normal get request to https://myinstants.com/accounts/login and is diffrent every time you load the page
.add("login", username)
.add("password", password)
.add("remember", "on")
.add("next", "/new/")
.build();
Request login = new Request.Builder()
.url("https://www.myinstants.com/accounts/login/?next=/new/")
.addHeader("cookie", CookieHandler.getCookie()) // cookie that is generated from the "set-cookie" response headers of the get request to https://myinstants.com/accounts/login
.addHeader("content-type", "application/x-www-form-urlencoded")
.post(loginBody)
.build();
Response response = new OkHttpClient().newCall(login).execute();
According to the chrome dev tools, the response of the above post request should have a couple of set-cookie response headers, but they are not present for me.
I don't think the issue is with the cookie I'm using because when comparing to what is found in the chrome dev tools, the cookie matches that exaclty (except for some things that are new every time you visit the site), so I think the issue is with the form post. Any ideas what I am doing wrong?
Some servers block the requests if they see you're making the requests from outside a browser.
What often works (but not always) is to try tricking the server into thinking you're using a browser. You can do this by setting the "User-Agent" header.
To do so, open the dev tools of your browser (F12), access the "Network" tab and make a request to any site. Then, look into the "Request Headers" section for the "User-Agent" value. Just copy it and send it with your request.
If all this fail, it's possible that the site has a bot protection based in Javascript. In this kind of site, the login page contains a javascript that triggers right before the login process and generates a random token that you need to send along with the credentials in order to login successfully. Since you're accessing without a browser, you can't run JavaScript and thus, you can't generate this token.
If that's the case, the best thing you can do is to use a real browser controlled programmatically. For Java you can use Selenium, but I personally prefer to use Puppeteer from NodeJS. In essence they're the same thing, an API to remote control a modified version of the chromium/chrome browser.
But with Puppeteer you have a little more flexibility than with Java because you don't need to convert between Java and Javsacritp objects and vice-versa.
I don't know what may be wrong and I don't have time to test your code now. But as a suggestion, you could try to make the upload using another library and see if it works.
I recommend Apache Fluent API:
https://mvnrepository.com/artifact/org.apache.httpcomponents/fluent-hc
I'm testing with JMeter and I need the HTTP request data to make that.
I tried to see that information in F12 Network of Chrome browser, but it doesn't appear the information there.
Someone knows how can I get that information?
You can't get the request because the browser refreshes the network tab when you go to another page. But you can persist these requests marking the option Preserve Logs, like so:
Image before request, with the checkbox checked:
After request, persistent logs:
You can see more information about the network tab here
The easiest way is to capture HAR file in the Chrome Developer Tools, it will have all the information for the request data including parameters, headers, cookies, etc.
Once done you will be able to inspect it with i.e. HAR Analyzer or simply convert it into a JMeter .jmx script
The best way to capture HTTP Requests is by using JMeter's Proxy.
If you try to inspect network in browser and construct JMeter script manually it takes lot of efforts, Alternatively you can just set your JMeter as a proxy server and capture your browsers network traffic.
Follow this post for more information on how to setup proxy in JMeter and record web applications.
I want to "fill in" a form on a website from java. I used charles proxy to determine that pressing the "Register" button on this form sends a URL encoded Http post request.
For example, if the website were:
https://apps.business.com/register
Then the http post was sent to:
https://apps.business.com/registersend
I then used the Advanced Rest Client plug-in for chrome to send a sample registration request successfully.
However, I now want to manually send this request from Java. I thought that as it was URL encoding, it would be a simple case of appending a URL encoded string to the rest of the URL. Something like:
https://apps.business.com/registersend/?email=me%40hotmail.com
etc., but I can't seem to get this to work and assume I'm missing something.
Thanks for the help.
To send HTTP POST you will need something like the HTTP Client API.
http://hc.apache.org/httpclient-3.x/
I created a swing client application which calls Restful api. I want to display both request and response. I could display response but request since it is a URI i could not print it in a xml format. How to display the request
There are plenty of browser plugins that'll let you construct a request to RESTful services. My favorite is Postman for Chrome
It'll let you type in the URI, required headers, and body, and displayes the results. Very useful for debugging RESTFul services.
I have found RESTClient to be nice. Its available in both chrome and firefox.
https://addons.mozilla.org/en-us/firefox/addon/restclient/
https://chrome.google.com/webstore/detail/advanced-rest-client/hgmloofddffdnphfgcellkdfbfbjeloo
Personally I've only used the one on Firefox since there's where I do any of my debugging.
I'm simulating an HTTPS process with Google App Engine's URL fetch API. The process has 2 steps: first, a GET request will return an URL with URL-encoded session information and a cookie; and second, a POST with some payload to the returned URL.
I have used Firebug to capture the headers of the 2 requests, e.g User-agent, Keep-alive, Connection, Cookie. I used these same headers in my code (the cookie value is updated according to the response). Testing on my computer is successful but the code always fails at the POST step on Google's server. On my development box, the remote .NET app website replies to the POST request with a 200-OK with the information that I want, but on Google side, the remote .NET app website also give a 200-OK response but with a "Session timeout" message (which I don't want). So what have I missed?
Are you connectiong to the GAE applictation through appspot.com domain or a custom domain? SSL is supported only on appspot.com, so maybe this the reason?