I want to read two numbers (randomly generated) from a website which are then used in order to compute a result and then submit the result using a POST request. To do so, I will also need to submit the cookie of that session so that the system is aware of the random numbers which have been produced in that particular session.
In order to read the numbers I am using Jsoup:
Document document = Jsoup.parse(Jsoup.connect("http://website.com/getNumbers").get().select("strong").text());
String[] numbers = document.text().split(" ");
String answer = methodThatComputesTheDesiredOutput(numbers);
Now I want to send a POST request that includes answer and cookies of that session. Here's a partially implemented POST request that includes only the one parameter (answer):
HttpClient httpClient = HttpClients.createDefault();
HttpPost httpPost = new HttpPost("http://website.com/submitAnswer");
List<NameValuePair> params = new ArrayList<NameValuePair>(1);
params.add(new BasicNameValuePair("answer", answer);
httpPost.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));
How can I obtain the cookie when reading the document and then use it as a parameter of the POST request?
Extract cookies using jsoup in the following way:
Response response = Jsoup.connect("http://website.com/getNumbers").execute();
Map<String, String> cookies = response.cookies();
Document document = Jsoup.parse(response.body());
Create a BasicCookieStore using the cookies extracted using jsoup. Create a HttpContext containing the cookie store and pass it while executing your next request.
BasicCookieStore cookieStore = new BasicCookieStore();
for (Entry<String, String> cookieEntry : cookies.entrySet()) {
BasicClientCookie cookie = new BasicClientCookie(cookieEntry.getKey(), cookieEntry.getValue());
cookie.setDomain(".example.com");
cookie.setPath("/");
cookieStore.addCookie(cookie);
}
HttpContext localContext = new BasicHttpContext();
localContext.setAttribute(HttpClientContext.COOKIE_STORE, cookieStore);
HttpPost httpPost = new HttpPost("http://website.com/submitAnswer");
List<NameValuePair> params = new ArrayList<NameValuePair>(1);
params.add(new BasicNameValuePair("answer", answer);
httpPost.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));
httpClient.execute(httpPost, localContext);
Send your first request like below :-
Response res = Jsoup.connect("login Site URL")
.method(Method.GET)
.execute();
Now get the cookies and send new request with the cookies something like below :-
CookieStore cookieStore = new BasicCookieStore();
for (String key : cookies.keySet()) {
Cookie cookie = new Cookie(key, cookies.get(key));
cookieStore.addCookie((org.apache.http.cookie.Cookie) cookie);
}
HttpClientContext context = HttpClientContext.create();
context.setCookieStore(cookieStore);
HttpClient httpClient = HttpClients.createDefault();
HttpPost httpPost = new HttpPost("http://website.com/submitAnswer");
httpClient.execute(httpPost,context);
Related
I have the code below, and my problem is that the response.status is 200 (is ok) and my response body is empty and it should return a json response. Is the problem the length maybe?:`
HttpPost post = new HttpPost("http://localhost:8080/rest/login");
httpClient = HttpClients.createDefault();
List<NameValuePair> formParams = new ArrayList<NameValuePair>();
NameValuePair pair = new BasicNameValuePair("token",token);
formParams.add(pair);
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formParams);
post.setEntity(entity);
HttpResponse response = httpClient.execute(post);
this.code = response.getStatusLine().getStatusCode();
InputStream body = response.getEntity().getContent();
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));`
I had similar problem. In my case it was also very strange, that when url contained IP instead of hostname, it worked (I get the nonempty response).
However, setting netscape cookie specs helped in my case.
httpClientBuilder.setDefaultRequestConfig(RequestConfig.custom()
.setCookieSpec(CookieSpecs.NETSCAPE)
.build());
I'd like to be able to view the complete HttpClient / HttpPost URI with params. I'm not sure how to output it to my console.
HttpClient httpclient = HttpClients.createDefault();
HttpPost httppost = new HttpPost(url);
httppost.setHeader("ContentType","application/x-www-form-urlencoded");
httppost.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));
System.out.println(httppost.getURI());
Printing getURI only outputs the baseURI and not the params.
Could someone help me with what I'm missing?
The entity has a method to serialize itself to an output stream writeTo(), so you can create one and dump the entity there. This is not the whole request, only the UrlEncodedFormEntity exposing the encoded parameters:
ByteArrayOutputStream outs = new ByteArrayOutputStream();
httppost.getEntity().writeTo(outs);
System.out.println(outs.toString("UTF-8"));
outs.close();
result is one line like this:
foul=play&foo=bar&baz=bam
In java while doing an HTTP post request using nameValuePairs we write the following code!
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("http://sometesturl.com");
JSONObject json = new JSONObject();
json.put("id",1);
json.put("name","john");
try {
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("data", "abc"));
nameValuePairs.add(new BasicNameValuePair("samplejson", json.toString()));
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
}
As seen over here we use the UrlEncodedFormEntity to encode our request body. Similarly, I need to do the same in Javascript. I have seen the EncodeURIComponent method but that doesn't seem to encode the request body. Instead it encodes the URL.
Can someone tell me how to encode the request body in javascript?
I am creating a 3rd party java application (desktop) that needs to connect to a php-based site and log in to gather pertinent data. There is no accessible web service, no API, and every user will have their own secure login. The site uses dojo (if that matters), and I am using Java HttpClient to send the post.
HttpPost httppost = new HttpPost("https://thewebsite.net/index/login"); // .php ?
// Create a new HttpClient and Post Header
HttpClient httpclient = new DefaultHttpClient();
//initialize the response string
String nextpage = "";
try {
// Add nvps
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(3);
nameValuePairs.add(new BasicNameValuePair("", ""));
nameValuePairs.add(new BasicNameValuePair("login", "USER"));
nameValuePairs.add(new BasicNameValuePair("", ""));
nameValuePairs.add(new BasicNameValuePair("pass", "PASSWORD"));
nameValuePairs.add(new BasicNameValuePair("Submit", ""));
httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = httpclient.execute(httppost);
userID = EntityUtils.toString(response.getEntity());
System.out.println(nextpage);
httppost.releaseConnection();
}
...
Now, the issue I'm having is that the response given to me is a validation jscript for the user / pass fields through dojo.
<script type='text/javascript'>
dojo.require("dojox.validate._base");
function validate_RepeatPassword(val, constraints)
{
var isValid = false;
if(constraints) {
var otherInput = dijit.byId(constraints[0]);
if(otherInput) {
var otherValue = otherInput.value;
isValid = (val == otherValue);
}
}
return isValid;
}
</script>
I simply want to connect, parse an html response, and close the connection.
When I use firebug, I get this as the post method, but I can't seem to get it to run:
Referer https://thewebsite.net/index/login
Source login=USER&pass=PASSWORD
When I use the HttpPost client to construct a direct post url without namevaluepairs:
HttpPost httppost = new HttpPost("https://thewebsite.net/index/login?login=USER&pass=PASSWORD");
, I get an error response that states "the user and pass fields cannot be left blank."
My question is: Is there a direct method to log in that is simpler that I'm missing that will allow me to successfully continue past log in?
Thanks - I love the SO community; hope you can help.
I think best library for doing this is jsoup
Connection.Response res =
Jsoup.connect("https://thewebsite.net/index/login?login=USER&pass=PASSWORD")
.method(Method.POST)
.execute();
After this you need to make verification also. You need to read cookies, request parameters and header parameters and this will work.
I didn't end up using your exact code (with the post parameters), but JSoup was the fix.
here's what I used:
`res = Jsoup.connect("https://thewebsite.net/index/login")
.data("login", User).data("pass", Pass)
.userAgent("Chrome").method(Method.POST).execute();
//then I grabbed the cookie and sent the next post for the data
Document t = res.parse(); //for later use
SessionID = res.cookie("UNIQUE_NAME");
//the JSON
Connection.Response driverx = Jsoup.connect("https://thewebsite.net/datarequest/data").cookie("UNIQUE_NAME",SessionID).userAgent("Chrome").method(Method.POST).execute();`
I need to work with REST api in android application which is created by my client. Below text is just copied from the pdf the client provides us.
--
In this example, a new user is created.
The parts of a possible request to the server is shown below:
Message part Contents
Header POST {url-prefix}/rest/user
Content-Type: application/xml
Content-Length: 205
Body <request>
<client>
<id>XY</id>
<name>myName</name>
<password>myPassword</password>
</client>
<user>
<name>myUserName</name>
<password>myUserPassword</password>
<groupId>12345</groupId>
</user>
</request>
--
After searching and studying, I come to know that, the possible request code (in Java) might be:
URL url=new URL("http://api.example.com/rest/user/?name=myUserName&password=myUserPassword&groupId=12345");
HttpURLConnection conn=(HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("Post");
OutputStreamWriter out=new OutputStreamWriter(conn.getOutputStream());
out.write("respose content:");
out.close();
From the pdf manual they provide, I got to know, for every request to the server, the client (thats me) has to transmit the authentication data.
My question is, where do I put the authentication data in the query string? Please help me on this.
Edit:After posting the below code as request:
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("http://api.example.com/rest/user/?name=Foysal&password=123456&groupid=12345");
httpPost.addHeader("Accept", "text/xml");
httpPost.setHeader("Content-Type","application/xml;charset=UTF-8");
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>();
nameValuePairs.add(new BasicNameValuePair("name", "APIappDevAccount"));
nameValuePairs.add(new BasicNameValuePair("password", "123456"));
httpPost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpParams params = new BasicHttpParams();
HttpConnectionParams.setStaleCheckingEnabled(params, false);
HttpConnectionParams.setConnectionTimeout(params, 5000);
HttpConnectionParams.setSoTimeout(params, 5000);
httpClient.setParams(params);
httpClient.getParams().setParameter(ClientPNames.COOKIE_POLICY, CookiePolicy.RFC_2109);
HttpResponse response = httpClient.execute(httpPost);
InputStream is = response.getEntity().getContent();
ByteArrayOutputStream os = new ByteArrayOutputStream();
byte[] buf;
int ByteRead;
buf = new byte[1024];
String xmldata = null;
double totalSize = 0;
while ((ByteRead = is.read(buf, 0, buf.length)) != -1) {
os.write(buf, 0, ByteRead);
totalSize += ByteRead;
}
xmldata = os.toString();
os.close();
is.close();
But I got the response as:
404
Not Found
Not Found The requested
URL /rest/user/ was not found on this
server. Apache/2.2.6
(Fedora) DAV/2 mod_ssl/2.2.6
OpenSSL/0.9.8b Server at
api.example.com Port 80
Looks to me like they want you to POST an XML document and put the authentication in that. Not much of a REST API (most REST APIS don't require an XML document).
You need to use conn.getOutputStream() to send that doc to the server and use conn.getInputStream() to read the response.
So you would have to create the XML doc like the one they show:
<request>
<client>
<id>XY</id>
<name>myName</name>
<password>myPassword</password>
</client>
<user>
<name>myUserName</name>
<password>myUserPassword</password>
<groupId>12345</groupId>
</user>
</request>
And then send it in your POST:
conn.setRequestProperty ( "Content-Type", "text/xml" );
out.write(requestDoc); //where requestDoc is the String containing the XML.
out.flush();
out.close();
You may execute a POST request like shown here: http://www.androidsnippets.com/executing-a-http-post-request-with-httpclient and put the authentication data as name value pairs:
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost("http://www.yoursite.com/script.php");
try {
// Add your data
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
nameValuePairs.add(new BasicNameValuePair("name", "myUserName"));
nameValuePairs.add(new BasicNameValuePair("password", "myUserPassword"));
httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
// Execute HTTP Post Request
HttpResponse response = httpclient.execute(httppost);
} catch (Exception e) {
// ... handle exception here
}