I am using a Http Get to request a website with a total of 7 Html Inputs on it.
However, when I get the page and output it, only 5 inputs appear (in either my console or outputted to a text file).
The website I'm trying to get the 7 inputs for is an intranet site so it'd be of no use to provide the address.
This is my code/ "http getting" method
//GET a web page and store as string in "htmlpage"
DefaultHttpClient httpclient = new DefaultHttpClient();
//Sometimes I need the below code, but not this time
//httpclient.setRedirectStrategy(new RedirectStrategy());
HttpGet httget = new HttpGet("example-website.aspx");
HttpResponse response = httpclient.execute(httget);
HttpEntity entity = response.getEntity();
InputStream in = entity.getContent();
StringBuffer charBuf = new StringBuffer();
do{char c = (char)in.read();
charBuf.append(c);
}while (charBuf.length() < entity.getContentLength());
String htmlpage = charBuf.toString();
charBuf.delete(0, charBuf.length()-1);
in.close();
EntityUtils.consume(entity);
httget.abort();
FileOutputStream fo = new FileOutputStream("please_have_7_this_time.html");
fo.write(htmlpage.getBytes());
fo.flush();
fo.close();
Related
I am trying to submit a form on this website, and get back the resulting misspellings from the text area as a string (only the "Reverse letters" checkbox should be selected). I have the code below, adapted from here:
private static void sendPost() throws Exception {
String url = "http://tools.seobook.com/spelling/keywords-typos.cgi";
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
post.setHeader("User-Agent", "Mozilla/5.0"); // add header
List<NameValuePair> urlParameters = new ArrayList<NameValuePair>();
//the input text area
urlParameters.add(new BasicNameValuePair("user_input", "tomato potato"));
//the checkbox
urlParameters.add(new BasicNameValuePair("reverse_letters", "reverse_letters"));
//the submit button (?)
urlParameters.add(new BasicNameValuePair("", "generate typos"));
post.setEntity(new UrlEncodedFormEntity(urlParameters));
HttpResponse response = client.execute(post);
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + post.getEntity());
System.out.println("Response Code : " +
response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(new InputStreamReader(
response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line + "\n");
}
System.out.println(result.toString());
}
If I copy and paste the lines from the console, and search through it in an editor for the misspellings, I do in fact have the input text and resulting text area text contained in the huge string. The string contains all html however, and I would like only the misspellings as a string. How would I extract only the resulting misspellings from this site, perhaps with a method as part of the Apache HttpClient Library, or I am taking the wrong approach?
Thanks, Dan
I think you are trying to put a square peg in a round hole, Selenium would probably be a better bet.
Apache http client is best used for request and response header handling not for processing the body of a response
An over complicated way would be to split the "result" variable using regex's
I have Tornado web service which returns a link after creating a CSV file as shown below.
http://10.0.2.2:8000/uploads/16165159GyjFImAYZssLEmn/16165159GyjFImAYZssLEmn.csv
Also I have outputted this URL within the Android application successfully. My question is how to parse the data of this file using the URL shown above. I have tried numerous ways and could not get it done. Can some one please help me to solve this problem. The code I have so far is as follows.
Also I have referred to this post as well, however still no success.
HttpClient httpclient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(modifyURL);
HttpResponse httpResponse = httpclient.execute(httpPost);
responseEntity = httpResponse.getEntity();
transformedImageURL = EntityUtils.toString(responseEntity);
URL url = new URL(transformedImageURL);
InputStream stream = url.openStream();
BufferedInputStream bis = new BufferedInputStream(stream);
ByteArrayBuffer baf = new ByteArrayBuffer(200);
nt current = 0;
while((current - bis.read()) != -1){
baf.append((byte) current);
}
String stockText = new String(baf.toByteArray());
String[] tokens = stockText.split(",");
String testData = tokens[0];
Thank you for your time
I am trying to send a http request to a website which is supposed to return a json response. The problem is that i am not getting the json data. But when i paste the url in a browser it displays the json output. Am a newbie. Kindly help.
Here is my code
HttpClient client = new DefaultHttpClient();
String url="http://directclientvendors.com/news24/api/get.php?type=news";
HttpGet request = new HttpGet(url);
HttpResponse response;
response = client.execute(request);
BufferedReader br =
new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line = "";
while(br.ready())
{
line+=br.readLine();
}
System.out.println("line "+line);
You should be executing a GET request and not a POST. Please change the request type to HttpGet. The browser executes a GET on the URL when you paste it on the address bar and hit enter.
Additionally use a Reader + StringBuilder / JsonReader / GSON to read from the URL's response content. String concatenation leads to the creation of additional objects unnecessarily.
[EDIT]
To my astonishment the API call works even when a POST call is made to get the resource. The problem must be in your parsing logic. Using a JsonReader works fine for me. This is just template code, but you can fill in the rest to get the other JSON elements. Regardless of whether POST works or not, you should still use GET for this call.
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet("http://directclientvendors.com/news24/api/get.php?type=news");
HttpResponse response = client.execute(request);
InputStream content = response.getEntity().getContent();
JsonReader jsonReader = new JsonReader(new InputStreamReader(content, "UTF-8"));
jsonReader.beginObject();
if(jsonReader.hasNext())
{
System.out.println(jsonReader.nextName()); // prints 'news'
// BEGIN_ARRAY etc to parse the rest
}
// END_OBJECT and cleanup
I have searched for a while and I am not finding a clear answer. I am trying to log into a webstie.
https://hrlink.healthnet.com/
This website redirects to a login page that is not consitent. I have to post my login credentials to the redirected URL.
Im am trying to code this in Java but I do not understand how to get the URL from the response. It may look a bit messy but I have it this way while I am testing.
HttpGet httpget = new HttpGet("https://hrlink.healthnet.com/");
HttpResponse response = httpclient.execute(httpget);HttpEntity entity = response.getEntity();
String redirectURL = "";
for(org.apache.http.Header header : response.getHeaders("Location")) {
redirectURL += "Location: " + header.getValue()) + "\r\n";
}
InputStream is;
is = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"),8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
String result = sb.toString();
I know i get redirected because my result string shows be the actual login page but I am not able to get the new URL.
In FireFox I am using TamperData. When I navigate to this website https://hrlink.healthnet.com/ I have a GET with a 302 - Found and the Location of the Login Page. Then another GET to the actual Login Page
Any help is greatly appreciated thank you.
Check out w3c documentation:
10.3.3 302 Found
The temporary URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).
If the 302 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.
One solution is to use POST method to break auto-redirecting at client side:
HttpPost request1 = new HttpPost("https://hrlink.healthnet.com/");
HttpResponse response1 = httpclient.execute(request1);
// expect a 302 response.
if (response1.getStatusLine().getStatusCode() == 302) {
String redirectURL = response1.getFirstHeader("Location").getValue();
// no auto-redirecting at client side, need manual send the request.
HttpGet request2 = new HttpGet(redirectURL);
HttpResponse response2 = httpclient.execute(request2);
... ...
}
Hope this helps.
Just wondering if anyone knows how to determine when a HTTP PUT request is complete. For eg:
HttpClient http = new DefaultHttpClient();
HttpPut putmethod = new HttpPut("http://abc.com/SETTINGS.TXT");
putmethod.setEntity(new StringEntity(data));
HttpResponse response = http.execute(putmethod);
How can I tell when the file has completely transferred/written. Do I need to monitor the HttpResponse? If so, what I am looking for?
Thanks
If your request is successfully complete then http client will return the success code 200 if it fails then it returns the another code (401 page not fount etc).
so you can check the code with the response and log appropriate message .
Example
HttpClient http = new DefaultHttpClient();
HttpPut putmethod = new HttpPut("http://abc.com/SETTINGS.TXT");
putmethod.setEntity(new StringEntity(data));
HttpResponse response = http.execute(putmethod);
if (response.getStatusLine().getStatusCode() == 200)
{
is = response.getEntity().getContent();
int ch;
sb = new StringBuffer();
while ((ch = is.read()) != -1) {
sb.append((char) ch);
}
// Log sb . it prints the response you get.
}
if you want check that file has completely transferred then try this..
String data = EntityUtils.toString(response.getEntity());
System.out.println("Data in .."+data);
in data you will get response from server...