I'm trying to get the whole html from a web page in Android.
In java console aplication I used to do like this:
DefaultHttpClient httpclient = new DefaultHttpClient();
String busca = "kindle";
HttpGet httpGet = new HttpGet("http://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords="+busca);
try {
ResponseHandler<String> manipulador = new BasicResponseHandler();
String resposta = httpclient.execute(httpGet,manipulador);
}
} finally {
httpGet.releaseConnection();
}
I tried to do the same in my Android aplication but I didn't work!
This library works in Android?
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
Is there in better way to get a page html code in a string on Android?
Thks for the help!
I did it with another URl and it worked :)
Maybe the HTML code of that page I was using as to big to save in a String or to show in a Text
You can have a loot at this :
HttpClient 4.0.1 - how to release connection?
HttpRequestBase.releaseConnection() is introduced in Version 4.2
Related
I'm trying to get the text response from the following URL:
http://translate.google.cn/translate_a/single?client=t&sl=zh-CN&tl=en&dt=t&tk=265632.142896&q=%E4%BD%A0%E5%A5%BD
The response is the following:
[[["Hello there","你好",,,1]],,"zh-CN"]
(You can verify this response by entering the address into your browser.)
Here is a simplified version of my code that tries to download this text:
import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
public class Test {
public static String downloadString() {
String url = "http://translate.google.cn/translate_a/single?client=t&sl=zh-CN&tl=en&dt=t&tk=265632.142896&q=%E4%BD%A0%E5%A5%BD";
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
ResponseHandler<String> handler = new BasicResponseHandler();
try {
return client.execute(request, handler);
} catch (Exception e) {
return "GET request failed.";
}
}
}
When I call Test.downloadString(), I get the following (incorrect) response:
[[["Huan Chai Sunsolt","浣犲ソ",,,0]],,"zh-CN"]
I'm guessing that there is some sort of encoding problem behind the scenes somewhere in the request process (there are six bytes that should be interpreted as two Chinese characters, but are instead interpreted as three Japanese characters), but I can't seem to pinpoint the exact cause. What am I doing wrong in my code?
It's strange, but adding the User-Agent header fixed the problem:
request.addHeader("User-Agent", "Mozilla/5.0 (X11; Linux x86_64; rv:33.0) Gecko/20100101 Firefox/33.0");
Android 6.0 release removes support for the Apache HTTP client. If your app is using this client and targets Android 2.3 (API level 9) or higher, use the HttpURLConnection class instead.
here: http://developer.android.com/about/versions/marshmallow/android-6.0-changes.html#behavior-apache-http-client
Is there a way to make get & put calls over HTTP in java ? I also need to automate any user inputs like a button click on the target web-page(any web-page, not just yahoo finance)
I tried using the apache commons library & couldn't quite crack it:
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
public class Fin {
/**
* #param args
*/
public static void main(String[] args) {
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet("http://finance.yahoo.com");
try {
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
} catch (Exception e) {
e.printStackTrace();
} finally {
httpget.releaseConnection();
}
}
}
I keep getting 'java.net.ConnectException: Connection refused', though i can see it in the browser.
If you really want to automate browser-based interactions, you could go further and use Watij, which runs a browser via the JVM and is driven via a browser-based API (I.e. you identify the button you want to press and it will actually do this)
Otherwise a library like the one you've identified will normally work. You have to watch out for client-side JavaScript interactions driving the requests, and configure proxies etc (I suspect this is your problem in the above)
I need to talk to an obscure webserver which requires authentication. If I don't supply credentials, a login form is displayed. However, if I do supply unsolicited Basic Authentication credentials, I get directly to the desired content.
wget supports this directly:
# this fails and downloads a form:
wget https://weird.egg/data.txt --http-user=me --http-password=shhh
# this works and downloads the document:
wget https://weird.egg/data.txt --http-user=me --http-password=shhh --auth-no-challenge
Now my question: How can I make the download in Java using Apache's HTTP Components?
Here's what I got so far. (There's also a proxy in place, and I use -Y on in wget, and I have a matching https_proxy environment variable.)
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.conn.params.ConnRoutePNames;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import java.net.URI;
// ...
DefaultHttpClient hc = new DefaultHttpClient();
hc.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, new HttpHost(proxy_name, proxy_port));
URI uri = new URI("https://weird.egg/data.txt");
hc..getCredentialsProvider().setCredentials(new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, AuthScope.ANY_REALM, AuthScope.ANY_SCHEME), new UsernamePasswordCredentials("me", "shh"));
hc.execute(new HttpGet(uri)); // etc
However, I only end up with the login form page, not the actual document. I'm suspecting that the DefaultHttpClient isn't sending the credentials unsolicited, in the way that wget does. Is there a way to make the Java program send the credentials?
Never mind. I solved the problem by not trying to use any library authentication methods, but just brute-forcing the Basic Authentication header into the request:
HttpGet get = new HttpGet(uri);
String basic_auth = new String(Base64.encodeBase64((username + ":" + password).getBytes()));
get.addHeader("Authorization", "Basic " + basic_auth);
hc.execute(get); // etc
(This needs the additional import org.apache.commons.codec.binary.Base64;, but in turn we can remove the credential-related imports.)
How to make a cross-domain call from GWT?
I found JSONPRequestBuilder as a solution, but it can only create GET request not POST. I am trying to call URL shortner service ("http://goo.gl/api/shorten") of google.
From servlet on GAE you can call external http services via URLFetch.
From client side GWT you can directly call Google Shortener API via gwt-google-apis. See the shortener example at the end of page.
Got it through URLFetch. Below is my code:
//Classes to import
import com.google.appengine.api.urlfetch.HTTPMethod;
import com.google.appengine.api.urlfetch.HTTPRequest;
import com.google.appengine.api.urlfetch.HTTPResponse;
import com.google.appengine.api.urlfetch.URLFetchService;
import com.google.appengine.api.urlfetch.URLFetchServiceFactory;
//Shortening download URL
URL url=new URL("http://goo.gl/api/shorten");
HTTPRequest req=new HTTPRequest(url,HTTPMethod.POST);
req.setPayload(("url=www.google.com").getBytes());
URLFetchService service = URLFetchServiceFactory.getURLFetchService();
HTTPResponse response = service.fetch(req);
byte[] content = response.getContent();
String urlshort=new String(content); //here is the JSON data from goo.gl
im trying to implement a simple comet app to just send data and recive data, my client side is written on java and the server is in node.js, im trying to implement it from the client side with HttpUrlConnection but it seems that when i try to write to the server it doesnt respond me. so how can code the server to respond? (currently using http.createServer(function(req, res){...}).listen(8124);
I'm doing something similar to that with an Android app. I can't say that this is the "right" way to do it, but I'm using these classes in my project (to make POSTs and check the responses):
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
To implement I'm doing something to the effect of
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(/*...(String) url...*/);
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
nameValuePairs.add(new BasicNameValuePair("key", "value"));
httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = httpclient.execute(httppost); // <-- this has useful info
To actually get useful stuff from the server I'm using JSON because, IMHO, XML parsing is a huge pain in Java.
So these are the classes I use for parsing the JSON that nodejs spits out at me.
import org.json.JSONArray;
import org.json.JSONObject;
...
JSONObject jObject = new JSONObject(EntityUtils.toString(response.getEntity()));
JSONArray datasets = jObject.getJSONArray("blahblahblah");
Having used it (quite easily) in my own Android app, I recommend Socket.IO-client Java.
It's a "full-featured Socket.IO Client Library for Java, which is compatible with Socket.IO v1.0 and later."