Reading over HTTP & (hopefully) automating actions using Java - java

Is there a way to make get & put calls over HTTP in java ? I also need to automate any user inputs like a button click on the target web-page(any web-page, not just yahoo finance)
I tried using the apache commons library & couldn't quite crack it:
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
public class Fin {
/**
* #param args
*/
public static void main(String[] args) {
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet("http://finance.yahoo.com");
try {
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
} catch (Exception e) {
e.printStackTrace();
} finally {
httpget.releaseConnection();
}
}
}
I keep getting 'java.net.ConnectException: Connection refused', though i can see it in the browser.

If you really want to automate browser-based interactions, you could go further and use Watij, which runs a browser via the JVM and is driven via a browser-based API (I.e. you identify the button you want to press and it will actually do this)
Otherwise a library like the one you've identified will normally work. You have to watch out for client-side JavaScript interactions driving the requests, and configure proxies etc (I suspect this is your problem in the above)

Related

Connection Failed: Timeout for Testing API using Java, Apache HTTP Client

I am trying to test API using Java. I am using Java 8, Apache HTTP client 4.5.3 to test it. I tried many different ways to testing using Java .net class, Apache HTTP client but every time same issue;
Exception in thread "main"
org.apache.http.conn.HttpHostConnectException: Connect to
api.github.com:443 [api.github.com/192.30.253.116,
api.github.com/192.30.253.117] failed: Connection timed out: connect
at
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159)
Everytime I am getting time out. But if I use same URL in Browser I am getting result.
Can someone help me to point out issue? Whether its setup issue or code issue?
Tried almost all codes available on internet. I am beginner for API testing and don't have knowledge of in depth of HTTP workflow.
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpUriRequest;
import org.apache.http.impl.client.HttpClientBuilder;
import java.io.IOException;
import java.net.*;
public class API {
public static void main(String args[]) throws IOException, URISyntaxException {
HttpUriRequest request = new HttpGet( "https://api.github.com" );
// When
HttpResponse response = HttpClientBuilder.create().build().execute( request );
System.out.println(response.getStatusLine().getStatusCode());
}
}
Using Java .net package
import java.io.IOException;
import java.net.*;
public class API {
public static void main(String args[]) throws IOException, URISyntaxException {
URL url = new URL("http://maps.googleapis.com/maps/api/geocode/json?address=chicago&sensor=false");
//URL url = uri.toURL();
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/xml");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("HTTP error code : "
+ conn.getResponseCode());
}
}
}
If the same URL works in browser then there are only three possibilities.
The URL expects headers like User-Agent. You can set request headers needed like this:
request.setHeader("User-Agent", "Mozilla");
You are in a corporate or restricted environment and need a proxy to connect to external URLs. Your browser might already be setup to use proxy server. In this case, you will need to pass proxy credentials to http client API.
Example: https://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientProxyAuthentication.java
All outgoing requests are blocked in your environment by firewall or something. In this case, you will need to ask your network admin to allow network connection.

HttpGet sends request with the wrong encoding

I'm trying to get the text response from the following URL:
http://translate.google.cn/translate_a/single?client=t&sl=zh-CN&tl=en&dt=t&tk=265632.142896&q=%E4%BD%A0%E5%A5%BD
The response is the following:
[[["Hello there","你好",,,1]],,"zh-CN"]
(You can verify this response by entering the address into your browser.)
Here is a simplified version of my code that tries to download this text:
import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
public class Test {
public static String downloadString() {
String url = "http://translate.google.cn/translate_a/single?client=t&sl=zh-CN&tl=en&dt=t&tk=265632.142896&q=%E4%BD%A0%E5%A5%BD";
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet(url);
ResponseHandler<String> handler = new BasicResponseHandler();
try {
return client.execute(request, handler);
} catch (Exception e) {
return "GET request failed.";
}
}
}
When I call Test.downloadString(), I get the following (incorrect) response:
[[["Huan Chai Sunsolt","浣犲ソ",,,0]],,"zh-CN"]
I'm guessing that there is some sort of encoding problem behind the scenes somewhere in the request process (there are six bytes that should be interpreted as two Chinese characters, but are instead interpreted as three Japanese characters), but I can't seem to pinpoint the exact cause. What am I doing wrong in my code?
It's strange, but adding the User-Agent header fixed the problem:
request.addHeader("User-Agent", "Mozilla/5.0 (X11; Linux x86_64; rv:33.0) Gecko/20100101 Firefox/33.0");
Android 6.0 release removes support for the Apache HTTP client. If your app is using this client and targets Android 2.3 (API level 9) or higher, use the HttpURLConnection class instead.
here: http://developer.android.com/about/versions/marshmallow/android-6.0-changes.html#behavior-apache-http-client

Apache Http Client in Android

I'm trying to get the whole html from a web page in Android.
In java console aplication I used to do like this:
DefaultHttpClient httpclient = new DefaultHttpClient();
String busca = "kindle";
HttpGet httpGet = new HttpGet("http://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords="+busca);
try {
ResponseHandler<String> manipulador = new BasicResponseHandler();
String resposta = httpclient.execute(httpGet,manipulador);
}
} finally {
httpGet.releaseConnection();
}
I tried to do the same in my Android aplication but I didn't work!
This library works in Android?
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
Is there in better way to get a page html code in a string on Android?
Thks for the help!
I did it with another URl and it worked :)
Maybe the HTML code of that page I was using as to big to save in a String or to show in a Text
You can have a loot at this :
HttpClient 4.0.1 - how to release connection?
HttpRequestBase.releaseConnection() is introduced in Version 4.2

Java class for embedded HTTP server in Swing app

I wish to embed a very light HTTP server in my Java Swing app which just accepts requests, performs some actions, and returns the results.
Is there a very light Java class that I can use in my app which listens on a specified port for HTTP requests and lets me handle requests?
Note, that I am not looking for a stand-alone HTTP server, just a small Java class which I can use in my app.
Since Java 6, the JDK contains a simple HTTP server implementation.
Example usage:
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.util.Iterator;
import java.util.List;
import java.util.Set;
import java.util.concurrent.Executors;
import com.sun.net.httpserver.Headers;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
public class HttpServerDemo {
public static void main(String[] args) throws IOException {
InetSocketAddress addr = new InetSocketAddress(8080);
HttpServer server = HttpServer.create(addr, 0);
server.createContext("/", new MyHandler());
server.setExecutor(Executors.newCachedThreadPool());
server.start();
System.out.println("Server is listening on port 8080" );
}
}
class MyHandler implements HttpHandler {
public void handle(HttpExchange exchange) throws IOException {
String requestMethod = exchange.getRequestMethod();
if (requestMethod.equalsIgnoreCase("GET")) {
Headers responseHeaders = exchange.getResponseHeaders();
responseHeaders.set("Content-Type", "text/plain");
exchange.sendResponseHeaders(200, 0);
OutputStream responseBody = exchange.getResponseBody();
Headers requestHeaders = exchange.getRequestHeaders();
Set<String> keySet = requestHeaders.keySet();
Iterator<String> iter = keySet.iterator();
while (iter.hasNext()) {
String key = iter.next();
List values = requestHeaders.get(key);
String s = key + " = " + values.toString() + "\n";
responseBody.write(s.getBytes());
}
responseBody.close();
}
}
}
Or you can use Jetty for that purpose. It’s quite lightweight and perfectly fits this purpose.
You can use jetty as embedded server, its fairly light weight. Other option is check this out for a simple java class to handle http requests http://java.sun.com/developer/technicalArticles/Networking/Webserver/.
Other way is in Java 6 you can use com.sun.net.httpserver.HttpServer
Sun embedded web server is useful, but com.sun.net package could be dropped without notice.
A better alternative are
http://tjws.sourceforge.net/ 100kb very small and jdk 1.6-aware
http://winstone.sourceforge.net/ bigger but a good shot
http://www.eclipse.org/jetty/ Jetty, very good in developement, support SPDY and websocket
If you're not using Java 6, then I would certainly recommend Jetty. That works very well and has a decent programming interface.
You said "very light" twice, so I think JLHTTP might be a good match for you. You can embed it as a single source file or a ~35K/50K jar file, yet it supports most functionality you'd need in an HTTP server out of the box.
Disclaimer: I'm the author. But check it out for yourself and see what you think :-)

Calling a Web Service from Seam

A simple question, but could someone provide sample code as to how would someone call a web service from within the JBoss Seam framework, and process the results?
I need to be able to integrate with a search platform being provided by a private vendor who is exposing his functionality as a web service. So, I'm just looking for some guidance as to what the code for calling a given web service would look like.
(Any sample web service can be chosen as an example.)
There's roughly a gajillion HTTP client libraries (Restlet is quite a bit more than that, but I already had that code snippet for something else), but they should all provide support for sending GET requests. Here's a rather less featureful snippet that uses HttpClient from Apache Commons:
HttpClient client = new HttpClient();
HttpMethod method = new GetMethod("http://api.search.yahoo.com/WebSearchService/V1/webSearch?appid=restbook&query=HttpClient");
client.executeMethod(method);
import org.restlet.Client;
import org.restlet.data.Protocol;
import org.restlet.data.Reference;
import org.restlet.data.Response;
import org.restlet.resource.DomRepresentation;
import org.w3c.dom.Node;
/**
* Uses YAHOO!'s RESTful web service with XML.
*/
public class YahooSearch {
private static final String BASE_URI = "http://api.search.yahoo.com/WebSearchService/V1/webSearch";
public static void main(final String[] args) {
if (1 != args.length) {
System.err.println("You need to pass a search term!");
} else {
final String term = Reference.encode(args[0]);
final String uri = BASE_URI + "?appid=restbook&query=" + term;
final Response response = new Client(Protocol.HTTP).get(uri);
final DomRepresentation document = response.getEntityAsDom();
document.setNamespaceAware(true);
document.putNamespace("y", "urn:yahoo:srch");
final String expr = "/y:ResultSet/y:Result/y:Title/text()";
for (final Node node : document.getNodes(expr)) {
System.out.println(node.getTextContent());
}
}
}
}
This code uses Restlet to make a request to Yahoo's RESTful search service. Obviously, the details of the web service you are using will dictate what your client for it looks like.
final Response response = new Client(Protocol.HTTP).get(uri);
So, if I understand this correctly, the above line is where the actual call to the web service is being made, with the response being converted to an appropriate format and manipulated after this line.
Assuming I were not using Restlet, how would this line differ?
(Of course, the actual processing code would be significantly different as well, so that's a given.)

Categories