I'm trying to parse an html document in a webservice. According to google, jsoup seem to be the faster and easier html parser, so I included in my project but I get the exception "Exception: java.lang.reflect.InvocationTargetException Message: java.lang.reflect.InvocationTargetException" I have tried everything, but nothings give results. Please help
I add jsoup.jar in my project's libray classpath.
I am using Eclipse Luna on Windows XP
Java 1.7 apache tomcat 7.0
this is my code:
try {
url = new URL("http://consulta.muniguate.com/emetra/despliega.php?tplaca="+tplaca+"&nplaca="+nplaca);
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
while ((line = rd.readLine()) != null) {
result += line;
}
Document doc = Jsoup.connect(result).get();
String title= doc.title();
System.out.println(title);
rd.close();
} catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
This is the full code:
package clases;
import java.io.BufferedReader;
import javax.jws.WebService;
import javax.jws.WebMethod;
import javax.jws.WebParam;
import java.io.IOException;
import java.io.InputStreamReader;
import java.lang.reflect.InvocationTargetException;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URL;
import org.jsoup.Connection.Method;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
#WebService(serviceName = "Transito")
public class Transito {
#WebMethod(operationName = "consultar_saldo")
public String consultar_saldo(String tplaca, int nplaca) throws InvocationTargetException {
String result = "";
try {
Document doc= Jsoup.connect("http://www.muniguate.com/utilities/remisiones.htm?tplaca="+tplaca+"&nplaca="+nplaca).userAgent("Mozilla").get();
String result = doc.title();
System.out.println(result);
} catch (Exception e){
e.getCause();
}
return result;
}
}
Jsoup.connect() accepts a url string, not response content
"I have tried everything, but nothings give results."
In that case you are doomed since there is nothing left for us to try.
But lets assume that you didn't try everything. Lets assume that documentation of Jsoup.connect() is actually telling the true, and this method is used only to create Connection to resource which should be parsed, not to parse it. Its get() method job to connect to resource from created Connection, parse it and return it as Document.
So this method instead of HTML text of resource, will need information required for connection like URL.
So instead of manually creating HttpURLConnection and reading its HTML code, pass string representing URL to Jsoup.connect() and then using get() connect and parse to this resource.
So instead of
URL url = new URL("http://consulta.muniguate.com/emetra/despliega.php?tplaca="+tplaca+"&nplaca="+nplaca);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
BufferedReader rd = new BufferedReader(new InputStreamReader(
conn.getInputStream()));
String line = null;
String result = "";
while ((line = rd.readLine()) != null) {
result += line;
}
Simply use
Document doc = Jsoup.connect("http://consulta.muniguate.com/emetra/despliega.php?tplaca="+tplaca+"&nplaca="+nplaca).get();
Now you should be able to use
String title = doc.title();
System.out.println(title);
Related
The requirement is to validate whether zip code really exists or not in a specific country??
Country = US
State = California
I wants to know how to call geonames API using REST to get country code by zip code? I would prefer an output in a JSON format over XML.
Answer :
Refer
http://www.mkyong.com/webservices/jax-rs/restfull-java-client-with-java-net-url/
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
public class NetClientGet {
// http://localhost:8080/RESTfulExample/json/product/get
public static void main(String[] args) {
try {
URL url = new URL("http://api.geonames.org/postalCodeSearchJSON?postalcode=94536&maxRows=1&username=demo&country=US");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ conn.getResponseCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader(
(conn.getInputStream())));
String output;
System.out.println("Output from Server .... \n");
while ((output = br.readLine()) != null) {
System.out.println(output);
}
conn.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Thanks Mkyong!
They have a file a country info file,There are regexes for about 150 countries.
You need to create GeoNames model. This model will handle creating, storing, and returning the value of our GeoNames WebServices request.
For more info - http://www.geonames.org/export/web-services.html
So in a nutshell, I'm just trying to get a small working skeleton program that I can use to sort of learn about Http communication and "feel" my way around to figure out what I will eventually need for a bigger program I am working on. This particular code here is actually just a chopped up version of an example from the Apache libraries. I could compile the examples listed on the Apache website, but they didn't run properly, giving a "java.net.ConnectException". I figured it had to do with Windows c-blocking a program like this from making a connection, and that I would need to run it as an administrator. I then tried taking the code and throwing it into an executable jar file, but I get a Cannot-find-or-load-main-class error. Am I an idiot or is the Apache library a little outdated/not fit for Win 8/something else?
Code below:
package NewProject;
import java.net.Socket;
import org.apache.http.ConnectionReuseStrategy;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.impl.DefaultBHttpClientConnection;
import org.apache.http.impl.DefaultConnectionReuseStrategy;
import org.apache.http.message.BasicHttpRequest;
import org.apache.http.protocol.HttpCoreContext;
import org.apache.http.protocol.HttpProcessor;
import org.apache.http.protocol.HttpProcessorBuilder;
import org.apache.http.protocol.HttpRequestExecutor;
import org.apache.http.protocol.RequestConnControl;
import org.apache.http.protocol.RequestContent;
import org.apache.http.protocol.RequestExpectContinue;
import org.apache.http.protocol.RequestTargetHost;
import org.apache.http.protocol.RequestUserAgent;
import org.apache.http.util.EntityUtils;
class NewProject
{
public static void main(String[] args) throws Exception
{
HttpProcessor httpproc = HttpProcessorBuilder.create()
.add(new RequestContent())
.add(new RequestTargetHost())
.add(new RequestConnControl())
.add(new RequestUserAgent("Test/1.1"))
.add(new RequestExpectContinue(true)).build();
HttpRequestExecutor httpexecutor = new HttpRequestExecutor();
HttpCoreContext coreContext = HttpCoreContext.create();
HttpHost host = new HttpHost("localhost", 8080);
coreContext.setTargetHost(host);
Out os = new Out("TestOut.txt");
DefaultBHttpClientConnection conn = new DefaultBHttpClientConnection(8 * 1024);
ConnectionReuseStrategy connStrategy = DefaultConnectionReuseStrategy.INSTANCE;
try
{
String[] targets =
{
"http://www.google.com/"
};
for (int i = 0; i < targets.length; i++)
{
if (!conn.isOpen())
{
Socket socket = new Socket(host.getHostName(), host.getPort());
conn.bind(socket);
}
BasicHttpRequest request = new BasicHttpRequest("GET", targets[i]);
os.println(">> Request URI: " + request.getRequestLine().getUri());
httpexecutor.preProcess(request, httpproc, coreContext);
HttpResponse response = httpexecutor.execute(request, conn, coreContext);
httpexecutor.postProcess(response, httpproc, coreContext);
os.println("<< Response: " + response.getStatusLine());
os.println(EntityUtils.toString(response.getEntity()));
os.println("==============");
if (!connStrategy.keepAlive(response, coreContext))
{
conn.close();
}
else
{
os.println("Connection kept alive...");
}
}
}
catch (IndexOutOfBoundsException iob)
{
os.println("What happened here?");
}
finally
{
conn.close();
}
return;
}
}
... they didn't run properly, giving a "java.net.ConnectException"
That could be caused by lots of things. There are clues in the exception message ... which you chose not to share with us.
... "Cannot find or load Main-Class"
Again multiple possible causes, and there are clues in the exception message ... which you chose not to share with us.
But the fact that you have created a JAR file plus the "Main-Class" hint in the error message fragment you provided suggest that you made a mistake in the creation of the JAR file; i.e. you used the wrong name for the "Main-Class" attribute.
Given that source code, the "Main-Class" attribute should be "NewProject.NewProject". I suspect you set it to something else.
A second possibility is that you haven't handled the dependency on the Apache library correctly. The Apache classes need to be on the classpath specified by the JAR file. (You can't use a -cp argument or $CLASSPATH when you launch with java -jar.)
Am I an idiot or is the Apache library a little outdated/not fit for Win 8/something else?
There is nothing wrong with the Apache library.
The code you posted seems a little low level (e.g. interacting directly with Socket connections). The code posted below should give you what it sounds like you are looking for. The classes used also give you a lot of inroads into setting and getting http parameters (e.g. headers, time-outs, etc).
package org.yaorma.example.http.client;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.URL;
public class HttpClientExample {
public static void main(String[] args) throws Exception {
String response;
response = get("http://www.google.com");
System.out.println("RESPONSE FROM GET -----------------------------------------");
System.out.println(response);
response = post("http://httpbin.org/post", "This is the message I posted to httpbin.org/post");
System.out.println("RESPONSE FROM POST -----------------------------------------");
System.out.println(response);
}
/**
* Method to post a request to a given URL.
*/
public static String post(String urlString, String message) {
try {
// get a connection
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
// set the parameters
conn.setRequestMethod("POST");
conn.setDoOutput(true);
// send the message
OutputStream os = conn.getOutputStream();
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(os, "UTF-8"));
writer.write(message);
writer.flush();
writer.close();
os.close();
// get the response
conn.connect();
InputStream content = (InputStream) conn.getInputStream();
// read the response
BufferedReader in = new BufferedReader(new InputStreamReader(content));
String rtn = "";
String line;
while ((line = in.readLine()) != null) {
rtn += line + "\n";
}
return rtn;
} catch (Exception exp) {
throw new RuntimeException(exp);
}
}
/**
* Method to do a get from a given URL.
*/
public static String get(String urlString) {
try {
// get a connection
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
// set the parameters
conn.setRequestMethod("GET");
conn.setDoOutput(true);
// get the response
conn.connect();
InputStream content = (InputStream) conn.getInputStream();
// read the response
BufferedReader in = new BufferedReader(new InputStreamReader(content));
String rtn = "";
String line;
while ((line = in.readLine()) != null) {
rtn += line + "\n";
}
return rtn;
} catch (Exception exp) {
throw new RuntimeException(exp);
}
}
}
When I type this following URL into my browser, Bugzilla answers with XML:
http://bugzilla.mycompany.local/buglist.cgi?ctype=rdf&bug_status=CONFIRMED&product=MyProduct
I want to process this XML in a Java program. But when I use the exact same URL in my Java program, Bugzilla answers with HTML instead of XML.
This is my program:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
public class Test {
public static void main(String[] args)
throws IOException {
URL url = new URL("http://bugzilla.mycompany.local/buglist.cgi?ctype=rdf&bug_status=CONFIRMED&product=MyProduct");
URLConnection connection = url.openConnection();
final StringBuilder response = new StringBuilder(1024);
try(InputStreamReader isr = new InputStreamReader(connection.getInputStream())) {
try(BufferedReader reader = new BufferedReader(isr)) {
String inputLine = null;
while((inputLine = reader.readLine()) != null) {
response.append(inputLine);
response.append('\n');
}
}
}
System.out.println(response);
}
}
What am I doing wrong?
The resulting HTML is not the result of the query. It's Bugzillas log-in form. Duh!
I am developing a Java application to be the server in a Google Cloud Messaging Android app.
I have been following a tutorial and I managed to do rest of the tutorial with out a trouble.
My Java application has three classes which are Content.java, POST2GCM.java, App.java. These classes do what the name describes.
Content.java class is below.
package com.hmkcode.vo;
import java.io.Serializable;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
public class Content implements Serializable {
private List<String> registration_ids;
private Map<String,String> data;
public void addRegId(String regId){
if(registration_ids == null)
registration_ids = new LinkedList<String>();
registration_ids.add(regId);
}
public void createData(String title, String message){
if(data == null)
data = new HashMap<String,String>();
data.put("title", title);
data.put("message", message);
}
}
App.java class is below
package com.hmkcode.vo;
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.hmkcode.vo.Content;
public class App
{
public static void main( String[] args )
{
System.out.println( "Sending POST to GCM" );
String apiKey = "AIzaSyB8azikXJKi_NjpWcVNJVO0d........";
Content content = createContent();
POST2GCM.post(apiKey, content);
}
public static Content createContent(){
Content c = new Content();
c.addRegId("APA91bFqnQzp0z5IpXWdth1lagGQZw1PTbdBAD13c-UQ0T76BBYVsFrY96MA4SFduBW9RzDguLaad-7l4QWluQcP6zSoX1HSUaAzQYSmI93....");
c.createData("Test Title", "Test Message");
return c;
}
}
POST2GCM.java class is below
package com.hmkcode.vo;
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import com.fasterxml.jackson.databind.*;
public class POST2GCM {
public static void post(String apiKey, Content content){
try{
// 1. URL
URL url = new URL("https://android.googleapis.com/gcm/send");
// 2. Open connection
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
// 3. Specify POST method
conn.setRequestMethod("POST");
// 4. Set the headers
conn.setRequestProperty("Content-Type", "application/json");
conn.setRequestProperty("Authorization", "key="+apiKey);
conn.setDoOutput(true);
// 5. Add JSON data into POST request body
//`5.1 Use Jackson object mapper to convert Content object into JSON
ObjectMapper mapper = new ObjectMapper();
// 5.2 Get connection output stream
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
// 5.3 Copy Content "JSON" into
mapper.writeValue(wr,content);
// 5.4 Send the request
wr.flush();
// 5.5 close
wr.close();
// 6. Get the response
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
// 7. Print result
System.out.println(response.toString());
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
The problem arises in the POST2GCM.java class, in the line
mapper.writeValue(wr,content);
Where the suggestions are to add try catch block,Add exception to method signature, Add catch clauses(s).
I did all the suggestions which did not solve the problem.
What would be the problem here?
You need to add the jackson-core-2.4.3.jar library file to your project.
Add it to your java build path too.
Of course ... 2.4.3 is the version I used, but it should work with previous versions.
I have set up a login area on my PHP server. The members.php file requires login; after I login a session is created. The session lasts for a while. I want to make this work so that while the session is still valid the java applet should be able to access members.php page.
I have embedded the Java applet into the members.php page. It makes a HttpURLConnection request, however when I get the response I find that it was redirected by the PHP server to the login page.
How do I set this up correctly?
Here is the Java Applet code:
import java.applet.Applet;
import java.awt.Dimension;
import java.io.BufferedReader;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import java.util.List;
import java.util.Map;
import javax.swing.JFrame;
import javax.swing.JTextArea;
public class phpConnectApplet extends Applet {
private static final long serialVersionUID = 1L;
public void init() {
URL url = null;
try {
url = new URL("http://www.example.com/members.php");
URLConnection urlConn = url.openConnection();
HttpURLConnection httpConn = (HttpURLConnection) urlConn;
httpConn.setDoOutput(true);
httpConn.setDoInput(true);
httpConn.setUseCaches(false);
httpConn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
DataOutputStream output = new DataOutputStream(httpConn.getOutputStream());
String content = "action=blah"; //just to test the PHP file
output.writeBytes(content);
output.flush();
output.close();
DataInputStream in = new DataInputStream(urlConn.getInputStream());
BufferedReader input = new BufferedReader(new InputStreamReader(in));
String str, result = "";
while ((str = input.readLine()) != null) {
result = result + str + "\n";
}
input.close();
Map<String, List<String>> headers = httpConn.getHeaderFields();
List<String> values = headers.get("Set-Cookie");
String cookieValue = null;
for (String v:values) {
if (cookieValue == null)
cookieValue = v;
else
cookieValue = cookieValue + ";" + v;
}
System.out.println(cookieValue);
JFrame f = new JFrame("App Title");
f.add(new JTextArea(result));
f.setMaximumSize(new Dimension(400,300));
f.pack();
f.setLocationRelativeTo(null);
f.setVisible(true);
} catch (MalformedURLException me) {
me.printStackTrace();
} catch(IOException ie) {
ie.printStackTrace();
} catch(Exception e) {
e.printStackTrace();
}
}
}
The output for this is a JFrame with one JTextField which contains the output HTML of the login page.
You need to capture the value of your PHPSESSID cookie (or whatever you are using for the cookie name) and add it to the request in the HttpURLConnection. This MAY come through as a system parameter, but if not, you can embed the session ID as an applet attribute on the page launching the applet. (I haven't experimented with this part specifically)
Here's a tutorial that explains how to send cookies in the URLConnection class: http://www.hccp.org/java-net-cookie-how-to.html
Specifically see the section titled Setting a cookie value in a request.