We Keep Coding

How to execute rest api from java and capture the response?

Executing this url on my browser: http://localhost:3161/devices/simulator/stop
I don't need login for it. It returns this rest api xml:
<response>
<type>response</type>
<ts>1463749194000</ts>
<status>OK</status>
<msg-version>2.3.0</msg-version>
<op>stop</op>
<data/>
</response>
How can I execute this from JAVA and then capture the xml response?

As other mentioned in this post, it is generic thing, you would be able to find it online already..
I know there are clients to call the REST services from Java. Two of them are listed for your case.
case -1 : if you are using Jersey REST API. Here to capture XML , you can go with your own way,for example use JAXB and XML elements to Java Bean properties.
import java.io.IOException;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.UriBuilder;
import org.apache.http.client.ClientProtocolException;
import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.WebResource;
import com.sun.jersey.api.client.config.ClientConfig;
import com.sun.jersey.api.client.config.DefaultClientConfig;
public class Test {
public static void main(String[] args) throws ClientProtocolException, IOException {
ClientConfig config = new DefaultClientConfig();
Client client = Client.create(config);
WebResource service = client.resource(UriBuilder.fromUri('http://localhost:3161/devices/simulator/stop').build());
// getting XML data
System.out.println(service. path('restPath').path('resourcePath').accept(MediaType.APPLICATION_JSON).get(String.class));
// getting JSON data
System.out.println(service. path('restPath').path('resourcePath').accept(MediaType.APPLICATION_XML).get(String.class));
}
}
Case 2: using HTTP method, it is simple method but parse XML instead of printing it here
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
public class Test {
public static void main(String[] args) throws ClientProtocolException, IOException {
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet('http://localhost:3161/devices/simulator/stop');
HttpResponse response = client.execute(request);
BufferedReader rd = new BufferedReader (new InputStreamReader(response.getEntity().getContent()));
String line = '';
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
}
}

cannot download csv files from morningstar.com using authentication

I have a premium account with morningstar and I tried to download a few csv files from the premium content area. For some reason I cannot get those premium content. For example, with premium account I can get 10 year financial statement data, but I've tried all the sample authentication java code from apache httpcomponents-client. All of them can only get me content that does not need authentication. How can I tell what authentication protocol morningtar is using and authenticate successfully? I tried the example code from org.apache.http.examples.client, including clientAuthentication.java, clientKerberosAuthentication.java, clientInteractiveAuthentication.java . If I log in in morningstar account in Chrome and paste this URL, I can get 10 years data csv, but if I access through java I only get 5 years data. Below are one of sample codes I tried. I didn't get exceptions or errors, but I only got 5 years data instead of 10.
import java.io.BufferedReader;
import java.io.InputStreamReader;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
public class ClientAuthentication {
public static void main(String[] args) throws Exception {
DefaultHttpClient httpclient = new DefaultHttpClient();
try {
httpclient.getCredentialsProvider().setCredentials(
new AuthScope("morningstar.com", 443),
new UsernamePasswordCredentials("xxx#gmail.com", "xxxx")); //anonymized this before posting to stackoverflow
HttpGet httpget = new HttpGet("http://financials.morningstar.com/ajax/ReportProcess4CSV.html?t=aapl&region=usa&culture=en_US&reportType=is&period=12&dataType=A&order=asc&columnYear=10&rounding=3&view=raw&productCode=USA&r=199209&denominatorView=raw&number=3");
System.out.println("executing request" + httpget.getRequestLine());
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
BufferedReader in;
in = new BufferedReader(new InputStreamReader(entity.getContent()));
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
if (entity != null) {
System.out.println("Response content length: " + entity.getContentLength());
int linenum = 0;
while (true){
String line = in.readLine();
if (line == null) break;
linenum++;
if (linenum>1)
System.out.println(line);
}
}
EntityUtils.consume(entity);
} finally {
// When HttpClient instance is no longer needed,
// shut down the connection manager to ensure
// immediate deallocation of all system resources
httpclient.getConnectionManager().shutdown();
}
}
}

sending parameters across applications

Is there a way I can send attributes across applications that may or may not be on the same machine ?
For example :
// IN APPLICATION 1 (APP-1)
request.setAttribute("Truth","Ghazal is the food for the soul of separation");
RequestDispatcher rd = request.getRequestDispatcher("http://IP/App-2/servlet");
rd.forward(request,response);
// IN APPLICATION 2'S (APP-2) SERVLET
String truth = request.getAttribute("Truth").toString();
// NOW USE THIS STRING
Let us suppose that IP on which app-1 is deployed is not the same as the IP on which the app-2 is deployed.
Is there any way I can send parameters like these across applications that are hosted far away from each other ? When I tried I couldn't do this way,but may be there is a way around.
Both the applications use Tomcat.

If you are going to be sharing state between a variable number of machines, then using HTTP as the method to store that state is not very reliable.
"Attributes" are not transmitted over HTTP, they are merely shared state that reside on the application for the given session. Attributes are 100% purely server-side information.
From the Javadocs:
"It is warned that when the request is dispatched from the servlet
resides in a different web application by RequestDispatcher, the
object set by this method may not be correctly retrieved in the caller
servlet."

you can create a base package to be used through the application
package base;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.ArrayList;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
public class GetXMLTask
{
static double longitute;
static double latitude;
public ArrayList<JSONObject> getOutputFromUrl1(String url)
{
ArrayList<JSONObject> output = new ArrayList<JSONObject>();
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(url);
HttpResponse response;
StringBuilder builder= new StringBuilder();
JSONObject myjson ;
JSONArray the_json_array;
try
{
response = httpClient.execute(httpPost);
BufferedReader in = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
char[] buf = new char[8000];
int l = 0;
while (l >= 0)
{
builder.append(buf, 0, l);
l = in.read(buf);
}
myjson = new JSONObject("{child:"+builder.toString()+"}");
JSONObject mmm = new JSONObject(builder.toString());
JSONArray mmmArr = mmm.getJSONArray("status");
the_json_array = myjson.getJSONArray("child");
for (int i = 0; i < the_json_array.length(); i++)
{
JSONObject another_json_object = the_json_array.getJSONObject(i);
output.add(another_json_object);
}
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}
return output;
}
}
now from your application call this method by
ArrayList<JSONObject> obj = new GetXMLTask().getOutputFromUrl1("url for the other application method which responds");

How to consume REST in Java [duplicate]

This question already has answers here:
How do you create a REST client for Java? [closed]
(18 answers)
Closed 2 years ago.
Using Java tools,
wscompile for RPC
wsimport for Document
etc..
I can use WSDL to generate the stub and Classes required to hit the SOAP Web Service.
But I have no idea how I can do the same in REST.
How can I get the Java classes required for hitting the REST Web Service.
What is the way to hit the service anyway?
Can anyone show me the way?

Working example, try this:
package restclient;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class NetClientGet {
public static void main(String[] args) {
try {
URL url = new URL("http://localhost:3002/RestWebserviceDemo/rest/json/product/dynamicData?size=5");//your url i.e fetch data from .
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP Error code : "
+ conn.getResponseCode());
}
InputStreamReader in = new InputStreamReader(conn.getInputStream());
BufferedReader br = new BufferedReader(in);
String output;
while ((output = br.readLine()) != null) {
System.out.println(output);
}
conn.disconnect();
} catch (Exception e) {
System.out.println("Exception in NetClientGet:- " + e);
}
}
}

As others have said, you can do it using the lower level HTTP API, or you can use the higher level JAXRS APIs to consume a service as JSON. For example:
Client client = ClientBuilder.newClient();
WebTarget target = client.target("http://host:8080/context/rest/method");
JsonArray response = target.request(MediaType.APPLICATION_JSON).get(JsonArray.class);

Its just a 2 line of code.
import org.springframework.web.client.RestTemplate;
RestTemplate restTemplate = new RestTemplate();
YourBean obj = restTemplate.getForObject("http://gturnquist-quoters.cfapps.io/api/random", YourBean.class);
Ref. Spring.io consuming-rest

The code below will help to consume rest api via Java.
URL - end point rest
If you dont need any authentication you dont need to write the authStringEnd variable
The method will return a JsonObject with your response
public JSONObject getAllTypes() throws JSONException, IOException {
String url = "/api/atlas/types";
String authString = name + ":" + password;
String authStringEnc = new BASE64Encoder().encode(authString.getBytes());
javax.ws.rs.client.Client client = ClientBuilder.newClient();
WebTarget webTarget = client.target(host + url);
Invocation.Builder invocationBuilder = webTarget.request(MediaType.APPLICATION_JSON).header("Authorization", "Basic " + authStringEnc);
Response response = invocationBuilder.get();
String output = response.readEntity(String.class
);
System.out.println(response.toString());
JSONObject obj = new JSONObject(output);
return obj;
}

Just make an http request to the required URL with correct query string, or request body.
For example you could use java.net.HttpURLConnection and then consume via connection.getInputStream(), and then covnert to your objects.
In spring there is a restTemplate that makes it all a bit easier.

If you also need to convert that xml string that comes as a response to the service call, an x object you need can do it as follows:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.StringReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;
import javax.xml.bind.JAXB;
import javax.xml.bind.JAXBException;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import org.w3c.dom.CharacterData;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
public class RestServiceClient {
// http://localhost:8080/RESTfulExample/json/product/get
public static void main(String[] args) throws ParserConfigurationException,
SAXException {
try {
URL url = new URL(
"http://localhost:8080/CustomerDB/webresources/co.com.mazf.ciudad");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/xml");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ conn.getResponseCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader(
(conn.getInputStream())));
String output;
Ciudades ciudades = new Ciudades();
System.out.println("Output from Server .... \n");
while ((output = br.readLine()) != null) {
System.out.println("12132312");
System.err.println(output);
DocumentBuilder db = DocumentBuilderFactory.newInstance()
.newDocumentBuilder();
InputSource is = new InputSource();
is.setCharacterStream(new StringReader(output));
Document doc = db.parse(is);
NodeList nodes = ((org.w3c.dom.Document) doc)
.getElementsByTagName("ciudad");
for (int i = 0; i < nodes.getLength(); i++) {
Ciudad ciudad = new Ciudad();
Element element = (Element) nodes.item(i);
NodeList name = element.getElementsByTagName("idCiudad");
Element element2 = (Element) name.item(0);
ciudad.setIdCiudad(Integer
.valueOf(getCharacterDataFromElement(element2)));
NodeList title = element.getElementsByTagName("nomCiudad");
element2 = (Element) title.item(0);
ciudad.setNombre(getCharacterDataFromElement(element2));
ciudades.getPartnerAccount().add(ciudad);
}
}
for (Ciudad ciudad1 : ciudades.getPartnerAccount()) {
System.out.println(ciudad1.getIdCiudad());
System.out.println(ciudad1.getNombre());
}
conn.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public static String getCharacterDataFromElement(Element e) {
Node child = e.getFirstChild();
if (child instanceof CharacterData) {
CharacterData cd = (CharacterData) child;
return cd.getData();
}
return "";
}
}
Note that the xml structure that I expected in the example was as follows:
<ciudad><idCiudad>1</idCiudad><nomCiudad>BOGOTA</nomCiudad></ciudad>

Look at Jersey. Again, REST is all about the data. And a tutorial here

JAX-RS but you can also use regular DOM that comes with standard Java

From your question its not clear whether you are using any frameworks.For REST you will be getting an WADL & Apache CXF recently added support for WADL-first development of REST services.Please go through http://cxf.apache.org/docs/index.html

You can able to consume a Restful Web service in Spring using RestTemplate.class.
Example :
public class Application {
public static void main(String args[]) {
RestTemplate restTemplate = new RestTemplate();
ResponseEntity<String> call= restTemplate.getForEntity("http://localhost:8080/SpringExample/hello",String.class);
System.out.println(call.getBody())
}
}
Reference

Apache Http Client APIs are very commonly used for calling HTTP Rest services.
Here is one of example of consuming HTTP GET call.
import java.io.IOException;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpUriRequest;
import org.apache.http.impl.client.HttpClientBuilder;
public class CallHTTPGetService {
public static void main(String[] args) throws ClientProtocolException, IOException {
HttpClient client = HttpClientBuilder.create().build();
HttpUriRequest httpUriRequest = new HttpGet("URL");
HttpResponse response = client.execute(httpUriRequest);
System.out.println(response);
}
}
Use following maven dependency if using Maven project.
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpmime -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
<version>4.5.1</version>
</dependency>

Automate HTML form submission using Java to find grocery hours

I'm trying to automate form submission using Java to get the hours of a grocery store here:
www.giantfood.com
I've posted the query and the hidden miles and storeType fields of the form, but my output.html is just the original web header and footer with an error message in the body. What am I doing wrong?
import java.io.*;
import java.net.*;
public class PostHTML
{
public static void main(String[] args)
{
try
{
URL url = new URL( "http://www.giantfood.com/our_stores/locator/store_search.htm" );
HttpURLConnection hConnection = (HttpURLConnection)
url.openConnection();
HttpURLConnection.setFollowRedirects( true );
hConnection.setDoOutput( true );
hConnection.setRequestMethod("POST");
PrintStream ps = new PrintStream( hConnection.getOutputStream() );
ps.print("groceryStoreAddress=20814&groceryStoreMiles=10&storeType=GROCERY");
ps.close();
hConnection.connect();
if( HttpURLConnection.HTTP_OK == hConnection.getResponseCode() )
{
InputStream is = hConnection.getInputStream();
OutputStream os = new FileOutputStream("output.html");
int data;
while((data=is.read()) != -1)
{
os.write(data);
}
is.close();
os.close();
hConnection.disconnect();
}
}
catch(Exception ex)
{
ex.printStackTrace();
}
}
}
UPDATE
Thanks! Using &'s worked. I'm trying to use HttpClient but I'm getting another error now:
package clientwithresponsehandler;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.HttpClient;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
import java.util.ArrayList;
import java.util.List;
import org.apache.http.NameValuePair;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.protocol.HTTP;
/**
* This example demonstrates the use of the {#link ResponseHandler} to simplify
* the process of processing the HTTP response and releasing associated resources.
*/
public class ClientWithResponseHandler {
public static void main(String[] args) throws Exception {
HttpClient httpclient = new DefaultHttpClient();
try {
HttpPost httpost = new HttpPost("http://www.giantfood.com/our_stores/locator/store_search.htm");
System.out.println("executing request " + httpost.getURI());
List <NameValuePair> nvps = new ArrayList <NameValuePair>();
nvps.add(new BasicNameValuePair("groceryStoreAddress", "20878"));
nvps.add(new BasicNameValuePair("groceryStoreMiles", "10"));
nvps.add(new BasicNameValuePair("storeType", "GROCERY"));
httpost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));
// Create a response handler
ResponseHandler<String> responseHandler = new BasicResponseHandler();
String responseBody = httpclient.execute(httpost, responseHandler);
System.out.println("----------------------------------------");
System.out.println(responseBody);
System.out.println("----------------------------------------");
} finally {
// When HttpClient instance is no longer needed,
// shut down the connection manager to ensure
// immediate deallocation of all system resources
httpclient.getConnectionManager().shutdown();
}
}
}
Output:
run:
executing request http://www.giantfood.com/our_stores/locator/store_search.htm
Exception in thread "main" org.apache.http.client.HttpResponseException: Moved Temporarily
at org.apache.http.impl.client.BasicResponseHandler.handleResponse(BasicResponseHandler.java:67)
at org.apache.http.impl.client.BasicResponseHandler.handleResponse(BasicResponseHandler.java:55)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:945)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:919)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:910)
at clientwithresponsehandler.ClientWithResponseHandler.main(ClientWithResponseHandler.java:39)
Java Result: 1
BUILD SUCCESSFUL (total time: 1 second)
I don't understand the Moved Temporarily error.

try to use
ps.print("groceryStoreAddress=20814&groceryStoreMiles=10&storeType=GROCERY")
instead
BTW, it's easier to use some http-library, like Apache HttpClient

Solved the Moved Temporarily by learning about HTML Redirects:
Httpclient 4, error 302. How to redirect?