So I'm trying to access data frim a rest api using java code and I'm not very experienced in getting data from an api using java. I had found the code below on another question. This code was able to output all the data from the link but I'm a bit confused on how to get specific values from the link. The link in the code below shows the nutrition info for an apple and what I'm looking for is being able to output specific values such as the fdcId or the description.
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class Main {
public static void main(String[] args) {
try {
URL url = new URL("https://api.nal.usda.gov/fdc/v1/food/1750339?api_key=DEMO_KEY");//your url i.e fetch data from .
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP Error code : "
+ conn.getResponseCode());
}
InputStreamReader in = new InputStreamReader(conn.getInputStream());
BufferedReader br = new BufferedReader(in);
String output;
while ((output = br.readLine()) != null) {
System.out.println(output);
}
conn.disconnect();
} catch (Exception e) {
System.out.println("Exception in NetClientGet:- " + e);
}
}
}
I haven't really tried much with the code. I tried looking for the answer for this online and didn't find much
You need to look how to parse Json in Java. This way you can take any data you need from that Json file. Some explanations for the similar question are here.
Usually Spring or other frameworks used for this purposes. In your example you can save JSON response to string like this:
String reponse = "";
String line;
while ((line = br.readLine()) != null) {
reponse += line;
}
And than parse this JSON, f.e. using Jackson ObjectMapper convert it into your dto.
There is an example here: https://www.baeldung.com/jackson-object-mapper-tutorial
Your question should be divided in two parts:
How to read info from API
How to parse it.
The first part you already did. But you can do it with less code by far. You can use 3d party HTTP clients. The most popular are Apache Http Client with good tutorial - Apache HttpClient Tutorial and OK Http client with good tutorial - A Guide to OkHttp. However, I wrote my own Http client that is not widely known but very simple to use.
The second part is how to parse Json. And for that you can use also 3d party Json parsers. The most popular ones would be Json Jackson or Gson (of Google). And again I also wrote my own thin wrapper over Json-Jackson that allows you to parse Json very simply. Here is the code example that uses my own utilities:
HttpClient httpClient = new HttpClient();
try {
httpClient.setConnectionUrl("https://api.nal.usda.gov/fdc/v1/food/1750339?api_key=DEMO_KEY");
httpClient.setRequestHeader("Accept", "application/json");
String jsonResponse = httpClient.sendHttpRequest(HttpClient.HttpMethod.GET);
Map<String, Object> map = JsonUtils.readObjectFromJsonString(jsonResponse, Map.class);
System.out.println("fdcid: " + map.get("fdcId"));
System.out.println("description: " + map.get("description"));
} catch (Exception e) {
System.out.println(httpClient.getLastResponseCode() + " "
+ httpClient.getLastResponseMessage() + TextUtils.getStacktrace(e, false));
}
If you run this code this would be the output:
fdcid: 1750339
description: Apples, red delicious, with skin, raw
The classes HttpClient, JsonUtils and TextUtils all come as part of MgntUtils Open Source library written and maintained by me. Here is Javadoc for the library If you want the source code of the whole library you can get it on Github here, and just the library as Maven artifact is available from Maven Central here
Related
I added an open extension to an event in a calendar and am trying to read it back.
Here is the url:
https://graph.microsoft.com/v1.0/users/{userid}/calendars/{calendarId}=/events?$expand=Extensions($filter=Id eq 'c.i.m.p.server.entities.outlook.Event')
I cannot get this to work in a Java program. The following combinations do work:
It works my Java program if I remove the $expand... parameter. I can also ask for certain fields, that works too.
The request works in Postman (I just have to set the token)
The request works in Graph Explorer when I log in as the owner of the calendar
Here is the extension (inside one of the events) when I use Postman to read the event. It is the last item in the event:
"extensions#odata.context": "https://graph.microsoft.com/v1.0/$metadata#users('{userid}')/calendars('{calendarId}')/events('{eventId})/extensions",
"extensions": [
{
"#odata.type": "#microsoft.graph.openTypeExtension",
"id": "Microsoft.OutlookServices.OpenTypeExtension.c.i.m.p.server.entities.outlook.Event",
"extensionName": "c.i.m.p.server.entities.outlook.Event",
"adherentId": "12346",
"timeSlotID": "346463"
}
]
Here is the Java code (Java 8, using java.io and java.net libraries):
private static void doSomething(String _accessToken) throws IOException {
String urlString = "https://graph.microsoft.com/v1.0/users/{userId}/calendars/{calendarId}/events?$expand=Extensions($filter=Id eq 'c.i.m.p.server.entities.outlook.Event')";
URL url = new URL(urlString);
Proxy webProxy
= new Proxy(Proxy.Type.HTTP, new InetSocketAddress({proxy-address}, {port}));
HttpURLConnection connection = (HttpURLConnection) url.openConnection(webProxy);
// Set the appropriate header fields in the request header.
connection.setRequestProperty("Authorization", "Bearer " + _accessToken);
connection.setRequestProperty("Accept", "application/json");
connection.setDoOutput(true);
connection.setReadTimeout(5000);
connection.setRequestMethod(HttpMethod.GET);
try {
connection.connect();
int responseCode = connection.getResponseCode();
System.out.println("execute(), response code = " + responseCode);
String responseMessage = connection.getResponseMessage();
System.out.println("execute(), response Message = " + responseMessage);
String responseString = null;
try {
InputStream ins = connection.getInputStream();
BufferedReader br=new BufferedReader(new InputStreamReader(ins));
StringBuffer sb=new StringBuffer();
String line;
while ((line=br.readLine()) != null) {
sb.append(line);
}
responseString = sb.toString();
} catch (Exception e) {
System.out.println("Could not get input stream from response, error is " + e.toString());
}
System.out.println("execute(), httpResult = " + responseString);
} catch (IOException e) {
System.out.println(".execute(), IOException : " + e.toString());
} finally {
connection.disconnect();
}
}
How do I fix this? Thanks!
400 means bad request. It could be because of url encoding. Url encode the query string.
Something like
String query = "Extensions($filter=Id eq 'c.i.m.p.server.entities.outlook.Event'";
String url = "https://graph.microsoft.com/v1.0/users/{userId}/calendars/{calendarId}/events?
$expand=" + URLEncoder.encode(query, StandardCharsets.UTF_8.name());
Alternatively you could use graph service java api based on your need which will help abstract all the interactions for you or you could use any of the rest clients available.
First of all, you should provide more info on the error - Stacktrace and error message. But 400 code indicates that was a user mistake, meaning that you are sending an invalid request. Since you say that postman request works then compare all the headers that are sent by postman and see if your code misses some hearer. As for the code, instead of coding your own Http client functionality I would suggest using 3d party Http client. Here are a few suggestions:
Apache Http client - very popular and well known 3d party Http Client
OK Http client - Open-source Http client. Here is tutorial
MgntUtils Http client - very simple 3d party HttpClient: Provided in MgntUtils Open source library (written by me). Very simple in use. Take a look at Javadoc. Library itself provided as Maven artifacts and on Git (including source code and Javadoc).
I have a dialogflow project that I'm trying to access from Java with a rest call.
It is giving me an authentication issue.
I have followed all online instructions (and many forum suggestions) to no avail.
I have tried generating the key json, as per the instructions here:
https://dialogflow.com/docs/reference/v2-auth-setup
and setting my environment variable as described, but nothing seems to work.
I have checked my projectID, and am running off the same machine with the environment variable, and have double, triple and quadruple checked it's name and location, but I still get the following error:
java.net.HttpRetryException: cannot retry due to server authentication, in streaming mode
Here is my code (though it's a REST call, so I don't know if it's so relevant):
String url = https://dialogflow.googleapis.com/v2/projects/MYPROJECT/agent/sessions/SESSION_NUM:detectIntent
URL url = new URL(full_url);
String inText = "Hello World";
String outText = "";
HttpURLConnection con = (HttpURLConnection) url.openConnection();
try {
con.setDoOutput(true);
con.setRequestMethod("POST");
// set body of http post
Map<String,String> arguments = new HashMap<>();
JSONObject inTextJsn = new JSONObject();
inTextJsn.append("text",inText);
inTextJsn.append("languageCode","en");
JSONObject fieldJsn = new JSONObject();
fieldJsn.append("text", inTextJsn);
arguments.put("queryInput", fieldJsn.toString());
StringJoiner sj = new StringJoiner("&");
for(Map.Entry<String,String> entry : arguments.entrySet())
sj.add(URLEncoder.encode(entry.getKey(), "UTF-8") + "="
+ URLEncoder.encode(entry.getValue(), "UTF-8"));
// post http post as bytes
byte[] bytes_out = sj.toString().getBytes(StandardCharsets.UTF_8);
con.setFixedLengthStreamingMode(bytes_out.length);
con.connect();
try (OutputStream os = con.getOutputStream()) {
os.write(bytes_out);
}
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream(),
"UTF-8"));
// read all lines to a string
String line;
String response = "";
while ((line = reader.readLine()) != null) {
response += line;
}
JSONObject responseJsn = new JSONObject(response);
outText = responseJsn.get("fulfillmentText").toString();
} catch (Exception e) {
System.err.println(e);
} finally {
con.disconnect();
}
return restResponse;
The gist of the code is to simply send a message ("Hello World!") to my dialogflow, and get back my agent's response (the code may have bugs - it's a bit hard to test when I can't get passed this authentication issue, so please help with the authentication, not code bugs).
Thanks all!
The directions at that page assume you're going to use the gcloud program to generate a currently valid bearer token, which is then sent along with the HTTP headers. That page illustrates
Your code doesn't seem to be generating an Authorization HTTP header at all, which is why you're getting the error you do.
Since you're using Java, you should look at the google-auth-library-java library, which will give you the tools to generate the token you need to provide in the Authorization header.
You may also wish to check out the google-cloud-java library. This contains Java classes to directly perform operations against Dialogflow instead of coding the REST/HTTP calls yourself. (However, it is still at an Alpha level for Dialogflow, so may not be stable or forwards compatible.)
I have to test the EPA's Data Exchange Web Services. Since it is difficult to create 100 accounts, buildings, energy usage distributions, etc. I want to automate the process. I searched for code examples to do a simple GET. The best one I found was at http://pic.dhe.ibm.com/infocenter/tivihelp/v10r1/index.jsp?topic=%2Fcom.ibm.taddm.doc_7.2%2FSDKDevGuide%2Ft_cmdbsdk_restapi_java.html. I modified this for my purposes.
With the certificate, it is throwing an error at that line
Without the certificate (commented out), the connection is timing out and throwing the exception at getResponseCode().
I'm not sure:
What is the correct way of submitting a certificate
If I am sending the credentials correctly
If my code is incomplete, and therefore, the application is unable to get the response code
I should be using Eclipse EE (with Web Tools Platform) and create Project > Web Application, instead of Eclipse Juno (without WTP)
Thank you in advance.
package Package1;
import java.io.*;
import java.util.*;
import java.lang.StringBuffer;
import java.net.*;
import java.net.HttpURLConnection;
import javax.net.ssl.HttpsURLConnection;
public class Class1 {
public static void main (String args[]){
try{
// set this property to the location of the cert file
System.setProperty("javax.net.ssl.trustStore","C:/Documents and Settings/bhattdr/Desktop/-.energystar.gov.der");
String username = "yy777PPP";
String password = "yy777PPP";
String userpass = "";
URL url = new URL("https://portfoliomanager.energystar.gov/wstest/account");
// URLConnection uc = url.openConnection();
HttpsURLConnection uc = (HttpsURLConnection) url.openConnection();
userpass = username + ":" + password;
String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary(userpass.getBytes());
System.out.println("sending request...");
uc.setRequestMethod("GET");
uc.setAllowUserInteraction(false);
uc.setDoOutput(true);
uc.setRequestProperty( "Content-type", "text/xml" );
uc.setRequestProperty( "Accept", "text/xml" );
uc.setRequestProperty ("Authorization", basicAuth);
System.out.println(uc.getRequestProperties());
// uc.setRequestProperty( "authorization", "Basic " + encode("administrator:collation"));
// Map headerFields = uc.getHeaderFields();
// System.out.println("header fields are: " + headerFields);
int rspCode = uc.getResponseCode();
if (rspCode == 200) {
InputStream is = uc.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String nextLine = br.readLine();
while (nextLine != null) {
System.out.println(nextLine);
nextLine = br.readLine();
}
}
}
catch(IOException e) {
e.printStackTrace();
}
}
}
You don't need to roll your own.
If you want to write something, you can use Jersey, which has existing classes to act as Rest Clients (Rest clients for Java?)
There are plenty of apps which exercise rest apis which you can use if you don't want to write something. Google turns up plenty (like http://code.google.com/p/rest-client/)
You're using a DER file as your key store which is not supported by Java
Crypto normally. Use the keytool to create a JKS or some other supported keystore and then refer to it.
AMong all the frameworks for REST-Clients... did you try OpenFeign? It's a components from the NetFlix stack. Easy to use and fits into all the other
components of NetFlix.
Give it a try: https://github.com/OpenFeign/feign
I have a Servlet that sends back a JSON Object and I would like to use this servlet in another Java project. I have this method that gets me the results:
public JSONArray getSQL(String aServletURL)
{
JSONArray toReturn = null;
String returnString = "";
try
{
URL myUrl = new URL(aServletURL);
URLConnection conn = myUrl.openConnection();
conn.setDoOutput(true);
BufferedReader in = new BufferedReader( new InputStreamReader( conn.getInputStream() ) );
String s;
while ((s = in.readLine()) != null )
returnString += s;
in.close();
toReturn = new JSONArray(returnString);
}
catch(Exception e)
{
return new JSONArray();
}
return toReturn;
}
This works pretty will, but the problem I am facing is the following:
When I do several simultaneous requests, the results get mixed up and I sometimes get a Response that does not match the request I send.
I suspect the problem to be related to the way I get the response back: The Reader reading a String from the InputStream of the connection.
How can I make sure that I get one reques -> one corresponding reply ?
Is there a better way to retrieve my JSON object from my servlet ?
Cheers,
Tim
When I do several simultaneous requests, the results get mixed up and I sometimes get a Response that does not match the request I send.
Your servlet is not thread safe. I'd bet that you've improperly assigned request scoped data either directly or indirectly as instance or class variables of the servlet. This is a common beginner's mistake.
Carefully read this How do servlets work? Instantiation, sessions, shared variables and multithreading and fix your servlet code accordingly. The problem is not in the URLConnection code shown so far, although it indicates that you're doing exactly the same job in both doGet() and doPost(), which in turn is already a smell as to how the servlet is designed.
Try removing setDoOutput(true), you are using the connection only for input and so you shouldn't use it.
Edit: alternatively try using HttpClient, it's much nicer that using "raw" Java.
I would like to upload files from java application/applet using POST http event. I would like to avoid to use any library not included in SE, unless there is no other (feasible) option.
So far I come up only with very simple solution.
- Create String (Buffer) and fill it with compatible header (http://www.ietf.org/rfc/rfc1867.txt)
- Open connection to server URL.openConnection() and write content of this file to OutputStream.
I also need to manually convert binary file into POST event.
I hope there is some better, simpler way to do this?
You need to use the java.net.URL and java.net.URLConnection classes.
There are some good examples at http://java.sun.com/docs/books/tutorial/networking/urls/readingWriting.html
Here's some quick and nasty code:
public void post(String url) throws Exception {
URL u = new URL(url);
URLConnection c = u.openConnection();
c.setDoOutput(true);
if (c instanceof HttpURLConnection) {
((HttpURLConnection)c).setRequestMethod("POST");
}
OutputStreamWriter out = new OutputStreamWriter(
c.getOutputStream());
// output your data here
out.close();
BufferedReader in = new BufferedReader(
new InputStreamReader(
c.getInputStream()));
String s = null;
while ((s = in.readLine()) != null) {
System.out.println(s);
}
in.close();
}
Note that you may still need to urlencode() your POST data before writing it to the connection.
You need to learn about the chunked encoding used in newer versions of HTTP. The Apache HttpClient library is a good reference implementation to learn from.