We Keep Coding

Java executing curl command using HttpURLConnection returns 204 (HTTP_NO_CONTENT)

I am using Java 11. I have the the following curl command, when I execute on the command line:
curl --location --request GET 'http://xxx.xxx.co.za:8080/document/details/Select+docId+From+%27Workflow+Club%2FCustomer+Invoices%27+where+recursive+%3D+true+and+invoice_number+%3D%271221669023%27' --header 'Authorization: Basic xxx'
Returns the following:
{errorMessage: 'PaperTrail API only available in enterprise edition'}
However, when I try execute the same URL in a Java application using HttpURLConnection, it returns a blank response.
private static final String USER_AGENT = "Mozilla/5.0";
private static final String GET_URL = "http://xxx.xxx.co.za:8080/document/details/";
private static final String GET_URL_QRY = "Select docId From 'Workflow Club/Customer Invoices' where recursive = true and invoice_number =':1'";
private static final String GET_AUTH_ENC = "Basic xxx";
#Override
public String getDocId(Long invoiceNumber) {
String get_url_qry = StringUtils.replace(GET_URL_QRY, ":1", Long.toString(invoiceNumber));
get_url_qry = URLEncoder.encode(get_url_qry, StandardCharsets.UTF_8);
final String get_url = GET_URL+get_url_qry;
try {
URL url = new URL(get_url);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestProperty("Authorization", GET_AUTH_ENC);
con.setRequestMethod("GET");
con.setRequestProperty("User-Agent", USER_AGENT);
int responseCode = con.getResponseCode();
logger.info(get_url+" -> GET Response Code :: " + responseCode);
if (responseCode == HttpURLConnection.HTTP_OK || responseCode == HttpURLConnection.HTTP_NO_CONTENT) { // success
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
String resp = response.toString();
logger.info(responseCode+" Response: '"+resp+"'.");
} else {
logger.error("GET request did not work (responseCode: "+responseCode+").");
}
} catch (MalformedURLException e) {
logger.error("MalformedURLException creating URL '"+get_url+"'. "+e.getMessage());
} catch (IOException e) {
logger.error("IOException creating connection from URL '"+get_url+"'. "+e.getMessage());
}
return null;
}
Outputs the following with a blank response:
204 Response: ''.
Question
How do I get the Java application to also return the same as the command line call?
UPDATE
I have a different POST url, that I need to call too, and I can call it successfully. So there's something wrong with my GET call.
private static final String USER_AGENT = "Mozilla/5.0";
E.g. GET call that returns a 204, with no content.
private String getDocId(Long invoiceNumber) {
String get_url_qry = StringUtils.replace(GET_URL_QRY, ":1", Long.toString(invoiceNumber));
get_url_qry = URLEncoder.encode(get_url_qry, StandardCharsets.UTF_8);
final String get_url = GET_URL+get_url_qry;
try {
URL url = new URL(get_url);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestProperty("Authorization", GET_AUTH_ENC);
con.setRequestMethod("GET");
con.setRequestProperty("User-Agent", USER_AGENT);
con.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
Map<String,String> data = handleResponse(con);
return data.get("docId");
} catch (MalformedURLException e) {
logger.error("MalformedURLException creating URL '"+get_url+"'. "+e.getMessage());
} catch (IOException e) {
logger.error("IOException creating connection from URL '"+get_url+"'. "+e.getMessage());
}
return null;
}
The POST call, that returns a 200, and the expected content.
private String getDocLink(String docId) {
if (StringUtils.isNotBlank(docId)) {
try {
URL url = new URL(POST_URL);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestProperty("Authorization", GET_AUTH_ENC);
con.setRequestMethod("POST");
con.setRequestProperty("User-Agent", USER_AGENT);
con.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
byte[] postDataBytes = getPostData(docId);
con.setRequestProperty("Content-Length", String.valueOf(postDataBytes.length));
con.setDoOutput(true);
con.getOutputStream().write(postDataBytes);
Map<String,String> data = handleResponse(con);
return data.get("url");
} catch (IOException e) {
logger.error("IOException creating connection from URL '"+POST_URL+"'. "+e.getMessage());
}
} else {
logger.error("No docId provided when trying to get a document link.");
}
return null;
}
So seeing the that POST call works, I think I must be doing something wrong with the GET call.

Did you try, setting the same user agent in your Java Code, cURL would use? something like curl/7.37.0?
As far as I can tell, that should be all, what differs. Aside cURL following redirects. But as there is no redirect, I guess it might be the User Agent making a difference.
There are a lot of server applications, behaving differently, when they are called by a browser (Like you make it think by setting the User-Agent to Mozilla/5.0), instead of some other application, like cURL.

From what you describe, the original call produces an error.
Assuming that you are also getting some form of error in the java code of your GET, you will catch the exception and simply log it. Then you will return null instead of a string, which will cause your response to have no content, i.e. 204.

how to do a POST correctly with HttpUrlConnection?

I've been looking for methods to do send data to a web form or a web service, principally php servers, but there's a lot of answers with deprecated methods, I have to post a single value in a form, and another more elaborated webservice.
So there are like 3 questions:
I want to know if is posible to post data in a form and submit it, and if is possible how to do that. The form isn't a webservice, is a form to user's input (this one).
How to use correctly the HttpUrlConnection to call a webservice, (post data and get response), the webservice can use JSON or simple 2 string values like user and password.
Here's the code that I've found, basically:
private class PostTask extends AsyncTask<String, Void, Boolean> {
#Override
protected Boolean doInBackground(String... params) {
HttpURLConnection conn=null;
try {
URL url = new URL(params[0]);
conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
//conn.setReadTimeout(10000);
//conn.setConnectTimeout(15000);
//conn.setDoInput(true);
String body = "4";
conn.connect();
OutputStream output = new BufferedOutputStream(conn.getOutputStream());
output.write(body.getBytes());
output.flush();
} catch (Exception e) {
e.printStackTrace();
}finally {
assert conn != null;
conn.disconnect();
}
return true;
}
}

Yes is possible to pass data from a form as url params using HttpUrlConnection.
You 've almost got it
In a GET request, the parameters are sent as part of the URL.
In a POST request, the parameters are sent as a body of the request,
after the headers.
the way to do it:
public void sendForm(String formParam1, String formParam2, String formParam3){
String urlParameters = "param1="+formParam1+"&param2="+formParam2+"&param3="+formParam3;
try {
byte[] postData = urlParameters.getBytes( StandardCharsets.UTF_8 );
int postDataLength = postData.length;
String request = "http://sicenet.com.mx/ws/index.php/inicio/editar_clave";
URL url = new URL( request );
HttpURLConnection conn= (HttpURLConnection) url.openConnection();
conn.setDoOutput( true );
conn.setInstanceFollowRedirects( false );
conn.setRequestMethod( "POST" );
conn.setRequestProperty( "Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty( "charset", "utf-8");
conn.setRequestProperty( "Content-Length", Integer.toString( postDataLength ));
conn.setUseCaches( false );
try(DataOutputStream wr = new DataOutputStream( conn.getOutputStream())) {
wr.write( postData );
}
}catch (Exception e){
e.printStackTrace();
}
}
Not tested if it will work.
found in this StackOverFlow answer

Posting to a webservice using httpurlconnection

How come I am only allowed to make posts to .com url's but not .asmx url's? Im a bit confused as what I want to generally do is send xml content to a .asmx url web service eventually. Can anyone supply me with tips why this doesn't work, and how I can post to a .asmx file?
public class POSTSenderExample {
public String echoCuties(String query) throws IOException {
// Encode the query
String encodedQuery = URLEncoder.encode(query, "UTF-8");
// This is the data that is going to be send to itcuties.com via POST request
// 'e' parameter contains data to echo
String postData = "e=" + encodedQuery;
URL url = new URL("http://echo.itgeeeks.asmx");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.setRequestProperty("Content-Length", String.valueOf(postData.length()));
// Write data
OutputStream os = connection.getOutputStream();
os.write(postData.getBytes());
// Read response
StringBuilder responseSB = new StringBuilder();
BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line;
while ( (line = br.readLine()) != null)
responseSB.append(line);
// Close streams
br.close();
os.close();
return responseSB.toString();
}
// Run this example
public static void main(String[] args) {
try {
System.out.println(new POSTSenderExample().echoCuties("Hi there!"));
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
}

Using "POST" is correct.
Instead of calling
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
you have to call
connection.setRequestProperty("Content-Type", "text/xml; charset=utf-8");
(if you are using utf-8 encoding which is probably the case).
You also have to set the SOAP Action in the http- Header:
connection.setRequestProperty("SOAPAction", SOAPAction);
You can find the SOAP Action eihter in the wsdl- file. What I did to find out all expected Parameters: I used a working WS Client, and traced the TCP traffic in order to find out the expected HTTP headers.

Java Post Request Not Working

Why does this not work? I have also tried using HttpURLConnection class however this did not work either. The php page cannot find the posted data.
Note: im new to post requests with java.
public static String GetURL(String inUrl, String post) {
String inputLine = "";
try {
if (!inUrl.contains("http")) {
throw new Exception("Invalid URL");
} else {
Log.writeLog(inUrl);
URL url = new URL(inUrl);
//send post
URLConnection connection = url.openConnection();
//connection.setRequestMethod("POST");
connection.setDoOutput(true);
connection.setRequestProperty("Accept-Charset", "UTF-8");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");
connection.connect();
OutputStream wr = connection.getOutputStream();
wr.write(post.getBytes());
wr.flush();
wr.close();
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
inputLine = in.readLine();
in.close();
}
} catch (Exception ex) {
ex.printStackTrace();
Log.writeLog("Error getting url.");
}
return inputLine;
}

Since you didn't specify the error, it is hard for me to see where you went wrong.
However, I think it would be a lot easier to use an abstraction like HttpClient. Your code would look like this:
URI uri = new URIBuilder()
.setURI(inUrl)
.addHeader("Accept-Charset", "UTF-8")
.addHeader("Content-Type", "application/x-www-form-urlencoded")
.build();
HttpPost httpPost = new HttpPost(uri);
httpPost.setEntity(post);
Much cleaner than dealing with the low-level details of streams and connections.

Making PUT request with JSON data using HttpURLConnection is not working

I'm trying to make PUT request with JSON data using HttpURLConnection in Java. The way I do it doesn't work. I get no errors so I don't know what the problem is.
public static void main(String[] args) {
URL url;
try {
url = new URL("http://fltspc.itu.dk/widget/515318fe17450f312b00153d/");
HttpURLConnection hurl = (HttpURLConnection) url.openConnection();
hurl.setRequestMethod("PUT");
hurl.setDoOutput(true);
hurl.setRequestProperty("Content-Type", "application/json");
hurl.setRequestProperty("Accept", "application/json");
String payload = "{'pos':{'left':45,'top':45}}";
OutputStreamWriter osw = new OutputStreamWriter(hurl.getOutputStream());
osw.write(payload);
osw.flush();
osw.close();
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
And here is the request I'm actually trying to make:
I was already making GET requests to the resource within the same app and it worked fine. I would be very grateful for all tips on how can I debug that or how can I try to do it some other way. So far I tried only using OutputStream instead of OutputStreamWriter but it doesn't work neither.

The Sun (Oracle) implementation of HttpURLConnection caches the content of your post unless you tell it to be in streaming mode. The content will be sent if you start interaction with the response such as:
hurl.getResponseCode();
Also, according to RFC 4627 you can not use single quotes in your json (although some implementations seem to not care).
So, change your payload to:
String payload = "{\"pos\":{\"left\":45,\"top\":45}}";
This example works for me
public class HttpPut {
public static void main(String[] args) throws Exception {
Random random = new Random();
URL url = new URL("http://fltspc.itu.dk/widget/515318fe17450f312b00153d/");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("PUT");
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "application/json");
connection.setRequestProperty("Accept", "application/json");
OutputStreamWriter osw = new OutputStreamWriter(connection.getOutputStream());
osw.write(String.format("{\"pos\":{\"left\":%1$d,\"top\":%2$d}}", random.nextInt(30), random.nextInt(20)));
osw.flush();
osw.close();
System.err.println(connection.getResponseCode());
}
}