I'm making a POST request using Java 8 like this:
URL url = new URL("http://target.server.com/doIt");
URLConnection connection = url.openConnection();
HttpURLConnection httpConn = (HttpURLConnection) connection;
byte[] soapBytes = soapRequest.getBytes();
httpConn.setRequestProperty("Host", "target.host.com");
httpConn.setRequestProperty("Content-Length", soapBytes.length+"");
httpConn.setRequestProperty("Content-Type", "application/soap+xml; charset=utf-8");
httpConn.setRequestMethod("POST");
httpConn.setConnectTimeout(5000);
httpConn.setReadTimeout(35000);
httpConn.setDoOutput(true);
httpConn.setDoInput(true);
OutputStream out = httpConn.getOutputStream();
out.write(soapBytes);
out.close();
int statusCode;
try {
statusCode = httpConn.getResponseCode();
} catch (IOException e) {
InputStream stream = httpConn.getErrorStream();
if (stream == null) {
throw e;
} else {
// this never happens
}
}
My soap request contains a document ID and the target server (which hosts a third-party service that I do not own or have access to) returns a PDF document that matches the supplied ID.
Most of the time, the server returns a PDF doc and occasionally the status code is 500 when the document is not available. However, sometimes the call to getResponseCode() throws an IOException with "Invalid Http response".
I thought that a server would always have some response code to return, no matter what happens.
Does this mean that server is returning complete garbage that doesn't
match the expected format of a HTTP response?
Is there a way to get any more information about the actual response?
Is there a way to retrieve the raw textual response (if any)?
As AxelH points out, there must be something wrong when connecting with the remote server, and in this case you just can't get a valid response.
If you are in a testing environment, you can monitorize the connection at TCP level (not at HTTP level): Put a monitor between your client and the remote server which monitorizes all the TCP traffic exchanged between the two peers. If you are using Eclipse, you can create a TCP monitor.
Related
Am making webservice calls to HTTPS server from an android application. Below is the code snippet, with which am able to make web service calls successfully and getting response.
My Question is, do we need to perform any additional step to encrypt data before making call to HTTPS server?
Because, from android profiler am able to see all my Web Requests in plain text format. My understanding is that request will gets encrypted before making HTTPS call.
public static WebServiceResp makeWebServiceCall(String XML, String urlPath) throws IOException{
//Code to make a web service HTTP request
String responseString = "";
String outputString = "";
String wsURL = urlPath;
URL url = new URL(wsURL);
URLConnection connection = url.openConnection();
HttpsURLConnection httpConn = (HttpsURLConnection)connection;
ByteArrayOutputStream bout = new ByteArrayOutputStream();
//System.out.println(XML);
byte[] buffer = new byte[XML.length()];
buffer = XML.getBytes();
bout.write(buffer);
byte[] b = bout.toByteArray();
// Set the appropriate HTTP parameters.
httpConn.setRequestProperty("Content-Length",
String.valueOf(b.length));
httpConn.setRequestProperty("Content-Type", "text/xml; charset=utf-8");
httpConn.setRequestMethod("POST");
httpConn.setRequestProperty("Cache-Control", "no-cache");
httpConn.setDoOutput(true);
httpConn.setDoInput(true);
OutputStream out = httpConn.getOutputStream();
//Write the content of the request to the outputstream of the HTTP Connection.
out.write(b);
out.close();
//Ready with sending the request.
//Check the status
int status = httpConn.getResponseCode();
Log.d(TAG, "makeWebServiceCall: "+"Processing Status: "+status);
BufferedReader in;
if (status <= 200) {
//Read the response.
Log.d(TAG, "makeWebServiceCall: Getting Input Stream");
InputStreamReader isr =
new InputStreamReader(httpConn.getInputStream());
in = new BufferedReader(isr);
}else{
//Read the response.
Log.d(TAG, "makeWebServiceCall: Getting Error Stream");
InputStreamReader isr =
new InputStreamReader(httpConn.getErrorStream());
in = new BufferedReader(isr);
}
//Write the SOAP message response to a String.
while ((responseString = in.readLine()) != null) {
outputString = outputString + responseString;
}
Log.d(TAG, "makeWebServiceCall: WebServiceResponse " + outputString);
//Parse the String output to a org.w3c.dom.Document and be able to reach every node with the org.w3c.dom API.
Document document = Utils.parseXmlFile(outputString);
//NodeList nodeLst = document.getElementsByTagName("GetWeatherResult");
// String weatherResult = nodeLst.item(0).getTextContent();
//System.out.println("Weather: " + weatherResult);
//Write the SOAP message formatted to the console.
WebServiceResp webServiceResp = new WebServiceResp();
webServiceResp.setDocument(document);
webServiceResp.setStatus(status);
return webServiceResp;
}
No. If you're sending it to an https website, the encryption is done as part of the protocol. You don't need to do any additional work.
No. The encryption that you see is on the network layer. The client which initiates the https call see what was sent and what was received. That is how https works.
When you look at chrome browser's network tab, you see what was sent and what was received. Now this is not a security problem, https is more about you doing things which make its difficult for anyone between the network to eavesdrop your data.
Now if you still want an additional level of security you can use certificate pinning
https://www.owasp.org/index.php/Certificate_and_Public_Key_Pinning
https://medium.com/#appmattus/android-security-ssl-pinning-1db8acb6621e
How can you add to network_security_config from MainActivity
So in this technique you basically say that the certificate hash that you expected is to have this content. And then if someone uses a trusted proxy with trusted CAs on the system, even after generating a valid certificate for the given domain the connections will not be established.
HTTPS is transparent to your application, all of the magic happens between Transport Layer(so it calls 'Transport Layer Security'), you may imagine encrypted telegrams in the old days, generals tell telegrapher messages in plain text, and telegrapher send them in encrypted form(maybe use some kind of codebook), anyone who didn't have the same codebook can't decrypt the message easily, and anyone who uses telegrams didn't care about the codebook(or even known about it, except those telegraphers on both side of the 'Transport Layer').
The encryption/decryption is done by built-in network client module provided by OS. So you needn't worry about it.
You can view plain texts with some client tools as they know exactly what they are sending/receiving. E.g. chrome developer tool. (Actually they don't care about encryption/decryption either).
I've got a simple Tomcat-based Java app that functions as a sort of firewall - I take requests from the "outside", reroute them to resources on the "inside", and return the result to the "outside."
This works fine for GETs, but I'm trying to add a POST function for a different request and I cannot get it working. The "inside" remote server is password protected and I cannot get the remote server to accept the authentication credentials (they work for the GET so the credentials are fine.) Instead, the Tomcat server calls the Authenticator over and over, and finally fails. Here's the error I'm getting:
java.net.ProtocolException: Server redirected too many times (20)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1848)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at com.mystuff.house.server.MyServlet.doPost(MyServlet.java:191)
I'm sure I'm doing something stupid, but I can't see where it is. Here's the guts of the servlet doPost() routine:
URL url = new URL("HTTP", "10.10.1.101", -1, "/myresource");
URLConnection con = url.openConnection();
HttpURLConnection http = (HttpURLConnection) con;
http.setRequestMethod("POST");
http.setDoOutput(true);
String encoded = String.valueOf(Base64.getEncoder().encode((a.getUsername().concat(":").concat(a.getPassword())).getBytes()));
http.setRequestProperty("Authorization", "Basic "+encoded);
http.setRequestProperty("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");
// Read the POST payload from the front end post, write to back end post
InputStream r = request.getInputStream();
OutputStream os = http.getOutputStream();
int j = 0;
while ((j = r.read()) != -1) {
os.write((byte) j);
}
http.connect();
// Try reading the result from the back end, push it back to the front end
try {
InputStream i = http.getInputStream();
OutputStream o = response.getOutputStream();
// read/write bytes until EOF
j = 0;
while ((j = i.read()) != -1) {
o.write((byte) j);
}
} catch (Exception ex) {
System.out.println("AIEEEE! Error receiving page from HTTP call");
ex.printStackTrace();
}
The problem with this, after some investigation, turned out to be that the authentication was not valid for the specific URL that I was trying to hit on the remote server.
I had expected to get a 403, 401 or 407 back from the remote server but that never happened, instead this "redirect" happened. So that's something to be aware of if you are trying to hit password-protected URLs from Java code.
Anyone knows why i get this error. I am trying to send a POST request, and here's the error message that i get.
Server response:
Error while dispatching hrxml [ Server was unable to process request. --> Procedure or function 'sp__LogMessage' expects parameter '#pi_ClientID', which was not supplied. at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at DispatchService.HRISMessageRouter.MessageRouter.Route(String HRXML)
at DispatchService.DispatchMessage.Dispatch(String HRXML)]
My Code :
URL link = new URL("https://example.com/example.asp");
HttpsURLConnection com = (HttpsURLConnection) link.openConnection();
String l;
con.setRequestMethod("POST");
con.setDoInput(true);
con.setDoOutput(true);
con.setRequestProperty("name", "rrrrr");
con.setRequestProperty("pwd", "ffff");
OutputStream os = con.getOutputStream();
os.flush();
InputStream is = con.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
StringBuffer r = new StringBuffer();
while((l = rd.readLine()) != null) {
r.append(l);
r.append('\r');
}
rd.close();
System.out.println("out "+ r.toString());
I have tried debugging the code etc, but still unable to find a possible reason to why this hapend . Can anyone help me to figure out the reason and a possible solution for this issue ?
The error message is:
Server was unable to process request. --> Procedure or function 'sp__LogMessage' expects parameter '#pi_ClientID', which was not supplied.
This looks like a SQL Stored Procedure on the server. Check to make sure that a client ID is being supplied to it.
Judging from the stacktrace:
Your application is talking to a SOAP service
The SOAP service is expecting an (XML encoded) request which includes a parameter that doesn't exist.
But this doesn't mesh with what you client code is doing. In fact, you are sending a POST request with parameters, and that will turn into a request body which is probably encoded as application/x-www-form-urlencoded ... not XML. That ain't going to work.
I need a monitor class that regularly checks whether a given HTTP URL is available. I can take care of the "regularly" part using the Spring TaskExecutor abstraction, so that's not the topic here. The question is: What is the preferred way to ping a URL in java?
Here is my current code as a starting point:
try {
final URLConnection connection = new URL(url).openConnection();
connection.connect();
LOG.info("Service " + url + " available, yeah!");
available = true;
} catch (final MalformedURLException e) {
throw new IllegalStateException("Bad URL: " + url, e);
} catch (final IOException e) {
LOG.info("Service " + url + " unavailable, oh no!", e);
available = false;
}
Is this any good at all (will it do what I want)?
Do I have to somehow close the connection?
I suppose this is a GET request. Is there a way to send HEAD instead?
Is this any good at all (will it do what I want?)
You can do so. Another feasible way is using java.net.Socket.
public static boolean pingHost(String host, int port, int timeout) {
try (Socket socket = new Socket()) {
socket.connect(new InetSocketAddress(host, port), timeout);
return true;
} catch (IOException e) {
return false; // Either timeout or unreachable or failed DNS lookup.
}
}
There's also the InetAddress#isReachable():
boolean reachable = InetAddress.getByName(hostname).isReachable();
This however doesn't explicitly test port 80. You risk to get false negatives due to a Firewall blocking other ports.
Do I have to somehow close the connection?
No, you don't explicitly need. It's handled and pooled under the hoods.
I suppose this is a GET request. Is there a way to send HEAD instead?
You can cast the obtained URLConnection to HttpURLConnection and then use setRequestMethod() to set the request method. However, you need to take into account that some poor webapps or homegrown servers may return HTTP 405 error for a HEAD (i.e. not available, not implemented, not allowed) while a GET works perfectly fine. Using GET is more reliable in case you intend to verify links/resources not domains/hosts.
Testing the server for availability is not enough in my case, I need to test the URL (the webapp may not be deployed)
Indeed, connecting a host only informs if the host is available, not if the content is available. It can as good happen that a webserver has started without problems, but the webapp failed to deploy during server's start. This will however usually not cause the entire server to go down. You can determine that by checking if the HTTP response code is 200.
HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();
connection.setRequestMethod("HEAD");
int responseCode = connection.getResponseCode();
if (responseCode != 200) {
// Not OK.
}
// < 100 is undetermined.
// 1nn is informal (shouldn't happen on a GET/HEAD)
// 2nn is success
// 3nn is redirect
// 4nn is client error
// 5nn is server error
For more detail about response status codes see RFC 2616 section 10. Calling connect() is by the way not needed if you're determining the response data. It will implicitly connect.
For future reference, here's a complete example in flavor of an utility method, also taking account with timeouts:
/**
* Pings a HTTP URL. This effectively sends a HEAD request and returns <code>true</code> if the response code is in
* the 200-399 range.
* #param url The HTTP URL to be pinged.
* #param timeout The timeout in millis for both the connection timeout and the response read timeout. Note that
* the total timeout is effectively two times the given timeout.
* #return <code>true</code> if the given HTTP URL has returned response code 200-399 on a HEAD request within the
* given timeout, otherwise <code>false</code>.
*/
public static boolean pingURL(String url, int timeout) {
url = url.replaceFirst("^https", "http"); // Otherwise an exception may be thrown on invalid SSL certificates.
try {
HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();
connection.setConnectTimeout(timeout);
connection.setReadTimeout(timeout);
connection.setRequestMethod("HEAD");
int responseCode = connection.getResponseCode();
return (200 <= responseCode && responseCode <= 399);
} catch (IOException exception) {
return false;
}
}
Instead of using URLConnection use HttpURLConnection by calling openConnection() on your URL object.
Then use getResponseCode() will give you the HTTP response once you've read from the connection.
here is code:
HttpURLConnection connection = null;
try {
URL u = new URL("http://www.google.com/");
connection = (HttpURLConnection) u.openConnection();
connection.setRequestMethod("HEAD");
int code = connection.getResponseCode();
System.out.println("" + code);
// You can determine on HTTP return code received. 200 is success.
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
if (connection != null) {
connection.disconnect();
}
}
Also check similar question How to check if a URL exists or returns 404 with Java?
Hope this helps.
You could also use HttpURLConnection, which allows you to set the request method (to HEAD for example). Here's an example that shows how to send a request, read the response, and disconnect.
The following code performs a HEAD request to check whether the website is available or not.
public static boolean isReachable(String targetUrl) throws IOException
{
HttpURLConnection httpUrlConnection = (HttpURLConnection) new URL(
targetUrl).openConnection();
httpUrlConnection.setRequestMethod("HEAD");
try
{
int responseCode = httpUrlConnection.getResponseCode();
return responseCode == HttpURLConnection.HTTP_OK;
} catch (UnknownHostException noInternetConnection)
{
return false;
}
}
public boolean isOnline() {
Runtime runtime = Runtime.getRuntime();
try {
Process ipProcess = runtime.exec("/system/bin/ping -c 1 8.8.8.8");
int exitValue = ipProcess.waitFor();
return (exitValue == 0);
} catch (IOException | InterruptedException e) { e.printStackTrace(); }
return false;
}
Possible Questions
Is this really fast enough?Yes, very fast!
Couldn’t I just ping my own page, which I want
to request anyways? Sure! You could even check both, if you want to
differentiate between “internet connection available” and your own
servers beeing reachable What if the DNS is down? Google DNS (e.g.
8.8.8.8) is the largest public DNS service in the world. As of 2013 it serves 130 billion requests a day. Let ‘s just say, your app not
responding would probably not be the talk of the day.
read the link. its seems very good
EDIT:
in my exp of using it, it's not as fast as this method:
public boolean isOnline() {
NetworkInfo netInfo = connectivityManager.getActiveNetworkInfo();
return netInfo != null && netInfo.isConnectedOrConnecting();
}
they are a bit different but in the functionality for just checking the connection to internet the first method may become slow due to the connection variables.
Consider using the Restlet framework, which has great semantics for this sort of thing. It's powerful and flexible.
The code could be as simple as:
Client client = new Client(Protocol.HTTP);
Response response = client.get(url);
if (response.getStatus().isError()) {
// uh oh!
}
I'm a problem with a HttpsURLConnection that I can't seem to solve. Basically, I'm sending up some info to a server and if some of that data is wrong, the server sends me a 500 response code. However, it also sends a message in the response telling me which bit of data was wrong. The problem is that the message is always empty when I read it in. I think this is because a filenotfound exception always gets thrown before the stream can be read. Am I right? I tried reading the errorstream as well but this is always empty. Here's a snippet:
conn = (HttpsURLConnection) connectURL.openConnection();
conn.setDoOutput(true);
conn.setConnectTimeout(30000);
conn.setReadTimeout(30000);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Length",
Integer.toString(outString.getBytes().length));
DataOutputStream wr = new DataOutputStream(conn
.getOutputStream());
wr.write(outString.getBytes());
wr.flush();
wr.close();
if(conn.getResponseCode>400{
String response = getErrorResponse(conn);
public String getErrorResponse(HttpsURLConnection conn) {
Log.i(TAG, "in getResponse");
InputStream is = null;
try {
//is = conn.getInputStream();
is = conn.getErrorStream();
// scoop up the reply from the server
int ch;
StringBuffer sb = new StringBuffer();
while ((ch = is.read()) != -1) {
sb.append((char) ch);
}
//System.out.println(sb.toString());
return sb.toString();
// return conferenceId;
}
catch (Exception e){
e.printStackTrace();
}
}
So just to follow up on this, here is how I solved it:
public static String getResponse(HttpsURLConnection conn) {
Log.i(TAG, "in getResponse");
InputStream is = null;
try {
if(conn.getResponseCode()>=400){
is = conn.getErrorStream();
}
else{
is=conn.getInputStream();
}
...read stream...
}
It seems that calling them like this produced an error stream with a message. Thanks for the suggestions!
Try setting content-type request property to "application/x-www-form-urlencoded"
The same is mentioned on this link:
http://developers.sun.com/mobility/midp/ttips/HTTPPost/
The Content-Length and Content-Type headers are critical because they tell the web server how many bytes of data to expect, and what kind, identified by a MIME type.
In MIDP clients the two most popular MIME types are application/octet-stream, to send raw binary data, and application/x-www-form-urlencoded, to send name-value pairs
Are you in control of the server? In other words, did you write the process that runs on the server and listens to the port you're trying to access?
If you did, then you should also be able to debug it and see why your process returns 404.
If you didn't, then describe your architecture (HTTP server, the component it invokes to respond to your HTTP(S) request, etc) and we'll take it from there.
In the very simplest case, of an HTTP server being an Apache server yielding control to some PHP script, it means that Apache couldn't assign your request to anything. Most likely a Web server misconfiguration. Provide some more details and we'll help you out.