We Keep Coding

SOAP message to webservice - HTTP response code: 403 for URL

I try to send a SOAP message in an XML file to a webservice and than grab the binary output and decode it. Endpoint uses HTTPS protocol, so I used TrustManager in my code to avoid PKIX problems. You can see my code here:
import javax.net.ssl.*;
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.security.cert.X509Certificate;
public class Main{
public static void sendSoapRequest() throws Exception {
String SOAPUrl = "URL HERE";
String xmlFile2Send = ".\\src\\request.xml";
String responseFileName = ".\\src\\response.xml";
String inputLine;
TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() { return null; }
public void checkClientTrusted(X509Certificate[] certs, String authType) { }
public void checkServerTrusted(X509Certificate[] certs, String authType) { }
} };
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
// Create all-trusting host name verifier
HostnameVerifier allHostsValid = new HostnameVerifier() {
public boolean verify(String hostname, SSLSession session) { return true; }
};
// Install the all-trusting host verifier
HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);
// Create the connection with http
URL url = new URL(SOAPUrl);
URLConnection connection = url.openConnection();
HttpURLConnection httpConn = (HttpURLConnection) connection;
FileInputStream fin = new FileInputStream(xmlFile2Send);
ByteArrayOutputStream bout = new ByteArrayOutputStream();
copy(fin, bout);
fin.close();
byte[] b = bout.toByteArray();
StringBuffer buf=new StringBuffer();
String s=new String(b);
b=s.getBytes();
// Set the appropriate HTTP parameters.
httpConn.setRequestProperty("Content-Length", String.valueOf(b.length));
httpConn.setRequestProperty("Content-Type", "text/xml; charset=utf-8");
httpConn.setRequestProperty("SOAPAction", "");
httpConn.setRequestMethod("POST");
httpConn.setDoOutput(true);
OutputStream out = httpConn.getOutputStream();
out.write(b);
out.close();
// Read the response.
httpConn.connect();
System.out.println("http connection status :"+ httpConn.getResponseMessage());
InputStreamReader isr = new InputStreamReader(httpConn.getInputStream());
BufferedReader in = new BufferedReader(isr);
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
FileOutputStream fos=new FileOutputStream(responseFileName);
copy(httpConn.getInputStream(),fos);
in.close();
}
public static void copy(InputStream in, OutputStream out) throws IOException {
synchronized (in) {
synchronized (out) {
byte[] buffer = new byte[256];
while (true) {
int bytesRead = in.read(buffer);
if (bytesRead == -1)
break;
out.write(buffer, 0, bytesRead);
}
}
}
}
public static void main(String args[]) throws Exception {
sendSoapRequest();
}
}
I get following error code, when I execute this.
Exception in thread "main" java.io.IOException: Server returned HTTP
response code: 403 for URL

Your implementation is alright, the problem is related to your Content-Type header, in fact.
The value text/xml; charset=utf-8 is the default Content-Type of SOAP 1.1, which is probably not the version of yours. SOAP 1.2 expects a header of type application/soap+xml; charset=utf-8, so changing your line of code to this one below is gonna make it working:
httpConn.setRequestProperty("Content-Type", "application/soap+xml; charset=utf-8");
In SoapUI, it's possible to check the headers calling the request and going to the Headers tab on the bottom of the window:
Then, you can compare the differences between your application configs and the SoapUI ones.

403 error might be related to your soap request headers being sent to the server.
All Hosts valid will allow your Java App to trust the SSL Cert for the URL.
Check if your server is expecting soap header with username/password. If you have access to this server, you can check through the web server logs on where your request is failing. Error code points to to missing Soap Header particularly Soap Headers with username and password

Wonder if your SOAP request contains any kind of authentication information in headers like SAML. One option is, in your above code where you read the file and send the data to server, instead of sending it to server you dump it to another file. Dump that byteoutputstream. Then copy text from that file and put it in SOAP UI and try running that. Does that work?

In a similar situation we have been some time before, and as long as trying TrustManager didn't work as expected, we managed to overcome this problem by installing the certificate from server to JVM's keystore (JVM used to run the application). More information about how to do it you can find in several posts, like
How to import a .cer certificate into a java keystore?
I am aware that it is a try to force JVM to accept SSL certificates, and this functionality would be better to live in application context, but as long as we were building a web application which ran in specific application servers, the implemented solution was an accepted one.

How to force Commons HTTPClient 3.1 to use TLS 1.2 only for HTTPS?

I wish to force Apache Commons HTTP-Client (version 3.1) to use TLS 1.2 as the only protocol for HTTPS.
This is due to the server supposedly being upgraded to TLS 1.2 and not accepting any older protocol anymore (causing 'Connection Reset' to be returned).
For further context, probably irrelevant, the HTTP-Client is used along with Axis2 to make a SOAP; some of the code used for setting up the HttpClient is below:
MultiThreadedHttpConnectionManager connMgr = new MultiThreadedHttpConnectionManager();
this.httpClient = new HttpClient(connMgr);
// initialize HttpClient parameters
HttpClientParams hcParams = this.httpClient.getParams();
// Maximum time to wait to receive connection from pool
hcParams.setConnectionManagerTimeout(this.maxWait);
hcParams.setSoTimeout(this.timeout);
hcParams.setParameter(HttpMethodParams.RETRY_HANDLER, new DefaultHttpMethodRetryHandler(this.retryCount, false));
// Initialize global Connection manager parameters
HttpConnectionManagerParams cmParams = connMgr.getParams();
cmParams.setDefaultMaxConnectionsPerHost(this.maxActive);
cmParams.setStaleCheckingEnabled(this.checkStaleConnections);
cmParams.setConnectionTimeout(this.timeout);
Thanks a lot for the help!

Too bad nobody answered; I was able to do it, first you write a CustomHttpSocketFactory, then you do:
String scheme = "https";
Protocol baseHttps = Protocol.getProtocol(scheme);
int defaultPort = baseHttps.getDefaultPort();
ProtocolSocketFactory baseFactory = baseHttps.getSocketFactory();
ProtocolSocketFactory customFactory = new CustomHttpsSocketFactory(baseFactory);
Protocol customHttps = new Protocol(scheme, customFactory, defaultPort);
Protocol.registerProtocol(scheme, customHttps);
A sample custom socket factory code is found here, but instead I did:
public class CustomHttpsSocketFactory implements SecureProtocolSocketFactory
{
private final SecureProtocolSocketFactory base;
public CustomHttpsSocketFactory(ProtocolSocketFactory base)
{
if(base == null || !(base instanceof SecureProtocolSocketFactory)) throw new IllegalArgumentException();
this.base = (SecureProtocolSocketFactory) base;
}
private Socket acceptOnlyTLS12(Socket socket)
{
if(!(socket instanceof SSLSocket)) return socket;
SSLSocket sslSocket = (SSLSocket) socket;
sslSocket.setEnabledProtocols(new String[]{"TLSv1.2" });
return sslSocket;
}
#Override
public Socket createSocket(String host, int port) throws IOException
{
return acceptOnlyTLS12(base.createSocket(host, port));
}
#Override
public Socket createSocket(String host, int port, InetAddress localAddress, int localPort) throws IOException
{
return acceptOnlyTLS12(base.createSocket(host, port, localAddress, localPort));
}
#Override
public Socket createSocket(String host, int port, InetAddress localAddress, int localPort, HttpConnectionParams params) throws IOException
{
return acceptOnlyTLS12(base.createSocket(host, port, localAddress, localPort, params));
}
#Override
public Socket createSocket(Socket socket, String host, int port, boolean autoClose) throws IOException
{
return acceptOnlyTLS12(base.createSocket(socket, host, port, autoClose));
}
}

You need a Socket reference in your code. Then you can set enabled protocols on it like this:
if (socket != null && (socket instanceof SSLSocket)) {
((SSLSocket)socket).setEnabledProtocols(new String[] {"TLSv1.2"});
}

It depends on how you are writing your clients and what JRE versions you are using:
If you are using JRE8 (unless you have replaced the default SunJSSE that comes with JRE8), there is a system property "jdk.tls.client.protocols". By default, whatever you mention here will be used for all client communication.
If you are using HttpsURLConnection object for client connection, u can use the system property "https.protocols". This will work for all JRE versions, not just JRE8.
If you don't specify anything, for TLS clients, in JRE8, TLSv1, v1.1 and v1.2 are enabled, so it will work with a server what supports any one of this versions. However in JRE7 by default TLSv1 alone is enabled.
In your code u can always override the default or what u pass through the system properties. What u set in the code will take higher precedence. To override in the code...
1) If you are using raw socket and SSLEngine, u can set the protocol and ciphers in the SSLEngine (sslEngine.setEnabledProtocols(..)
2) If you are using SSLSocket, u can set the protocol and ciphers in the SSLSocket (sslSocket.setEnabledProtocols(..)
You can also get an SSLContext with the required protocol enabled and use that for whatever SSL components you use. SSLContext.getInstance("TLSvx.x"). Note that by default it will return a context with all the protocols lesser that TLSvx.x enabled. If u have configured "jdk.tls.client.protocols", this will return a context with those protocols enabled.
It would not be a good idea to hard coded the protocols in the code. Quite often, we will encounter certain customers want specific version either because they use old servers or some serious vulnerabilities are encountered in some TLS versions. Either set it through the system properties or even if you are explicitly setting in sslsocket or sslengine, read that from some conf file.
Also refer:
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html
http://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html

How to create a custom secure socket connection between client/server?

I am developing a client/server application based on Java sockets and I would like some guide lines on how to make those secure/encrypted.
So, I have been reading the last two weeks about SSL, security, public and private key encryption, symmetric and asymmetric methodologies and I would like to implement a safe connection between client/server with the approach below:
Server: Create a public key for encryption and private key for decryption
Client: Uses Server's public key to encrypt his credentials along with a timestamp of request so as the encrypted value won't be the same every time he wants to authenticate himself. Attached to this message clients also includes the shared key that both client/server should connect (symmetric encryption).
Server: Decrypts the message with his private key, check user's credentials and timestamp. If credentials are correct, then it uses the symmetric key to send a response of success.
Rest of communication continues with client's symmetric key.
The below Java code is an example of how SSL communication can be achieved with SSL sockets. clientKeyStore.jks is the Client's Java Keystore which holds the server's public key. When a socket is created, I guess, that the client uses the public key to initiate the communication. My question is where the exchange of symmetric key happens, (if it happens?). Are there any vulnerabilities that I should have in mind, or is this safe from Man-In-The-Middle-Attack?
Thank you in advance
SSLSocketServer
public class SSLSocketServer {
private SSLServerSocket serverSocket;
public SSLSocketServer(int port) throws IOException {
System.setProperty("javax.net.ssl.keyStore","serverKeyStore.jks");
System.setProperty("javax.net.ssl.keyStorePassword", "serverKeyStorePass");
SSLServerSocketFactory factory = (SSLServerSocketFactory) SSLServerSocketFactory.getDefault();
serverSocket = (SSLServerSocket) factory.createServerSocket(port);
start();
}
public void start() {
while (true) {
try {
// Client is connected
SSLSocket clientSocket = (SSLSocket) serverSocket.accept();
System.out.println("User connected successfully?");
} // try
catch (IOException e) {
e.printStackTrace();
} // catch
} // while
} // run
} // class SSLSocketServer
SSLSocketClient
public class SSLSocketClient {
public SSLSocketClient(String serverHost, int serverPort) throws UnknownHostException, IOException {
System.setProperty("javax.net.ssl.trustStore","clientKeyStore.jks");
System.setProperty("javax.net.ssl.trustStorePassword", "clientKeyStorePass");
SSLSocketFactory sslsocketfactory = (SSLSocketFactory) SSLSocketFactory.getDefault();
SSLSocket sslsocket = (SSLSocket) sslsocketfactory.createSocket(serverHost, serverPort);
} // SocketClient()
} // class SSLSocketClient

How to use a custom socketfactory in Apache HttpComponents

I have been trying to use a custom SocketFactory in the httpclient library from the Apache HTTPComponents project. So far without luck. I was expecting that I could just set a socket factory for a HttpClient instance, but it is obviously not so easy.
The documentation for HttpComponents at http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html does mention socket factories, but does not say how to use them.
Does anybody know how this is done?

oleg's answer is of course correct, I just wanted to put the information directly here, in case the link goes bad. In the code that creates a HttpClient, I use this code to let it use my socket factory:
CustomSocketFactory socketFactory = new CustomSocketFactory();
Scheme scheme = new Scheme("http", 80, socketFactory);
httpclient.getConnectionManager().getSchemeRegistry().register(scheme);
CustomSocketFactory is my own socket factory, and I want to use it for normal HTTP traffic, that's why I use "http" and 80 as parameters.
My CustomSchemeSocketFactory looks similar to this:
public class CustomSchemeSocketFactory implements SchemeSocketFactory {
#Override
public Socket connectSocket( Socket socket, InetSocketAddress remoteAddress, InetSocketAddress localAddress, HttpParams params ) throws IOException, UnknownHostException, ConnectTimeoutException {
if (localAddress != null) {
socket.setReuseAddress(HttpConnectionParams.getSoReuseaddr(params));
socket.bind(localAddress);
}
int connTimeout = HttpConnectionParams.getConnectionTimeout(params);
int soTimeout = HttpConnectionParams.getSoTimeout(params);
try {
socket.setSoTimeout(soTimeout);
socket.connect(remoteAddress, connTimeout );
} catch (SocketTimeoutException ex) {
throw new ConnectTimeoutException("Connect to " + remoteAddress + " timed out");
}
return socket;
}
#Override
public Socket createSocket( HttpParams params ) throws IOException {
// create my own socket and return it
}
#Override
public boolean isSecure( Socket socket ) throws IllegalArgumentException {
return false;
}
}

We use a custom socket factory to allow HttpClient connections to connect to HTTPS URLs with untrusted certificates.
Here is how we did it:
We adapted implementations of both the 'EasySSLProtocolSocketFactory' and 'EasyX509TrustManager' classes from the examples source directory referenced by Oleg.
In our HttpClient startup code, we do the following to enable the new socket factory:
HttpClient httpClient = new HttpClient();
Protocol easyhttps = new Protocol("https", new EasySSLProtocolSocketFactory(), 443);
Protocol.registerProtocol("https", easyhttps);
So that any time we request an https:// URL, this socket factory is used.

TLS connection using SSLSocket is slow in Android OS

I'm developing an Android app which uses SSLSocket to connect to a server. This is the code I'm using:
// Connect
if (socket == null || socket.isClosed() || !socket.isConnected()) {
if (socket != null && !socket.isClosed())
socket.close();
Log.i(getClass().toString(), "Connecting...");
if (sslContext == null) {
sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, trustAllCerts, new SecureRandom());
}
SSLSocketFactory socketFactory = sslContext.getSocketFactory();
socket = (SSLSocket)socketFactory.createSocket(host, port);
socket.setSoTimeout(20000);
socket.setUseClientMode(true);
connected = true;
Log.i(getClass().toString(), "Connected.");
}
// Secure
if (connected) {
Log.i(getClass().toString(), "Securing...");
SSLSession session = socket.getSession();
secured = session.isValid();
if (secured) {
Log.i(getClass().toString(), "Secured.");
}
else
Log.i(getClass().toString(), "Securing failed.");
}
The problem is that it takes about 5 seconds or event more to do the TLS handshake in the line below:
SSLSession session = socket.getSession();
I have made a similar iPhone app, the handshake takes just 1 second there, so I think the problem is not in the server I'm connecting to, it's maybe in the code above. The connection itself is fast enough, just the TLS handshake is slow.
Does anybody know if it's normal in Android, or if it is not, how to make it faster?
Thank you.
EDITED on 21.01.11:
I have found out, that the handshake is fast when I connect to another server, for example paypal.com:443.
But I had been connecting to another server before - a .NET service written by me. As I had said before, I did not think the problem was in that server because if I connect to it with my iPhone App the handshake is fast. Now I don't know why it is fast on iPhone and slow on Android. After the connection is established, the only thing I do in the .NET server is:
Console.WriteLine("New client connected.");
this.sslStream = new SslStream(tcpClient.GetStream(), true);
this.sslStream.ReadTimeout = 15000;
this.sslStream.WriteTimeout = 15000;
Console.WriteLine("Beginning TLS handshake...");
this.sslStream.AuthenticateAsServer(connection.ServerCertificate, false, SslProtocols.Tls, false);
Console.WriteLine("TLS handshake completed.");

There was a bug on earlier versions of the Android SDK. Apparently, it's doing an unnecessary DNS reverse lookup. You need to prevent this from happening. Here's a workaround that worked for me. It used to take 15 seconds, now it takes 0-1 seconds. Hope it helps.
Here's the link to the Google issue.
boolean connected = false;
if (socket == null || socket.isClosed() || !socket.isConnected()) {
if (socket != null && !socket.isClosed()) {
socket.close();
}
Log.i(getClass().toString(), "Connecting...");
messages.getText().append("Connecting...");
final KeyStore keyStore = KeyStore.getInstance("BKS");
keyStore.load(getResources().openRawResource(R.raw.serverkey), null);
final KeyManagerFactory keyManager = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keyManager.init(keyStore, null);
//keyManager.init(null, null);
final TrustManagerFactory trustFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustFactory.init(keyStore);
sslContext = SSLContext.getInstance("TLS");
sslContext.init(keyManager.getKeyManagers(), trustFactory.getTrustManagers(), rnd);
final SSLSocketFactory delegate = sslContext.getSocketFactory();
SocketFactory factory = new SSLSocketFactory() {
#Override
public Socket createSocket(String host, int port)
throws IOException, UnknownHostException {
InetAddress addr = InetAddress.getByName(host);
injectHostname(addr, host);
return delegate.createSocket(addr, port);
}
#Override
public Socket createSocket(InetAddress host, int port)
throws IOException {
return delegate.createSocket(host, port);
}
#Override
public Socket createSocket(String host, int port, InetAddress localHost, int localPort)
throws IOException, UnknownHostException {
return delegate.createSocket(host, port, localHost, localPort);
}
#Override
public Socket createSocket(InetAddress address, int port, InetAddress localAddress, int localPort)
throws IOException {
return delegate.createSocket(address, port, localAddress, localPort);
}
private void injectHostname(InetAddress address, String host) {
try {
Field field = InetAddress.class.getDeclaredField("hostName");
field.setAccessible(true);
field.set(address, host);
} catch (Exception ignored) {
}
}
#Override
public Socket createSocket(Socket s, String host, int port, boolean autoClose) throws IOException {
injectHostname(s.getInetAddress(), host);
return delegate.createSocket(s, host, port, autoClose);
}
#Override
public String[] getDefaultCipherSuites() {
return delegate.getDefaultCipherSuites();
}
#Override
public String[] getSupportedCipherSuites() {
return delegate.getSupportedCipherSuites();
}
};
socket = (SSLSocket)factory.createSocket("192.168.197.133", 9999);
socket.setSoTimeout(20000);
socket.setUseClientMode(true);
connected = true;
Log.i(getClass().toString(), "Connected.");
messages.getText().append("Connected.");
}
// Secure
if (connected) {
Log.i(getClass().toString(), "Securing...");
messages.getText().append("Securing...");
SSLSession session = socket.getSession();
boolean secured = session.isValid();
if (secured) {
Log.i(getClass().toString(), "Secured.");
messages.getText().append("Secured.");
}
}

You are using a new SecureRandom per connection, instead of using a single static pre-initialized SecureRandom. Everytime you create a new SecureRandom(), you need to gather entropy for seeding (a slow process).
SecureRandom does not self-seed until it is first used, which is why the delay does not occur until the call to getSession()

I have done something similar to this and it is slower than an unsecured connection. Granted my case was https vs http and it is a little different the SSL/TLS factor will add slowness to the deal.
I have two identical apps that comunicate with the same protocol to the same server, one in android and one in iPhone, both using https. When I tested them both in http I would see more or less the same response time, in https iOS was slightly faster in my case, but not terribly.

The problem is most likely in the way the device validates server certificates. Validation can involve contacting third-party for CRLs and OCSP responses. If this happens, it takes time. iPhone probably just doesn't do this (at least by default) which is a security hole BTW.