I am trying to make a Soap request using Spring WS. The request needs to send across client certificate to the server. I have figured out the configuration that makes the correct request to the server. Code presented below
#Bean
public HttpsUrlConnectionMessageSender messageSender() throws Exception {
HttpsUrlConnectionMessageSender httpsUrlConnectionMessageSender = new HttpsUrlConnectionMessageSender();
httpsUrlConnectionMessageSender.setTrustManagers(<TrustManager>);
httpsUrlConnectionMessageSender.setKeyManagers(<KeyManager>);
httpsUrlConnectionMessageSender.setHostnameVerifier(new HostnameVerifier() {
public boolean verify(String hostname, SSLSession session) {
return true;
}
});
return httpsUrlConnectionMessageSender;
}
#Bean
#Scope
public WebServiceTemplate webServiceTemplate() throws Exception {
WebServiceTemplate webServiceTemplate = new WebServiceTemplate(<messageFactory>);
webServiceTemplate.setMarshaller(<jaxb2Marshaller>);
webServiceTemplate.setUnmarshaller(<jaxb2Marshaller>);
webServiceTemplate.setDefaultUri(<URL>);
webServiceTemplate.setMessageSender(());
return webServiceTemplate;
}
Now I was wondering how does HttpsUrlConnectionMessageSender manage connections? Does it create new connection for every request or it has internal connection pooling. How do we use external connection pooling here?
The client has default connection pool of 20 and default connection pool per request 2.
But you can overwrite it like this:
public HttpComponentsMessageSender httpComponentsMessageSender(Integer readTimeout, Integer connectionTimeOut,
Integer throttlingLimit) {
HttpComponentsMessageSender sender = new HttpComponentsMessageSender();
sender.setReadTimeout(readTimeout);
sender.setConnectionTimeout(connectionTimeOut);
PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
CloseableHttpClient poolingHttpSoapClient = HttpClients.custom().setConnectionManager(connManager)
.addInterceptorFirst(new RemoveSoapHeadersInterceptor()).setMaxConnPerRoute(throttlingLimit)
.setMaxConnTotal(throttlingLimit + 1).build();
sender.setHttpClient(poolingHttpSoapClient);
return sender;
}
Related
I'm very new to automation and I'm trying to write a code in Java to setup remote connection using proxy and okhttp. But, it keeps throwing error:
org.openqa.selenium.SessionNotCreatedException: Unable to create a new remote session. Original error: Too many tunnel connections attempted: 21
Getting the error on the last line of the code snippet, i.e. on the return statement.
Request your help in finding the issue in my code. Below is my code snippet:
Function is being called as below:
General.driver = connectViaProxy(caps);
Implementation of connectViaProxy:
public static AndroidDriver<AndroidElement> connectViaProxy(DesiredCapabilities caps) throws FileNotFoundException, IOException {
String proxyHost = Config.getValue("proxy.host");
int proxyPort = Config.getValueint("proxy.port");
String proxyUserDomain = Config.getValue("proxy.user-domain");
String proxyUser = Config.getValue("proxy.user");
String proxyPassword=EncryptionUtil.passwordDecoder(Config.getValue("proxy.encrypted-password").getBytes());
URL url;
try {
url = new URL("https://"+Config.getValue("BrowserStack.userName")+":"+Config.getValue("BrowserStack.accessKey")+"#hub-cloud.browserstack.com/wd/hub");
} catch (MalformedURLException e) {
throw new RuntimeException(e.getMessage(), e);
}
Authenticator proxyAuthenticator = new Authenticator()
{
#Override
public Request authenticate(Route route, Response response) throws IOException {
String credential = Credentials.basic(proxyUserDomain + "\\" + proxyUser, proxyPassword);
return response.request().newBuilder()
.header("Proxy-Authorization", credential)
.build();
}
};
okhttp3.OkHttpClient.Builder client = new okhttp3.OkHttpClient.Builder()
.connectTimeout(60, TimeUnit.SECONDS)
.writeTimeout(60, TimeUnit.SECONDS)
.readTimeout(60, TimeUnit.SECONDS)
.proxy(new Proxy(Proxy.Type.HTTP, new InetSocketAddress(proxyHost, proxyPort)))
.proxyAuthenticator(proxyAuthenticator);
Factory factory = new MyHttpClientFactory(new org.openqa.selenium.remote.internal.OkHttpClient(client.build(), url));
HttpCommandExecutor executor = new AppiumCommandExecutor(MobileCommand.commandRepository, url, factory);
return new AndroidDriver<AndroidElement>(executor, caps);
}
I believe the proxy in your network is not passing the first request made by your test script to establish connection to BrowserStack.
Try to whitelist *.browserstack.com in your proxy for ports 80 and 443.
If that does not work, use local testing and follow the steps from the below documentation so that the request to BrowserStack is by-passed by the proxy.
https://www.browserstack.com/docs/automate/selenium/test-behind-proxy/configure-settings
I am having a problem in my Spring Boot application where I am using a RestTemplate which is autowired for making POST calls to a REST service. The code where I am initializing the REstTemplate bean is as follows:
#Bean(name = "template")
public RestTemplate restTemplate(RestTemplateBuilder builder) throws NoSuchAlgorithmException, KeyManagementException {
TrustManager[] trustAllCerts = new TrustManager[] {
new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[0];
}
public void checkClientTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
}
};
SSLContext sslContext = SSLContext.getInstance("SSL");
sslContext.init(null, trustAllCerts, new java.security.SecureRandom());
HostnameVerifier hostnameVerifier = SSLConnectionSocketFactory.getDefaultHostnameVerifier();
// Conn Pool
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(
RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", new SSLConnectionSocketFactory(sslContext, hostnameVerifier))
.build());
cm.setMaxTotal(Integer.MAX_VALUE);
cm.setDefaultMaxPerRoute(50);
CloseableHttpClient httpClient = HttpClients.custom()
.setSSLContext(sslContext)
.setConnectionManager(cm)
.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
.build();
HttpComponentsClientHttpRequestFactory customRequestFactory = new HttpComponentsClientHttpRequestFactory();
customRequestFactory.setHttpClient(httpClient);
return builder.requestFactory(() -> customRequestFactory).build();
}
The RestTemplate is then autowired:
#Autowired
#Qualifier("template")
private RestTemplate restTemplate;
And the RestTemplate is used thus:
restResponse = restTemplate.postForObject(targetUrl, obj, CustomRestResponse.class);
I am deploying the application to PCF where it is running with 6 instances. It's then giving me an error like this:
org.springframework.web.client.RestClientException: Error while extracting response for type [class com.test.model.CustomRestResponse] and content type [application/json;charset=UTF-8]; nested exception is org.apache.http.ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected
To replicate the issue in my local environment, I created a multithreaded application which will spawn multiple threads and call the REST service simultaneously:
ExecutorService service = Executors.newFixedThreadPool(10);
IntStream.range(0, 100).forEach(i -> service.submit(new RestTemplateTesterRunnable(i)));
And I was able to get the exception in my local system as well (Note that I don't have this multithreading code in the actual application. This is just for replicating the problem in my local since I can't run 6 instances automatically). The full stacktrace is as follows:
org.springframework.web.client.RestClientException: Error while extracting response for type [class com.test.model.CustomRestResponse] and content type [application/json;charset=UTF-8]; nested exception is org.apache.http.ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:119)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:744)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:677)
at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:421)
at com.test.RestTemplateTester$RestTemplateTesterRunnable.run(RestTemplateTester.java:237)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.http.ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected
at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:263)
at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:183)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:210)
at org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:312)
at org.apache.http.impl.execchain.ResponseEntityProxy.streamClosed(ResponseEntityProxy.java:142)
at org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
at org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
at java.io.PushbackInputStream.close(PushbackInputStream.java:379)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._closeInput(UTF8StreamJsonParser.java:252)
at com.fasterxml.jackson.core.base.ParserBase.close(ParserBase.java:369)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4210)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3258)
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:239)
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:227)
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:104)
... 9 more
However, I am not being able to solve the problem. I referred this question:
How to efficiently use RestTemplate in multithreading application?
and this one:
How to use RestTemplate efficiently in Multithreaded environment?
and in both of them, people are saying that RestTemplate is thread-safe and there shouldn't be a problem in using it in a multithreaded environment. However, what I am seeing is that this exception is still happening for me. I thought at first that this was happening due to my use of the CloseableHttpClient, as is given by this post:
ConnectionClosedException: Premature end of chunk coded message body with apache hhtpclient 4.5.5
However, as I am not directly using the client, which is internally used by Spring Boot, I didn't have much to do about it. But I looked into the internal working of the RestTemplate's postForObject() method in any case, which led me to this method:
#Nullable
protected <T> T doExecute(URI url, #Nullable HttpMethod method, #Nullable RequestCallback requestCallback,
#Nullable ResponseExtractor<T> responseExtractor) throws RestClientException {
Assert.notNull(url, "URI is required");
Assert.notNull(method, "HttpMethod is required");
ClientHttpResponse response = null;
try {
ClientHttpRequest request = createRequest(url, method);
if (requestCallback != null) {
requestCallback.doWithRequest(request);
}
response = request.execute();
handleResponse(url, method, response);
return (responseExtractor != null ? responseExtractor.extractData(response) : null);
}
catch (IOException ex) {
String resource = url.toString();
String query = url.getRawQuery();
resource = (query != null ? resource.substring(0, resource.indexOf('?')) : resource);
throw new ResourceAccessException("I/O error on " + method.name() +
" request for \"" + resource + "\": " + ex.getMessage(), ex);
}
finally {
if (response != null) {
response.close();
}
}
}
I am seeing that before returning, the response object is being closed, as was being told in the previous StackOverflow post I linked. Can this be the reason for the ConnectionClosedException in a multithreaded environment? If so, what is the solution? I tried to use evictIdleConnections() as given by the Javadocs:
https://hc.apache.org/httpcomponents-client-4.5.x/current/httpclient/apidocs/org/apache/http/impl/client/HttpClientBuilder.html#evictIdleConnections(long,%20java.util.concurrent.TimeUnit)
So the code became:
CloseableHttpClient httpClient = HttpClients.custom()
.setSSLContext(sslContext)
.setConnectionManager(cm)
.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
.evictIdleConnections(10, TimeUnit.MILLISECONDS)
.build();
However, this didn't help. I also saw that some people are using org.apache.commons.httpclient.MultiThreadedHttpConnectionManager, but I found that this is a deprecated connection manager, and the HttpComponentsClientHttpRequestFactory does not support it. Nevertheless, I somehow used it also, but with no results. Please hellp me in getting the actual reason for this exception, and how to resolve it.
I'm using Jedis, I cannot connect directly to Redis, I have to use a proxy.
Can I connect to Redis with Jedis using a socks proxy?
Can you help me, please.
Regards.
I was looking for a solution but could n't find any. So I have done the following changes to fix this issue:
#NotNull
private ProxySelector proxySelector() {
return new ProxySelector() {
#Override
public List<Proxy> select(URI uri) {
List<Proxy> proxies = new LinkedList<>();
InetSocketAddress socketAddress =
new InetSocketAddress(proxyHost, proxyPort);
proxies.add(new Proxy(Type.HTTP, socketAddress));
return proxies;
}
#Override
public void connectFailed(URI uri, SocketAddress sa, IOException e) {
log.error(String.format(
"Connection to proxy server %s with socket address %s failed with error. %s",
uri, sa, e));
}
};
}
and then set the default proxy selector:
ProxySelector.setDefault(proxySelector());
How can I use Socks5 proxy in Okhttp to start http request ?
My code:
Proxy proxy = new Proxy(Proxy.Type.SOCKS, InetSocketAddress.createUnresolved(
"socks5host", 80));
OkHttpClient client = new OkHttpClient.Builder()
.proxy(proxy).authenticator(new Authenticator() {
#Override
public Request authenticate(Route route, Response response) throws IOException {
if (HttpUtils.responseCount(response) >= 3) {
return null;
}
String credential = Credentials.basic("user", "psw");
if (credential.equals(response.request().header("Authorization"))) {
return null; // If we already failed with these credentials, don't retry.
}
return response.request().newBuilder().header("Authorization", credential).build();
}
}).build();
Request request = new Request.Builder().url("http://google.com").get().build();
Response response = client.newCall(request).execute(); <--- **Here, always throw java.net.UnknownHostException: Host is unresolved: google.com**
System.out.println(response.body().string());
How to avoid UnknownHostException?
Any example ?
Thanks!
I found a solution: When create a OkHttpClient.Builder(), set a new socketFactory instead of set proxy, and return a sock5 proxy inside socketFactory createSocket.
I think it's the easiest working soulution. But it seems to me that it can be not 100% safe. I took this code from this code from here and modified it because my proxy's RequestorType is SERVER.
Actually, java has a strange api for proxies, you should to set auth for proxy through system env ( you can see it from the same link)
final int proxyPort = 1080; //your proxy port
final String proxyHost = "your proxy host";
final String username = "proxy username";
final String password = "proxy password";
InetSocketAddress proxyAddr = new InetSocketAddress(proxyHost, proxyPort);
Proxy proxy = new Proxy(Proxy.Type.SOCKS, proxyAddr);
Authenticator.setDefault(new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
if (getRequestingHost().equalsIgnoreCase(proxyHost)) {
if (proxyPort == getRequestingPort()) {
return new PasswordAuthentication(username, password.toCharArray());
}
}
return null;
}
});
OkHttpClient client = new OkHttpClient.Builder()
.proxy(proxy)
.build();
I have been trying to use a custom SocketFactory in the httpclient library from the Apache HTTPComponents project. So far without luck. I was expecting that I could just set a socket factory for a HttpClient instance, but it is obviously not so easy.
The documentation for HttpComponents at http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html does mention socket factories, but does not say how to use them.
Does anybody know how this is done?
oleg's answer is of course correct, I just wanted to put the information directly here, in case the link goes bad. In the code that creates a HttpClient, I use this code to let it use my socket factory:
CustomSocketFactory socketFactory = new CustomSocketFactory();
Scheme scheme = new Scheme("http", 80, socketFactory);
httpclient.getConnectionManager().getSchemeRegistry().register(scheme);
CustomSocketFactory is my own socket factory, and I want to use it for normal HTTP traffic, that's why I use "http" and 80 as parameters.
My CustomSchemeSocketFactory looks similar to this:
public class CustomSchemeSocketFactory implements SchemeSocketFactory {
#Override
public Socket connectSocket( Socket socket, InetSocketAddress remoteAddress, InetSocketAddress localAddress, HttpParams params ) throws IOException, UnknownHostException, ConnectTimeoutException {
if (localAddress != null) {
socket.setReuseAddress(HttpConnectionParams.getSoReuseaddr(params));
socket.bind(localAddress);
}
int connTimeout = HttpConnectionParams.getConnectionTimeout(params);
int soTimeout = HttpConnectionParams.getSoTimeout(params);
try {
socket.setSoTimeout(soTimeout);
socket.connect(remoteAddress, connTimeout );
} catch (SocketTimeoutException ex) {
throw new ConnectTimeoutException("Connect to " + remoteAddress + " timed out");
}
return socket;
}
#Override
public Socket createSocket( HttpParams params ) throws IOException {
// create my own socket and return it
}
#Override
public boolean isSecure( Socket socket ) throws IllegalArgumentException {
return false;
}
}
We use a custom socket factory to allow HttpClient connections to connect to HTTPS URLs with untrusted certificates.
Here is how we did it:
We adapted implementations of both the 'EasySSLProtocolSocketFactory' and 'EasyX509TrustManager' classes from the examples source directory referenced by Oleg.
In our HttpClient startup code, we do the following to enable the new socket factory:
HttpClient httpClient = new HttpClient();
Protocol easyhttps = new Protocol("https", new EasySSLProtocolSocketFactory(), 443);
Protocol.registerProtocol("https", easyhttps);
So that any time we request an https:// URL, this socket factory is used.