We Keep Coding

AnnotatedConnectException - Springboot - Microservices - Distributed Server using eureka server

Eureka server is running in different server and customer service running in different server. The customer service is registered and available for feign call as well, but randomly customer service is de-registering from eureka server due below exception and going offline.
io.netty.channel.AbstractChannel$AnnotatedConnectException
message
syscall:getsockopt(..) failed: Connection refused
Whether any configuration missed.
Error Log
com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out
at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187)
at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27)
at com.sun.jersey.api.client.Client.handle(Client.java:652)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.put(WebResource.java:529)
at com.netflix.discovery.shared.transport.jersey.AbstractJerseyEurekaHttpClient.sendHeartBeat(AbstractJerseyEurekaHttpClient.java:102)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.MetricsCollectingEurekaHttpClient.execute(MetricsCollectingEurekaHttpClient.java:73)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.RedirectingEurekaHttpClient.execute(RedirectingEurekaHttpClient.java:89)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:120)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.DiscoveryClient.renew(DiscoveryClient.java:846)
at com.netflix.discovery.DiscoveryClient$HeartbeatThread.run(DiscoveryClient.java:1405)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:933)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:161)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:82)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:276)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:294)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:257)
at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:230)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:679)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:481)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:173)

api unable to send the traces to aws-xray deamon in docker container

I have two docker containers for rest API and x-ray daemon up and running. In my local, API sends the traces to x-ray daemon successfully. But unable to achieve in docker.
These steps I have followed.
added was x-ray filter to API to instrument HTTP request
configured 0.0.0.0:2000 as x-ray deamon address to listen all request
pulled AWS x-ray daemon official docker image and run the image in docker
whenever I am trying to hit my API,I am getting exception like this
2020-06-03 08:08:33.799 INFO 1 --- [pool-1-thread-1] c.a.x.s.sampling.pollers.RulePoller : Polling sampling rules.2020-06-03 08:08:34.413 ERROR 1 --- [pool-1-thread-1] c.a.x.s.sampling.pollers.RulePoller : Encountered error polling GetSamplingRules:
com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to 0.0.0.0:2000 [/0.0.0.0] failed: Connection refused (Connection refused)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1207) ~[aws-java-sdk-core-1.11.787.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1153) ~[aws-java-sdk-core-1.11.787.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802) ~[aws-java-sdk-core-1.11.787.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) ~[aws-java-sdk-core-1.11.787.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) ~[aws-java-sdk-core-1.11.787.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) ~[aws-java-sdk-core-1.11.787.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) ~[aws-java-sdk-core-1.11.787.jar!/:na]
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) ~[aws-java-sdk-core-1.11.787.jar!/:na] at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) ~[aws-java-sdk-core-1.11.787.jar!/:na] at com.amazonaws.services.xray.AWSXRayClient.doInvoke(AWSXRayClient.java:1607) ~[aws-java-sdk-xray-1.11.787.jar!/:na]
at com.amazonaws.services.xray.AWSXRayClient.invoke(AWSXRayClient.java:1574) ~[aws-java-sdk-xray-1.11.787.jar!/:na]
at com.amazonaws.services.xray.AWSXRayClient.invoke(AWSXRayClient.java:1563) ~[aws-java-sdk-xray-1.11.787.jar!/:na]
at com.amazonaws.services.xray.AWSXRayClient.executeGetSamplingRules(AWSXRayClient.java:800) ~[aws-java-sdk-xray-1.11.787.jar!/:na]
at com.amazonaws.services.xray.AWSXRayClient.getSamplingRules(AWSXRayClient.java:771) ~[aws-java-sdk-xray-1.11.787.jar!/:na]
at com.amazonaws.xray.strategy.sampling.pollers.RulePoller.pollRule(RulePoller.java:65) ~[aws-xray-recorder-sdk-core-2.4.0.jar!/:na]
at com.amazonaws.xray.strategy.sampling.pollers.RulePoller.lambda$start$0(RulePoller.java:46) ~[aws-xray-recorder-sdk-core-2.4.0.jar!/:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_212]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_212]
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 0.0.0.0:2000 [/0.0.0.0] failed: Connection refused (Connection refused)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156) ~[httpclient-4.5.10.jar!/:4.5.10]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:374) ~[httpclient-4.5.10.jar!/:4.5.10]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_212]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_212]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212]
at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76) ~[aws-java-sdk-core-1.11.787.jar!/:na]
at com.amazonaws.http.conn.$Proxy64.connect(Unknown Source) ~[na:na]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[httpclient-4.5.10.jar!/:4.5.10]
10]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.10.jar!/:4.5.10]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.10.jar!/:4.5.10]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.10.jar!/:4.5.10]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.10.jar!/:4.5.10]
at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) ~[aws-java-sdk-core-1.11.787.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1330) ~[aws-java-sdk-core-1.11.787.jar!/:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145) ~[aws-java-sdk-core-1.11.787.jar!/:na]
... 21 common frames omitted
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_212]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_212]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:204) ~[na:1.8.0_212]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_212]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_212]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_212]
at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75) ~[httpclient-4.5.10.jar!/:4.5.10]
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.10.jar!/:4.5.10]
... 37 common frames omitted

I solved the problem with this solution:
start the X-Ray container
docker inspect on this container and check which is the IP address
Create an environment variable AWS_XRAY_DAEMON_ADDRESS with the value obtained in previous step (ex: 172.XX.0.X:2000)
start the application container adding -e AWS_XRAY_DAEMON_ADDRESS to pass the variable to the container (ex: docker run -e AWS_XRAY_DAEMON_ADDRESS --name my_containter image_name)
Steps 3 and 4 can be combined by directly passing the IP address to the container (docker run -e AWS_XRAY_DAEMON_ADDRESS=ip_address --name my_containter image_name)

This could be because the app running in the Docker container cannot connect to the X-Ray daemon running locally on the host.
You might find this answer helpful. There is a simple Docker platform example that runs the daemon and the app side-by-side in separate containers. It uses docker-compose networking and the AWS_XRAY_DAEMON_ADDRESS environment variable to connect the app container to the daemon.

On Linux, starting containers in host network solves the issue (using --network host flag when starting container).
Other than that, the AWS_XRAY_DAEMON_ADDRESS environment variable can be used for setting XRay container IP address, as mentioned by #G. Ionut in his answer.

Is there way to check correctness of mongoDB connection?

In my spring boot application i use spring-boot-starter-data-mongodb:2.1.3 to get coonection for MongoDB. I have some properties file to config mongoDB:
spring.data.mongodb.host=localhost
spring.data.mongodb.port=27017
spring.data.mongodb.database=database
If i set incorrect host name (spring.data.mongodb.host=incorrect host) or port my application starts successfully. But i want that application to fail as same as when i set host name with wrong format (spring.data.mongodb.host=hxxt://wrongFormat)
Caused by: com.mongodb.MongoException: host and port should be specified in host:port format
How can i do that?
Example:
application.propertires
#Wrong host
spring.data.mongodb.host=www.google.com
spring.data.mongodb.port=27017
spring.data.mongodb.database=database
DemoApplication.java:
#SpringBootApplication
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
}
Config.java:
#Configuration
public class Config {
#Bean
public CommandLineRunner commandLineRunner(JobRepository jobRepository ){
return args -> jobRepository.findById("1");
}
}
JobRepository.java:
public interface JobRepository extends MongoRepository<Job, String> {}
Job.java:
#Getter
#Setter
#EqualsAndHashCode
#NoArgsConstructor
#Document(collection = "Jobs")
public class Job {
#Id
private String id = null;
private String field;
}

Your issue is that the connection check on mongo happens asynchronously. So if the input is correct, the beans are all created by spring boot and the connection fails at a later point. You can force it to explode by defining a bean that checks the connection at startup. For example, I use this:
/**
* This is a testing bean. It is only here to explode when the database is not available.
* This is to counter act default spring boot behaviour where the Mongo configuration
* will connect to a localhost DB regardless of whether it exists or not.
*
*/
public class MongoDatabaseVerifier {
private static final Logger log = LoggerFactory.getLogger(MongoDatabaseVerifier.class);
public MongoDatabaseVerifier(MongoClient mongoClient) {
checkConnectionOrThrow(mongoClient);
}
/**
* Calls {#link MongoClient#getAddress()} and throws the result away if successful
* #param client
*/
void checkConnectionOrThrow(MongoClient client) {
client.getAddress(); // this will either succeed super fast or explode
log.info("Database fully started ...");
}
}
Since this bean does its check on creation, spring will only continue the startup if it succeeds. Otherwise a bean creation exception will be thrown and the startup aborted.
I hope that helps

I tried to reproduce the issue. Though it says application is started but it is getting aborted after waiting for timeout period of 30000 ms settings to connect to the cluster.
As pointed out by others, you can write some custom logic to check the connectivity or try to set decrease connectivity timeout to fail fast.
sample project code
2019-05-20 10:28:07.381 INFO 86945 --- [ restartedMain] com.demo.Application : Started Application in 5.882 seconds (JVM running for 6.893)
2019-05-20 10:28:07.435 INFO 86945 --- [ restartedMain] org.mongodb.driver.cluster : Cluster description not yet available. Waiting for 30000 ms before timing out
2019-05-20 10:28:26.253 INFO 86945 --- [oogle.com:27017] org.mongodb.driver.cluster : Exception in monitor thread while connecting to server www.google.com:27017
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:67) ~[mongodb-driver-core-3.8.2.jar:na]
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:126) ~[mongodb-driver-core-3.8.2.jar:na]
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:117) ~[mongodb-driver-core-3.8.2.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_101]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_101]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_101]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_101]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_101]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_101]
at com.mongodb.internal.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:64) ~[mongodb-driver-core-3.8.2.jar:na]
at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:62) ~[mongodb-driver-core-3.8.2.jar:na]
... 3 common frames omitted
2019-05-20 10:28:37.444 INFO 86945 --- [ restartedMain] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2019-05-20 10:28:37.452 ERROR 86945 --- [ restartedMain] o.s.boot.SpringApplication : Application run failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:816) [spring-boot-2.1.3.RELEASE.jar:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:797) [spring-boot-2.1.3.RELEASE.jar:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:324) [spring-boot-2.1.3.RELEASE.jar:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) [spring-boot-2.1.3.RELEASE.jar:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) [spring-boot-2.1.3.RELEASE.jar:2.1.3.RELEASE]
at com.demo.Application.main(Application.java:12) [classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_101]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_101]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_101]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_101]
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49) [spring-boot-devtools-2.1.3.RELEASE.jar:2.1.3.RELEASE]
Caused by: org.springframework.dao.DataAccessResourceFailureException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=www.google.com:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: connect timed out}}]; nested exception is com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=www.google.com:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: connect timed out}}]
at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:90) ~[spring-data-mongodb-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:2774) ~[spring-data-mongodb-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.data.mongodb.core.MongoTemplate.executeFindOneInternal(MongoTemplate.java:2629) ~[spring-data-mongodb-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.data.mongodb.core.MongoTemplate.doFindOne(MongoTemplate.java:2353) ~[spring-data-mongodb-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.data.mongodb.core.MongoTemplate.findById(MongoTemplate.java:843) ~[spring-data-mongodb-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.data.mongodb.repository.support.SimpleMongoRepository.findById(SimpleMongoRepository.java:118) ~[spring-data-mongodb-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_101]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_101]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_101]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_101]
at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:359) ~[spring-data-commons-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:200) ~[spring-data-commons-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:644) ~[spring-data-commons-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.5.RELEASE.jar:5.1.5.RELEASE]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:608) ~[spring-data-commons-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.lambda$invoke$3(RepositoryFactorySupport.java:595) ~[spring-data-commons-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:595) ~[spring-data-commons-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.5.RELEASE.jar:5.1.5.RELEASE]
at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:59) ~[spring-data-commons-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.5.RELEASE.jar:5.1.5.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93) ~[spring-aop-5.1.5.RELEASE.jar:5.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.5.RELEASE.jar:5.1.5.RELEASE]
at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:61) ~[spring-data-commons-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.5.RELEASE.jar:5.1.5.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.5.RELEASE.jar:5.1.5.RELEASE]
at com.sun.proxy.$Proxy61.findById(Unknown Source) ~[na:na]
at com.demo.TestConfig.lambda$commandLineRunner$0(TestConfig.java:15) ~[classes/:na]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:813) [spring-boot-2.1.3.RELEASE.jar:2.1.3.RELEASE]
... 10 common frames omitted
Caused by: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=www.google.com:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.SocketTimeoutException: connect timed out}}]
at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:179) ~[mongodb-driver-core-3.8.2.jar:na]
at com.mongodb.internal.connection.MultiServerCluster.getDescription(MultiServerCluster.java:54) ~[mongodb-driver-core-3.8.2.jar:na]
at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:136) ~[mongodb-driver-3.8.2.jar:na]
at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:94) ~[mongodb-driver-3.8.2.jar:na]
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.getClientSession(MongoClientDelegate.java:249) ~[mongodb-driver-3.8.2.jar:na]
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:172) ~[mongodb-driver-3.8.2.jar:na]
at com.mongodb.client.internal.FindIterableImpl.first(FindIterableImpl.java:198) ~[mongodb-driver-3.8.2.jar:na]
at org.springframework.data.mongodb.core.MongoTemplate$FindOneCallback.doInCollection(MongoTemplate.java:2813) ~[spring-data-mongodb-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.data.mongodb.core.MongoTemplate$FindOneCallback.doInCollection(MongoTemplate.java:2788) ~[spring-data-mongodb-2.1.5.RELEASE.jar:2.1.5.RELEASE]
at org.springframework.data.mongodb.core.MongoTemplate.executeFindOneInternal(MongoTemplate.java:2626) ~[spring-data-mongodb-2.1.5.RELEASE.jar:2.1.5.RELEASE]
... 35 common frames omitted
2019-05-20 10:28:37.457 INFO 86945 --- [ restartedMain] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'applicationTaskExecutor'
Process finished with exit code 0

Upon deployment JBoss EAP 6.4.0 cannot find (remote) host names provided in standalone.xml

I am trying to deploy an application provided to me. It should deploy an EAR application on a JBoss EAP server. I ran a configuration script (also given) which applies the following to the standalone.xml file:
For sake of anonymity I have changed the host/-app names to NAME_ONE, NAME_TWO, and APP_NAME.
Mail target:
<outbound-socket-binding name="mail-smtp">
<remote-destination host="NAME_ONE" port="25"/>
</outbound-socket-binding>
Database:
<connection-url>jdbc:postgresql://NAME_TWO/APP_NAME</connection-url>
When I try to run the server it gives me following exceptions:
ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1) MSC000001: Failed to start service jboss.mail-session.java:jboss/mail/Default: org.jboss.msc.service.StartException in service jboss.mail-session.java:jboss/mail/Default: Failed to start service
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1936) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_91]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_91]
Caused by: java.lang.RuntimeException: JBAS015451: Unknown host for outbound socket binding configuration 'mail-smtp'.
at org.jboss.as.mail.extension.MailSessionService.getServerSocketAddress(MailSessionService.java:169)
at org.jboss.as.mail.extension.MailSessionService.setServerProps(MailSessionService.java:133)
at org.jboss.as.mail.extension.MailSessionService.getProperties(MailSessionService.java:91)
at org.jboss.as.mail.extension.MailSessionService.start(MailSessionService.java:65)
at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1980) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1913) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
... 3 more
Caused by: java.net.UnknownHostException: NAME_ONE
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) [rt.jar:1.8.0_91]
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) [rt.jar:1.8.0_91]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) [rt.jar:1.8.0_91]
at java.net.InetAddress.getAllByName0(InetAddress.java:1276) [rt.jar:1.8.0_91]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) [rt.jar:1.8.0_91]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) [rt.jar:1.8.0_91]
at java.net.InetAddress.getByName(InetAddress.java:1076) [rt.jar:1.8.0_91]
at org.jboss.as.network.OutboundSocketBinding.getDestinationAddress(OutboundSocketBinding.java:146)
at org.jboss.as.mail.extension.MailSessionService.getServerSocketAddress(MailSessionService.java:167)
... 8 more
When I try to deploy the application a similar exception occurs for both the mail target and the database:
18:30:17,071 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (ServerService Thread Pool -- 53) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: Could not create connection
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:356)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:304)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:834)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:379)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:453)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:425)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:354)
at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:510)
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143)
at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:70) [hibernate-entitymanager-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.hibernate.engine.jdbc.internal.JdbcServicesImpl$ConnectionProviderJdbcConnectionAccess.obtainConnection(JdbcServicesImpl.java:242) [hibernate-core-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:117) [hibernate-core-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.hibernate.service.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:85) [hibernate-core-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:184) [hibernate-core-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:156) [hibernate-core-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.hibernate.cfg.Configuration.buildTypeRegistrations(Configuration.java:1827) [hibernate-core-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1785) [hibernate-core-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.hibernate.ejb.EntityManagerFactoryImpl.<init>(EntityManagerFactoryImpl.java:96) [hibernate-entitymanager-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:915) [hibernate-entitymanager-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:900) [hibernate-entitymanager-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.hibernate.ejb.HibernatePersistence.createContainerEntityManagerFactory(HibernatePersistence.java:76) [hibernate-entitymanager-4.2.18.Final-redhat-2.jar:4.2.18.Final-redhat-2]
at org.jboss.as.jpa.service.PersistenceUnitServiceImpl.createContainerEntityManagerFactory(PersistenceUnitServiceImpl.java:200) [jboss-as-jpa-7.5.0.Final-redhat-21.jar:7.5.0.Final-redhat-21]
at org.jboss.as.jpa.service.PersistenceUnitServiceImpl.access$600(PersistenceUnitServiceImpl.java:57) [jboss-as-jpa-7.5.0.Final-redhat-21.jar:7.5.0.Final-redhat-21]
at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:99) [jboss-as-jpa-7.5.0.Final-redhat-21.jar:7.5.0.Final-redhat-21]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_91]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_91]
at org.jboss.threads.JBossThread.run(JBossThread.java:122)
Caused by: org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:150)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64)
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:123)
at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:28)
at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:20)
at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)
at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:22)
at org.postgresql.Driver.makeConnection(Driver.java:391)
at org.postgresql.Driver.connect(Driver.java:265)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:328)
... 28 more
Caused by: java.net.UnknownHostException: NAME_TWO
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184) [rt.jar:1.8.0_91]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) [rt.jar:1.8.0_91]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.8.0_91]
at java.net.Socket.connect(Socket.java:589) [rt.jar:1.8.0_91]
at java.net.Socket.connect(Socket.java:538) [rt.jar:1.8.0_91]
at java.net.Socket.<init>(Socket.java:434) [rt.jar:1.8.0_91]
at java.net.Socket.<init>(Socket.java:211) [rt.jar:1.8.0_91]
at org.postgresql.core.PGStream.<init>(PGStream.java:60)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:74)
... 37 more
In a desperate attempt I tried to add both names to the hosts file but that did not work. If it helps in any way: the server does say that the PostgreSQL driver has been deployed successfully on it in its management console.

Apparently the database I was trying to connect to did not exist. I received the database and then used pgAdmin to restore it. I then changed the location to my localhost:port and it was able to connect to the database.

BoneCP will not return or throw if connections are invalid

Using BoneCp 0.7.1
I setup my application and everything is working fine, but then the DB went offline.
On the next application request getConnection() started throwing internal exceptions and the retry mechanism kicked in but getConnection() never returned leaving the thread hanging.
The below exception gets thrown over and over but you would expect getConnection() to return NULL or throw an SQLException at least so you can handle it.
This seems similar: http://jolbox.com/forum/viewtopic.php?t=339&p=1390 But my problem arise after connections are established.
Also saw this... BoneCP doesn't recover from broken connection
I will try 8.0-rc1
10:03:58.302 [BoneCP-pool-watch-thread-com.xxx.xxxjdbc-28eb0780-683c-4fa8-92dc-57b2d071af2a] ERROR com.jolbox.bonecp.ConnectionHandle - Database access problem.
Killing off all remaining connections in the connection pool. SQL State = 08S01
10:03:58.318 [BoneCP-pool-watch-thread-com.xxx.xxxjdbc-28eb0780-683c-4fa8-92dc-57b2d071af2a] ERROR com.jolbox.bonecp.PoolWatchThread - Error in trying to obtain
a connection. Retrying in 500ms
java.sql.SQLException: Network error IOException: Connection refused: connect
at net.sourceforge.jtds.jdbc.JtdsConnection.<init>(JtdsConnection.java:434) ~[jtds-1.3.0.jar:1.3.0]
at net.sourceforge.jtds.jdbc.Driver.connect(Driver.java:183) ~[jtds-1.3.0.jar:1.3.0]
at java.sql.DriverManager.getConnection(DriverManager.java:579) ~[na:1.7.0_17]
at java.sql.DriverManager.getConnection(DriverManager.java:221) ~[na:1.7.0_17]
at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:256) ~[bonecp-0.7.1.RELEASE.jar:0.7.1.RELEASE]
at com.jolbox.bonecp.ConnectionHandle.obtainInternalConnection(ConnectionHandle.java:211) ~[bonecp-0.7.1.RELEASE.jar:0.7.1.RELEASE]
at com.jolbox.bonecp.ConnectionHandle.<init>(ConnectionHandle.java:170) ~[bonecp-0.7.1.RELEASE.jar:0.7.1.RELEASE]
at com.jolbox.bonecp.PoolWatchThread.fillConnections(PoolWatchThread.java:101) [bonecp-0.7.1.RELEASE.jar:0.7.1.RELEASE]
at com.jolbox.bonecp.PoolWatchThread.run(PoolWatchThread.java:82) [bonecp-0.7.1.RELEASE.jar:0.7.1.RELEASE]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_17]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_17]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_17]
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method) ~[na:1.7.0_17]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) ~[na:1.7.0_17]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) ~[na:1.7.0_17]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) ~[na:1.7.0_17]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:157) ~[na:1.7.0_17]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) ~[na:1.7.0_17]
at java.net.Socket.connect(Socket.java:579) ~[na:1.7.0_17]
at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_17]
at java.lang.reflect.Method.invoke(Method.java:601) ~[na:1.7.0_17]
at net.sourceforge.jtds.jdbc.SharedSocket.createSocketForJDBC3(SharedSocket.java:300) ~[jtds-1.3.0.jar:1.3.0]
at net.sourceforge.jtds.jdbc.SharedSocket.<init>(SharedSocket.java:253) ~[jtds-1.3.0.jar:1.3.0]
at net.sourceforge.jtds.jdbc.JtdsConnection.<init>(JtdsConnection.java:329) ~[jtds-1.3.0.jar:1.3.0]
... 11 common frames omitted
Tried 8.0-rc2 same issue getConnection() wont return.

Old question, but may I suggest trying HikariCP for improved recoverability.