We Keep Coding

Mongodb Atlas "Got Socket exception on Connection To Cluster"

I'm using Java & Springboot and MongoDB Atlas and created a database which response to many Object's CURD
When I do the post on uploadingImage, I got this error Got Socket exception on Connection [connectionId{localValue:4, serverValue:114406}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017
However when I call other object's CRUD, it works totally fine. I don't why it raise this exception. BTW all my CRUD operation of all objects works well on localhost when not connecting to MongoDB Atlas, That means my ImageDAO should be fine, I just used mongoTemplate.insert(Image).
I search online, and they said might be IP whitelist of Atlas, So I setup my Cluster open to any IP Address.
Also I set my timeout and socket configure like this in my .properties file:
spring.data.mongodb.uri=mongodb+srv://username:password#cluster0.1c6kg.mongodb.net/database?retryWrites=true&w=majority&keepAlive=true&pooSize=30&autoReconnect=true&socketTimeoutMS=361000000&connectTimeoutMS=3600000
it still not work, I think the problem definitely related to the timeout of socket, But I don't know where else I can config

The Error log is here:
2020-11-01 12:25:34.275 WARN 20242 --- [nio-8088-exec-1] org.mongodb.driver.connection : Got socket exception on connection [connectionId{localValue:4, serverValue:114406}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017. All connections to cluster0-shard-00-02.1c6kg.mongodb.net:27017 will be closed.
2020-11-01 12:25:34.283 INFO 20242 --- [nio-8088-exec-1] org.mongodb.driver.connection : Closed connection [connectionId{localValue:4, serverValue:114406}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017 because there was a socket exception raised by this connection.
2020-11-01 12:25:34.295 INFO 20242 --- [nio-8088-exec-1] org.mongodb.driver.cluster : No server chosen by WritableServerSelector from cluster description ClusterDescription{type=REPLICA_SET, connectionMode=MULTIPLE, serverDescriptions=[ServerDescription{address=cluster0-shard-00-00.1c6kg.mongodb.net:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=8, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=46076648, setName='atlas-d9ovwb-shard-0', canonicalAddress=cluster0-shard-00-00.1c6kg.mongodb.net:27017, hosts=[cluster0-shard-00-00.1c6kg.mongodb.net:27017, cluster0-shard-00-01.1c6kg.mongodb.net:27017, cluster0-shard-00-02.1c6kg.mongodb.net:27017], passives=[], arbiters=[], primary='cluster0-shard-00-02.1c6kg.mongodb.net:27017', tagSet=TagSet{[Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='US_WEST_2'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=null, setVersion=1, lastWriteDate=Sun Nov 01 12:25:29 PST 2020, lastUpdateTimeNanos=104428017411386}, ServerDescription{address=cluster0-shard-00-02.1c6kg.mongodb.net:27017, type=UNKNOWN, state=CONNECTING}, ServerDescription{address=cluster0-shard-00-01.1c6kg.mongodb.net:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=8, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=41202444, setName='atlas-d9ovwb-shard-0', canonicalAddress=cluster0-shard-00-01.1c6kg.mongodb.net:27017, hosts=[cluster0-shard-00-00.1c6kg.mongodb.net:27017, cluster0-shard-00-01.1c6kg.mongodb.net:27017, cluster0-shard-00-02.1c6kg.mongodb.net:27017], passives=[], arbiters=[], primary='cluster0-shard-00-02.1c6kg.mongodb.net:27017', tagSet=TagSet{[Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='US_WEST_2'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=null, setVersion=1, lastWriteDate=Sun Nov 01 12:25:29 PST 2020, lastUpdateTimeNanos=104428010234368}]}. Waiting for 30000 ms before timing out
2020-11-01 12:25:34.316 INFO 20242 --- [ngodb.net:27017] org.mongodb.driver.cluster : Discovered replica set primary cluster0-shard-00-02.1c6kg.mongodb.net:27017
2020-11-01 12:25:34.612 INFO 20242 --- [nio-8088-exec-1] org.mongodb.driver.connection : Opened connection [connectionId{localValue:5, serverValue:108547}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017
2020-11-01 12:25:34.838 WARN 20242 --- [nio-8088-exec-1] org.mongodb.driver.connection : Got socket exception on connection [connectionId{localValue:5, serverValue:108547}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017. All connections to cluster0-shard-00-02.1c6kg.mongodb.net:27017 will be closed.
2020-11-01 12:25:34.838 INFO 20242 --- [nio-8088-exec-1] org.mongodb.driver.connection : Closed connection [connectionId{localValue:5, serverValue:108547}] to cluster0-shard-00-02.1c6kg.mongodb.net:27017 because there was a socket exception raised by this connection.
2020-11-01 12:25:34.876 INFO 20242 --- [ngodb.net:27017] org.mongodb.driver.cluster : Discovered replica set primary cluster0-shard-00-02.1c6kg.mongodb.net:27017
2020-11-01 12:25:34.878 ERROR 20242 --- [nio-8088-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.data.mongodb.UncategorizedMongoDbException: Exception sending message; nested exception is com.mongodb.MongoSocketWriteException: Exception sending message] with root cause

Elasticsearch RestClient Connection reset by peer

I have in my AWS VPC a cluster of ES with 2 nodes. On top of those nodes I have a load balancer. In the same vpc I have a microservice that accesses Elasticsearch via RestHighLevelClient version 7.5.2 .
I create the client in the following manner :
public class ESClientWrapper {
#Getter
private RestHighLevelClient client;
public ESClientWrapper() throws IOException {
FileInputStream propertiesFile = new FileInputStream("/var/elastic.properties");
Properties properties = new Properties();
properties.load(propertiesFile );
RestClientBuilder builder = RestClient.builder(new HttpHost(
properties .getProperty("host"),
Integer.parseInt(properties.getProperty("port"))
));
this.client = new RestHighLevelClient(builder);
}
}
When my micro service doesn't get requests for a long time (12h..) there are occurrences when the first response that is sent (or a few after..) are getting the following error:
2020-09-09 07:03:13.106 INFO 1 --- [nio-8080-exec-1] c.a.a.services.CustomersMetadataService : Trying to add the following role : {role=a2}
2020-09-09 07:03:13.106 INFO 1 --- [nio-8080-exec-1] c.a.a.e.repositories.ESRepository : Trying to insert the following document to app-index : {role=a2}
2020-09-09 07:03:13.109 ERROR 1 --- [nio-8080-exec-1] c.a.a.e.dal.ESRepository : Failed to add customer : {role=a2}
java.io.IOException: Connection reset by peer
at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:828) ~[elasticsearch-rest-client-7.5.2.jar!/:7.5.2]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:248) ~[elasticsearch-rest-client-7.5.2.jar!/:7.5.2]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235) ~[elasticsearch-rest-client-7.5.2.jar!/:7.5.2]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1514) ~[elasticsearch-rest-high-level-client-7.5.2.jar!/:7.5.2]
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1484) ~[elasticsearch-rest-high-level-client-7.5.2.jar!/:7.5.2]
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1454) ~[elasticsearch-rest-high-level-client-7.5.2.jar!/:7.5.2]
at org.elasticsearch.client.RestHighLevelClient.index(RestHighLevelClient.java:871) ~[elasticsearch-rest-high-level-client-7.5.2.jar!/:7.5.2]
....
....
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-embed-core-9.0.35.jar!/:9.0.35]
at java.base/java.lang.Thread.run(Thread.java:836) ~[na:na]
Caused by: java.io.IOException: Connection reset by peer
at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:na]
at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:na]
at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276) ~[na:na]
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:245) ~[na:na]
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223) ~[na:na]
at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:358) ~[na:na]
at org.apache.http.impl.nio.reactor.SessionInputBufferImpl.fill(SessionInputBufferImpl.java:231) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
at org.apache.http.impl.nio.codecs.AbstractMessageParser.fillBuffer(AbstractMessageParser.java:136) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:241) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) ~[httpasyncclient-4.1.4.jar!/:4.1.4]
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) ~[httpasyncclient-4.1.4.jar!/:4.1.4]
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ~[httpcore-nio-4.4.13.jar!/:4.4.13]
... 1 common frames omitted
2020-09-09 07:06:55.109 INFO 1 --- [nio-8080-exec-2] c.a.a.services.MyService : Trying to add the following role : {role=a2}
2020-09-09 07:06:55.109 INFO 1 --- [nio-8080-exec-2] c.a.a.e.repositories.ESRepository : Trying to insert the following document to index app-index: {role=a2}
2020-09-09 07:06:55.211 INFO 1 --- [nio-8080-exec-2] c.a.a.e.dal.ESRepository : IndexResponse[index=app-index,type=_doc,id=x532323272533321870287,version=1,result=created,seqNo=70,primaryTerm=1,shards={"total":2,"successful":2,"failed":0}]
As you can see, 3 minutes after the failed request the next request was successfully handeled by ES. What can kill the request ? I checked Elasticsearch logs and didn't see any indication for killing connection. The MS is in the same vpc as elastic so it isn't passing through any firewall that might kill it.
I found the following issue in github that suggested to increase the default connection timeout but I'm wondering if the issue here is really a timeout problem and if increasing the default time is really the best solution..
Also, I found this bug opened in their repo regarding the same problem but without any answers.
UPDATE
I noticed that even after 10 minutes my service is up this happens. My service started and sent a query to ES and everything worked well. After 10 minutes I sent insert request and it failed on connection reset by peer.

In the end I didn't find a problem in my configuration/implementation. It seems like a bug in the implementation of Elasticsearch's RestHighLevelClient.
I implemented a retry mechanism that wraps the RestHighLevelClient and retries the query if I get the same error. I used Spring #Retry annotation for this solution.

I was facing the same issue. Everything worked fine, but after some time a single request got refused.
The solution (in my case) was to set the keepalive property of the tcp connection with:
final RestClientBuilder restClientBuilder = RestClient.builder(...);
restClientBuilder.setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder.setDefaultIOReactorConfig(IOReactorConfig.custom()
.setSoKeepAlive(true)
.build()))
Found here:
https://github.com/elastic/elasticsearch/issues/65213

Solace JMS Channel closing on it's own when running on a Kubernetes cluster (Spring Boot)

So I have a very simple JMS Listener running on Spring Boot, and running on a Kubernetes cluster on Google Cloud.
The only thing i've defined is the following in my configuration class:
#Bean
public DefaultJmsListenerContainerFactory cFactory(ConnectionFactory connectionFactory, JmsErrorHandler errorHandler) {
logger.info("cFactory called...");
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setErrorHandler(errorHandler);
factory.setTransactionManager(null);
factory.setSessionTransacted(false);
return factory;
}
My Application.properties:
solace.jms.host=tcps://[host]
solace.jms.clientUsername=[username]
solace.jms.clientPassword=[password]
solace.jms.msgVpn=[msgvpn]
solace.jms.queueName=[queuename]
server.port=8080
logging.level.com.solacesystems=INFO
JMS Listener:
#JmsListener(destination="${solace.jms.queueName}", containerFactory = "cFactory")
public void onMessage(Message message) {
[Do stuff with message]
}
I have this issue in the logs that says the following:
2020-02-28 21:08:40.247 INFO 1 --- [nio-8080-exec-6] c.s.j.protocol.impl.TcpClientChannel : Connecting to host 'orig=tcps://[host goes here], scheme=tcps://, host=[host], port=55443' (host 1 of 1, smfclient 294, attempt 1 of 1, this_host_attempt: 1 of 1)
2020-02-28 21:07:21.968 INFO 1 --- [enerContainer-1] c.m.a.notam.listener.JmsMessageListener : Message Received and processed
2020-02-28 21:07:20.572 INFO 1 --- [nio-8080-exec-8] c.s.jcsmp.protocol.smf.SSLSmfClient : closeOutbound() : isSslDowngradeEnabled: false, mSslEngineClosed: false
2020-02-28 21:07:20.572 INFO 1 --- [nio-8080-exec-8] c.s.j.protocol.impl.TcpClientChannel : Channel Closed (smfclient 262)
2020-02-28 21:07:20.535 INFO 1 --- [nio-8080-exec-8] c.s.j.protocol.impl.TcpClientChannel : Connected to host 'orig=tcps://[host]:55443, scheme=tcps://, host=[host], port=55443' (smfclient 262)
It basically loops like this all day long and I can't figure out why. When I run this locally on my development machine the connection remains open and messages are streaming in just fine.
There is no other log to give me a clue why the channel is closing on its own like this.
Any one have any ideas what the problem might be?

Neo4J connection pool is closing after application starts

I'm running my application and Neo4j 4 in Docker compose environment. After starting my application I'm getting some weird logs that connection pool is closing connection to DB (Closing connection pool towards graphdb(172.21.0.4):7687) and after this neo4jClient is unable to query DB (logs below). What is the reason of this behaviour?
BTW. I created Spring Health Check (driver.verifyConnectivity()), but it always return OK (no error is thrown).
Any ideas?
#Configuration
class Neo4jConfiguration {
private val logger = LoggerFactory.getLogger(Neo4jConfiguration::class.java)
#Bean
fun neo4jDriver(
#Value("\${spring.data.neo4j.host}") host: String?,
#Value("\${spring.data.neo4j.port}") port: Int?): Driver {
val connectionUrl = "neo4j://$host:$port"
logger.info("Connecting to Neo4j on `$connectionUrl`")
return GraphDatabase.driver(connectionUrl/*, AuthTokens.basic("neo4j", "secret")*/)
}
#Bean
fun neo4jClient(): ReactiveNeo4jClient = ReactiveNeo4jClient.create(neo4jDriver(null, null))
#Bean
fun neo4jTransactionManager() = ReactiveNeo4jTransactionManager(neo4jDriver(null, null))
}
Docker compose:
version: '3.7'
services:
graphdb:
image: neo4j:4.0.0
ports:
- 7474:7474
- 7687:7687
environment:
NEO4J_AUTH: none
NEO4J_dbms_connectors_default__listen__address: 0.0.0.0
volumes:
- ./docker/neo4j/data:/data
networks:
- things
networks:
things:
name: things
Full logs:
2020-02-24 20:57:32.922 INFO 1 --- [ restartedMain] c.t.r.repo.neo4j.Neo4jConfiguration : Connecting to Neo4j on `neo4j://graphdb:7687`
2020-02-24 20:57:33.321 INFO 1 --- [ restartedMain] Driver : Routing driver instance 656417291 created for server address graphdb:7687
2020-02-24 20:57:43.329 INFO 1 --- [o4jDriverIO-2-3] LoadBalancer : Routing table for database 'system' is stale. Ttl 1582577863326, currentTime 1582577863328, routers AddressSet=[], writers AddressSet=[], readers AddressSet=[], database 'system'
2020-02-24 20:57:43.437 INFO 1 --- [o4jDriverIO-2-2] ConnectionPool : Closing connection pool towards graphdb(172.21.0.4):7687, it has no active connections and is not in the routing table registry.
2020-02-24 20:57:43.440 INFO 1 --- [o4jDriverIO-2-2] LoadBalancer : Updated routing table for database 'system'. Ttl 1582578163422, currentTime 1582577863439, routers AddressSet=[0.0.0.0:7687], writers AddressSet=[0.0.0.0:7687], readers AddressSet=[0.0.0.0:7687], database 'system'
2020-02-24 20:58:02.694 INFO 1 --- [ault-executor-1] LoadBalancer : Routing table for database '<default database>' is stale. Ttl 1582577882693, currentTime 1582577882694, routers AddressSet=[], writers AddressSet=[], readers AddressSet=[], database '<default database>'
2020-02-24 20:58:02.777 INFO 1 --- [o4jDriverIO-2-2] ConnectionPool : Closing connection pool towards graphdb(172.21.0.4):7687, it has no active connections and is not in the routing table registry.
2020-02-24 20:58:02.777 INFO 1 --- [o4jDriverIO-2-2] LoadBalancer : Updated routing table for database '<default database>'. Ttl 1582578182776, currentTime 1582577882777, routers AddressSet=[0.0.0.0:7687], writers AddressSet=[0.0.0.0:7687], readers AddressSet=[0.0.0.0:7687], database '<default database>'
2020-02-24 20:58:02.803 WARN 1 --- [o4jDriverIO-2-2] LoadBalancer : Failed to obtain a connection towards address 0.0.0.0:7687
org.neo4j.driver.exceptions.SessionExpiredException: Server at 0.0.0.0:7687 is no longer available
at org.neo4j.driver.internal.cluster.loadbalancing.LoadBalancer.lambda$acquire$9(LoadBalancer.java:204) ~[neo4j-java-driver-4.0.0.jar:4.0.0-d03d93ede8ad65657eeb90ed890757203ecfaa7a]
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(Unknown Source) ~[na:na]
[... removed ...]
Caused by: org.neo4j.driver.exceptions.ServiceUnavailableException: Unable to connect to 0.0.0.0:7687, ensure the database is running and that there is a working network connection to it.
at org.neo4j.driver.internal.async.connection.ChannelConnectedListener.databaseUnavailableError(ChannelConnectedListener.java:76) ~[neo4j-java-driver-4.0.0.jar:4.0.0-d03d93ede8ad65657eeb90ed890757203ecfaa7a]
at org.neo4j.driver.internal.async.connection.ChannelConnectedListener.operationComplete(ChannelConnectedListener.java:70) ~[neo4j-java-driver-4.0.0.jar:4.0.0-d03d93ede8ad65657eeb90ed890757203ecfaa7a]
at org.neo4j.driver.internal.async.connection.ChannelConnectedListener.operationComplete(ChannelConnectedListener.java:37) ~[neo4j-java-driver-4.0.0.jar:4.0.0-d03d93ede8ad65657eeb90ed890757203ecfaa7a]
[... removed ...]
... 7 common frames omitted
Caused by: org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /0.0.0.0:7687
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:na]
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source) ~[na:na]
[... removed ...]

I have just come across the same problem, after a little experimenting turns out that the ip should be "bolt://$host:$port" and not "neo4j://$host:$port"
Seems like some of the spring.io tutorials are out of date.

Apache NIFI PutHiveStreaming Issue

I am using the NIFI PutHiveStreaming processor to write records into HDFS and I keep running into this issue. I am not able to make much of it because it seems as if I have adhered to all the configuration requirements. Any pointers from someone has successfully resolved this issue? (Nifi - 1.4.0 and Hive - 2.3.3)
2018-04-18 09:03:49,997 INFO [Timer-Driven Process Thread-5] hive.metastore Trying to connect to metastore with URI thrift://hive-metastore:9083
2018-04-18 09:03:49,999 INFO [Timer-Driven Process Thread-5] hive.metastore Connected to metastore.
2018-04-18 09:03:50,486 WARN [Timer-Driven Process Thread-5] o.a.h.h.m.RetryingMetaStoreClient MetaStoreClient lost connection. Attempting to reconnect.
org.apache.thrift.transport.TTransportException: null
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
2018-04-18 09:03:51,487 INFO [Timer-Driven Process Thread-5] hive.metastore Trying to connect to metastore with URI thrift://hive-metastore:9083
2018-04-18 09:03:51,491 INFO [Timer-Driven Process Thread-5] hive.metastore Connected to metastore.
2018-04-18 09:03:51,505 ERROR [Timer-Driven Process Thread-5] o.a.n.processors.hive.PutHiveStreaming PutHiveStreaming[id=1b8f6a4a-e456-3c2b-74be-bc9a0927a43b] Failed to create HiveWriter for endpoint: {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }: org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
at org.apache.nifi.util.hive.HiveWriter.(HiveWriter.java:79)
at org.apache.nifi.util.hive.HiveUtils.makeHiveWriter(HiveUtils.java:46)
at org.apache.nifi.processors.hive.PutHiveStreaming.makeHiveWriter(PutHiveStreaming.java:968)
Caused by: org.apache.nifi.util.hive.HiveWriter$TxnBatchFailure: Failed acquiring Transaction Batch from EndPoint: {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
at org.apache.nifi.util.hive.HiveWriter.nextTxnBatch(HiveWriter.java:264)
at org.apache.nifi.util.hive.HiveWriter.(HiveWriter.java:73)
... 24 common frames omitted
Caused by: org.apache.hive.hcatalog.streaming.TransactionError: Unable to acquire lock on {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransactionImpl(HiveEndPoint.java:578)
at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransaction(HiveEndPoint.java:547)
at org.apache.nifi.util.hive.HiveWriter.nextTxnBatch(HiveWriter.java:261)
... 25 common frames omitted
Caused by: org.apache.thrift.transport.TTransportException: null
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
I have set the concurrency and txn parameters in hive-site.xml as well.