We Keep Coding

hikari oracle connection Failing (spring boot)

I am trying to connect my oracle database using Hikari,
but for some reason on the line HikariDataSource ds = new HikariDataSource(dbConfig);
It is throwing me error that states
2020-10-29 00:06:51.496 [restartedMain] INFO com.zaxxer.hikari.HikariDataSource.<init>(80) - HikariPool-1 - Starting...
2020-10-29 00:06:52.608 [restartedMain] ERROR com.zaxxer.hikari.pool.HikariPool.throwPoolInitializationException(593) - HikariPool-1 - Exception during pool initialization.
java.sql.SQLException: Listener refused the connection with the following error:
ORA-12514, TNS:listener does not currently know of service requested in connect descriptor
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:458)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
HikariConfig dbConfig = new HikariConfig();
dbConfig.setJdbcUrl("jdbc:oracle:thin:#ip:1521/tabs1");
dbConfig.setUsername("usernmae");
dbConfig.setPassword("password");
dbConfig.setDriverClassName("oracle.jdbc.OracleDriver");
dbConfig.setMinimumIdle(1);
dbConfig.setMaximumPoolSize(3);
dbConfig.setAutoCommit(false);
log.info("Configs are :" +dbConfig);
HikariDataSource ds = new HikariDataSource(dbConfig);
I have done a lot of research on it but unable to find the exact problem for the solution.

Please use one of the following
jdbc:oracle:thin:#HOST:PORT:SID
or
jdbc:oracle:thin:#//HOST:PORT/SERVICE
As far as I can see you miss the first two Backslash and end up in a Connect try to use a SID while Service Name not recognized.

Apache NIFI PutHiveStreaming Issue

I am using the NIFI PutHiveStreaming processor to write records into HDFS and I keep running into this issue. I am not able to make much of it because it seems as if I have adhered to all the configuration requirements. Any pointers from someone has successfully resolved this issue? (Nifi - 1.4.0 and Hive - 2.3.3)
2018-04-18 09:03:49,997 INFO [Timer-Driven Process Thread-5] hive.metastore Trying to connect to metastore with URI thrift://hive-metastore:9083
2018-04-18 09:03:49,999 INFO [Timer-Driven Process Thread-5] hive.metastore Connected to metastore.
2018-04-18 09:03:50,486 WARN [Timer-Driven Process Thread-5] o.a.h.h.m.RetryingMetaStoreClient MetaStoreClient lost connection. Attempting to reconnect.
org.apache.thrift.transport.TTransportException: null
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
2018-04-18 09:03:51,487 INFO [Timer-Driven Process Thread-5] hive.metastore Trying to connect to metastore with URI thrift://hive-metastore:9083
2018-04-18 09:03:51,491 INFO [Timer-Driven Process Thread-5] hive.metastore Connected to metastore.
2018-04-18 09:03:51,505 ERROR [Timer-Driven Process Thread-5] o.a.n.processors.hive.PutHiveStreaming PutHiveStreaming[id=1b8f6a4a-e456-3c2b-74be-bc9a0927a43b] Failed to create HiveWriter for endpoint: {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }: org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
at org.apache.nifi.util.hive.HiveWriter.(HiveWriter.java:79)
at org.apache.nifi.util.hive.HiveUtils.makeHiveWriter(HiveUtils.java:46)
at org.apache.nifi.processors.hive.PutHiveStreaming.makeHiveWriter(PutHiveStreaming.java:968)
Caused by: org.apache.nifi.util.hive.HiveWriter$TxnBatchFailure: Failed acquiring Transaction Batch from EndPoint: {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
at org.apache.nifi.util.hive.HiveWriter.nextTxnBatch(HiveWriter.java:264)
at org.apache.nifi.util.hive.HiveWriter.(HiveWriter.java:73)
... 24 common frames omitted
Caused by: org.apache.hive.hcatalog.streaming.TransactionError: Unable to acquire lock on {metaStoreUri='thrift://hive-metastore:9083', database='default', table='bi_events_identification_carrier', partitionVals=[2018, 3, 28] }
at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransactionImpl(HiveEndPoint.java:578)
at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransaction(HiveEndPoint.java:547)
at org.apache.nifi.util.hive.HiveWriter.nextTxnBatch(HiveWriter.java:261)
... 25 common frames omitted
Caused by: org.apache.thrift.transport.TTransportException: null
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
I have set the concurrency and txn parameters in hive-site.xml as well.

datastax driver connection = apache ignite and cassandra(you may want to increase the driver number of per-host connections)

Components: apache ignite + apache cassandra. Use defaut datastax driver.
After doing several operation(about 3-5 billions entities put to cache) we get a situation when datastax driver always reconnects to cassandra from ignite.
2017-02-16 13:29:21.287 INFO 160487 --- [ sys-#441%null%] m.r.t.d.c.m.p.c.c.p.RetryPolicyImpl : init cluster
2017-02-16 13:29:21.288 INFO 160487 --- [ sys-#441%null%] com.datastax.driver.core.Cluster : New Cassandra host <our host> added
2017-02-16 13:29:21.307 INFO 160487 --- [ sys-#441%null%] m.r.t.d.c.m.p.c.c.p.RetryPolicyImpl : close cluster
2017-02-16 13:29:23.516 INFO 160487 --- [ sys-#441%null%] com.datastax.driver.core.ClockFactory : Using native clock to generate timestamps.
2017-02-16 13:29:23.537 INFO 160487 --- [ sys-#441%null%] c.d.d.c.p.DCAwareRoundRobinPolicy : Using data-center name 'datacenter1' for DCAw
And this process is endless and it can be interrupted by server restarted.
Infrastructure :
1 server ignite - ~Xmx30g and 8 cores.
25 clients ignites ~Xmx1g and 8 cores.
1 node cassandra.
Batch size(entities which will be put to cache and then cassandra) is about 1-2M.
Config datasource =>
public DataSource dataSource() {
DataSource dataSource = new DataSource();
dataSource.setUser(login);
dataSource.setPassword(pass);
dataSource.setPort(port);
dataSource.setContactPoints(host);
dataSource.setRetryPolicy(retryPolicy);
dataSource.setFetchSize(10_000);
dataSource.setReconnectionPolicy(new ConstantReconnectionPolicy(1000));
dataSource.setLoadBalancingPolicy(DCAwareRoundRobinPolicy.builder().withUsedHostsPerRemoteDc(0).build());
dataSource.setSocketOptions(new SocketOptions().setReadTimeoutMillis(100_000).setConnectTimeoutMillis(100_00));
return dataSource;
}
config cache =>
CacheConfiguration<KeyIgnite, Long> cfg = new CacheConfiguration<>();
cfg
.setName(area)
.setRebalanceMode(CacheRebalanceMode.SYNC)
.setStartSize(1_000_000)
.setAtomicityMode(CacheAtomicityMode.ATOMIC)
.setIndexedTypes(KeyIgnite.class, Long.class)
.setCacheMode(CacheMode.PARTITIONED)
.setBackups(0);
if (!clientMode) {
CassandraCacheStoreFactory<KeyIgnite, Long> csFactory = new CassandraCacheStoreFactory<>();
csFactory.setDataSource(ds);
csFactory.setPersistenceSettings(kv);
// CassandraCacheStoreFactoryDwh<KeyIgnite, Long> csFactory = new CassandraCacheStoreFactoryDwh<>(ds, kv,params);
cfg
.setCacheStoreFactory(csFactory)
.setReadThrough(true)
.setWriteThrough(true);
}
cfg.setExpiryPolicyFactory(TouchedExpiryPolicy.factoryOf(new Duration(TimeUnit.DAYS, 5)));
return cfg;
ignite config =>:
TcpDiscoveryMulticastIpFinder finder = new TcpDiscoveryMulticastIpFinder();
finder.setAddresses(adresses);
return Ignition.start(
new IgniteConfiguration()
.setClientMode(clientMode)
.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(finder).setNetworkTimeout(10000))
.setFailureDetectionTimeout(50000)
.setPeerClassLoadingEnabled(false)
.setLoadBalancingSpi(new RoundRobinLoadBalancingSpi())
);
When we have done several iteration of putting items to cache we have gotten this case.
after including debug level i have gotten this record:
2017-02-17 17:48:41.570 DEBUG 24816 --- [ sys-#184%null%] com.datastax.driver.core.RequestHandler : [1071994376-1] Error querying ds-inmemorydb-02t/10.216.28.34:9042 : com.datastax.driver.core.exceptions.DriverException: Timeout while trying to acquire available connection (you may want to increase the driver number of per-host connections)

The timeout increase on the driver helped

SolrJ hanging when connecting to zookeeper

I have a local two instance Solr Cloud setup with a single zookeeper instance. I am trying to connect via SolrJ to execute a query however my code hangs for 2mins or so when executing the query and then fails. I have followed the basic example on the Solr wiki. The logs/code is below
2016-07-24 13:29:01.932 INFO 83666 --- [qtp699221219-28] org.apache.zookeeper.ZooKeeper : Initiating client connection, connectString=localhost:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$3#496eab9
2016-07-24 13:29:01.948 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ConnectionManager : Waiting for client to connect to ZooKeeper
2016-07-24 13:29:01.953 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-07-24 13:29:01.955 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Socket connection established to localhost/127.0.0.1:2181, initiating session
2016-07-24 13:29:01.967 INFO 83666 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1561cdd875e0004, negotiated timeout = 10000
2016-07-24 13:29:01.972 INFO 83666 --- [back-3-thread-1] o.a.solr.common.cloud.ConnectionManager : Watcher org.apache.solr.common.cloud.ConnectionManager#4bb95d56 name:ZooKeeperConnection Watcher:localhost:2181 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
2016-07-24 13:29:01.972 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ConnectionManager : Client is connected to ZooKeeper
2016-07-24 13:29:01.973 INFO 83666 --- [qtp699221219-28] o.apache.solr.common.cloud.SolrZkClient : Using default ZkACLProvider
2016-07-24 13:29:01.974 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Updating cluster state from ZooKeeper...
2016-07-24 13:29:01.990 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Loaded empty cluster properties
2016-07-24 13:29:01.995 INFO 83666 --- [qtp699221219-28] o.a.solr.common.cloud.ZkStateReader : Updated live nodes from ZooKeeper... (0) -> (2)
2016-07-24 13:31:24.653 ERROR 83666 --- [qtp699221219-28] o.a.s.client.solrj.impl.CloudSolrClient : Request to collection foo failed due to (0) java.net.ConnectException: Operation timed out, retry? 0
and my code is:
String zkHostString = "localhost:2181";
CloudSolrClient solr = new CloudSolrClient.Builder().withZkHost(zkHostString).build();
solr.setDefaultCollection("foo");
SolrQuery query = new SolrQuery();
query.set("q", "*:*");
QueryResponse response = null;
try {
response = solr.query(query);
} catch (SolrServerException e) {
return null;
}
//Do Something with the results...

Urgh, I'm an idiot, the zookeeper instance and solr instances are inside docker, the code posted above is not. So Zookeeper reported back the solr urls using the docker containers ip...The host needs to connect via localhost and not the docker container ip.
Eg: Zookeeper responds [http://172.17.0.5:8983/solr/foo_shard1_replica2, http://172.17.0.6:8984/solr/foo_shard1_replica1]
but my code needs to call [http://localhost:8983/solr/foo_shard1_replica2, http://localhost:8984/solr/foo_shard1_replica1]

connecting to mongo shard via java driver 3.2.2

I am trying to connect to mongo query routers in a test environment (I setup just one query router for test -> pointing to a config server (instead of three) which in turn points to a two node shard with no replicas). I can insert/fetch documents using the mongo shell (and have verified that the documents are going to the sharded nodes). However, when I try to test the connection to the mongo database, I get the output copied below (code being used is also copied underneath). I am using mongo database v3.2.0 and java driver v3.2.2 (I am trying to use the async api).
[info] 14:34:44.562 227 [main] MongoAuthentication INFO - testing 1
[info] 14:34:44.595 260 [main] cluster INFO - Cluster created with settings {hosts=[192.168.0.1:27018], mode=MULTIPLE, requiredClusterType=SHARDED, serverSelectionTimeout='30000 ms', maxWaitQueueSize=30}
[info] 14:34:44.595 260 [main] cluster INFO - Adding discovered server 192.168.0.1:27018 to client view of cluster
[info] 14:34:44.652 317 [main] cluster DEBUG - Updating cluster description to {type=SHARDED, servers=[{address=192.168.0.1:27018, type=UNKNOWN, state=CONNECTING}]
[info] Outputting database names:
[info] 14:34:44.660 325 [main] cluster INFO - No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=SHARDED, connectionMode=MULTIPLE, all=[ServerDescription{address=192.168.0.1:27018, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
[info] Counting the number of documents
[info] 14:34:44.667 332 [main] cluster INFO - No server chosen by ReadPreferenceServerSelector{readPreference=primaryPreferred} from cluster description ClusterDescription{type=SHARDED, connectionMode=MULTIPLE, all=[ServerDescription{address=192.168.0.1:27018, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
[info] - Count result: 0
[info] 14:34:45.669 1334 [cluster-ClusterId{value='577414c420055e5bc086c255', description='null'}-192.168.0.1:27018] connection DEBUG - Closing connection connectionId{localValue:1}
part of the code being used
final MongoClient mongoClient = MongoClientAccessor.INSTANCE.getMongoClientInstance();
final CountDownLatch listDbsLatch = new CountDownLatch(1);
System.out.println("Outputting database names:");
mongoClient.listDatabaseNames().forEach(new Block<String>() {
#Override
public void apply(final String name) {
System.out.println(" - " + name);
}
}, new SingleResultCallback<Void>() {
#Override
public void onResult(final Void result, final Throwable t) {
listDbsLatch.countDown();
}
});
The enum being used is responsible for reading config options and passing a MongoClient reference to its caller. The enum itself calls other classes which I can copy as well if needed. I have the following option configured for ReadPreference:
mongo.client.readPreference=PRIMARYPREFERRED
Any thoughts on what I might be doing wrong or might have misinterpreted? The goal is to connect to the shard via the mongos (query router) so that I can insert/fetch documents in the Mongo shard.

The mongo shard setup (query router, config and shard with replica sets) was not correctly configured. Ensure that the config server(s) replica set is launched first, mongos (query router) is brought up and points to these config servers, the mongo shards are brought up and then the shards are registered via the query router (mongos) as well as the collection is enabled for sharding. Obviously, make sure that the driver is connecting to the mongos (query router) process.