We Keep Coding

multiple Threads read to the same table in database by using the same connection in java?

I have defined three transaction in which select operations and SELECT operations are happening on the different parameter passed . I try to invoke this method concurrently . I am get an error:
o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 0, SQLState: null
Aug 25, 2020 # 12:16:39.000 2020-08-25 06:46:39.388 ERROR 1 --- [o-9003-exec-630] o.h.engine.jdbc.spi.SqlExceptionHelper : Hikari - Connection is not available, request timed out after 60000ms.
And sometimes
org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserved for non-replication superuser connections
I am new to java. Please guide me to solve this issue. Do I need to write multithreading to access number of resources or configuration issue?
hikari:
poolName: Hikari
autoCommit: false
minimumIdle: 5
connectionTimeout: 60000
maximumPoolSize: 80
idleTimeout: 60000
maxLifetime: 240000
leakDetectionThreshold: 300000

Multiple Threads read to the same table in database by using the same connection in java?
This is generally speaking not going to work. The JDBC API types Connection, Statement, ResultSet and so on are not generally thread-safe1. You should not try to use on instance in multiple threads.
If you want to avoid having multiple connections open the normal approach is to use a JDBC connection pool to manage the connections. When a thread needs to talk to the database, it gets a connection from the pool. When it has finished talking to the database, it releases it back to the pool.
In the PostgreSQL / Hikari case:
For PostgreSQL - "Using the driver in a multi-threaded or a servlet environment"
For Hikari - the getConnection() call is thread-safe, but I couldn't find anything that explicitly talked about the thread-safety of the connection object when shared by multiple threads.
1 - I have seen it stated that a spec compliant JDBC driver should be thread-safe, but I could not see where the JDBC spec actually requires this to be so. But even assuming that it does say that somewhere, the threads sharing a connection would need to coordinate very carefully to avoid things like one thread causing another thread's resultset to "spontaneously" close.

Hive2 JDBC exception: org.apache.hadoop.security.AccessControlException Permission denied

I'm running hive 2.1.1, hadoop 2.7.3 on Ubuntu 16.04.
It is totally fine to do HQL in the hive terminal.
hive> show databases;
OK
default
Time taken: 0.799 seconds, Fetched: 1 row(s)
hive> show tables;
OK
Time taken: 0.027 seconds
hive> create table test1(id int, name string);
OK
Time taken: 0.928 seconds
hive> show tables;
OK
test1
Time taken: 0.021 seconds, Fetched: 1 row(s)
hive>
This is the error I get from JDBC in beeline and Java:
Error: Error while processing statement: FAILED: Execution Error,
return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:Got exception:
org.apache.hadoop.security.AccessControlException Permission denied:
user=hive2, access=WRITE,
inode="/user/hive/warehouse/test2":server:supergroup:drwxrwxr-x
I get the same error when I try insert query in both beeline and Java. It seems I can only read but not write through JDBC.
Here's how I log in in beeline:
Beeline version 2.1.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: hive2
Enter password for jdbc:hive2://localhost:10000: ********
Here's piece of my Java code:
try(
Connection con = DriverManager.getConnection("jdbc:hive2://localhost:10000/default", "hive2", "password");
Statement stmt = con.createStatement();) {
...
}
I tried to set up the hive.exec.stagingdir property in hive-site.xml:
<property>
<name>hive.exec.stagingdir</name>
<value>/tmp/hive-staging</value>
</property>
Unfortunately it doesn't solve the problem.
Another solution I found is to log-in as a root user. But how could I log-in as a root user instead of user [hive2]?
How could I execute the write query (create, insert) through JDBC without getting the Permission denied error?
Thank you!

Getting occasional "Failed to validate a newly established connection" SQL Exception on a Spring JDBCTemplate in TomEE

I have a small Spring (4.3.5) app running in TomEE (7.0.1), talking to an Oracle (11) db.
I have a single DAO class that can run two different queries.
I have two query processor tasks running these queries, on a scheduler, one every two minutes, one every three minutes. When I first saw the problem I'm going to describe, I had a single scheduled task running both queries.
Both queries are similar, just returning different columns and aggregates on the same table. The two queries are assembled in code, but they aren't really dynamic. The components are relatively static. I've verified that both queries are fine by running in the debugger and pasting the generated query into my SQL browser, and they both work fine.
When I start up the app, I get results from both queries a couple of times. Then, I started to see the following in the log:
Jan 19, 2017 9:21:05 AM org.apache.tomcat.jdbc.pool.PooledConnection validate
WARNING: SQL Validation error
java.lang.NullPointerException
at oracle.jdbc.driver.T4C8Oall.getNumRows(T4C8Oall.java:973)
at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:1041)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1329)
at oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1909)
at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1871)
at oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:318)
at org.apache.tomcat.jdbc.pool.PooledConnection.validate(PooledConnection.java:509)
at org.apache.tomcat.jdbc.pool.PooledConnection.validate(PooledConnection.java:443)
at org.apache.tomcat.jdbc.pool.ConnectionPool.testAllIdle(ConnectionPool.java:1049)
at org.apache.tomcat.jdbc.pool.ConnectionPool$PoolCleaner.run(ConnectionPool.java:1371)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Jan 19, 2017 9:21:05 AM org.apache.tomcat.jdbc.pool.PooledConnection validate
WARNING: SQL Validation error
java.sql.SQLException: Invalid SQL type: sqlKind = UNINITIALIZED
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:63)
at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:1033)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1329)
at oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1909)
at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1871)
at oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:318)
at org.apache.tomcat.jdbc.pool.PooledConnection.validate(PooledConnection.java:509)
at org.apache.tomcat.jdbc.pool.PooledConnection.validate(PooledConnection.java:443)
at org.apache.tomcat.jdbc.pool.ConnectionPool.testAllIdle(ConnectionPool.java:1049)
at org.apache.tomcat.jdbc.pool.ConnectionPool$PoolCleaner.run(ConnectionPool.java:1371)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
This is clearly in pool validation, not in running my query.
My Datasource in tomee.xml looks like this:
<Resource id="tst36" type="DataSource">
testOnBorrow = true
testOnReturn = true
testWhileIdle = true
logValidationErrors = true
validationQuery = "select 1 from dual"
JdbcDriver = oracle.jdbc.OracleDriver
MaxActive = 10
MinIdle = 2
MaxIdle = 2
MaxWait = 10000
JdbcUrl = jdbc:...
UserName = ...
Password = ...
</Resource>
What can I do here?

are you sure
validationQuery = "select 1 from dual"
is what you meant? Isn't it
validationQuery = select 1 from dual

How to identify remote Type 4 JDBC workload on IBM Mainframe zos DB2

I need to be able to identify Type 4 jdbc workload on an IBM mainframe DB2 v10 zos database.
Our mainframe db2 monitor can filter on the following fields
SYSTEM ID
DB2 SUBSYSTEM ID
DATA SHARING GROUP
AUTHORIZATION ID
PLAN NAME
CONNECTION ID
OPERATOR ID
CORRELATION ID
DBRM/PACKAGE ID
BUFFER POOL ID
COLLECTION ID
LOCAL LOCATION
REQUESTING LOCATION
OTHER LOCATION
DATABASE.PAGESET
LOCK RESOURCE
I am guessing many of these values are not available to be changed.
However these items seems likely candidates
CONNECTION ID
CORRELATION ID
REQUESTING LOCATION
OTHER LOCATION
my questions are:-
i). Can the "likely candidates" be set in my java client jdbc code?
ii). How can I set these "likely candidates"?
If i cannot use any of these fields, then I have managed to amend the clientInfo associated with my JDBC connection, as shown in this snippet of jdbc trace
[jcc][Time:2015-12-10-14:39:24.851][Thread:main][Connection#3b6eb2ec] getClientInfo () called
[jcc][Time:2015-12-10-14:39:24.851][Thread:main][Connection#3b6eb2ec] getClientInfo () returned {ClientUser=XXXXXX00, ApplicationName=db2jcc_application, ClientHostname=L0513039, ClientAccountingInformation=JCC04130L0513039 '}
[jcc][SystemMonitor:stop] core: 0.28737999999999997ms | network: 0.0ms | server: 0.0ms
[jcc][SystemMonitor:start]
[jcc][Time:2015-12-10-14:39:24.852][Thread:main][Connection#3b6eb2ec] setClientInfo ({ApplicationName=crsJCC_application}) called
[jcc][SystemMonitor:stop] core: 3.613203ms | network: 0.0ms | server: 0.0ms
[jcc][SystemMonitor:start]
[jcc][Time:2015-12-10-14:39:24.856][Thread:main][Connection#3b6eb2ec] getClientInfo () called
[jcc][Time:2015-12-10-14:39:24.856][Thread:main][Connection#3b6eb2ec] getClientInfo () returned {ClientUser=XXXXXX00, ApplicationName=xxxxxx_application, ClientHostname=L0513039, ClientAccountingInformation=JCC04130L0513039 '}
[jcc][SystemMonitor:stop] core: 0.24718099999999998ms | network: 0.0ms | server: 0.0ms
[jcc][Time:2015-12-10-14:39:24.857][Thread:main][Connection#3b6eb2ec] createStatement () called
[jcc][Time:2015-12-10-14:39:24.863][Thread:main][Connection#3b6eb2ec] createStatement () returned Statement#5ebec15
[jcc
In this case I amended the ApplicationName within ClientInfo, what I would like is that the initial value as customised, e.g. that the initial value WAS'NT "db2jcc_application" but "started out as "xxxxxx_application", is this possible?

Working with the DB2Driver you should be able to set application name like this:
Properties p= new Properties();
p.put("user", "admin");
p.put("password", "secret");
p.put("clientProgramName", "xxxx_application");
Connection conn = DriverManager.getConnection(
"jdbc:db2://localhost:50000/yourdb", props);
If you're using a DB2DataSource, check out this information from IBM:
com.ibm.db2.jcc.DB2DataSource ds =
new com.ibm.db2.jcc.DB2DataSource();
ds.setDriverType(4);
ds.setServerName("localhost");
ds.setPortNumber(50000);
ds.setDatabaseName("sample");
ds.setUser("username");
ds.setPassword("password");
ds.setClientProgramName("My application");
As stated here:
clientProgramName
Specifies an application ID that is fixed for the duration of a physical connection for a client. The value of this property becomes the correlation ID on a DB2 for z/OS server. Database administrators can use this property to correlate work on a DB2 for z/OS server to client applications. The data type of this property is String. The maximum length is 12 bytes. If this value is null, the IBM DB2 Driver for JDBC and SQLJ supplies a value of db2jccthread-name.

Unable to establish a JDBC connection to Hive from Eclipse

I am trying to establish a JDBC connection to Hive so that I can view and create tables and query Hive tables from Eclipse. I used HiveClient sample code: https://cwiki.apache.org/confluence/display/Hive/HiveClient
Then I added all the required jars to the java build path inside eclipse and started Hive Thrift Server. Port 10000 is listening. I am using Cloudera QuickstartVM 4.6.1 and the eclipse that comes with it. Here's the error that I get in the IDE when I try to run the code.
Exception in thread "main" java.sql.SQLException: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:191)
at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:127)
at org.apache.hadoop.hive.jdbc.HiveConnection.configureConnection(HiveConnection.java:108)
at org.apache.hadoop.hive.jdbc.HiveConnection.<init>(HiveConnection.java:103)
at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:104)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185)
at jdbc.Hive.main(Hive.java:24)
When I try connecting to Hive using beeline, I get the same error. However, when I eliminate the host name and port from the !connect command it works with the following error:
beeline> !connect jdbc:hive:// "" ""
scan complete in 4ms
Connecting to jdbc:hive://
14/03/21 18:42:03 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
14/03/21 18:42:03 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
14/03/21 18:42:04 INFO metastore.ObjectStore: ObjectStore, initialize called
14/03/21 18:42:05 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored.
What am I missing here!?

You have 2 options to connect hiveserver using jdbc
Option 1 : Hiveserver2
You are trying to connect hiveserver2, hiveserver version in cloudera manager is hivesever2, which is more secure than hiveserver. JDBC code you are using is hiveserver,Use the following code snippet for hiveserver2
Class.forName("org.apache.hive.jdbc.HiveDriver");
Connection con = DriverManager.getConnection("jdbc:hive2://localhost:10000/default", "hive", "");
Statement stmt = con.createStatement();
String tableName = "testHiveDriverTable";
stmt.execute("drop table if exists " + tableName);
stmt.execute("create table " + tableName + " (key int, value string)");
String sql = "show tables '" + tableName + "'";
If you look at the connection string, can see the hiveserver version 2(jdbc:hive2://localhost:10000/default", "", ""), second and third arguments are username and password, by default keep it empty string "".
For executing this program add hiveserver2 specific libraries.
Instead of writing your own programs for checking hiveserver2 jdbc connection, beeline hive client can be used as follows
> [testuser02#Abcd-Host1 ~]$ beeline
> beeline> !connect jdbc:hive2://Abcd-Host1:10000/default "" "" ""
>
> 0: jdbc:hive2://Abcd-Host1:10000/default> show tables;
+------------+
| tab_name |
+------------+
| sample_07 |
| sample_08 |
| test1 |
+------------+
3 rows selected (0.334 seconds)
Options 2: Hiveserver1
If you want to make use of your existing code(code for hiveserver1), which you are having https://cwiki.apache.org/confluence/display/Hive/HiveClient. You got to start a new hiveserver in your userspace in another port. Use the following command to start a hiveserver in a given port
nohup hive --service hiveserver -p 10001 &
Now change the port number to 10001 in jdbc connection and run it.