I'm running hive 2.1.1, hadoop 2.7.3 on Ubuntu 16.04.
It is totally fine to do HQL in the hive terminal.
hive> show databases;
OK
default
Time taken: 0.799 seconds, Fetched: 1 row(s)
hive> show tables;
OK
Time taken: 0.027 seconds
hive> create table test1(id int, name string);
OK
Time taken: 0.928 seconds
hive> show tables;
OK
test1
Time taken: 0.021 seconds, Fetched: 1 row(s)
hive>
This is the error I get from JDBC in beeline and Java:
Error: Error while processing statement: FAILED: Execution Error,
return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
MetaException(message:Got exception:
org.apache.hadoop.security.AccessControlException Permission denied:
user=hive2, access=WRITE,
inode="/user/hive/warehouse/test2":server:supergroup:drwxrwxr-x
I get the same error when I try insert query in both beeline and Java. It seems I can only read but not write through JDBC.
Here's how I log in in beeline:
Beeline version 2.1.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: hive2
Enter password for jdbc:hive2://localhost:10000: ********
Here's piece of my Java code:
try(
Connection con = DriverManager.getConnection("jdbc:hive2://localhost:10000/default", "hive2", "password");
Statement stmt = con.createStatement();) {
...
}
I tried to set up the hive.exec.stagingdir property in hive-site.xml:
<property>
<name>hive.exec.stagingdir</name>
<value>/tmp/hive-staging</value>
</property>
Unfortunately it doesn't solve the problem.
Another solution I found is to log-in as a root user. But how could I log-in as a root user instead of user [hive2]?
How could I execute the write query (create, insert) through JDBC without getting the Permission denied error?
Thank you!
Related
When trying to issue "SHOW POOLS" or any stats query command on pgbouncer database via JDBC, facing the below exception.
org.postgresql.util.PSQLException: ERROR: unsupported pkt type: 80
at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2270)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1998)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:570)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:406)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:286)
JDBC code:
String connectionUrl = "jdbc:postgresql://"+ipaddress+":"+port+"/"+database;
con = DriverManager.getConnection(connectionUrl, userName, password);
statement = con.createStatement(); statement.executeQuery("SHOW POOLS");
JDBC Driver Version: 42.2.14 ;
PgBouncer Version: 1.14.0 ;
Postgres Version: 11.4;
PS:
Manually able to connect to pgbouncer database and issue all pgbouncer admin commands like SHOW POOLS or SHOW STATS. Just not able to execute the same from JDBC.
JDBC use extended query protocol by default, try simple protocol for such query
String connectionUrl = "jdbc:postgresql://"+ipaddress+":"+port+"/"+database+"?preferQueryMode=simple";
PGBouncer currently supports only the simple protocol - the packet type 80 is for 'Parse', which is the first step in the extended protocol. The message you see in the exception PSQLException actually comes from PGBouncer.
I was currently using java spring jdbctemplate to insert and select record. However, Now the system have go for production, some issue has happen. This system will prevent record duplicate by select the record by certain key to check is it already exist. However, I notice the 1st transaction after inserted into database, then the 2nd transaction come in, jdbc template select still show that record was not found? I notice this will happen if the 2 transaction was submitted within a milliseconds different...
The record time was
e.g.
1st Record insert - 15/6/17 12:22:39,986 - insert success
2nd record select - 15/6/17 12:22:44,680 - search record show not found?
Sample insert log
15/6/17 12:22:39,937 DEBUG [30:http-nio-8080-exec-2] (org.springframework.jdbc.core.JdbcTemplate:869) Executing prepared SQL update
15/6/17 12:22:39,938 DEBUG [30:http-nio-8080-exec-2] (org.springframework.jdbc.core.JdbcTemplate:616) Executing prepared SQL statement [INSERT INTO ...;]
15/6/17 12:22:39,950 DEBUG [30:http-nio-8080-exec-2] (org.springframework.jdbc.core.JdbcTemplate:879) SQL update affected 1 rows
15/6/17 12:22:39,951 DEBUG [30:http-nio-8080-exec-2] (org.springframework.jdbc.core.JdbcTemplate:869) Executing prepared SQL update
15/6/17 12:22:39,952 DEBUG [30:http-nio-8080-exec-2] (org.springframework.jdbc.core.JdbcTemplate:616) Executing prepared SQL statement [INSERT INTO ...;]
15/6/17 12:22:39,965 DEBUG [30:http-nio-8080-exec-2] (org.springframework.jdbc.core.JdbcTemplate:879) SQL update affected 1 row
15/6/17 12:22:39,970 DEBUG [30:http-nio-8080-exec-2] (org.springframework.jdbc.datasource.DataSourceUtils:327) Returning JDBC Connection to DataSource
Sample select log
15/6/17 12:22:44,680 DEBUG [29:http-nio-8080-exec-1] (org.springframework.jdbc.core.JdbcTemplate:682) Executing prepared SQL query
15/6/17 12:22:44,680 DEBUG [29:http-nio-8080-exec-1] (org.springframework.jdbc.core.JdbcTemplate:616) Executing prepared SQL statement [SELECT * FROM ...]
Please note that this system was set in 2 different server by using load balancing. They both point to the same database. The insert was completed on server 1 because it already return the connection to the jdbc connection pool. A new transaction come in the server 2, select but no record found?
I wonder why it still cannot found event after insert was finish? Is it because of the time delay?
This question had been resolved. It was due to the server time not consistent. Thanks
RealSkeptic
I have web application. And I have jdbc database pooling system.
If my minIdle and initialSize value are 50, I have the following error:
[ WARN] [http-nio-8080-exec-25 03:11:33] (SqlExceptionHelper.java:logExceptions:144) SQL Error: 12519, SQLState: 66000
[ERROR] [http-nio-8080-exec-25 03:11:33] (SqlExceptionHelper.java:logExceptions:146) Listener refused the connection with the following error:
ORA-12519, TNS:no appropriate service handler found
might be this is because I should configure XE oracle to get many connections.
I do something like this:
ALTER SYSTEM SET processes=10000 scope=spfile
But this did not help me.
Also:
SQL> connect
Enter user-name: system
Enter password:
Connected.
SQL> select count(*) from v$process;
COUNT(*)
----------
44
SQL> show parameter processes;
NAME TYPE VALUE
------------------------------------ ----------- -----------------------
aq_tm_processes integer 0
db_writer_processes integer 1
gcs_server_processes integer 0
global_txn_processes integer 1
job_queue_processes integer 4
log_archive_max_processes integer 4
processes integer 100
SQL>
I have windows 7 x64. and this is Oracle Express edition.
If too many connection opened within a short time can cause this. You should check in your processes.
You may make system sleep for some time after some connection.
I am trying to establish a JDBC connection to Hive so that I can view and create tables and query Hive tables from Eclipse. I used HiveClient sample code: https://cwiki.apache.org/confluence/display/Hive/HiveClient
Then I added all the required jars to the java build path inside eclipse and started Hive Thrift Server. Port 10000 is listening. I am using Cloudera QuickstartVM 4.6.1 and the eclipse that comes with it. Here's the error that I get in the IDE when I try to run the code.
Exception in thread "main" java.sql.SQLException: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:191)
at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:127)
at org.apache.hadoop.hive.jdbc.HiveConnection.configureConnection(HiveConnection.java:108)
at org.apache.hadoop.hive.jdbc.HiveConnection.<init>(HiveConnection.java:103)
at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:104)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185)
at jdbc.Hive.main(Hive.java:24)
When I try connecting to Hive using beeline, I get the same error. However, when I eliminate the host name and port from the !connect command it works with the following error:
beeline> !connect jdbc:hive:// "" ""
scan complete in 4ms
Connecting to jdbc:hive://
14/03/21 18:42:03 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
14/03/21 18:42:03 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
14/03/21 18:42:04 INFO metastore.ObjectStore: ObjectStore, initialize called
14/03/21 18:42:05 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored.
What am I missing here!?
You have 2 options to connect hiveserver using jdbc
Option 1 : Hiveserver2
You are trying to connect hiveserver2, hiveserver version in cloudera manager is hivesever2, which is more secure than hiveserver. JDBC code you are using is hiveserver,Use the following code snippet for hiveserver2
Class.forName("org.apache.hive.jdbc.HiveDriver");
Connection con = DriverManager.getConnection("jdbc:hive2://localhost:10000/default", "hive", "");
Statement stmt = con.createStatement();
String tableName = "testHiveDriverTable";
stmt.execute("drop table if exists " + tableName);
stmt.execute("create table " + tableName + " (key int, value string)");
String sql = "show tables '" + tableName + "'";
If you look at the connection string, can see the hiveserver version 2(jdbc:hive2://localhost:10000/default", "", ""), second and third arguments are username and password, by default keep it empty string "".
For executing this program add hiveserver2 specific libraries.
Instead of writing your own programs for checking hiveserver2 jdbc connection, beeline hive client can be used as follows
> [testuser02#Abcd-Host1 ~]$ beeline
> beeline> !connect jdbc:hive2://Abcd-Host1:10000/default "" "" ""
>
> 0: jdbc:hive2://Abcd-Host1:10000/default> show tables;
+------------+
| tab_name |
+------------+
| sample_07 |
| sample_08 |
| test1 |
+------------+
3 rows selected (0.334 seconds)
Options 2: Hiveserver1
If you want to make use of your existing code(code for hiveserver1), which you are having https://cwiki.apache.org/confluence/display/Hive/HiveClient. You got to start a new hiveserver in your userspace in another port. Use the following command to start a hiveserver in a given port
nohup hive --service hiveserver -p 10001 &
Now change the port number to 10001 in jdbc connection and run it.
When i deploy the web app on test server (it works fine on localhost ), this BLO_BlOCKED_MOBILES table at the given line of code.
Query query = this.getSession().createSQLQuery(
strQuery.append(" INSERT INTO BLO_BlOCKED_MOBILES (BLO_CUSTOMER_ID_FK,BLO_MOBILE,BLO_NEWMOBILE,BLO_PASSPORT,BLO_STATUS,BLOCKED_BY,BLOCKED_AT ) "
+"VALUES( "+customer.getId() +", '"+ oldMobile +"','"+newMobile+"','"+customer.getPassportNo()+"','Y','user',CURRENT_TIMESTAMP)" ).toString());
mtmrsLogger.info(query.toString());
int affectedRows = query.executeUpdate();
The db is in mysql and app uses hibernate,struts 1 and spring. I created this new table ,
added this on hibernate.cfg.xml
< mapping class="com.mtmrs.model.branch.BlockedMobile"/>
and created the model file BlockedMobile.java
and when it is used, the below error comes.
WARN org.hibernate.util.JDBCExceptionReporter - SQL Error: 1146, SQLState: 42S02
ERROR org.hibernate.util.JDBCExceptionReporter - Table 'merchantrade.blo_blocked_mobiles' doesn't exist
Please let me know if the information is insufficient. I don't see what's wrong.
Peraphs the two DB (on your localhost server and on test server) are different. So the test server has not got your table.
According to the error message, either the table blo_blocked_mobiles or the scheme merchantrade does not exist on the MySQL database on the test server.