We Keep Coding

Authenticating an Impala JDBC Connection in a Kerberized environment

When trying to launch a Java Action within Oozie (CDH6.3.1) I get a GSS initiate failed. The code in question is pretty straight forward:
String impalaUrl = "jdbc:impala://pxyserver.global.ad:21050/default;SSL=1;AuthMech=1;KrbRealm=GLOBAL.AD;KrbHostFQDN=pxyserver.global.ad;KrbServiceName=Impala;sslTrustStore=/opt/cloudera/security/jks/truststore.jks;trustStorePassword=password";
Properties impalaProperties = new Properties();
impalaProperties.put("user", "svcaccount");
impalaProperties.put("password", "svcpassword");
impalaProperties.put("Driver", "com.cloudera.impala.jdbc41.Driver");
try {
setConnection(DriverManager.getConnection(impalaUrl , impalaProperties));
} catch (SQLException e) {
e.printStackTrace();
}
Running this gives the following stack trace:
java.sql.SQLException: [Cloudera][ImpalaJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][ImpalaJDBCDriver](500169) Unable to connect to server: GSS initiate failed.
at com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
at com.cloudera.impala.hivecommon.api.ServiceDiscoveryFactory.createClient(Unknown Source)
at com.cloudera.impala.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source)
at com.cloudera.impala.impala.core.ImpalaJDBCDSIConnection.establishConnection(Unknown Source)
at com.cloudera.impala.jdbc.core.LoginTimeoutConnection.connect(Unknown Source)
at com.cloudera.impala.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
at com.cloudera.impala.jdbc.common.AbstractDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
I can fix this by kiniting with a keytab. However in our production environment Oozie runs this on one of many worker nodes and they do not have valid kerberos tickets. What are my options here? If possible I'd rather not kinit every day on each of our 40 worker nodes, is there a parameter I am missing here?
Thanks

Microsoft SSAS OLAP Connection from Java Code

I am new to the concept of OLAP DB.
Please excuse me if I missed out something in the Connection String.
I am trying to connect to connect to Microsoft Analysis Server(which is in different Domain) from my System.
While Connecting I am getting below Error.
Can you please let me know, what am I doing wrong?
I have executed the Steps Mentioned in :
https://msdn.microsoft.com/en-us/library/gg492140.aspx
and referred to the below link for connection to the Cube:
http://www.matthewgodding.com/articles/2010/2/27/getting-data-from-sql-analysis-services-using-java-and-olap4.html
After connecting to the CUbe, I need to execute MDX Query.
Please find the snippet of the code I am using:
public static void main(String[] args) {
try {
Class.forName("org.olap4j.driver.xmla.XmlaOlap4jDriver");
final Connection connection = DriverManager
.getConnection("jdbc:xmla:Server=http://Remote_Server_IPAddress/OLAP/msmdpump.dll;"
+ "Catalog=MyCatalog;"
+ "User='Domian\\Username';" \\ This is the Username to login to the Server
+ "Password='Password';" \\ This is the Password to login to the Server
+ "Cube=Cube;"); \\This is the Cube Name
I get the below Exception when I execute the program:
Exception:
java.lang.RuntimeException: org.olap4j.OlapException: This connection encountered an exception while executing a query.
at org.olap4j.driver.xmla.DeferredNamedListImpl.getList(DeferredNamedListImpl.java:96)
at org.olap4j.driver.xmla.DeferredNamedListImpl.size(DeferredNamedListImpl.java:116)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.getOlapDatabase(XmlaOlap4jConnection.java:451)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.getDatabase(XmlaOlap4jConnection.java:444)
at org.sid.practise.CheckConnection.main(CheckConnection.java:79)
Caused by: org.olap4j.OlapException: This connection encountered an exception while executing a query.
at org.olap4j.driver.xmla.XmlaHelper.createException(XmlaHelper.java:43)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.executeMetadataRequest(XmlaOlap4jConnection.java:878)
at org.olap4j.driver.xmla.XmlaOlap4jDatabaseMetaData.getMetadata(XmlaOlap4jDatabaseMetaData.java:137)
at org.olap4j.driver.xmla.XmlaOlap4jDatabaseMetaData.getMetadata(XmlaOlap4jDatabaseMetaData.java:67)
at org.olap4j.driver.xmla.XmlaOlap4jDatabaseMetaData.getDatabaseProperties(XmlaOlap4jDatabaseMetaData.java:1044)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.makeConnectionPropertyList(XmlaOlap4jConnection.java:324)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.generateRequest(XmlaOlap4jConnection.java:1037)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.populateList(XmlaOlap4jConnection.java:849)
at org.olap4j.driver.xmla.DeferredNamedListImpl.populateList(DeferredNamedListImpl.java:136)
at org.olap4j.driver.xmla.DeferredNamedListImpl.getList(DeferredNamedListImpl.java:90)
... 4 more
Caused by: org.olap4j.driver.xmla.proxy.XmlaOlap4jProxyException: This proxy encountered an exception while processing the query.
at org.olap4j.driver.xmla.proxy.XmlaOlap4jHttpProxy.getResponse(XmlaOlap4jHttpProxy.java:181)
at org.olap4j.driver.xmla.proxy.XmlaOlap4jAbstractHttpProxy.get(XmlaOlap4jAbstractHttpProxy.java:181)
at org.olap4j.driver.xmla.proxy.XmlaOlap4jHttpProxy.get(XmlaOlap4jHttpProxy.java:42)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.executeMetadataRequest(XmlaOlap4jConnection.java:876)
... 12 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at org.olap4j.driver.xmla.proxy.XmlaOlap4jHttpProxy.getResponse(XmlaOlap4jHttpProxy.java:136)

It's difficult to know what went wrong for you. Few things you could do,
Don't specify cube in connection. This is not listed as property in XMLA driver.
You can install fiddler and see what's going on with the requests.
Not sure if single quotes are needed in user name and password.

db.lck appears while compiling a java program using a derby database

Recently I have been trying a lot and scratching everywhere to connect to a derby database using NetBeans 8.0. After a lot of research in SO (link1, link2, link3, link4) and everywhere else I am unable to understand that how to make it go away.I realize this is the reason :-(the db.lck) that I am not getting a connection to the embedded apache derby DB.
I notice that db.lck appears while I compile the Java program. Is that a reason that I am denied to a embedded apache derby connection?
N.B :-1. I connected to the JavaDB Server.2. Disconnected the connection to the DB while compiling the program.3. Added the relevant jar file derby.jar.4. Created the database with default schema APP used username and password both as app.
5. Removed the create=true option from the connection string.
6. I can view and manipulate the data through the SQL console in NetBeans 8.0
Providing the code below for further reference :
test.java
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
public class test
{
public static void main(String args [])
{
try {
Connection con = DriverManager.getConnection("jdbc:derby:C:/Users/Dhruvh/Documents/NetBeansProjects/ApacheDerby/derbytest","app","app");
PreparedStatement stmt=con.prepareStatement("select * from \"APP\".TABLE1");
ResultSet rs=stmt.executeQuery();
if(rs.next())
{
System.out.println("Id : "+rs.getInt(1) +" "+" name :"+rs.getString(2));
}
else
{
System.out.println("No word matching in database");
}
} catch (SQLException err) {
System.out.println(err.getMessage());
}
}
}
I have also tried using:-
PreparedStatement stmt=con.prepareStatement("select * from APP.TABLE1");
PreparedStatement stmt=con.prepareStatement("select * from TABLE1");
PreparedStatement stmt=con.prepareStatement("select * from app.table1"); //table was created in lower case
But everything I mention above gave the same error:-
Table/View 'APP.TABLE1' does not exist.
StackTrace:
java.sql.SQLSyntaxErrorException: Table/View 'APP.TABLE1' does not exist.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement42.<init>(Unknown Source)
at org.apache.derby.jdbc.Driver42.newEmbedPreparedStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)
at test.main(test.java:12)
Caused by: ERROR 42X05: Table/View 'APP.TABLE1' does not exist.
at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
at org.apache.derby.impl.sql.compile.FromBaseTable.bindTableDescriptor(Unknown Source)
at org.apache.derby.impl.sql.compile.FromBaseTable.bindNonVTITables(Unknown Source)
at org.apache.derby.impl.sql.compile.FromList.bindTables(Unknown Source)
at org.apache.derby.impl.sql.compile.SelectNode.bindNonVTITables(Unknown Source)
at org.apache.derby.impl.sql.compile.DMLStatementNode.bindTables(Unknown Source)
at org.apache.derby.impl.sql.compile.DMLStatementNode.bind(Unknown Source)
at org.apache.derby.impl.sql.compile.CursorNode.bindStatement(Unknown Source)
at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source)
Also proving a snapshot of the created DB
Please tell if any further references are needed to solve the problem.
Thanks for you patience

You gave derby location of a database that doesn't exist, so it tries to create the database itself.
jdbc:derby:C:/Users/Dhruvh/Documents/NetBeansProjects/ApacheDerby/src/derbytest;**create=true**
The bold part is a default argument which you leave out in your driver creation.
You can connect to the database but there doesn't exist a table you want to access. You have to fix the location of the database. Also if you are connected to the db through your program(looks like mplab x ?), you need to terminate that connection first.

SQL-Database and Java

I read quite a few articles here about how to connect from java to a mysql database. Somehow, this does not work for me, and I cannot find out what I do wrong. First, here is my code:
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.Connection;
import java.sql.DriverManager;
class sqltest{
private static int uid = -1;
protected static Connection dbConn = null;
public static void main (String args[]){
try {
Class.forName("org.gjt.mm.mysql.Driver").newInstance();
} catch (ClassNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (InstantiationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IllegalAccessException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
ResultSet rs = null;
try{
dbConn = DriverManager.getConnection("jdbc:mysql://XXX.XXX.XX.XX:XXXX/peXXXXX","XXXXXX","XXXXXX");
PreparedStatement ps = dbConn.prepareStatement("select id from supervisor where username = philipp");
rs = ps.executeQuery();
uid = rs.getInt("id");
}catch(SQLException e){
e.printStackTrace();
}
System.out.println(uid);
}
}
The connection details (user, password, adress) are correct. There is only one thing I don't understand: There is already a java-web-applet which is establishing such a connection, and it uses no port (is this possible?).
Now these are my errors that I get:
When I use no port or the default 3306 port, i get this error:
com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.net.ConnectException
MESSAGE: Connection timed out: connect
STACKTRACE:
java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:173)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:268)
at com.mysql.jdbc.Connection.createNewIO(Connection.java:2745)
at com.mysql.jdbc.Connection.<init>(Connection.java:1553)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:285)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at sqltest.main(sqltest.java:27)
** END NESTED EXCEPTION **
Last packet sent to the server was 28 ms ago.-1
at com.mysql.jdbc.Connection.createNewIO(Connection.java:2820)
at com.mysql.jdbc.Connection.<init>(Connection.java:1553)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:285)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at sqltest.main(sqltest.java:27)
I assume that this is caused by a wrong port. Is this right?
When I use the port that i assume is the right one, I get no error, but the program just runs endlessly and will not come to any result.
Thx for any help!

Make sure you're connecting to the right host/port and that these are accessible.
To do this, e.g. open a terminal and type telnet <host> <port> from the same
machine on which you're running the Java code.

I did a quick online port scan for that address and it looks like the server side is not listening on port xxxxx. Maybe firewall configuration? Or just stopped server/wrong address.

Yes,
a wrong port may cause this exception. The statement looks wrong. If you have more that a million records in the table you might wait days.

Running endlessly seems very unusual given there are no loops in your code. My guess is the SQL query is taking a very long time.
Warning: this code is wrong!
PreparedStatement ps = dbConn.prepareStatement("select id from supervisor where username = philipp");
I guess you'd need to test this against MySQL to be sure, but in most databases I have worked with, an unquoted string literal like that will throw an error.
Also, runtime variables in a query ought to use bind variables with a prepared statement.
In regard to the connection failure, I would say that if specifying the correct port number yields a different error than not specifying a port number, then the default port number is not the same as what you are using. That is surprising. I think 3306 is fairly well known as MySQL's default port. I am not familiar with this driver: org.gjt.mm.mysql.Driver. Try the standard MySQL driver first.
Next, since using the right port number connects, but you say it sits there, My guess is select id from supervisor where username = philipp is running long. I am still surprised it compiled given what I said earlier, but you can try running it in MySQL yourself. Bring up the mysql command-line client and run it.
If it runs slow there too, then you know it is not a Java issue. Do some tuning on it. Perhaps the username column is not indexed and you have millions of rows. Check the execution plan to know for sure.
[Edit 1]
A good technique when a program seems "stalled" is to add print statements at various points. I suggest putting a bunch of System.out.println calls to see between which lines of code are we hung? That will help you narrow the problem space.

Ok this question is answered:
The SQL-Database can be contacted from outside my university environment. But it is located on a server which is blocking my communication. Unfortunately, this was not written into the server's log-file.
Thank you guys for your help!

Database not found error when connecting to Derby network server

I'm having problems initializing my javadb network server and setting a connection to it. It's a JavaFX program.
This is what I have so far:
try {
Class.forName("org.apache.derby.jdbc.ClientDriver").newInstance();
javadbserver = new NetworkServerControl();
javadbserver.start(null);
} catch (ClassNotFoundException e) {
Logger.getLogger(MainGuiController.class.getName()).log(Level.SEVERE, null, ex);
System.out.println("Where is your JavaDB embedded Driver?");
return;
}
String dbName = "mydb";
String dbUser = "auser";
String dbPass = "password";
PreparedStatement prepstmt;
try {
this.conn = DriverManager.getConnection("jdbc:derby://localhost:1527/mydb;user=auser;password=password");
System.out.println("Went through!");
} catch (SQLException ex) {
Logger.getLogger(MainGuiController.class.getName()).log(Level.SEVERE, null, ex);
}
I always catch the second exception.
If I right click on the javadb service in netbeans and choose connect, everything runs smoothly. [Actually it'd be nice to know what code or program java runs in the background when I select that]
In my projects list under libraries I see derby.jar, derbyclient.jar and derbynet.jar
What am I doing wrong? Please help!
Here's the error I get
java.sql.SQLNonTransientConnectionException: The connection was refused because the database mydb was not found.
at org.apache.derby.client.am.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source)
at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:579)
at java.sql.DriverManager.getConnection(DriverManager.java:243)
at mydb.MainGuiController.initialize(MainGuiController.java:105)
at javafx.fxml.FXMLLoader.load(FXMLLoader.java:2152)
at javafx.fxml.FXMLLoader.load(FXMLLoader.java:2028)
at mydb.mydb.start(mydb.java:37)
at com.sun.javafx.application.LauncherImpl$5.run(LauncherImpl.java:319)
at com.sun.javafx.application.PlatformImpl$5.run(PlatformImpl.java:215)
at com.sun.javafx.application.PlatformImpl$4$1.run(PlatformImpl.java:179)
at com.sun.javafx.application.PlatformImpl$4$1.run(PlatformImpl.java:176)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.javafx.application.PlatformImpl$4.run(PlatformImpl.java:176)
at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
at com.sun.glass.ui.win.WinApplication.access$100(WinApplication.java:29)
at com.sun.glass.ui.win.WinApplication$3$1.run(WinApplication.java:73)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.derby.client.am.DisconnectException: The connection was refused because the database mydb was not found.
at org.apache.derby.client.net.NetConnectionReply.parseRDBNFNRM(Unknown Source)
at org.apache.derby.client.net.NetConnectionReply.parseAccessRdbError(Unknown Source)
at org.apache.derby.client.net.NetConnectionReply.parseACCRDBreply(Unknown Source)
at org.apache.derby.client.net.NetConnectionReply.readAccessDatabase(Unknown Source)
at org.apache.derby.client.net.NetConnection.readSecurityCheckAndAccessRdb(Unknown Source)
at org.apache.derby.client.net.NetConnection.flowSecurityCheckAndAccessRdb(Unknown Source)
at org.apache.derby.client.net.NetConnection.flowUSRIDPWDconnect(Unknown Source)
at org.apache.derby.client.net.NetConnection.flowConnect(Unknown Source)
at org.apache.derby.client.net.NetConnection.<init>(Unknown Source)
at org.apache.derby.client.net.NetConnection40.<init>(Unknown Source)
at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl40.newNetConnection(Unknown Source)

By the JDBC url, it's looks like you're trying to connect to a Derby server vs. an embedded instance. If you are trying to connect to a server instance, here are some considerations:
did you start the server yourself, did mydb already exists?
if not, did you pass in the correct params to create (e.g. ;create=true)
example: jdbc:derby://localhost:1527/dbname;create=true
if mydb did exists, are you pointing the server to the correct location?
also, depending on what was used to start derby (e.g. embedded vs network driver) default database locations are different as well. You can read about it here
Basically the exception you're getting is that Derby's saying it can't find your database - it's basically a path issue.

If you're using netbeans, you should fix this by going into the connection properties and adding a property. Under "property" type "create" and under "value" type "true".