Using a jdbc connection to hive I can't seem to get hive to actually find the keytab I want to connect with. I've written a custom krb5 login module and to point to the keytab and added it in the java options (-Djava.security.auth.login.config). Any thoughts on why it might not be taking?
Before try to establish the connection program, mush ensure that the keytab authentication renewed or not.
Check klist command in linux and check the expired date and time. If the auth. already expired, use kinit command to renew it before establish connection through jdbc.
Related
I am trying to connect to Impala using keytab file but I didn't find exact URL to connect with keytab. I have gone through Cloudera documentation, but it does not mention using keytab file with principal to connect to Impala. What will be the JDBC URL to connect to Impala using keytab file?
As described in the User Guide, section "Using Kerberos", Impala JDBC driver supports three methods of getting Kerberos Subject:
...The driver tries to obtain the Subject from the current thread's inherited AccessControlContext. If the AccessControlContext contains
multiple Subjects, the driver uses the most recent Subject.
The driver checks the java.security.auth.login.config system property for a JAAS configuration. If a JAAS configuration is
specified, the driver uses that information to create a LoginContext
and then uses the Subject associated with it.
The driver checks the KRB5_CONFIG and KRB5CCNAME system environment variables for a Kerberos ticket cache. The driver uses the information
from the cache to create a LoginContext and then uses the Subject
associated with it.
You can control which method driver will be using via KrbAuthType parameter in connection URL, otherwise it will try each method one by one.
So, all you should do to use keytab file (method# 2) is
create a jaas.conf file specifying keytab location, principal, etc.
set system property java.security.auth.login.config in your program (or use
JVM startup argument -Djava.security.auth.login.config=</some-path-to/jaas.conf>) pointing to the location of that jaas.conf, and
in your JDBC connection URL, set ...AuthMech=1;KrbRealm=<YOUR-REALM.COM>;KrbHostFQDN=your.impalad.host.com;KrbServiceName=<impala-service-principal> for Kerberos authentication
...before trying to connect.
Alternatively, kinit using your keytab so that ticket cache contains necessary TGT before calling your Java application (essentially, use method 3 above).
We have a remote PostgreSQL DB server which requires SSL certs to be provided to connect to it.
I have the following certs with me.
root.pem
intermediate.pem
private-chain.pem
public-chain.pem
certificate.cer
certificate.key
And I am able to connect to the remote database using psql as
psql "port=5432 host=remote-host user=username sslcert=public-chain.pem sslkey=certificate.key dbname=database sslmode=require"
Now, I need to connect to the same database from a Java Spring based Rest API. So, ideally this would require building keystrokes from the certs and using that to connect to the the db.
The issue is, I'm not able to! I've tried all combinations of cert, public-chain, private-chain, root ca, etc. in the keystore. And I've tried passing the keystore as JVM arguments when calling the jar file (-Djavax.net.ssl.trustStore -Djavax.net.ssl.trustStorePassword -Djavax.net.ssl.keyStore -Djavax.net.ssl.keyStorePassword).
I've also tried pointing to the keystrokes from PostgreSQL JDBC connection string (jdbc:postgresql://remote-host:5432/database?ssl=true&sslcert='filename'&sslkey='key'). Not really sure if this is the right way.
But I keep getting this error.
Caused by: org.postgresql.util.PSQLException: FATAL: connection requires a valid client certificate
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:473) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:205) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.Driver.makeConnection(Driver.java:452) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.Driver.connect(Driver.java:254) ~[postgresql-42.2.2.jar!/:42.2.2]
I am not able to figure out why it works with psql and not with jdbc.
EDIT 1:
Currently I am trying the following approach. This is the spring datasource configuration
spring.datasource.url=jdbc:postgresql://remote-host:5432/database?ssl=true&sslfactory=org.postgresql.ssl.NonValidatingFactory
spring.datasource.username=username
This is the cert config
export PGSSLCERT=/tmp/client.cer
export PGSSLKEY=/tmp/client.key
export PGSSLMODE=allow
I've also followed the steps described here to trust root ca. But I guess that is not necessary since I'm using org.postgresql.ssl.NonValidatingFactory
And I'm starting the application as
java -jar -Dspring.profiles.active=prod application.jar
Any insights into this are appreciated! Thanks.
What does your pb_hba.conf setting look like?
The JDBC driver only supports the trust, ident, password, md5, and crypt authentication methods.
So your java app will have to connect using password and certificate. You can specify that in your pb_hba.conf:
hostssl all all 0.0.0.0/0 md5 clientcert=1
I want to be able to connect to a SQL Server using jdbc and windows authentication.
I saw some answers on the internet saying i should add the following property to the connection string:
integratedSecurity=true;
And also add
sqljdbc_auth.dll
To the java path.
But this, as far as i understand applies only when i'm connecting from a Windows machine.
When i try this on a Linux machine i get:
java.sql.SQLException: This driver is not configured for integrated authentication
My question is how do I do it from a Linux machine.
Thanks
Well, eventually I answer my own question:
This is not possible to use Windows authentication from a linux machine using the Microsoft JDBC driver.
This is possible using the jTDS JDBC driver using the following connection string:
jdbc:jtds:sqlserver://host:port;databaseName=dbname;domain=domainName;useNTLMv2=true;
Thank you all for all the comments
TL;DR
It is not possible to use native Windows Authentication for JDBC connections to MSSQL from a JVM running on Linux.
This MSDN article explains the authentiation methods with JDBC on Linux, potential errors, and available options:
https://blogs.msdn.microsoft.com/psssql/2015/01/09/jdbc-this-driver-is-not-configured-for-integrated-authentication/
...in the JDBC 4.0 driver, you can use the authenticationScheme
connection property to indicate how you want to use Kerberos to
connect to SQL. There are two settings here.
NativeAuthentication (default) – This uses the sqljdbc_auth.dll and is specific to the Windows platform. This was the only option
prior to the JDBC 4.0 driver.
JavaKerberos – Makes use of the Java API’s to invoke kerberos and does not rely on the Windows Platform. This is java specific and not
bound to the underlying operating system, so this can be used on both
Windows and Linux platforms.
...
The following document outlines how to use Kerberos with the JDBC
Driver and walks through what is needed to get JavaKerberos working
properly.
Using Kerberos Integrated Authentication to Connect to SQL Server
http://msdn.microsoft.com/en-us/library/gg558122%28v=sql.110%29.aspx
For those who are using DBeaver the way to connect to the SQL Server Database is:
In order to connect to the SQL Server from Linux Debian using DBeaver
1.- Select SQL Server jTDS driver
2.- Enter the connection information
3.- Go to Driver Properties tab and add the domain, user, password
Just as a note, in some post I found that they needed to change the property USENTLMV2 to TRUE but it worked for me either by putting the USERTLNMV2 in true or false.
A problem that I found was that when I was trying to connect to the database using my user and password the next error was thrown:
Login failed. The login is from an untrusted domain and cannot be used with Windows authentication.
This error was thrown because of my user was about to expire. I tried with another AD user and it could connect.
I know this is kind of an older topic but in case Google sends people here:
There are two main JDBC drivers for SQL Server. One is from Microsoft and the other from jTDS. jTDS can, amazingly, connect using Windows auth (NTLM) from other platforms, including Linux, as described here: http://jtds.sourceforge.net/faq.html#windowsAuth. It can, of course, also use SQL-authenticated logins. SQL-authenticated logins are no harder to use from any OS than any other, so don't forget about those an option.
The version provided by Microsoft is the one from which #mjn provided a quote from the documentation. It is able to connect using Windows authentication by specifying integratedSecurity=true, authenticationScheme=javaKerberos, and authentication=NotSpecified.
It is tricky to get this working even if you don't go out of your way to find more confusion, so always keep in mind which driver you are using - and tell us in these posts so that you can get more specific help.
This JDBC URL is validated to work with latest Microsoft SQL Server JDBC driver:
jdbc:sqlserver://[server]:[port];database=[db\;trustServerCertificate=true;integratedSecurity=true;user=[user without domain];password=[pw];authenticationScheme=NTLM;domain=[domain];authentication=NotSpecified
Example:
jdbc:sqlserver://mysql.myorg.com:1433;database=mydb;trustServerCertificate=true;integratedSecurity=true;user=myuser;password=mypwd;authenticationScheme=NTLM;domain=ad.myorg.com;authentication=NotSpecified
I was able to connect to a SQL Server 2016 Data Mart and JDBC connection Microsoft JDBC Driver using Windows Authentication using the following script on a Ubuntu Linux Docker Image running on Windows 10.
# initializes spark session
from pyspark.sql import SparkSession
spark = SparkSession\
.builder\
.master('local[*]')\
.appName('FDM')\
.config("spark.driver.extraClassPath","pyspark_jars/*")\
.config('spark.executor.memory', '4g')\
.config('spark.driver.memory', '16g')\
.config('spark.executor.cores', '4')\
.getOrCreate()
jdbc_url = '''jdbc:sqlserver://SERVER;databaseName=DBNAME;trustServerCertificate=true;integratedSecurity=true;user=USERID;password=PASSWORD;authenticationScheme=NTLM;domain=US;authentication=NotSpecified'''
spark_df = spark.read\
.format("jdbc")\
.option("url", jdbc_url)\
.option("driver","com.microsoft.sqlserver.jdbc.SQLServerDriver")\
.option("query", 'select top(1000) * from SCHEMA.TABLE')\
.option("fetchsize", 100000)\
.load()
spark_df.write.csv('TEST.csv', mode = "overwrite", header=True)
Is it possible to update password in Active Directory where SSL is not enabled from a Java application?
It's not possible until you will have at least 128-bit connection. That is with TLS or SSL connection. You can use LDAPS to have secure connection, for example from here:
http://docs.oracle.com/javase/jndi/tutorial/ldap/security/ssl.html
Or you can just setup SSL checking this tutorial:
http://blogs.msdn.com/b/alextch/archive/2012/05/15/how-to-set-active-directory-password-from-java-application.aspx
I am writing a struts2 app and using hibernate for persistence. I deploy may app on heroku and everything works ok, but when ever I run it locally I get:
org.postgresql.util.PSQLException: FATAL:no pg_hba.conf entry for host "xx.xx.xxx.xxx", user "someuser", database "somedatabase", SSL off
I know the problem is I need to connect to the database over ssl but how can I set this up locally?
Add this to the end of your JDBC connection URL:
?ssl=true&sslfactory=org.postgresql.ssl.NonValidatingFactory
The key here is "SSL off".
You must use SSL to connect to Heroku. How to enable it depends on the client you are using, which I'm guessing is PgJDBC since you're using Java and Hibernate.
The SSL section of the manual for PgJDBC covers what you need.