I have this very interesting issue that has been making me tear down my hair for the past two days.
I am using jUnit and an embedded Tomcat to test some API endpoints (Jersey).
I'm not a big fan of mocking and I made this setup to test the API responses in conditions as close as possible to production.
When the API receives a call, it should supply a response accordingly (found, not found, etc). This is where Hibernate comes in.
When I run this on my Tomcat set up in eclipse, or when I deploy the build (Maven) on a standalone Tomcat on the remote server, everything works fine, but when the API is called on the embedded Tomcat during the test I get this error:
java.lang.ClassCastException: com.models.listing.Listing cannot be
cast to com.models.listing.Listing
Yes, it's the same class name.
To retrieve the Listing object I use the Hibernate standard of getting a persisted object by ID
Listing listing = session.get(Listing.class, ID);
This is how the embedded tomcat setup looks like:
public void start(String appName) throws Exception {
File root = getRootFolder();
System.setProperty("org.apache.catalina.startup.EXIT_ON_INIT_FAILURE", "true");
Path tempPath = Files.createTempDirectory("tomcat-base-dir");
tomcat = new Tomcat();
tomcat.setPort(0);
tomcat.enableNaming();
tomcat.setSilent(true);
tomcat.setBaseDir(tempPath.toString());
tomcat.getHost().setDeployOnStartup(true);
tomcat.getHost().setAutoDeploy(true);
tomcat.getHost().setAppBase(tempPath.toString());
File webContentFolder = new File(root.getAbsolutePath(), "src/main/webapp/");
if (!webContentFolder.exists()) {
webContentFolder = Files.createTempDirectory("default-doc-base").toFile();
}
StandardContext ctx = (StandardContext) tomcat.addWebapp("/" + appName, webContentFolder.getAbsolutePath());
//Disable TLD scanning by default
if (System.getProperty(Constants.SKIP_JARS_PROPERTY) == null ) {
System.out.println("disabling TLD scanning");
StandardJarScanFilter jarScanFilter = (StandardJarScanFilter) ctx.getJarScanner().getJarScanFilter();
jarScanFilter.setTldSkip("*");
}
System.out.println("configuring app with basedir: " + webContentFolder.getAbsolutePath());
// Declare an alternative location for your "WEB-INF/classes" dir
// Servlet 3.0 annotation will work
File additionWebInfClassesFolder = new File(root.getAbsolutePath(), "target/classes");
WebResourceRoot resources = new StandardRoot(ctx);
WebResourceSet resourceSet;
if (additionWebInfClassesFolder.exists()) {
resourceSet = new DirResourceSet(resources, "/WEB-INF/classes", additionWebInfClassesFolder.getAbsolutePath(), "/");
System.out.println("loading WEB-INF resources from as '" + additionWebInfClassesFolder.getAbsolutePath() + "'");
} else {
resourceSet = new EmptyResourceSet(resources);
}
resources.addPreResources(resourceSet);
ctx.setResources(resources);
//start tomcat
tomcat.start();
}
And this is how the Hibernate configuration looks like:
private static SessionFactory buildSessionFactory() {
// setup the session factory
Configuration configuration = new Configuration();
//add annotated classes
configuration.addAnnotatedClass(Listing.class);
//connection properties
configuration.setProperty("hibernate.dialect", "org.hibernate.dialect.MySQL5InnoDBDialect");
configuration.setProperty("hibernate.connection.driver_class", "com.mysql.jdbc.Driver");
configuration.setProperty("hibernate.connection.url", "jdbc:mysql://<some IP>:3306/<some db name>");
configuration.setProperty("hibernate.connection.release_mode", "auto");
configuration.setProperty("hibernate.connection.username", "<some username>");
configuration.setProperty("hibernate.connection.password", "<some password>");
configuration.setProperty("hibernate.connection.provider_class", "org.hibernate.connection.C3P0ConnectionProvider");
//sql properties
configuration.setProperty("show_sql", "true");
configuration.setProperty("format_sql", "true");
//misc properties
configuration.setProperty("hibernate.hbm2ddl.auto", "validate");
configuration.setProperty("hibernate.current_session_context_class", "thread");
//c3p0 properties
configuration.setProperty("hibernate.c3p0.min_size", "1");
configuration.setProperty("hibernate.c3p0.max_size", "10");
configuration.setProperty("hibernate.c3p0.timeout", "100");
configuration.setProperty("hibernate.c3p0.max_statements", "50");
configuration.setProperty("hibernate.c3p0.idle_test_period", "1000");
configuration.setProperty("hibernate.c3p0.validate", "true");
//return null
return configuration.buildSessionFactory();
}
To summarize, the test starts in jUnit, the embedded tomcat instance fires up and with a REST client I send a request to an API endpoint. The endpoint responds after it retrieves the resource from Hibernate.
Dependency versions:
Glassfish Jersey 2.23
jUnit 4.11
Tomcat embedded 8.5.3
Hibernate 5.2.1
My best bet is some issue with the class loader.
I know that the JVM sees classes as different if they were loaded with different class loaders even if it's basically the same class from the same package and so on, but I just don't seem to find a way to make this work.
Maybe my hypothesis is completely off and I am missing something here, so if anyone has encountered something like this or has some suggestions (I already tried tens of "solutions") please jump in.
Thanks everyone for the help in advance!
I still didn't figure out the mystery and I just let go after some other "trial and error" ideas with class loaders and so on.
I "fixed" it by downgrading to hibernate 4.3.11
Related
We are running glassfish 4 for our java web application and are running into an issue with timers. Normal servlet calls are able to enjoy as many different connections as they want, which makes integrations much easier. Once we add timers however, the datasources need to be "XA" datasources instead. We set one up as such below:
public XADataSource getNewConnection() {
Encapsulations encap = new Encapsulations();
XADataSource ds = null;
try {
Context ctx = new InitialContext();
if(!encap.getDataSource().equals("Production")){
ds = (XADataSource) ctx.lookup("jdbc/XA_TEST");
}else{
ds = (XADataSource) ctx.lookup("jdbc/XA");
}
} catch (Exception e) {
CatchException.logException(null, e);
String error = e.toString();
}
return ds;
}
The problem is that when the ds = (XADataSource) ctx.lookup("jdbc/XA_TEST") line runs we get this error:
java.lang.ClassCastException: com.sun.gjc.spi.jdbc40.DataSource40 cannot be cast to javax.sql.XADataSource
We use the sqljdbc42 jar for our normal connections, so it is a bit strange to see 40 in there. Anyone know what the problem is? The datasource we are using was set up as an XADataSource, other than downloading a different jar I don't know what is missing.
It's been a while since I don't use Glassfish, but as far as I remember you should install the jar for your database provider that has the implementation of XADataSource interface. In MS SQL Server I've used the jTDS driver.
Another thing to consider from your code snippet is that you should have different configurations of glassfish for development and production, you should leave that kind of things to the application server and not inside your code. Glassfish is a full JavaEE Application Server, it has all the benefits including database connection pooling.
Trying to migrate an application from WebLogic 12.2.1 to Tomcat 8.5.4, what under Weblogic was an entry as Foreign JNDI Providers for an LDAP connection has been migrated to a new Resource under Tomcat.
Following this advice on Stack Overflow, a custom LdapContextFactory has been packaged as a new jar file under Tomcat lib folder.
In the Tomcat server.xml file the following GlobalNamingResources/Resource has been configured:
<Resource name="ldapConnection"
auth="Container"
type="javax.naming.ldap.LdapContext"
factory="com.sample.custom.LdapContextFactory"
singleton="false"
java.naming.referral="follow"
java.naming.factory.initial="com.sun.jndi.ldap.LdapCtxFactory"
java.naming.provider.url="ldap://some.host:389"
java.naming.security.authentication="simple"
java.naming.security.principal="CN=some,OU=some,OU=some,DC=some,DC=a,DC=b"
java.naming.security.credentials="password"
com.sun.jndi.ldap.connect.pool="true"
com.sun.jndi.ldap.connect.pool.maxsize="10"
com.sun.jndi.ldap.connect.pool.prefsize="4"
com.sun.jndi.ldap.connect.pool.timeout="30000" />
The connection above works fine when browsing the LDAP directory via an LDAP browser like Apache Directory Studio / LDAP Browser embedded in Eclipse.
The custom com.sample.custom.LdapContextFactory is quite simple:
public class LdapContextFactory implements ObjectFactory {
public Object getObjectInstance(Object obj, Name name, Context nameCtx, Hashtable<?, ?> environment)
throws Exception {
Hashtable<Object, Object> env = new Hashtable<>();
Reference reference = (Reference) obj;
Enumeration<RefAddr> references = reference.getAll();
while (references.hasMoreElements()) {
RefAddr address = references.nextElement();
String type = address.getType();
String content = (String) address.getContent();
env.put(type, content);
}
return new InitialLdapContext(env, null);
}
}
However, at start-up Tomcat is throwing the following exception:
07-Sep-2016 15:04:01.064 SEVERE [main] org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.createMBeans Exception processing Global JNDI Resources
javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-031001E5, problem 2001 (NO_OBJECT), data 0, best match of:
''
]; remaining name ''
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3160)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3081)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2888)
at com.sun.jndi.ldap.LdapCtx.c_listBindings(LdapCtx.java:1189)
at com.sun.jndi.toolkit.ctx.ComponentContext.p_listBindings(ComponentContext.java:592)
at com.sun.jndi.toolkit.ctx.PartialCompositeContext.listBindings(PartialCompositeContext.java:330)
at com.sun.jndi.toolkit.ctx.PartialCompositeContext.listBindings(PartialCompositeContext.java:317)
at javax.naming.InitialContext.listBindings(InitialContext.java:472)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.createMBeans(GlobalResourcesLifecycleListener.java:136)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.createMBeans(GlobalResourcesLifecycleListener.java:145)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.createMBeans(GlobalResourcesLifecycleListener.java:110)
at org.apache.catalina.mbeans.GlobalResourcesLifecycleListener.lifecycleEvent(GlobalResourcesLifecycleListener.java:82)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:94)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:401)
at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:345)
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:784)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:152)
at org.apache.catalina.startup.Catalina.start(Catalina.java:655)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:355)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:495)
Similar questions and investigations suggest an invalid LDAP DN, but:
The same LDAP configuration works fine via an LDAP Client
No search is actually performed, at start-up time Tomcat throws this exception without any query
The error suggests an empty string '' as remaining name, hence not really something not found, apparently
Question(s): Is this the correct way to migrate an Foreign JNDI Providers entry from WebLogic to Tomcat? How to fix an invalid LDAP DN entry with an empty remaining name? Could it be a missing baseDN to configure somewhere?
Update
The same exact error happens when changing the LdapContextFactory to the following, as suggested via comments:
public Object getObjectInstance(Object obj, Name name, Context nameCtx, Hashtable<?, ?> environment)
throws Exception {
Hashtable<Object, Object> env = new Hashtable<>();
Reference reference = (Reference) obj;
Enumeration<RefAddr> references = reference.getAll();
String providerUrl = "no valid URL";
while (references.hasMoreElements()) {
RefAddr address = references.nextElement();
String type = address.getType();
String content = (String) address.getContent();
switch (type) {
case Context.PROVIDER_URL:
env.put(Context.PROVIDER_URL, content);
providerUrl = content;
break;
default:
env.put(type, content);
break;
}
}
InitialLdapContext context = null;
Object result = null;
try {
context = new InitialLdapContext(env, null);
LOGGER.info("looking up for " + providerUrl);
result = context.lookup(providerUrl);
} finally {
if (context != null) {
context.close();
}
}
LOGGER.info("Created new LDAP Context");
return result;
}
Change is confirmed via logging, to make sure it was deployed properly.
The involved listener is defined by default at the top of the server.xml file as
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
And cannot be disabled as per official documentation:
The Global Resources Lifecycle Listener initializes the Global JNDI resources defined in server.xml as part of the Global Resources element. Without this listener, none of the Global Resources will be available.
The same also happens on Tomcat version 8.5.5 and 7.0.69: simply adding the new global resource as above and the additional jar providing the factory above, the exception pointing at an empty remaining name will be thrown.
The stacktrace went away by appending to the java.naming.provider.url property the LDAP schema DN, using the first factory implementation provided in the question.
Below a screenshot of the LDAP client used in this context, the Apache Directory Studio / LDAP Browser embedded in Eclipse, from which it was possible to browse the concerned LDAP simply using the initial values of the question.
By appending the schema DN of the Root element to the connection URL, the exception went away and the LDAP resource is now shared via JNDI in Tomcat 8.
Further details as outcome of the troubleshooting:
In Tomcat 8 global resources are handled via a global resource listener, the GlobalResourcesLifecycleListener, defined by default in the server.xml file. Such a listener invokes a context.listBindings("") on bean creation, hence effectively browsing the LDAP directory.
This initial browsing may most probably be the difference between Tomcat and WebLogic, where LDAP is looked up via JNDI only when required, hence via direct query, rather than at start-up with a generic query. As such, in Tomcat the LDAP url would need further details, that is, a slightly different configuration as part of its url to directly point to a valid base DN.
From official WebLogic documentation:
On start up, WebLogic Server attempts to connect to the JNDI source. If the connection is successful, WebLogic Server sets up the requested objects and links in the local JNDI tree, making them available to WebLogic Server clients.
Hence, a connection is rather simpler than a listBindings:
Enumerates the names bound in the named context, along with the objects bound to them. The contents of any subcontexts are not included.
the code below allows me to create an embedded undertow servlet server, i have a problem setting the 'max-parameters' of the connector settings, the way i understand it undertow is normally configured via xml file.
public static String initCustomServer_(Servlet servlet,int preferedPort,String servletName,String[] resourceList,String... domainName){
String contextURL = null;
int curPort = preferedPort==-1?9001:preferedPort;
boolean initServ = false;
System.out.println("====servlet running in local mode====");
while(!initServ) {
try{
io.undertow.servlet.api.DeploymentInfo servletBuilder = io.undertow.servlet.Servlets.deployment()
.setClassLoader(servlet.getClass().getClassLoader())
.setContextPath(domainName.length==0?"/":"/"+domainName[0])
.setDeploymentName("test.war")
.addServlets(
io.undertow.servlet.Servlets.servlet(servletName, servlet.getClass()).addMapping("/"+servletName)
)
.setResourceManager(new io.undertow.server.handlers.resource.FileResourceManager(new File("src/dss_core/HTML5/webapp"), 1));
io.undertow.servlet.api.DeploymentManager manager = io.undertow.servlet.Servlets.defaultContainer().addDeployment(servletBuilder);
manager.deploy();
io.undertow.server.HttpHandler servletHandler = manager.start();
io.undertow.server.handlers.PathHandler path = io.undertow.Handlers.path(io.undertow.Handlers.redirect(domainName.length==0?"/":"/"+domainName[0]))
.addPrefixPath(domainName.length==0?"/":"/"+domainName[0], servletHandler);
io.undertow.Undertow server = io.undertow.Undertow.builder()
.addHttpListener(curPort, "localhost")
.setHandler(path)
.build();
server.start();
initServ = true;
contextURL = "http://localhost:"+curPort+(domainName.length==0?"":"/"+domainName[0])+"/"+servletName;
} catch (Exception ex) {
//creation of server at certain port fails therefore try again on another port
System.err.println(" server unable to initialize :" + ex.getMessage());
ex.printStackTrace();
curPort++;
}
}
return contextURL;
}
rather than using an xml like the one below how do i change configurations such as 'max-parameter' via embedded java code?
<server name="default-server">
<http-listener name="default" socket-binding="http" max-parameters="5000"/>
found here are list of stuff that i can configure via xml how can i set them via java code?
UPDATE 1: yay found some options in io.undertow.UndertowOptions, how ever this doesn't work as it is declared final, what now?
io.undertow.UndertowOptions.MAX_PARAMETERS = 10000;
after hours of research and trial and error finally i got it, my first idea was to simply get the code and compile it myself, negative side of that is that i'd have to download all source code then compile it, such proved to be trouble some and i decided to quit after seeing endless dependencies and hours of downloading their source code. configuring the server looked like this
io.undertow.Undertow server = io.undertow.Undertow.builder()
.addHttpListener(curPort, "localhost")
.setHandler(path)
.setServerOption(io.undertow.UndertowOptions.MAX_PARAMETERS, 10000)
.setServerOption(io.undertow.UndertowOptions.OPTION2, Value2)
.build();
setServerOption method and io.undertow.UndertowOptions class finally made sense, it's too bad undertow isn't very popular and not much sample code lying around, i hope i help anybody wishing to take the embedded road of undertow
I have some code like this for connecting to HornetQ.
Properties properties = new Properties();
properties.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");
properties.put(Context.PROVIDER_URL, "remote://127.0.0.1:4447");
properties.put(Context.SECURITY_PRINCIPAL, "user");
properties.put(Context.SECURITY_CREDENTIALS, "pwd");
ConnectionFactory connectionFactory = null;
Destination destination = null;
try {
Context context = new InitialContext(properties);
I inherited this, and am trying to get a better understanding of it. I haven't found documentation for the valid values where I have "remote://". I'm not sure if it's accurate to call that a protocol or not, but that's what it looks like. I've seen "jnp://" in other samples.
Is there an official list of valid values, and what they mean?
You may want to refer to specific JNDI Reference for specific versions. JBOSS AS 7.2 is covered here: https://docs.jboss.org/author/display/AS72/JNDI+Reference (note that in JBOSS AS 7.x, jnp is no longer supported, older JBOSS versions do support the jnp:// and access via the standard naming services).
Another link: https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/Naming_on_JBoss-The_Naming_InitialContext_Factories.html.
Hi I am trying to write a java client for secure hbase.
I want to do kinit also from code itself for that i`m using the usergroup information class.
Can anyone point out where am I going wrong here?
this is the main method that Im trying to connect o hbase from.
I have to add the configuration in the CONfiguration object rather than using the xml, because the client can be located anywhere.
Please see the code below:
public static void main(String [] args) {
try {
System.setProperty(CommonConstants.KRB_REALM, ConfigUtil.getProperty(CommonConstants.HADOOP_CONF, "krb.realm"));
System.setProperty(CommonConstants.KRB_KDC, ConfigUtil.getProperty(CommonConstants.HADOOP_CONF,"krb.kdc"));
System.setProperty(CommonConstants.KRB_DEBUG, "true");
final Configuration config = HBaseConfiguration.create();
config.set(CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHENTICATION, AUTH_KRB);
config.set(CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHORIZATION, AUTHORIZATION);
config.set(CommonConfigurationKeysPublic.FS_AUTOMATIC_CLOSE_KEY, AUTO_CLOSE);
config.set(CommonConfigurationKeysPublic.FS_DEFAULT_NAME_KEY, defaultFS);
config.set("hbase.zookeeper.quorum", ConfigUtil.getProperty(CommonConstants.HBASE_CONF, "hbase.host"));
config.set("hbase.zookeeper.property.clientPort", ConfigUtil.getProperty(CommonConstants.HBASE_CONF, "hbase.port"));
config.set("hbase.client.retries.number", Integer.toString(0));
config.set("zookeeper.session.timeout", Integer.toString(6000));
config.set("zookeeper.recovery.retry", Integer.toString(0));
config.set("hbase.master", "gauravt-namenode.pbi.global.pvt:60000");
config.set("zookeeper.znode.parent", "/hbase-secure");
config.set("hbase.rpc.engine", "org.apache.hadoop.hbase.ipc.SecureRpcEngine");
config.set("hbase.security.authentication", AUTH_KRB);
config.set("hbase.security.authorization", AUTHORIZATION);
config.set("hbase.master.kerberos.principal", "hbase/gauravt-namenode.pbi.global.pvt#pbi.global.pvt");
config.set("hbase.master.keytab.file", "D:/var/lib/bda/secure/keytabs/hbase.service.keytab");
config.set("hbase.regionserver.kerberos.principal", "hbase/gauravt-datanode2.pbi.global.pvt#pbi.global.pvt");
config.set("hbase.regionserver.keytab.file", "D:/var/lib/bda/secure/keytabs/hbase.service.keytab");
UserGroupInformation.setConfiguration(config);
UserGroupInformation userGroupInformation = UserGroupInformation.loginUserFromKeytabAndReturnUGI("hbase/gauravt-datanode2.pbi.global.pvt#pbi.global.pvt", "D:/var/lib/bda/secure/keytabs/hbase.service.keytab");
UserGroupInformation.setLoginUser(userGroupInformation);
User user = User.create(userGroupInformation);
user.runAs(new PrivilegedExceptionAction<Object>() {
#Override
public Object run() throws Exception {
HBaseAdmin admins = new HBaseAdmin(config);
if(admins.isTableAvailable("ambarismoketest")) {
System.out.println("Table is available");
};
HConnection connection = HConnectionManager.createConnection(config);
HTableInterface table = connection.getTable("ambarismoketest");
admins.close();
System.out.println(table.get(new Get(null)));
return table.get(new Get(null));
}
});
System.out.println(UserGroupInformation.getLoginUser().getUserName());
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
I`m getting the following exception:
Caused by: org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): GSS initiate failed
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:110)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:146)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupSaslConnection(RpcClient.java:762)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.access$600(RpcClient.java:354)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:883)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:880)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:880)
... 33 more
Any pointers would be helpful.
The above works nicely, but I've seen a lot of folks struggle with setting all of the right properties in the Configuration object. There's no de-facto list that I've found of exactly what you need and don't need and it is painfully dependent on your cluster configuration.
The surefire way is to have a copy of your HBase configurations in your classpath, since your client can be anywhere as you mentioned. Then you can add the resources to your object without having to specify all properties.
Configuration conf = HBaseConfiguration.create();
conf.addResource("core-site.xml");
conf.addResource("hbase-site.xml");
conf.addResource("hdfs-site.xml");
Here were some sources to back this approach:
IBM,
Scalding (Scala)
Also note that this approach doesn't limit you to actually use the internal Zookeeper principal and keytab, i.e. you can create keytabs for applications or Active Directory users and leave the internally generated keytabs for the daemons to authenticate amongst themselves.
Not sure if you still need help. I think setting the "hadoop.security.authentication" property is missing from your snippet.
I am using following code snippet to connect to secure HBase (on CDH5). You can give a try.
config.set("hbase.zookeeper.quorum", zookeeperHosts);
config.set("hbase.zookeeper.property.clientPort", zookeeperPort);
config.set("hadoop.security.authentication", "kerberos");
config.set("hbase.security.authentication", "kerberos");
config.set("hbase.master.kerberos.principal", HBASE_MASTER_PRINCIPAL);
config.set("hbase.regionserver.kerberos.principal", HBASE_RS_PRINCIPAL);
UserGroupInformation.setConfiguration(config);
UserGroupInformation.loginUserFromKeytab(ZOOKEEPER_PRINCIPAL,ZOOKEEPER_KEYTAB);
HBaseAdmin admins = new HBaseAdmin(config);
TableName[] tables = admins.listTableNames();
for(TableName table: tables){
System.out.println(table.toString());
}
in Jdk 1.8, you need set
"System.setProperty("javax.security.auth.useSubjectCredsOnly", "false");"
config.set("hbase.zookeeper.quorum", zookeeperHosts);
config.set("hbase.zookeeper.property.clientPort", zookeeperPort);
config.set("hadoop.security.authentication", "kerberos");
config.set("hbase.security.authentication", "kerberos");
config.set("hbase.master.kerberos.principal", HBASE_MASTER_PRINCIPAL);
config.set("hbase.regionserver.kerberos.principal", HBASE_RS_PRINCIPAL);
System.setProperty("javax.security.auth.useSubjectCredsOnly", "false");
UserGroupInformation.setConfiguration(config);
UserGroupInformation.loginUserFromKeytab(ZOOKEEPER_PRINCIPAL,ZOOKEEPER_KEYTAB);
HBaseAdmin admins = new HBaseAdmin(config);
TableName[] tables = admins.listTableNames();
for(TableName table: tables){
System.out.println(table.toString());
}
quote:
http://hbase.apache.org/book.html#trouble.client
question: 142.9
I think the best is https://scalding.io/2015/02/making-your-hbase-client-work-in-a-kerberized-environment/
To make the code work you don’t have to change any line from the one written in the top of this post, you just have to make your client able to access the full HBase configuration. This just implies to change your running classpath to:
/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hbase/conf:target/scala-2.11/hbase-assembly-1.0.jar
This will make everything run smoothly. It is specific for CDH 5.3 but you can adapt it for your cluster configuration.
PS No need in this:
conf.addResource("core-site.xml");
conf.addResource("hbase-site.xml");
conf.addResource("hdfs-site.xml");
Because HBaseConfiguration has
public static Configuration addHbaseResources(Configuration conf) {
conf.addResource("hbase-default.xml");
conf.addResource("hbase-site.xml");