ZooKeeper for Java/Spring Config? - java

Are there any well documented use cases of Apache ZooKeeper being used to distribute configuration of Java applications, and in particular Spring services?
Like many users of cloud services I have a requirement to change the configuration of a variable amount of Java services, preferably at run-time without needing to restart the services.
UPDATE
Eventually I ended up writing something that would load a ZooKeeper node as a properties file, and create a ResourcePropertySource and insert it into a Spring context. Note that this will not reflect changes in the ZooKeeper node after the context has started.
public class ZooKeeperPropertiesApplicationContextInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
private static final Logger logger = LoggerFactory.getLogger(ZooKeeperPropertiesApplicationContextInitializer.class);
private final CuratorFramework curator;
private String projectName;
private String projectVersion;
public ZooKeeperPropertiesApplicationContextInitializer() throws IOException {
logger.trace("Attempting to construct CuratorFramework instance");
RetryPolicy retryPolicy = new ExponentialBackoffRetry(10, 100);
curator = CuratorFrameworkFactory.newClient("zookeeper", retryPolicy);
curator.start();
}
/**
* Add a primary property source to the application context, populated from
* a pre-existing ZooKeeper node.
*/
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
logger.trace("Attempting to add ZooKeeper-derived properties to ApplicationContext PropertySources");
try {
populateProjectProperties();
Properties properties = populatePropertiesFromZooKeeper();
PropertiesPropertySource propertySource = new PropertiesPropertySource("zookeeper", properties);
applicationContext.getEnvironment().getPropertySources().addFirst(propertySource);
logger.debug("Added ZooKeeper-derived properties to ApplicationContext PropertySources");
curator.close();
} catch (IOException e) {
logger.error("IO error attempting to load properties from ZooKeeper", e);
throw new IllegalStateException("Could not load ZooKeeper configuration");
} catch (Exception e) {
logger.error("IO error attempting to load properties from ZooKeeper", e);
throw new IllegalStateException("Could not load ZooKeeper configuration");
} finally {
if (curator != null && curator.isStarted()) {
curator.close();
}
}
}
/**
* Populate the Maven artifact name and version from a property file that
* should be on the classpath, with values entered via Maven filtering.
*
* There is a way of doing these with manifests, but it's a right faff when
* creating shaded uber-jars.
*
* #throws IOException
*/
private void populateProjectProperties() throws IOException {
logger.trace("Attempting to get project name and version from properties file");
try {
ResourcePropertySource projectProps = new ResourcePropertySource("project.properties");
this.projectName = (String) projectProps.getProperty("project.name");
this.projectVersion = (String) projectProps.getProperty("project.version");
} catch (IOException e) {
logger.error("IO error trying to find project name and version, in order to get properties from ZooKeeper");
}
}
/**
* Do the actual loading of properties.
*
* #return
* #throws Exception
* #throws IOException
*/
private Properties populatePropertiesFromZooKeeper() throws Exception, IOException {
logger.debug("Attempting to get properties from ZooKeeper");
try {
byte[] bytes = curator.getData().forPath("/distributed-config/" + projectName + "/" + projectVersion);
InputStream in = new ByteArrayInputStream(bytes);
Properties properties = new Properties();
properties.load(in);
return properties;
} catch (NoNodeException e) {
logger.error("Could not load application configuration from ZooKeeper as no node existed for project [{}]:[{}]", projectName, projectVersion);
throw e;
}
}
}

You should consider Spring Cloud Config:
http://projects.spring.io/spring-cloud/
Spring Cloud Config Centralized external configuration management
backed by a git repository. The configuration resources map directly
to Spring Environment but could be used by non-Spring applications
if desired.
Source code available here:
https://github.com/spring-cloud/spring-cloud-config
Sample application here:
https://github.com/spring-cloud/spring-cloud-config/blob/master/spring-cloud-config-sample/src/main/java/sample/Application.java

I created a set of spring beans integration zookeeper and springframework as propertyplaceholderconfigurer, in github: https://github.com/james-wu-shanghai/spring-zookeeper.git
you can take a look.

Not spring in particular but for java generally, there is a CXF implementation of the distributed OSGI standard that uses ZooKeeper as the discovery server to push updated bundles down to the container : http://cxf.apache.org/dosgi-discovery.html.

Zookeeper can be very nicely leveraged with higher abstraction using Curator APIs for configuration management in distributed applications. To get started just follow these two steps.
STEP 1 : Start zookeper server and then start zookeeper cli and create some znodes. Znodes are nothing but UNIX like files which contain values, and name of files depict property name.
To create/fetch/update properties use these commands on zookeeper cli.
create /system/dev/example/port 9091
get /system/dev/example/port
set /system/dev/example/port 9092
To fetch these properties in java program refer this code snippet.
import java.util.HashMap;
import java.util.Map;
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.retry.ExponentialBackoffRetry;
public class App
{
public static void main( String[] args ) throws Exception
{
final String ZK = "localhost:2181";
final Map<String, String> data = new HashMap<String, String>();
CuratorFramework client = CuratorFrameworkFactory.newClient(ZK, new ExponentialBackoffRetry(100, 3));
client.start();
System.out.println(new String(client.getData().forPath("/system/dev/example/port")));
}
}

I was at an Apache Camel talk from James Strachen last week and he mentioned using ZooKeeper under the covers for their Java-based server in the cloud as the source of configuration info.
I've seen a talk from Adrian Colyer (CTO of SpringSource) about runtime config change in Spring, but does Spring support this today?
In my opinion, if you're starting from a typically architected Spring application, I don't see you having an easy job retrofitting dynamic config changes on top of it.

After finding a suggestion to use a FactoryBean to populate a regular PropertyPlaceholderConfigurer I've built this:
package fms;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.config.AbstractFactoryBean;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.util.Properties;
public class ZkPropertiesFactoryBean extends AbstractFactoryBean<Properties> implements Watcher {
private Logger LOGGER = LoggerFactory.getLogger(ZkPropertiesFactoryBean.class);
private String zkConnect;
private String path;
private int timeout = 1000;
#Override protected Properties createInstance() throws Exception {
long start = System.currentTimeMillis();
Properties p = new Properties();
p.load(new ByteArrayInputStream(loadFromZk()));
double duration = (System.currentTimeMillis() - start)/1000d;
LOGGER.info(String.format("Loaded %d properties from %s:%s in %2.3f sec", p.size(), zkConnect, path, duration));
return p;
}
#Override public Class<Properties> getObjectType() {
return Properties.class;
}
private byte[] loadFromZk() throws IOException, KeeperException, InterruptedException {Stat stat = new Stat();
ZooKeeper zk = new ZooKeeper(zkConnect, timeout, this);
return zk.getData(path, false, stat);
}
#Override public void process(WatchedEvent event) {}
public void setPath(String path) {this.path = path;}
public void setZkConnect(String zkConnect) {this.zkConnect = zkConnect;}
}
In the spring-config.xml you create the beans as follows:
<bean id="zkProperties" class="fms.ZkPropertiesFactoryBean" p:zkConnect="localhost:2181" p:path="/app/zk-properties"/>
<context:property-placeholder properties-ref="zkProperties"/>

Related

Remote Connect to Jboss Application-Server - NamingIOException cause by ClassNotFound

First of all this is my first question on StackOverflow and I'm an Intern in a Company in Germany, so My English is a little broken and my Knowledge might be limited.
I Try to connectio to a Jboss 6.1.0 eap remotely.
I'm using Eclipse as IDE for the EJB and the EAR but I run the Jboss form cmd
My ejb3 definition look like that:
package de.jack;
import javax.ejb.Remote;
#Remote
public interface TestServiceRemote {
public void sayRemote();
}
package de.jack;
import javax.ejb.Stateless;
/**
* Session Bean implementation class TestService
*/
#Stateless
public class TestService implements TestServiceRemote {
public TestService() { }
public void sayRemote() {
System.out.println("\n\nHello");
}
}
After gernerating the .ear file I deploy them in the JBoss AS and all that works fine
I can view them in the browser under localhost:9990 and check that they are deployed
Now to the Part where I fail - the Client:
public static void main(String argv[]){
Properties props = new Properties();
props.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");
props.put(Context.PROVIDER_URL, "remote://localhost:4447");
props.put(Context.SECURITY_PRINCIPAL, "jack");
props.put(Context.SECURITY_CREDENTIALS, "katze");
props.put("jboss.naming.client.ejb.context", true);
// create a context passing these properties
InitialContext context;
Object test = null;
try {
context = new InitialContext(props);
} catch (NamingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return;
}
try {
test =
context.lookup("ConnectorBean/TestService!de.jack.TestServiceRemote");
} catch (NamingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return;
}
On Run I get the exception:
org.jboss.naming.remote.protocol.NamingIOException: Failed to lookup [Root exception is java.io.IOException: java.lang.ClassNotFoundException: de.jack.TestServiceRemote]
at org.jboss.naming.remote.client.ClientUtil.namingException(ClientUtil.java:49)
at org.jboss.naming.remote.protocol.v1.Protocol$1.execute(Protocol.java:104)
at org.jboss.naming.remote.protocol.v1.RemoteNamingStoreV1.lookup(RemoteNamingStoreV1.java:95)
at org.jboss.naming.remote.client.HaRemoteNamingStore$1.operation(HaRemoteNamingStore.java:245)
...
I not sure what exactly I did wrong
one reason could be that I do not have admin-rights on the maschine or I mixed up the properties on Client side
Sorry for my bad english and I'm very thankful for any help!
modify TestService class
#Stateless
#Remote(TestServiceRemote.class)
public class TestService implements TestServiceRemote {
public TestService() { }
public void sayRemote() {
System.out.println("\n\nHello");
}
}
ensure remote client has a reference of TestServiceRemote.class
change lookup jndi name
// The app name is the application name of the deployed EJBs. This is typically the ear name
// without the .ear suffix. However, the application name could be overridden in the application.xml of the
// EJB deployment on the server.
// Since we haven't deployed the application as a .ear, the app name for us will be an empty string
final String appName = "";
// This is the module name of the deployed EJBs on the server. This is typically the jar name of the
// EJB deployment, without the .jar suffix, but can be overridden via the ejb-jar.xml
// In this example, we have deployed the EJBs in a jboss-as-ejb-remote-app.jar, so the module name is
// jboss-as-ejb-remote-app
final String moduleName = "jboss-as-ejb-remote-app";
// AS7 allows each deployment to have an (optional) distinct name. We haven't specified a distinct name for
// our EJB deployment, so this is an empty string
final String distinctName = "";
// The EJB name which by default is the simple class name of the bean implementation class
final String beanName = TestService.class.getSimpleName();
// the remote view fully qualified class name
final String viewClassName = TestServiceRemote.class.getName();
String jndiName= "ejb:" + appName + "/" + moduleName + "/" + distinctName + "/" + beanName + "!" + viewClassName;
TestServiceRemote service = (TestServiceRemote)context.lookup(jndiName);
Detail please reffer: https://docs.jboss.org/author/display/AS71/EJB+invocations+from+a+remote+client+using+JNDI
Server log should show correct global JNDI name for the bean. It should be like foo/EJB-NAME/remote. Then you need change it in context.lookup("ConnectorBean/TestService!de.jack.TestServiceRemote").
Please check -
http://docs.jboss.org/ejb3/docs/tutorial/1.0.7/html/JNDI_Bindings.html

Jini/JavaSpaces discovery error

On this article: http://java.sun.com/developer/technicalArticles/tools/JavaSpaces/ is a tutorial how to run JavaSpaces client. I wrote these classes in Eclipse, started Launch-All script and Run example. It works.
After that I exported these classes into executable jar (JavaSpaceClient.jar) and tried that jar with following command:
java -jar JavaSpaceClient.jar
It works fine, gives me result:
Searching for a JavaSpace...
A JavaSpace has been discovered.
Writing a message into the space...
Reading a message from the space...
The message read is: Здраво JavaSpace свете!
My problem is when I move this jar file on my other LAN computer, it shows me error when I type same command. Here is error:
cica#cica-System-Name:~/Desktop$ java -jar JavaSpaceClient.jar
Searching for a JavaSpace...
Jul 27, 2011 11:20:54 PM net.jini.discovery.LookupDiscovery$UnicastDiscoveryTask run
INFO: exception occurred during unicast discovery to biske-Inspiron-1525:4160 with constraints InvocationConstraints[reqs: {}, prefs: {}]
java.net.UnknownHostException: biske-Inspiron-1525
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:175)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
at java.net.Socket.connect(Socket.java:546)
at java.net.Socket.connect(Socket.java:495)
at com.sun.jini.discovery.internal.MultiIPDiscovery.getSingleResponse(MultiIPDiscovery.java:134)
at com.sun.jini.discovery.internal.MultiIPDiscovery.getResponse(MultiIPDiscovery.java:75)
at net.jini.discovery.LookupDiscovery$UnicastDiscoveryTask.run(LookupDiscovery.java:1756)
at net.jini.discovery.LookupDiscovery$DecodeAnnouncementTask.run(LookupDiscovery.java:1599)
at com.sun.jini.thread.TaskManager$TaskThread.run(TaskManager.java:331)
I just writes "Searching for JavaSpace..." and after a while prints these error messages.
Can someone help me with this error?
EDIT:
For discovery I am using LookupDiscovery class I've found on Internet:
import java.io.IOException;
import java.rmi.RemoteException;
import net.jini.core.lookup.ServiceRegistrar;
import net.jini.core.lookup.ServiceTemplate;
import net.jini.discovery.LookupDiscovery;
import net.jini.discovery.DiscoveryListener;
import net.jini.discovery.DiscoveryEvent;
/**
A class which supports a simple JINI multicast lookup. It doesn't register
with any ServiceRegistrars it simply interrogates each one that's
discovered for a ServiceItem associated with the passed interface class.
i.e. The service needs to already have registered because we won't notice
new arrivals. [ServiceRegistrar is the interface implemented by JINI
lookup services].
#todo Be more dynamic in our lookups - see above
#author Dan Creswell (dan#dancres.org)
#version 1.00, 7/9/2003
*/
public class Lookup implements DiscoveryListener {
private ServiceTemplate theTemplate;
private LookupDiscovery theDiscoverer;
private Object theProxy;
/**
#param aServiceInterface the class of the type of service you are
looking for. Class is usually an interface class.
*/
public Lookup(Class aServiceInterface) {
Class[] myServiceTypes = new Class[] {aServiceInterface};
theTemplate = new ServiceTemplate(null, myServiceTypes, null);
}
/**
Having created a Lookup (which means it now knows what type of service
you require), invoke this method to attempt to locate a service
of that type. The result should be cast to the interface of the
service you originally specified to the constructor.
#return proxy for the service type you requested - could be an rmi
stub or an intelligent proxy.
*/
Object getService() {
synchronized(this) {
if (theDiscoverer == null) {
try {
theDiscoverer =
new LookupDiscovery(LookupDiscovery.ALL_GROUPS);
theDiscoverer.addDiscoveryListener(this);
} catch (IOException anIOE) {
System.err.println("Failed to init lookup");
anIOE.printStackTrace(System.err);
}
}
}
return waitForProxy();
}
/**
Location of a service causes the creation of some threads. Call this
method to shut those threads down either before exiting or after a
proxy has been returned from getService().
*/
void terminate() {
synchronized(this) {
if (theDiscoverer != null)
theDiscoverer.terminate();
}
}
/**
Caller of getService ends up here, blocked until we find a proxy.
#return the newly downloaded proxy
*/
private Object waitForProxy() {
synchronized(this) {
while (theProxy == null) {
try {
wait();
} catch (InterruptedException anIE) {
}
}
return theProxy;
}
}
/**
Invoked to inform a blocked client waiting in waitForProxy that
one is now available.
#param aProxy the newly downloaded proxy
*/
private void signalGotProxy(Object aProxy) {
synchronized(this) {
if (theProxy == null) {
theProxy = aProxy;
notify();
}
}
}
/**
Everytime a new ServiceRegistrar is found, we will be called back on
this interface with a reference to it. We then ask it for a service
instance of the type specified in our constructor.
*/
public void discovered(DiscoveryEvent anEvent) {
synchronized(this) {
if (theProxy != null)
return;
}
ServiceRegistrar[] myRegs = anEvent.getRegistrars();
for (int i = 0; i < myRegs.length; i++) {
ServiceRegistrar myReg = myRegs[i];
Object myProxy = null;
try {
myProxy = myReg.lookup(theTemplate);
if (myProxy != null) {
signalGotProxy(myProxy);
break;
}
} catch (RemoteException anRE) {
System.err.println("ServiceRegistrar barfed");
anRE.printStackTrace(System.err);
}
}
}
/**
When a ServiceRegistrar "disappears" due to network partition etc.
we will be advised via a call to this method - as we only care about
new ServiceRegistrars, we do nothing here.
*/
public void discarded(DiscoveryEvent anEvent) {
}
}
My client program tries simply to search for JavaSpaces service write MessageEntry into and then retrieves message and prints it out. Here is client program:
import net.jini.space.JavaSpace;
public class SpaceClient {
public static void main(String argv[]) {
try {
MessageEntry msg = new MessageEntry();
msg.content = "Hello JavaSpaces wordls!";
System.out.println("Searching for JavaSpaces...");
Lookup finder = new Lookup(JavaSpace.class);
JavaSpace space = (JavaSpace) finder.getService();
System.out.println("JavaSpaces discovered.");
System.out.println("Writing into JavaSpaces...");
space.write(msg, null, 60*60*1000);
MessageEntry template = new MessageEntry();
System.out.println("Reading message from JavaSpaces...");
MessageEntry result = (MessageEntry) space.read(template, null, Long.MAX_VALUE);
System.out.println("Message: "+result.content);
} catch(Exception e) {
e.printStackTrace();
}
}
}
And of course this is MessageEntry class:
import net.jini.core.entry.*;
public class MessageEntry implements Entry {
public String content;
public MessageEntry() {
}
public MessageEntry(String content) {
this.content = content;
}
public String toString() {
return "MessageContent: " + content;
}
}
EDIT2:
I did discovery on two Windows computers.
After that I tried Windows - Ubuntu combiantion and it doesn't work. Maybe there are some network problems? When I ping each another everything is ok. Maybe there are some DNS issues on Ubuntu..
EDIT3:
Windows - Ubuntu combination works if JavaSpaces service is started up on Windows and client program is on Ubuntu. When I try to do reverse, to run JavaSpaces service on Ubuntu and run client on Windows error occurs.
Obviously there is some problem with Ubuntu. Ubuntu has installed OpenJDK installed by default. I installed Oracle JDK, and set JAVA_HOME and put JAVA_HOME/bin into PATH variable. I wonder maybe there is some problem with different versions of Java, maybe I am not using right one.
It is possible that the service registrar that you are running (on host biske-Inspiron-1525 at port 4160), is discovering it's hostname incorrectly (without domain name) and is therefore sending out the announcements with a short hostname. Therefore, after discovering the service registrar, it is possible that subsequently the client is trying to make a connection to the service registrar it cannot resolve the hostname if it is on a different domain.
To ensure that the service registrar is running with the correct hostname, try starting it with the following command line attribute:
-Dcom.sun.jini.reggie.unicastDiscoveryHost="biske-Inspiron-1525.and.its.domain"
It appears that you are doing unicast discovery to a specific host and port and that you can't look up that host.
Assuming you can resolve the name biske-Inspiron-1525 with DNS try removing the ":4160" part and see if the unicast lookup succeeds then.
Here is an example of the code I use to look up a service. It's a bit more complicated because I implement ServiceDiscoveryListener and handle service discovery that way. I actually keep a list of services and dynamically switch between then when one fails but I stripped that part out of the example. I am also using the Configuration part of Jini which I'll explain afterwards. The service interface I am using here is called "TheService":
public class JiniClient implements ServiceDiscoveryListener {
private TheService service = null;
private Class[] serviceClasses;
private ServiceTemplate serviceTemplate;
public JiniClient(String[] configFiles) throws ConfigurationException {
Configuration config = ConfigurationProvider.getInstance(configFiles,
getClass().getClassLoader());
// Set the security manager
System.setSecurityManager(new RMISecurityManager());
// Define the service we are interested in.
serviceClasses = new Class[] {TheService.class};
serviceTemplate = new ServiceTemplate(null, serviceClasses, null);
// Build a cache of all discovered services and monitor changes
ServiceDiscoveryManager serviceMgr = null;
DiscoveryManagement mgr = null;
try {
mgr = (DiscoveryManagement)config.getEntry(
getClass().getName(), // component
"discoveryManager", // name
DiscoveryManagement.class); // type
if (null == mgr) {
throw new ConfigurationException("entry for component " +
getClass().getName() + " name " +
"discoveryManager must be non-null");
}
} catch (Exception e) {
/* This will catch both NoSuchEntryException and
* ConfigurationException. Putting them both
* below just to make that clear.
*/
if( (e instanceof NoSuchEntryException) ||
(e instanceof ConfigurationException)) {
// default value
try {
System.err.println("Warning, using default multicast discover.");
mgr = new LookupDiscoveryManager(LookupDiscovery.ALL_GROUPS,
null, // unicast locators
null); // DiscoveryListener
} catch(IOException ioe) {
e.printStackTrace();
throw new RuntimeException("Unable to create lookup discovery manager: " + e.toString());
}
}
}
try {
serviceMgr = new ServiceDiscoveryManager(mgr, new LeaseRenewalManager());
} catch (IOException e) {
e.printStackTrace();
throw new RuntimeException("Unable to create service discovery manager: " + e.toString());
}
try {
serviceMgr.createLookupCache(serviceTemplate,
null, // no filter
this); // listener
} catch(Exception e) {
e.printStackTrace();
throw new RuntimeException("Unable to create serviceCache: " + e.getMessage());
}
}
public void serviceAdded(ServiceDiscoveryEvent evt) {
/* Called when a service is discovered */
ServiceItem postItem = evt.getPostEventServiceItem();
//System.out.println("Service appeared: " +
// postItem.service.getClass().toString());
if(postItem.service instanceof TheService) {
/* You may be looking for multiple services.
* The serviceAdded method will be called for each
* so you can use instanceof to figure out if
* this is the one you want.
*/
service = (TheService)postItem.service;
}
}
public void serviceRemoved(ServiceDiscoveryEvent evt) {
/* This notifies you of when a service goes away.
* You could keep a list of services and then remove this
* service from the list.
*/
}
public void serviceChanged(ServiceDiscoveryEvent evt) {
/* Likewise, this is called when a service changes in some way. */
}
The Configuration system allows you to dynamically configure the discovery method so you can switch to discover specific unicast systems or multicast without changing the app. Here is an example of a unicast discovery configuration file that you could pass to the above objects constructor:
import net.jini.core.discovery.LookupLocator;
import net.jini.discovery.LookupDiscoveryManager;
import net.jini.discovery.LookupDiscovery;
com.company.JiniClient {
discoveryManager = new LookupDiscoveryManager(
LookupDiscovery.ALL_GROUPS,
new LookupLocator[] { new LookupLocator("jini://biske-Inspiron-1525.mycompany.com")},
null,
this); // the current config
}
I found solution! That was dns issue. On Ubuntu my /etc/hosts file was:
192.168.1.3 biske-Inspiron-1525 # Added by NetworkManager
127.0.0.1 localhost.localdomain localhost
::1 biske-Inspiron-1525 localhost6.localdomain6 localhost6
127.0.1.1 biske-Inspiron-1525
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
I've just removed line 127.0.1.1 biske-Inspiron-1525 and now it works fine.
Little thing was destroyed million of my nerves :)

Easy way to start a standalone JNDI server (and register some resources)

For testing purposes, I'm looking for a simple way to start a standalone JNDI server, and bind my javax.sql.DataSource to "java:/comp/env/jdbc/mydatasource" programmatically.
The server should bind itself to some URL, for example: "java.naming.provider.url=jnp://localhost:1099" (doesn't have to be JNP), so that I can look up my datasource from another process. I don't care about which JNDI server implementation I'll have to use (but I don't want to start a full-blown JavaEE server).
This should be so easy, but to my surprise, I couldn't find any (working) tutorial.
The JDK contains a JNDI provider for the RMI registry. That means you can use the RMI registry as a JNDI server. So, just start rmiregistry, set java.naming.factory.initial to com.sun.jndi.rmi.registry.RegistryContextFactory, and you're away.
The RMI registry has a flat namespace, so you won't be able to bind to java:/comp/env/jdbc/mydatasource, but you will be able to bind to something so it will accept java:/comp/env/jdbc/mydatasource, but will treat it as a single-component name (thanks, #EJP).
I've written a small application to demonstrate how to do this: https://bitbucket.org/twic/jndiserver/src
I still have no idea how the JNP server is supposed to work.
I worked on the John´s code and now is working good.
In this version I'm using libs of JBoss5.1.0.GA, see jar list below:
jboss-5.1.0.GA\client\jbossall-client.jar
jboss-5.1.0.GA\server\minimal\lib\jnpserver.jar
jboss-5.1.0.GA\server\minimal\lib\log4j.jar
jboss-remote-naming-1.0.1.Final.jar (downloaded from http://search.maven.com)
This is the new code:
import java.net.InetAddress;
import java.util.Hashtable;
import java.util.concurrent.Callable;
import javax.naming.Context;
import javax.naming.InitialContext;
import org.jnp.server.Main;
import org.jnp.server.NamingBeanImpl;
public class StandaloneJNDIServer implements Callable<Object> {
public Object call() throws Exception {
setup();
return null;
}
#SuppressWarnings("unchecked")
private void setup() throws Exception {
//configure the initial factory
//**in John´s code we did not have this**
System.setProperty(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
//start the naming info bean
final NamingBeanImpl _naming = new NamingBeanImpl();
_naming.start();
//start the jnp serve
final Main _server = new Main();
_server.setNamingInfo(_naming);
_server.setPort(5400);
_server.setBindAddress(InetAddress.getLocalHost().getHostName());
_server.start();
//configure the environment for initial context
final Hashtable _properties = new Hashtable();
_properties.put(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
_properties.put(Context.PROVIDER_URL, "jnp://10.10.10.200:5400");
//bind a name
final Context _context = new InitialContext(_properties);
_context.bind("jdbc", "myJDBC");
}
public static void main(String...args){
try{
new StandaloneJNDIServer().call();
}catch(Exception _e){
_e.printStackTrace();
}
}
}
To have good logging, use this log4j properties:
log4j.rootLogger=TRACE, A1
log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
To consume the Standalone JNDI server, use this client class:
import java.util.Hashtable;
import javax.naming.Context;
import javax.naming.InitialContext;
/**
*
* #author fabiojm - Fábio José de Moraes
*
*/
public class Lookup {
public Lookup(){
}
#SuppressWarnings("unchecked")
public static void main(String[] args) {
final Hashtable _properties = new Hashtable();
_properties.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory");
_properties.put("java.naming.provider.url", "jnp://10.10.10.200:5400");
try{
final Context _context = new InitialContext(_properties);
System.out.println(_context);
System.out.println(_context.lookup("java:comp"));
System.out.println(_context.lookup("java:jdbc"));
}catch(Exception _e){
_e.printStackTrace();
}
}
}
Here's a code snippet adapted from JBoss remoting samples. The code that is
in the samples (version 2.5.4.SP2 ) no longer works. While the fix
is simple it took me more hours than I want to think about to figure it out.
Sigh. Anyway, maybe someone can benefit.
package org.jboss.remoting.samples.detection.jndi.custom;
import java.net.InetAddress;
import java.util.concurrent.Callable;
import org.jnp.server.Main;
import org.jnp.server.NamingBeanImpl;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class StandaloneJNDIServer implements Callable<Object> {
private static Logger logger = LoggerFactory.getLogger( StandaloneJNDIServer.class );
// Default locator values - command line args can override transport and port
private static String transport = "socket";
private static String host = "localhost";
private static int port = 5400;
private int detectorPort = 5400;
public StandaloneJNDIServer() {}
#Override
public Object call() throws Exception {
StandaloneJNDIServer.println("Starting JNDI server... to stop this server, kill it manually via Control-C");
//StandaloneJNDIServer server = new StandaloneJNDIServer();
try {
this.setupJNDIServer();
// wait forever, let the user kill us at any point (at which point, the client will detect we went down)
while(true) {
Thread.sleep(1000);
}
}
catch(Exception e) {
e.printStackTrace();
}
StandaloneJNDIServer.println("Stopping JBoss/Remoting server");
return null;
}
private void setupJNDIServer() throws Exception
{
// start JNDI server
String detectorHost = InetAddress.getLocalHost().getHostName();
Main JNDIServer = new Main();
// Next two lines add a naming implemention into
// the server object that handles requests. Without this you get a nice NPE.
NamingBeanImpl namingInfo = new NamingBeanImpl();
namingInfo.start();
JNDIServer.setNamingInfo( namingInfo );
JNDIServer.setPort( detectorPort );
JNDIServer.setBindAddress(detectorHost);
JNDIServer.start();
System.out.println("Started JNDI server on " + detectorHost + ":" + detectorPort );
}
/**
* Outputs a message to stdout.
*
* #param msg the message to output
*/
public static void println(String msg)
{
System.out.println(new java.util.Date() + ": [SERVER]: " + msg);
}
}
I know I'm late to the party, but I ended up hacking this together like so
InitialContext ctx = new InitialContext();
// check if we have a JNDI binding for "jdbc". If we do not, we are
// running locally (i.e. through JUnit, etc)
boolean isJndiBound = true;
try {
ctx.lookup("jdbc");
} catch(NameNotFoundException ex) {
isJndiBound = false;
}
if(!isJndiBound) {
// Create the "jdbc" sub-context (i.e. the directory)
ctx.createSubcontext("jdbc");
//parse the jetty-web.xml file
Map<String, DataSource> dataSourceProperties = JettyWebParser.parse();
//add the data sources to the sub-context
for(String key : dataSourceProperties.keySet()) {
DataSource ds = dataSourceProperties.get(key);
ctx.bind(key, ds);
}
}
Have you considered using Mocks? If I recall correctly you use Interfaces to interact with JNDI. I know I've mocked them out at least once before.
As a fallback, you could probably use Tomcat. It's not a full blown J2EE impl, it starts fast, and is fairly easy to configure JNDI resources for. DataSource setup is well documented. It's sub-optimal, but should work.
You imply you've found non-working tutorials; that may mean you've already seen these:
J2EE or J2SE? JNDI works with both
Standalone JNDI server using jnpserver.jar
I had a quick go, but couldn't get this working. A little more perseverance might do it, though.
For local, one process standalone jar purpouses I would use spring-test package:
SimpleNamingContextBuilder builder = new SimpleNamingContextBuilder();
SQLServerConnectionPoolDataSource myDS = new SQLServerConnectionPoolDataSource();
//setup...
builder.bind("java:comp/env/jdbc/myDS", myDS);
builder.activate();
startup log:
22:33:41.607 [main] INFO org.springframework.mock.jndi.SimpleNamingContextBuilder - Static JNDI binding: [java:comp/env/jdbc/myDS] = [SQLServerConnectionPoolDataSource:1]
22:33:41.615 [main] INFO org.springframework.mock.jndi.SimpleNamingContextBuilder - Activating simple JNDI environment
I have been looking for a similar simple starter solution recently. The "file system service provider from Sun Microsystems" has worked for me well. See https://docs.oracle.com/javase/jndi/tutorial/basics/prepare/initial.html.
The problem with the RMI registry is that you need a viewer - here you just need to look at file contents.
You may need fscontext-4.2.jar - I obtained it from http://www.java2s.com/Code/Jar/f/Downloadfscontext42jar.htm

SAP JCo connector loaded forever in GlassFish v2.1 (can't unload)

I've a problem with GlassFish and SAP JCo connector (sapjco3.jar)
I load it at start up of an J2EE application (jwm.ear) and initialize it wrapped in a singleton the first time a connection to SAP is needed.
The problem is that this jar remains always initialized in memory, I need to restart glassfish to unload the initialized connections if I need to change a single parameter. Stopping or undeploying the app doesn't unload sapjco.jar and furter redeployments of the app never get the new connection parameters, the first initialization remains until GlassFish restart.
Does anybody knows how to unload or reinitialize this library? preferably even without redeploying the app, first time the app is activated I have a reference to jcoProvider, next activations get a null reference to jcoProvider, but a jcoProvider continues instantiated in memory with initial values.
Regards!
Notes:
GlassFish is version 2.1 in Windows 2008 server, jdk is 1.6.0.14
sapjco3.jar and sapjco3.dll are copied to \domains\domain1\lib\ext and are version 3 of SAP java connector.
Singleton to get SAP connections:
package es.grupotec.ejb.SAP;
import com.sap.conn.jco.JCoDestination;
import com.sap.conn.jco.JCoDestinationManager;
import com.sap.conn.jco.JCoException;
import com.sap.conn.jco.ext.DestinationDataProvider;
import com.sap.conn.jco.ext.Environment;
import es.grupotec.ejb.util.ConexionSAPException;
import java.util.Properties;
public final class SAP {
private static String SAP_SERVER = "JWM";
private static SAP instance = null;
private static JCOProvider jcoProvider = null;
private SAP() {
// Exists only to defeat instantiation.
}
// Get SAP connection
public static synchronized JCoDestination getDestination() throws ConexionSAPException {
JCoDestination jcoDestination = null;
if (Environment.isDestinationDataProviderRegistered()) {
try {
jcoDestination = JCoDestinationManager.getDestination(SAP_SERVER);
return jcoDestination;
} catch (JCoException ex) {
throw new ConexionSAPException(ex.getMessage());
}
}
// Create new connection
if(jcoProvider == null) init();
// Get connection
try {
jcoDestination = JCoDestinationManager.getDestination(SAP_SERVER);
return jcoDestination;
} catch (JCoException ex) {
throw new ConexionSAPException(ex.getMessage());
}
}
// Initialize connection to SAP
public static synchronized void init() throws ConexionSAPException {
SAPVO sap = new SAPVO();
Properties properties = new Properties();
if(jcoProvider == null) {
// Get SAP config from database
try {
sap = SAPDAO.getSAPConfig();
} catch (Exception ex) {
throw new ConexionSAPException(ex.getMessage());
}
// Create connection object
jcoProvider = new JCOProvider();
}
properties.setProperty(DestinationDataProvider.JCO_ASHOST, sap.getJCO_ASHOST());
properties.setProperty(DestinationDataProvider.JCO_SYSNR, sap.getJCO_SYSNR());
properties.setProperty(DestinationDataProvider.JCO_CLIENT, sap.getJCO_CLIENT());
properties.setProperty(DestinationDataProvider.JCO_USER, sap.getJCO_USER());
properties.setProperty(DestinationDataProvider.JCO_PASSWD, sap.getJCO_PASSWD());
properties.setProperty(DestinationDataProvider.JCO_LANG, sap.getJCO_LANG());
try {
jcoProvider.changePropertiesForABAP_AS(properties);
} catch (Exception e) {
throw new ConexionSAPException(e.getMessage());
}
}
public static synchronized void change(SAPVO sap) throws ConexionSAPException {
Properties properties = new Properties();
// If connection is null create a new one
if(jcoProvider == null) jcoProvider = new JCOProvider();
properties.setProperty(DestinationDataProvider.JCO_ASHOST, sap.getJCO_ASHOST());
properties.setProperty(DestinationDataProvider.JCO_SYSNR, sap.getJCO_SYSNR());
properties.setProperty(DestinationDataProvider.JCO_CLIENT, sap.getJCO_CLIENT());
properties.setProperty(DestinationDataProvider.JCO_USER, sap.getJCO_USER());
properties.setProperty(DestinationDataProvider.JCO_PASSWD, sap.getJCO_PASSWD());
properties.setProperty(DestinationDataProvider.JCO_LANG, sap.getJCO_LANG());
try {
jcoProvider.changePropertiesForABAP_AS(properties);
} catch (Exception e) {
throw new ConexionSAPException(e.getMessage());
}
}
// Prevent instantiation by clone
#Override
public Object clone() throws CloneNotSupportedException {
throw new CloneNotSupportedException();
}
}
JCo provider implementation:
package es.grupotec.ejb.SAP;
import com.sap.conn.jco.ext.DestinationDataEventListener;
import com.sap.conn.jco.ext.DestinationDataProvider;
import com.sap.conn.jco.ext.Environment;
import es.grupotec.ejb.util.ConexionSAPException;
import java.util.Properties;
public class JCOProvider implements DestinationDataProvider {
private String SAP_SERVER = "JWM";
private DestinationDataEventListener eventListener;
private Properties ABAP_AS_properties;
public JCOProvider(){
}
public JCOProvider(SAPVO sap){
ABAP_AS_properties = new Properties();
ABAP_AS_properties.setProperty(DestinationDataProvider.JCO_ASHOST, sap.getJCO_ASHOST());
ABAP_AS_properties.setProperty(DestinationDataProvider.JCO_SYSNR, sap.getJCO_SYSNR());
ABAP_AS_properties.setProperty(DestinationDataProvider.JCO_CLIENT, sap.getJCO_CLIENT());
ABAP_AS_properties.setProperty(DestinationDataProvider.JCO_USER, sap.getJCO_USER());
ABAP_AS_properties.setProperty(DestinationDataProvider.JCO_PASSWD, sap.getJCO_PASSWD());
ABAP_AS_properties.setProperty(DestinationDataProvider.JCO_LANG, sap.getJCO_LANG());
ABAP_AS_properties.setProperty(DestinationDataProvider.JCO_POOL_CAPACITY, sap.getJCO_POOL_CAPACITY());
ABAP_AS_properties.setProperty(DestinationDataProvider.JCO_PEAK_LIMIT, sap.getJCO_PEAK_LIMIT());
try {
if (!Environment.isDestinationDataProviderRegistered())
Environment.registerDestinationDataProvider(this);
else changePropertiesForABAP_AS(ABAP_AS_properties);
} catch (Exception ex) {
String msg = ex.getMessage();
}
}
#Override
public Properties getDestinationProperties(String name) {
if (name.equals(SAP_SERVER) && ABAP_AS_properties!=null) return ABAP_AS_properties;
else return null;
}
#Override
public boolean supportsEvents() {
return true;
}
#Override
public void setDestinationDataEventListener(DestinationDataEventListener eventListener) {
this.eventListener = eventListener;
}
public void changePropertiesForABAP_AS(Properties properties) throws ConexionSAPException {
try {
if (!Environment.isDestinationDataProviderRegistered()) {
if (ABAP_AS_properties == null) ABAP_AS_properties = properties;
Environment.registerDestinationDataProvider(this);
}
if (properties == null) {
if (eventListener != null) eventListener.deleted(SAP_SERVER);
ABAP_AS_properties = null;
} else {
ABAP_AS_properties = properties;
if (eventListener != null) eventListener.updated(SAP_SERVER);
}
} catch (Exception ex) {
throw new ConexionSAPException(ex.getMessage());
}
}
}
Your problem is probably related to the fact that there is some native code involved here. This is even true for JCo 3. Whereas JCo 3 does not make usage of the native RFC library anymore, it still requires JNI to communicate with the CPIC layer.
Getting a JVM to unload a native library is an exercise in utmost frustration. The JNI Specification states that a native library will be unloaded when the ClassLoader associated with the class it provides the implementation to is unloaded, but trying to force a ClassLoader to unload is virtually impossible within the JVM.
If your EAR file includes the sapjco3.jar it will be reloaded each time your code is reloaded. This very likely will causes exceptions as the native library can't be loaded more than once and there is practically no way to unload native code. So you might consider to place the sapjco3.jar outside of the J2EE container and let your J2EE engine load that library once at start time rather than putting it into the EAR that gets reloaded over and over.
Which release of SAP are you intending to connect to? We had several problems with the Java Connector, it was not really threadsafe and could not be embedded into a EJB application properly. Same problems came with SAP's seculib for single sign on. It either didn't work. The only solution was to load it outside of the J2EE engine.
Have you ever thought of replacing the JCO with webservices? Of course it's slightly slower as the data has to go through the ICF but it's more robust. We switched all our integrations to this solution.

Custom gRPC command for Dropwizard exits immediately

I've created an application using Dropwizard that starts a gRPC server. I do not use the regular server, and want to start my application using java -jar my-fat.jar grpc config.yml instead.
I've come as far as to add the command as the only available command during startup by overriding the corresponding method in the application class:
public class App extends Application<Configuration> {
public static void main(String[] args) throws Exception {
new App().run(args);
}
#Override
protected void addDefaultCommands(final Bootstrap<Configuration> bootstrap) {
bootstrap.addCommand(new GrpcCommand(this));
}
}
I can launch my application using java -jar my-fat.jar grpc config.yml. My command looks like this:
public class GrpcCommand extends EnvironmentCommand<Configuration> {
public GrpcCommand(Application<Configuration> application) {
this(application, "grpc", "Runs the Dropwizard application as a gRPC server");
}
/**
* Creates a new environment command.
*
* #param application the application providing this command
* #param name the name of the command, used for command line invocation
* #param description a description of the command's purpose
*/
protected GrpcCommand(final Application<Configuration> application, final String name, final String description) {
super(application, name, description);
}
#Override
protected void run(final Environment environment, final Namespace namespace, final Configuration configuration) throws Exception {
final var certificateService = AzureCertificateService.createWithClients(
AzureSecretClient.create(configuration.getKeyVaultConfiguration()),
AzureCertificateClient.create(configuration.getKeyVaultConfiguration())
);
final var validationService = CertificateValidationService.create(certificateService);
final var signingService = CertificateSigningService.create(certificateService);
final Pair<X509Certificate, KeyPair> certificate = certificateService.getSigningCertificateWithKeyPair();
final BaseApiImpl baseApi = new BaseApiImpl(validationService, signingService);
final GrpcServer grpcServer = GrpcServer.newBuilder()
.withBaseApi(baseApi)
.withConfiguration(configuration.getGrpcConfiguration())
.withCertificate(certificate.getLeft())
.withPrivateKey(certificate.getRight().getPrivate())
.build();
new Thread(() -> {
try {
grpcServer.start();
} catch (Exception e) {
e.printStackTrace();
}
}).run();
environment.healthChecks().register("grpc-server", new GrpcServerHealthCheck(grpcServer));
}
}
The way that thread is started is not for production use, I'm just trying to get forward. The start method for the GrpcServer class:
#Override
public void start() throws Exception {
final NettyServerBuilder nettyServerBuilder = NettyServerBuilder.forPort(configuration.getPort())
.addService(baseApi)
.intercept(new OriginInterceptor());
if (certificate != null && privateKey != null) {
LOG.info("Got certificate and private key, enabling SSL");
nettyServerBuilder.sslContext(buildSslContext());
}
server = nettyServerBuilder
.build()
.start();
LOG.info("Server started at port {}", server.getPort());
}
And I see the message GrpcServer: Server started at port 50441 in my log when I start. However, the application does not stay open. What am I missing? Shouldn't my use of the thread create a thread that stops the application from exiting? How can I keep the application running after the gRPC server has started?
When I disabled the server command, Jetty isn't started either (of course), which kept the application alive previously.
I found the simplest solution in the gRPC Hello World Example.
My start method now looks like this:
public void start() throws Exception {
// Everything else as above
Runtime.getRuntime().addShutdownHook(new Thread(GrpcServer.this::stop));
LOG.info("Server started at port {}", server.getPort());
blockUntilShutdown();
}
private void blockUntilShutdown() throws InterruptedException {
if (server != null) {
server.awaitTermination();
}
}

Categories