UPDATE: See my blog post on this topic about a year after this was written: http://blog.ringerc.id.au/2012/07/java-ee-7-needs-improvements-in-app.html
... for references to the Java EE 7 planning discussion on this topic.
I've mostly finished writing a small Java EE 6 application, and am in the process of replacing the hard-coded preferences with a proper dynamic configuration interface.
I'm not sure how - or, more specifically, where - to store settings. There must be some obvious, "standard" way to do this that's expected to "just work" across various frameworks and containers, but for the life of me I cannot find it.
What I want is a simple way to load and store settings, one that works across different app servers and OSes, doesn't require any confguration by the user, and actually works properly. The Java Preferences API would be ideal - but seems broken under Glassfish 3.1.
Options for storing configuration would theoretically include:
Using context-parameters from the container environment
Storing them using the Java Preferences API
Reading/writing a properties file ... somewhere
Using JPA to store them in a JavaDB provided by the container
Putting it in a properties file that's loaded off the classpath
Using system properties to set configuration options, or path to a .properties file
This would seem to be a basic requirement that'd be well catered for in an environment where the container supposedly provides you with all the core services you might need - but all these approaches have issues.
A bug in glassfish renders (1) unworkable, and in any case the Glassfish web admin user interface lacks any way to configure context parameters, so you have to use `asadmin' and some less than lovely command line syntax to do it. Context parameters can only be accessed via the ServletContext - which isn't accessible in a consistent way between frameworks like JSF2, JAX-RS, and raw servlets - but at least Seam Servlet handles that.
What appears to be another bug in glassfish was a library version conflict between the deployed app and Glassfish breaks (2). The preferences backend fails to flush preferences to disk, so the stored preferences data is lost when the application server is restarted. The Java Preferences API also seems to be considered a J2SE/desktop thing, despite its inclusion in the Java EE 6 specs.
(3) might work - but there's no way to know where your app has read/write access on the file system and where it should look. You can't make this configurable, as it becomes a chicken-and-egg problem then. Various platform-specific guesses could be applied, but would break in the presence of a SecurityManager.
(4) would work, but it's nuking a fly. It requires that a JavaDB service be running and forces the user to make sure the JDBC and pool resources in the app server are configured properly. It's big and complicated for a simple job, and entity modelling isn't a lovely fit for preferences storage anyway, as it'll mostly land up being key/value structured.
(5) would work, but requires users to know where to put the config file where it'll be found under various different app servers. It also makes it hard for the app to provide any kind of configuration UI because it can't necessarily find the local path to the config file or open it for writing, especially in the presence of a SecurityManager.
(6) would also work, but forces the user to configure the configuration system before they can configure the application. Needless to say, that doesn't excite me, given how relatively complicated deploying the app and creating the resources already is for users who don't already know Glassfish/EE.
So ... how are you handling configuration and storage of options? Have you found a way that lets you "just do it" without the user having to configure anything to allow your app to store its configuration?
The problem with the preferences API was caused by the inclusion of jaxb and stax implementation jars on in the application's war, pulled in by jersey-json . With these excluded (as they're provided by the app server anyway) the preferences API resumed functioning correctly.
It looks like the prefs API with custom UI for setup appears to be the best way to go.
Though not the environment you spoke about: http://www.osgi.org/javadoc/r4v42/org/osgi/service/cm/ConfigurationAdmin.html
Related
What is the best way to store parameters and data for an EE7 application. I have to provide the web applications with information like a member fee or similar data (which may/can be altered several times in a year). The owner of the application should also have a central place where these data are stored and an application to change them.
Thanks in advance for any input
Franz
This is one question we are currently struggling with as we re-architect some of our back-end systems here, and I do agree with the comment from #JB Nizet that it should be stored on the database, however I will try to add some additional considerations and options to help you make the decision that is right for you. The right option will depend on a few factors though.
If you are delivering source code and automation to build and deploy your software, the configuration can be stored in a source code repository (i.e. as YAML or XML) and bundled with your deployable during the build process. This is a bit archaic but certainly widely adopted practice and works well, for the most part.
If you are delivering deployable binaries, you have a couple of options.
First one is to have a predetermined place in the file system where your application will look for an "override" configuration file (i.e. home directory of the user used to run your application server). This way you can have your binary deployable file completely separate from your configuration, but you will still need to build some sort of automation and version control for that configuration file so that your customer can roll back versions if/when necessary. This can also be one or many configuration files (i.e. separate files for your app server, vs. the application itself).
The option we are contemplating currently is having a configuration database where all of our applications can query for their own configuration. This can either be a very simple or complex solution depending on your particular needs - for us these are internal applications and we manage the entire lifecycles ourselves, but we have a need to have a central repository since we have tens of services and applications running with a good number of common configuration keys, and updating these keys independently can be error prone.
We are looking at a few different solutions, but I would certainly not store the configuration in our main database as: 1) I don't think SQL is best repository for configuration, 2) I believe we can get better performance from NoSQL databases which can be critical if you need to load some of those configuration keys for every request.
MongoDB and CouchDB both come to mind as good candidates for storing the our configuration keys if you need clearly defined hierarchy for you options, whereas Redis or Memcached are great options if you just need a key-value storage for your configuration (faster than document based too). We will also likely build a small app to help up configure and version the configuration and push changes to existing/active servers, but we haven't spec'd out all the requirements for that.
There are also some OSS solutions that may work for you, although some of them add too much complexity for what we are trying to achieve at this point. If you are using springframework, take a look at the Spring Cloud Config Project, it is very interesting and worth looking into.
This is a very interesting discussion and I am very willing to continue it if you have more questions on how to achieve distributed configurations. Food for thought, here are some of my personal must haves and nice to haves for our new configuration architecture design:
Global configuration per environment (dev,staging,prod)
App specific configuration per environment (dev,staging,prod)
Auto-discovery (auto environment selection depending on requestor)
Access control and versioning
Ability to push updates live to different services
Roger,thanks a lot. Do you have an example for the version predetermined place in the file system"predetermined place in the file system"? Does it make sense to use a singleton which reads the configuration file (using Startup annotation) and provides then the configuration data? But this does not support a dynamic solution.kind regards Franz
We have a legacy application that uses Struts 1.2.9. The app is currently internationalized the standard way - .properties files for all UI labels, errors, messages, etc; <message-resouces> definition for each .properties file in struts-config.xml using default Factory & MessageResources definitions; <bean:message> usage in all JSPs. This has worked great till now, but for the fact that the application itself a framework for services used by a few hundred (yes 100's!) other applications internally.
We have a requirement to extend the i18n functionality as follows:
Define a custom directory for .properties files - so this would be outside the scope of the classpath; basically not inside the .war package. The idea is to support just message string changes w/o really having to redeploy the entire application.
This custom directory will also contain per supported applications messages - this could be just a subset of the existing ones or the entire set of resources tailored specifically to that application.
Custom way of supporting per request basis Locale setting - barring all other considerations (default stack, classpath/package lookups, etc.) this is analogous to the way I18nInterceptor works in Struts2 with the requestOnlyParameterName attribute set to true.
Yes, I do understand that a few 100 bundles loaded at the same time will be memory intensive, but that is acceptable in our case.
Any help is appreciated - be it direction, sample code, etc.
Note: I completely agree that moving onto a newer UI platform is probably the best solution. But we can't.
TIA.
I had a similar requirement in a spring project, not only for i18n, also web services endpoints and other kind of properties.
We accomplish that requirement by adding that directory, in which we place the properties files, into the classpath in the server start configuration file.
Tested and working in weblogic 11g (Preproduction and production) and in a tomcat server (development environment).
Hope helps
Our company is currently using RAD to develop our Java apps, but we're looking to move to Eclipse with the WebSphere Developer Tools. The pilot for our transition is going pretty well, except we're running into a classloader policy issue for new applications that are originally created in Eclipse, not RAD. Our projects that were originally created by RAD are deployed with the correct classloader policy (PARENT_LAST) when published via Eclipse because we originally used the Deployment Descriptor Editor in RAD which set the proper classloader policy in /src/main/application/META-INF/ibmconfig/cells/defaultCell/applications/defaultApp/deployments/defaultApp/deployment.xml. But now with Eclipse & WebSphere Developer Tools, we no longer have the nice Deployment Descriptor Editor UI to create or modify this file for us (apparently it's not included with the WDT plugin).
So, my question then is what is the best way to go about setting this classloader policy? We still need some new apps to have the classloader policy of PARENT_LAST set when we deploy them to our local servers. Our team has though about this a bit and we can see 4 options at the moment.
Open the Admin Console after every publish and change it. This would be a huge pain, and is pretty much not even a real option.
Change the server profile setting to use a PARENT_LAST classloader policy for all apps. This however is not the case for all the apps at our company, and would not work for all groups.
Run a jython script after every publish to set the classloader policy. This is slightly better than option 1, but not by much.
Manually create a deployment.xml file in the same location as the other apps created by RAD with the same structure as the deployment.xml files created by RAD, and modify it as necessary for each app.
Option 4 seems to be the best of the bunch, but it's still a manual process and somewhat error prone. Even if most of our developers can grok this approach for new apps, it would be most ideal if this were a simple one button click type process.
So given the fact that IBM has omitted the Deployment Descriptor Editor from the WDT plugin it would seem as if option 4 is our only hope, but I'll ask once more, is there any other better way to set a WebSphere classloader policy to PARENT_LAST for an app when that app is created in Eclipse? Any help is appreciated, thanks.
Well, Eclipse is free, while Rational Application Developer costs about $5,000 per year (per developer). The nice Deployment Editor (which wasn't that nice. It tends to include all sorts of things that aren't needed. Who needs that Derby DataSource defined there, anyway?) is one of the things you have to give up for saving tons of cash on an annual basis.
I'm digressing.
Option (1) is a complete no-no. You don't want to rely on manual steps for deployments; you should strive to automate deployments to the extent possible.
Option (2) might do. I am not sure which flavour of WebSphere you're using, but if you're using the Network Deployment edition, then you can design a WebSphere topology that consists of multiple servers and clusters. You could, theoretically, come up with such a topology whereby PARENT_LAST applications run on a specific server (or cluster) and PARENT_FIRST applications run on another server (or cluster).
You may be able to combine option (2) with a technical initiative to have all of your applications work with PARENT_LAST. This is the recommended approach if your application is using popular third-party libraries that WebSphere happens to use as well (for its own internal purposes). For example, if you're using Commons Lang, then you're already recommended to switch to PARENT_LAST because WebSphere uses its own internal copy of Commons Lang that might conflict with yours.
Option (3) - it's of course better than option (1) but isn't necessarily worse than option (2) if you can get your WebSphere topology right.
Option (4) is harder to implement but I believe it's the best approach overall:
It's a one-time setup effort for each EAR (and for each WAR that exists within the EAR).
Once it's done, deployment can easily be automated as no extra steps are needed.
If you're working with a local test environment to test your code, and you're rapidly publishing applications from your workspace into your local test environment, then this approach is the only approach (other than option (2)) that will work for you without extra manual work.
If none works... consider paying $5,000 per year (per user) and get option (5) - use IBM's editor. Or, better off... hire someone to design an Eclipse plugin that will do that for you. Shouldn't take more than a week or two to develop.
um nether answer is useful.
Go into WAS console and pick your application; example:
Enterprise Applications > my_application_ear > Class loader AND change the "class loader order and WAR class loader policy"
Open the admin console within eclipse click server >> your server >> scroll down and under server infrastructure >> java process management select class loader >> select new you can change it here
I have an application that consists of approx. 20 java components.
About half of the components are servers and the other half batch programs.
Almost all of them talk directly to an oracle database (jdbc via some of our infrastructure code jars) the other couple of components talk to some of the servers which talk to the database.
Anyway, each component is configured with numerous XML configuration files.
These are becoming almost impossible to maintain.
Some of the configuration is specific to a component others are similar (database URLs, connectors etc)
What is worse is that the application is not installed in many environments - in fact only about 10 environments (qa, dev, production etc etc).
But the people who own these environments don't seem able to maintain the configs correctly.
In particular whenever there is an upgrade there is invariably configuration errors.
I have even started checking in some of the environments configurations into SVN along with the code.
I tried an xml schema validator at one point (it consisted of defining the valid XML in .xsd files and then throwing an error if the schema rules were breached but that didnt work)
I'm thinking I am missing something basic here - perhaps there is a tool to manage this or perhaps I should be storing the configuration in the database.
The application was largely designed by a colleague but I feel myself that it's overly configurable - in fact many of the config actually refers to classes - i.e. one can choose handlers and parsers etc - the XML config almost looks like code.
Any advice greatly appreciated
Peter
Substituting XML for code is usually a bad idea; things that are declarative are probably OK, but things that are procedural probably aren't.
If all that configuration was defined in Java code, a lot of the upgrade issues would turn into compilation issues. The compiler would pick them out for you, and you could correct them.
So you've got a multi-part problem. You need to rationalize your configuration information into a set of partitions (per-component, per-installation, global). You need to try to verify configuration information at compile-time, where possible. And you need to write validation for the loaded configurations, to sanity check them.
To the extent possible, shift configuration relatively static stuff into Guice (at least, it's what I prefer). A lot of things happen in a nice, type-safe way with it.
Consider running a WebDAV server for each instance of the app, and storing configuration into it. Each can hit a simple URL to pull the current versions of the configuration files.
Or, stand up a lightweight XML database like BaseX with its REST capability, then store and load your configuration information there. Use JSLP or something like it to have your components find the central configuration repository.
An additional advantage to using an XML DB is that you'll be able to do a lot of sanity checking and updating by querying across the set of all configuration files. For example, if a given instance of the application should have the same JDBC parameters in each configuration file, a simple xquery will tell you if that's true.
If you don't have the ability to modify the applications that are pulling the configuration file (the config file format is fixed), then consider writing a query servlets for the XML database that assemble the required configuration information, from nested blocks or templates. That will allow you to figure out what's common between the configuration files and dynamically generate parameterized versions of those blocks.
Sounds like the key here is making incremental improvements. Allow the old way to configure, but have the configuration load look for a central config source first.
I don't think that the syntax of the configuration files is at the heart of the problem: using Java properties files instead of XML would leave you with exactly the same issues. There may be an issue that the configuration information is too dispersed - it's hard to tell. The main issue seems to be that the whole thing is too fragile - the application is too dependent on manual configuration, and it seems that the configuration for each environment needs to be different. You should try to focus on reducing the number of configuration parameters that need to be set to make the system work (without necessarily reducing the options available for diagnostics etc for use when they are really needed.), on having intelligent defaults and self-configuration. Perhaps even invest in creating an installation wizard.
As you have some Oracle databases handy why not store your configuration in there?
Then you only need one or two configuration parameters to point to an Oracle database suitable for that environment and download the rest of the configuration from the database.
The contents of the configuration table should be pretty static for any given environment so there should be no need to amend anything except the jdbc connection when you migrate your software through its life cycle.
I'd like to use something like the Filtering Classloader to prevent specific packages from creeping into the application context and becoming visible to Spring.
Changing the classloader order causes all sorts of nasty problems so I´d like to try this route.
Is it possible to achieve this with Websphere 6? If not, can I replace my own application classolader and implement the filter myself?
There is no such filtering mechanism in WebSphere, and there is no way to replace the application class loader. You'll have to use PARENT_LAST to override classes, sorry.
bkail's answer is right, WAS doesn't have such feature even in its latest public version (8.5.5).
I just created a RFE requesting such feature so whoever is interested in this, please vote for it which may increase the possibility of this being implemented:
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=43936
(IBM ID required)
In the meantime, you may use isolated shared libraries to override any particular classes (the above mentioned class loading order control - like parent_last - is too rough as it affects the class loading order of the whole application or module)
Create a shared library with desired jars on the classpath, configure it as isolated shared library, reference it from the deployed application (or module).
See here for complete documentation
http://pic.dhe.ibm.com/infocenter/wasinfo/v8r5/topic/com.ibm.websphere.base.doc/ae/tcws_sharedlib.html
I was just about to post the same question. But the answer was quite unsatisfying. I however checked the request from Petr H at the IBM developerworks and IBM did implement this feature (Huge thanks Petr!):
"WebSphere Application Server V8.5.5.7 (=Fixpack 7) gained the ability to prevent packages from the server classloader being visible to applications. This was delivered in the document "ISOLATE DEPLOYED ARTIFACTS FROM OSS PACKAGES" and is documented in 'Isolating open source software packages'.
The supplied links describe the mechanism by configuring always-protected packages. You basically have to do the following:
Under Server Infrastructure on the server settings page in the administrative console, click Java and process management > Process definition.
Select Java virtual machine.
Define the following system properties in the JVM generic arguments section as follows:
-Dcom.ibm.ws.classloader.server.alwaysProtectedPackages=org.bouncycastle.
Please not that the final dot "." is really important otherwise everything will be ignored! Several packages can be added by comma ","
Click Apply, OK and save the changes. Make sure that a file synchronization is done before you restart the servers. Restart WebSphere Application Server for the changes to take effect.
Examine the native_stdout.log and find the system properties that are previously defined. For example, when you specify always-protected package org.bouncycastle., statements such as the following might appear:
ProtectionMetaData.clinit: system property: com.ibm.ws.classloader.server.alwaysProtectedPackages=org.bouncycastle.