So here's my project:
I am building a central interface/dashboard to present the test data for several test types of multiple product versions. We're using TestNG on our massive product, and while not enough tests are being written, that's a discussion for another topic. Here's what the directory structure looks like:
Filesystem/productVersion+testType/uniqueDateAndBuildID/testng-results.xml
That results.xml file contains tags with child test tags, which correspond to a filesystem directory and then xml files containing actual test case results (pass, fail, etc)
The XML parsing and filesystem traversal is all well and good/reliable.
Flow of control:
Client accesses main page --> server opens properties file --> server checks for web server property (either Websphere or Tomcat, if I'm working locally) --> server sets bunch of constants based on that. Constants include: root filesystem directory, filesystem separator (translation), "like types (basically same tests on different platforms)", and a base URL to append onto. --> server then reads properties file some more and does all of its XML processing. Results are cached in memory as well as to the filesystem using ObjectOutputStream. --> A big list of results is sent back to the client to do the UI processing/display.
Here's where I run into a problem: I can't access those Global variables (contained/set in a Globals class...bad I know :-/ ) back on the client, even though they're in the shared folder. If you're wondering why I can't just load the properties again, it's because the client is GWT-ified Javascript which doesn't include File(). So my next thought, having done a little bit of upper level Java reading was to maybe use a Globals singleton object and pass that back too..but it seems like that's just as bad if not impossible. Suggestions here would be great.
This whole thing is pretty tightly coupled, something my previous Java education hadn't really gotten into yet. And since this is just an internal portal for devs to check, there doesn't seem to be much of a point in actually testing my code. As long as it displays correctly, logs properly, and handles errors gracefully, right? All in all it's <15 classes, so it's not really a big big deal I guess. Should I refactor to clean it all up and make it "better Java", comment everything to clearly delineate flow of control, or not worry too much about it because it's small? I know in the future to think more about things before I design them, but I really didn't know a large amount of the higher Java principles I've been exposed to since starting.
edit after doing a bit of thinking, came up with a possible workaround. What about, instead of passing back only a list of results, I passed back some other custom list implementation that included a globals 'header' object? I could preserve state.
A simple solution would be the Dictionary class:
Provides dynamic string lookup of
key/value string pairs defined in a
module's host HTML page. Each unique
instance of Dictionary is bound to a
named JavaScript object that resides
in the global namespace of the host
page's window object. The bound
JavaScript object is used directly as
an associative array.
You just need to add some dynamic content to your host HTML page - make the server print the values read from the properties file in the form of a JavaScript object:
var GlobalProperties = {
property1: "value1",
property2: "value2"
};
Then, use Dictionary in your code to read those values:
Dictionary globalProperties = Dictionary.getDictionary("GlobalProperties");
String property1 = globalProperties.get("property1");
PS: If you are looking for good ideas/advices on how to make your code less coupled -> more testable, I'd recommend Misko Hevery's blog. He's got many interesting posts, like why singletons are usually bad (global state, not the pattern itself). But most importantly - it has the awesome guide to writing testable code (some guidelines used internally in Google).
You could pass those Global variables using a simple object with a HashMap thought a GWT-RPC call or just include this Hashmap with the result you already retrieve in the first place (along the "big list of results [that] is sent back to the client to do the UI processing/display.")
You can't access serverside singletons from the compiled javascript.
You have two options basically. You can make a Serializable class in the client code, that represents the global variables, or pass your global variables object, but this is a rather inefficient solution.
The simplest is to use a HashMap<String, String> in a serializable object, which you can retrieve with an RPC call:
public class GwtGlobalVariables implements Serializable {
private HashMap<String, String> map = new HashMap<String, String>();
public void put(// a delegate put method of choice
public void setMap() // a getter / setter for the map if you need it
}
Ensure the class is within a GWT module's source folders, i.e. in the same place as your entry point maybe.
Fill the map out with the values needed, pass it through rpc and you have it in your client side code.
Related
I'm in the middle of a massive refactoring project, the code has a 5000 line main class which was injected into everything, stored everything and had all of the common code.
I'm no expert on analysis and design but I've separated out things to the best of my ability and I'm about 80% through refactoring the classes that depend on the main class to use the new classes I've created.
There are some types of data which are initialised when the application starts and accessed by pretty much everything throughout the life of the application. For instance there is a Config class which holds hundreds of parameters.
The approach I've taken is to create several singletons the two most central are GUIData and ClientData. GUIData contains a reference to the mainframe of the application and clientdata maintains references to the config and other similar classes.
This allows me to call ClientData.getInstance().getConfig().getParam("param") from anywhere in the code but I don't feel like this is the best approach.
I considered individual static classes instead of these data singletons which contain instances of the classes but some of the classes do need constructors.
I've been googling on and off for a week trying to find a better way to do this but somehow I always end up on threads talking about database caching
Immutable (configuration) instances provide "thread-safe application-wide data access".
Typesafe's config (as suggested in a comment by Brian Kent) does exactly that.
Note that this does not involve static classes or singletons. Static classes and singletons may serve your purposes now,
but they could prove bothersome in the future. They can be handy ofcourse, but try limiting their use.
Initialization will have to be done after reading and parsing the configuration data. It is typically done at application startup, before other processing threads are started. The initialization will have to validate the configuration data as much as possible in order to fail fast and terminate the program if the configuration data is no good.
Having a lot of configuration data bundled together can create "hidden lines of communication". E.g. you update one value and the application fails because it required updates to other values as well. It's perfectly fine to put all configuration data in one file and load it from there, but your application (with hundreds of configuration options) should divide the configuration data in sets that are used by different parts of your application. This improves isolation, helps unit-testing and makes it possible to change the application in the future without getting too many nasty surprises.
There are two ways to use a set of configuration data:
from within an object call a singleton Settings.getInstance().getConfigForThisModule().
provide each object that uses configuration data with the configuration data via the constructor or via setConfig(ConfigForThisModule config).
The first approach depends on a convention not to call Settings.getInstance().getConfigForACompletelyUnrelatedModule() which could be a weakness. The second approach is more in line with "dependency injection" and could be more future proof.
You could mix both approaches while you are refactoring, just make sure to be consistent (e.g. only use the singleton approach for configuration data that is used in all parts of the application).
To further improve your design for using the configuration data, keep the following (likely) future functional requirement in mind: when the configuration file is updated, configuration data is reloaded and used in the application. Most logging frameworks manage to support this functional requirement without affecting the performance of multi-threaded applications. Among other things, it requires the following of your application:
if the new configuation data is no good, the program is not terminated but an error is logged instead and the old configuration data remains in use. Your initialization procedure will need to handle both "load at fresh start" and "reload" scenarios. The main thing to take away from this is that your initialization procedure needs to be re-usable and should not affect other (running) parts of your application (isolation, again).
long-lived objects may not keep a local copy of configuration data or a reference to an instance of ConfigForThisModule, instead Settings.getInstance()... (or some other method that can return an updated instance) should be called regurarly.
replacing old configuration with new configuration may not result in errors. Technically, replacing the configuration is as simple as updating an AtomicReference with a new configuration instance returned with Settings.getInstance().... But this is also where the isolation of the configuration data sets are tested: there should be no problem using an old set in one module and a new set in another module at the same.
Configuration data can be seen as a sort of "global state". With that in mind, further design points on what to do and what to avoid (partially blatantly copied to this answer) are discussed in the following two questions:
Why is Global State so Evil?
How are globals any different from a database?
Sorry, the question is a bit vague, are you looking to store the config or the cached objects used by other parts of your program ?
Since you have 100s of params, start with splitting up the config into manageable blocks
1) Split up your configuration parameters into logical blocks that have 1:1 correspondence with a simple properties file -its going to take some time
2) These property files must be externalized so that you can change them at any point in time, make sure that you pass in the base location via a env variable to the program
3) Write a utility class (singleton) that wraps Apache commons configuration to hold your config. (read *.properties from the base location and merge the properties into one configuration object) this must be done before any threads are kicked off.
4) Refer to the configuration param in your code using config.getXXXX() methods
Apache commons config also has ability to reload the config when your properties file changes on the filesystem.
Once this is done, use a DI container like Spring or Guice to cache the configured objects.
If it's just String property values you need, you don't even need a class for that - a global facility exists for you already: System.getProperties()
All you need do is first load the property values on start up:
System.setProperty("myKey", "myValue"); // see below how load properties from a file
Then read it anywhere in your code:
String myValue = System.getProperty("myKey");
or
String myValue = System.getProperty("myKey", "my desired default");
If your container doesn't support property loading out of the box, to load properties from an external file that looks like this:
key1=value
key2=some other value
etc...
you can use this code:
Files.lines(Paths.get("path/to/file"))
.filter(line -> !line.startsWith("#") || !line.contains("=")) // ignore comment/blank
.map(line -> line.split("=", 2)) // split into key/value
.forEach(split -> System.setProperty(split[0], split[1])); // load as property
you can use the Java Properties class util, basically its a HashTable
reference : https://docs.oracle.com/javase/7/docs/api/java/util/Properties.html
you create a file fileName.properties and store your data in key value pairs, for example:
username=your name
port=8080
then you load it into Properties Object and get the data like the following:
Properties prop = new Properties();
load the file...
String userName = prop.getProperty("username")
String port = prop.getProperty("port")// you can parse it to int if needed
what i suggest is to create a property file for each type of configuration like:
clientData.properties
appConfig.properties
you can follow this simple tutorial
http://www.mkyong.com/java/java-properties-file-examples/
I'm using an API providing access to a special server environment. This API has a wide range of Data objects you can retrieve from it. For Example APICar
Now I'd like to have "my own" data object (MyCar) containing all information of that data object but i'd like to either leave out some properties, augment it, or simply rename some of them.
This is because i need those data objects in a JSON driven client application. So when someone changes the API mentioned above and changes names of properties my client application will break immediatly.
My question is:
Is there a best practice or a design pattern to copy objects like this? Like when you have one Object and want to transfer it into another object of another class? I've seen something like that in eclipse called "AdapterFactory" and was wondering if it's wide used thing.
To make it more clear: I have ObjectA and i need ObjectB. ObjectA comes from the API and its class can change frequently. I need a method or an Object or a Class somewhere which is capable of turning an ObjectA into ObjectB.
I think you are looking for Design Pattern Adapter
It's really just wrapping an instance of class A in an instance of class B, to provide a different way of using it / different type.
"I think" because you mention copying issues, so it may not be as much a class/type thing as a persistence / transmission thing.
Depending on your situation you may also be interested in dynamic proxying, but that's a Java feature.
I am looking for concrete ideas of how to manage a lot of different parameter settings for my java program. I know this question is a bit diffuse but I need some ideas about the big picture so that my code becomes more maintainable.
What my project does is to perform many processing steps on data, mostly text. These processing steps are algorithms of varying complexity that often have many settings. I would also like to change which processing steps are used by e.g. configuration files.
The reason for my program is to do repeatable experiments, and because of this I need to be able to get a complete view of all the parameters used in the different parts of the code, preferably in a nice format.
At this (prototype) stage I have the settings in source code like:
public static final param1=0.35;
and each class that is responsible for some processing step has its own hard coded settings. It is actually quite scary because there is no simple way to change things or to even see what is done and with what parameters/settings.
My idea is to have a central key/value store for all settings that also supports a dump of all settings. Example:
k:"classA_parameter1",v:"0.35"
k:"classC_parameter5",v:"false"
However, I would not really like to just store the parameters as strings but have them associated to an actual java class or object.
Is it smarter to have a singleton "SettingsManager" that manages everything. Or to have a SettingsManager object in each class that main has access to? I don't really like storing string descriptions of the settings but I cant see any other way (Lets say one setting is a SAXparser implementation that is used and another parameter is a double, e.g. percentage) since I really don't want to store them as Objects and cast them.
Experience and links to pages about relevant design patterns is greatly appreciated.
To clarify, my experiments could be viewed as a series of algorithms that are working on data from files/databases. These algorithms are grouped into different classes depending on their task in the whole process, e.g.
Experiment //main
InternetLookup //class that controls e.g. web scraping
ThreadedWebScraper
LanguageDetection //from "text analysis" package
Statistics //Calculate and store statistics
DatabaseAccess
DecisionMaking //using the data that we have processed earlier, make decisions (machine learning)
BuildModel
Evaluate
Each of the lowest level classes have parameters and are different but I still want a to get a view of everything that is going on.
You have the following options, starting with the simplest one:
A Properties file
Apache Commons Configuration
Spring Framework
The latter allows creation of any Java object from an XML config file but note that it's a framework, not a library: this means that it affects the design of the whole application (it promotes the Inversion of Control pattern).
This wheel has been invented multiple times already.
From the most basic java.util.Properties to the more advanced frameworks like Spring, which offers advanced features like value injection and type conversion.
Building it yourself is probably the worst approach.
Maybe not a complete answer to your question, but some points to consider:
Storing values as strings (and parsing the strings into other types via your SettingsManager) is the usual approach. If your configuration value is too complex to do this then it's probably not really a configuration value, but part of your implementation.
Consider injecting the individual configuration values required by each class via constructor arguments, rather than just passing in the whole SettingsManager object (see Law of Demeter)
Avoid creating a Singleton SettingsManager if possible, singletons harm testability and damage the design of your application in various ways.
If the number of parameters is big I would split them to several config files. Apache Commons Configuration, as mentioned by #Pino is really a nice library to handle them.
On the Java-side I would probably create one config-class per file and wrap Commons Configuration config to load settings, eg:
class StatisticsConfig {
private Configuration config = ... ;
public double getParameter1() {
return config.getDouble("classA_parameter1");
}
}
This may need quite a lot of boilerplate code if the number of parameters is big but I think it is quite clean solution (and easy to refactor).
From the business prespective, here's the problem
We have a number of shared folders that people use, let's call it //shared/the/drive. However, our server might know this shared drive as some other name, perhaps //ir83pn3br8mwhonamesthesethingsanyway/the/drive since the networking group insists on having incredibly messed up server names. For most of the servers, it works just fine to use the simple name, but on this one, it's just not working right. So the bandaid for our problem is, in our code, to just be like "Oh you're using shared - we'll replace that with stupid name from networking.
Okay - now on to the more technical side of things:
I have a FileItem (Apache commons FileUpload module) object that might have a name //shared/the/drive/stuff/plans.doc. I need to create a FileItem that references //stupidname/the/drive/stuff/plans.doc. What should I do?
Should I edit the request object in the JSP? That sounds like a bad idea.
Should I use reflection to edit the FileItem object? That sounds like an even worse idea.
I'm not a front end guy (note which tags I have votes in... haha), really... more of a server dude... this just got dropped onto my plate. Is it possible to intercept the text box before it gets to the request, moving the change to the client side?
I can't possibly have been the first person to come across this problem. I'm not looking for code necessarily (would I mind? No I wouldn't.) but a general approach of both what will work, and/or how this sort of thing (changing what a user inputs) is handled in a 'best practicey' kind of way is most welcome.
Its not uncommon when dealing with distributed file systems to have a "fake path" which the user sees and deals with and a backend path which represent the actual node that allows you to manipulate the file in context of the request you receive.
Every page you hit on the web is not represented by the physical URL you type into the browser. Files live on CDNs, in CMS systems, are dynamically created out of databases ....whatever.
Theres no need to hack on any objects. You just wrap them with another object that contains their transient properties such as where Im going to access that file THIS time.
In a Java EE environment, we are normally used to storing text in a property/resource file. And that property file is associated with some view HTML markup file. E.g. if your label 'First Name' changes to 'Full Name' on a HTML page, you could use the property to make that update.
firstName=First Name
someOtherData=This is the data to display on screen, from property file
If you are in an environment, where it is difficult to update those property files on a regular basis, what architecture are developers using to change text/label content that would normally reside in a property file? Or let's say you need to change that content before redeploying a property file change. A bad solution is to store that in a database? Are developers using memcache? Is that usually used for caching solutions?
Edit-1 A database is really not designed for this type of task (pulling text to display on the screen), but there are use-cases for a database. I can add a locale column or state field, also add a column filter by group. If I don't use a database or property file, what distributed key/value solution would allow me to add custom filters?
Edit-2 Would you use a solution outside of the java framework? Like a key/value datastore? memcachedb?
I want to assure you that if you need constant changes on localized texts, for example they tend to differ from deployment to deployment, database is the way to go. Well, not just the database, you need to cache your strings somehow. And of course you wouldn't want to totally re-build your resource access layer, I suppose.
For that I can suggest extending ResourceBundle class to automatically load strings from database and store it in WeakHashMap. I choose WeakHashMap because of its features - it removes a key from the map when it is no longer needed reducing memory footprint. Anyway, you need to create an accessor class. Since you mentioned J2EE, which is pretty ancient technology, I will give you Java SE 1.4 compatible example (it could be easily re-worked for newer Java, just put #Override when needed and add some String generalization to Enumeration):
public class WeakResourceBundle extends ResourceBundle {
private Map cache = new WeakHashMap();
protected Locale locale = Locale.US; // default fall-back locale
// required - Base is abstract
// #Override
protected Object handleGetObject(String key) {
if (cache.containsKey(key))
return cache.get(key);
String value = loadFromDatabase(key, locale);
cache.put(key, value);
return value;
}
// required - Base is abstract
// #Override
public Enumeration getKeys() {
return loadKeysFromDatabase();
}
// optional but I believe needed
// #Override
public Locale getLocale() {
return locale;
}
// dummy testing method, you need to provide your own
// should throw MissingResourceException if key does not exist
private String loadFromDatabase(String key, Locale aLocale) {
System.out.println("Loading key: " + key
+ " from database for locale:"
+ aLocale );
return "dummy_" + aLocale.getDisplayLanguage(aLocale);
}
// dummy testing method, you need to provide your own
private Enumeration loadKeysFromDatabase() {
return Collections.enumeration(new ArrayList());
}
}
Because of some strange ResourceBundle's loading rules, you would actually need to extend WeakResourceBundle class to create one class each for supported languages:
// Empty Base class for Invariant Language (usually English-US) resources
// Do not need to modify anything here since I already set fall-back language
package com.example.i18n;
public class MyBundle extends WeakResourceBundle {
}
One supported language each (I know it sucks):
// Example class for Polish ResourceBundles
package com.example.i18n;
import java.util.Locale;
public class MyBundle_pl extends WeakResourceBundle {
public MyBundle_pl() {
super();
locale = new Locale("pl");
}
}
Now, if you need to instantiate your ResourceBundle, you would only call:
// You probably need to get Locale from web browser
Locale polishLocale = new Locale("pl", "PL");
ResourceBundle myBundle = ResourceBundle.getBundle(
"com.example.i18n.MyBundle", polishLocale);
And to access the key:
String someValue = myBundle.getString("some.key");
Possible gotchas:
ResourceBundle requires Fully Qualified Class Name (thus the package name).
If you omit Locale parameter, default (which means Server) Locale would be used. Be sure to always pass Locale while instantiating ResourceBundle.
myBundle.getString() could throw MissingResourceException if you follow my suggestion. You would need to use try-catch block to avoid problems. Instead you may decide on returning some dummy string from database access layer in the event of missing key (like return "!" + key + "!") but either way it should probably be logged as an error.
You should always attempt to create Locale objects passing both language and country code. That is just because, languages like Chinese Simplified (zh_CN) and Chinese Traditional (zh_TW) for example, are totally different languages (at least in terms of writing) and you would need to support two flavors of them. For other countries, ResourceBundle will actually load correct language resource automatically (note that I have created MyBundle_pl.java, not MyBundle_pl_PL.java and it still works. Also, ResourceBundle would automatically fall-back to Enlish-US (MyBundle.java) if there is no resource class for given language (that is why I used such a strange class hierarchy).
EDIT
Some random thoughts about how to make it more awsome.
Static factories (avoid using ResourceBundle directly)
Instead of directly instantiating the bundles with ResourceBundle, you could add static factory method(s):
public static ResourceBundle getInstance(Locale aLocale) {
return ResourceBundle.getBundle("com.example.i18n.MyBundle", aLocale);
}
If you decide to change the name of WeakResourceBundle class to something more appropriate (I decided to use LocalizationProvider), you could now easily instantiate your bundles from consuming code:
ResourceBundle myBundle = LocalizationProvider.getInstance(polishLocale);
Auto-generated resource classes
Localized MyBundle classes could be easily generated via building script. The script could be either configuration file or database driven - it somehow needs to know which Locale are in use within the system. Either way, the classes share very similar code, so generating them automatically really makes sense.
Auto-detecting Locale
Since you are the one that implement the class, you have full control of its behavior. Therefore (knowing your application architecture) you can include Locale detection here and modify getInstance() to actually load appropriate language resources automatically.
Implement additional Localization-related methods
There are common tasks that needs to be done in Localized application - formatting and parsing dates, numbers, currencies, etc. are usual examples. Having end user's Locale in place, you can simply wrap such methods in LocalizationProvider.
Gee, I really love my job :)
You speak about property files, but at execution time, you are likely to have a resource bundle or something that want a list of key/value pairs (maybe even depending of the locale)
You can store data in whatever format and then use it to contruct the right ressource bundle with it. Even if it comes from memory. So database can perfectly do that, because, properties would all be loaded at startup, cached in JVM memory and that's all. (SELECT * FROM LOCALIZATION_DATA)
Don't use distributed cache for that, the data set you have is likely to be small anyway... what ? Maybe a few MB at worst. And access to that data must be instantaneous once loaded because all views will trigger access to it dozen, or even hundred of time per page.
If you want to update the data without restarting the application just add an administration screen somewhere with a "reload localization data", or even a screen that allow to update this type of data (but save to the file/DB/whatever)
From a workflow point of view, it depend of what you are trying to achieve.
The classic property file is the prefered way of doing this. You put it into versionning, together with the source code so you always have the translation up to date with the code. You want to debug V1.3.0.1 ? just get this version, and you'll use the property file that was used at this time. You added new code that require new keys ? Or just changed they key name for whatever reason ? You know that the code and your locatization information are linked into a coherant state. And this is automatic.
If your data is not under version control, you loose automatic versionning and history of your data. When you deploy/redeploy to a new machine, discrepancy can appear and even prevent the application from running propertly (if a new key is required but not added. This is not great, prone to errors and more manual interventions.
If you really need live updates, and really can't release new version for that, what i would do is to have two source for your data. The standard data, under version control, so your sure all is good for a new install from scratch. And the "customised data", in the server that can override standard values. The customized values are not lost when updating from version to version, because this is just the standard values that are updated.
If the change in the server is purely a one shoot customization, then you just go to the right admin webpage, use the customize localization data screen and that's all.
If the change is something that you'll want to keep for any new installation, you add it 2 time. One time in the server, one time in version control.
You could always use JNDI, or even consider a document repository like JCR for this sort of thing.
Not so sure a database couldn't handle this, I think what you are really looking for is a cache that can be invalidated when those properties change. Have you thought about using something like JBoss Infinispan (http://www.jboss.org/infinispan)? It's extremely simple to use, and can be distributed across multiple application servers.
Infinispan, once configured, can be used like a Map; keep in mind you can configure it to be distributed across a cluster!
If you don't mind using a NoSQL solution, I would recommend something like Redis or Memcache. Of course, I would advocate that you keep a local cache (why incur the cost of a network call, especially if these properties are not likely to change often?).
As requested by Berlin Brown, I add another answer, more focussed on it's specific needs :
From the amount of data you need (like a thousand of entries), you just need to load your property file at startup by specifying a remote URL.
Data is cached in JVM memory for maximum performance.
Depending on your workflow you then have a background process that check for update on a regalar basis (let say each minute, hour, whatever is enough for you) or you can have a "button" in administration "refresh localization data" developper use when an update is needed.
No need for database. No need for memcached, no need for NoSQL. a simple URL accessible from production server. In term of security and dev it is easier, faster and more flexible.
Implementation details: if you use the standard format, you'll have a file per language/contry. Don't forget to update for all languages or bundle them together (using a zip for exemple).