This is probably a basic question that has some sort of solution that I am not aware of, but basically I have a apache-tomcat web application that hosts a lot of different sites and each visitor needs access to the contents of an xml file. There are about 6 different xml files that this could be. If I allow the file to be accessed each time (the file is used in lots of included pages and assets) I get too many files open, if I store it in the sessions, I get too much memory usage.
What I would like is when I compile the classes to have one class read each of the files into memory and then to access that data like a constant. Is there an easy way of doing this?
This is the classic case where a singleton would be useful. A singleton is often used to load content only once.
A modified example from the wikipedia page on Singletons (http://en.wikipedia.org/wiki/Singleton_pattern):
public class Singleton {
private static final Singleton INSTANCE = new Singleton();
private String xmlFileContents;
private Singleton() {
// Call method to populate xmlFileContents field from XML file
}
public static Singleton getInstance() {
return INSTANCE;
}
public String getXMLFileContents() {
return xmlFileContents;
}
}
Related
I am starting with Java/Tomcat, and I am struggling with a problem that was very easy to solve with C++.
My webservice (sigle webapp) works by using the input values to lookup the numeric answer in a large, pre-calculated table. I am struggling with the initialization of this table.
Problem details:
The data table is huge (3000x3000);
The data is pre-computed and this computation is very costly (it
takes hours);
The data is static, it will never change after it is calculated for a
given instance;
In C++, I would just define a a static const array and initialize it inline. I was not able to do this in Java, apparently there's no concept of static data initialization in Java, it needs to generate initialization code and this code cannot be larger than 64k. In fact, I couldn't even load the file with the static initialization in Eclipse, it would hang-up.
So I need to initialize the table from a static file on disk. I tried to place a .csv file on WEB-INF/static, but found no way to open it reliably from inside my Java code (the absolute path will be in different places on my development and production environments, for example).
This is my current class definition (with mocked-up data for the initialization):
package com.hmt.restjersey;
public final class G {
static public final float[][] data = new float[3000][3000];
//TODO: actual initialization from file
static {
Logger.writeEventLog("Initializing G table...");
for (int alpha = 0; alpha < 3000; alpha++) {
for (int beta = 0; beta < 3000; beta++) {
data[alpha][beta] = 1.0f / (1 + alpha + beta);
}
}
Logger.writeEventLog("G table initialized.");
}
}
So, my questions:
How to reliably access the data file (WEB-INF/static/data.csv) to initialize the table?
Is a .csv file the best way to load numeric data efficiently?
Also, since the table is huge I would like to have a single instance of it in the server to save memory and speed up initialization. How do I assure that there will be a single instance shared by all servlet processes?
That's my two cents:
Regarding memory sharing, if all your servlets are in the same WAR (webapp) then they share static vars (because it's the same classloader), but it's even nicer to use ServletContext which is meant just for this, see ServletContext
As the ServletContext example (link above) shows, you don't necessarily need a static initializer - you can use ServletContextListener to init on application startup (btw you could also do initialization on-demand, in the 'getter' of your huge data).
If you'd like to share memory between 2 different WARs, I don't know a straightforward solution. Theoretically it can be shared if the class with the static var is in TOMCAT_HOME/lib, but iMHO it's confusing and weird
Putting the calculation in file/storage is a great idea, because you might find yourself restarting Tomcat!
As to how to locate the file, I agree with dmitrievanthony's comment regarding getResourceAsStream . Basically it allows you to take the file from your classpath (the same one used for locating code), one simple example would be putting it in /WEB-INF/classes/data.csv , see example code Here (I personally like when this approach is wrapped in "Resource" from Spring framework, but it could be an overkill).
Please note: As mentioned in my comment above, I tried to offer answers to your direct questions for the design you chose, but if I were in your shoes I'd stop to consider this design (e.g. is easy to distribute between servers? is it modular and unit-testable? Could "data.csv" be replaced with a database, or MongoDB, or even a separate "dataService" WAR?). But please ignore this remark if you're already considered it...
Edited: ServletContext example, without static fields:
// Class to encapsulate date:
public class G{
private double[][] data;
public static G loadData(){
data=...// complex loading
}
}
// Usage in ServletContextListener:
public class MyListener implements ServletContextListener{
public void contextInitialized(ServletContext ctx) {
G g= G.loadData();
ctx.put("myData", g);
}
// Usage is Servlet:
doGet(...){
G g=(G) getServletContext().getAttribute("myData");
}
Singleton pattern alternative (but I suggest care in terms of testability and modularity, you may also want to have a look at frameworks such as SpringMVC, but let's start simple):
// Singleton:
public class G{
private volatile double[][] data;
private G instance;
public static G getInstance(){
// I don't synchronize because I rely on ServletContextListener to initialize once
if(data==null)
data=... // complex loading
return data;
}
}
// ServletContextListener:
public void contextInitialized(ServletContext ctx) {
G.getInstance();
}
// Usage in servlet:
doGet(){
G g=G.getInstance(); // I don't like it in terms of OOD, but it works
}
Let us assume up to runtime we do not know what are the details of configuration(may user need to configure these parameters in config file before running the application.
I want to read those configuration details and need to reuse them wherever I need them in my application. For that I want to make them as global constants(public static final).
So, My doubt is, is there any performance implications if I read from config file directly from the required class? since,runtime values I can not directly put in separate Interface.
I am thinking it will impact performance.Please suggest me any better way to do this.
UPDATE: Can I use separate final class for configuration details?
putting all configuration details as constants in a separate public final class
(To read all configuration details at once from the configuration file and storing them as global constants for later use in application)
I am thinking it will impact performance.
I doubt that this will be true.
Assuming that the application reads the configuration file just once at startup, the time taken to read the file is probably irrelevant to your application's overall performance. Indeed, the longer the application runs, the less important startup time will be.
Standard advice is to only optimize for application performance when you have concrete evidence (i.e. measurements) to say that performance is a significant issue. Then, only optimize those parts of your code that profiling tells you are really a performance bottleneck.
Can I use separate final class for configuration details
Yes it is possible to do that. Nobody is going to stop you1.
However, it is a bad idea. Anything that means that you need to recompile your code to change configuration parameters is a bad idea. IMO.
To read all configuration details at once from the configuration file and storing them as global constants for later use in application.
Ah ... so you actually want to read the values of the "constants" instead of hard-wiring them.
Yes, that is possible. And it makes more sense than hard-wiring configuration parameters into the code. But it is still not a good idea (IMO).
Why? Well lets look at what the code has to look like:
public final class Config {
public static final int CONST_1;
public static final String CONST_2;
static {
int c1;
String c2;
try (Scanner s = new Scanner(new File("config.txt"))) {
c1 = s.nextInt();
c2 = s.next();
} catch (IOException ex) {
throw RuntimeException("Cannot load config properties", ex);
}
CONST_1 = c1;
CONST_2 = c2;
}
}
First observation is that makes no difference that the class is final. It is declaring the fields as final that makes them constant. (Declaring the class as final prevents subclassing, but that has no impact on the static fields. Static fields are not affected by inheritance.)
Next observation is that this code is fragile in a number of respects:
If something goes wrong in the static initializer block. the unchecked exception that is thrown by the block will get wrapped as an ExceptionInInitializerError (yes ... it is an Error!!), and the Config class will be marked as erroneous.
If that happens, there is no realistic hope of recovering, and it possibly even a bad idea to try and diagnose the Error.
The code above gets executed when the Config class is initialized, but determining when that happens can be tricky.
If the configuration filename is a parameter, then you have the problem of getting hold of the parameter value ... before the static initialization is triggered.
Next, the code is pretty messy compared with loading the state into a instance variables. And that messiness is largely a result of having to work within the constraints of static initializers. Here's what the code looks like if you use final instance variables instead.
public final class Config {
public final int CONST_1;
public final String CONST_2;
public Config(File file) throws IOException {
try (Scanner s = new Scanner(file)) {
CONST_1 = s.nextInt();
CONST_2 = s.next();
}
}
}
Finally, the performance benefits of static final fields over final fields are tiny:
probably one or two machine instructions each time you access one of the constants,
possibly nothing at all if the JIT compiler is smart, and you handle the singleton Config reference appropriately.
In either case, in the vast majority of cases the benefits will be insignificant.
1 - OK ... if your code is code-reviewed, then someone will probably stop you.
Have you ever heard of apache commons configuration http://commons.apache.org/proper/commons-configuration/ ?
It is the best configuration reader I have ever found and even am using it in my application which is running in production since 1 year. Never found any issues, very easy to understand and use, great performance. I know its a bit of dependency to your application but trust me you will like it.
All you need to do is
Configuration config = new ConfigSelector().getPropertiesConfiguration(configFilePath);
String value = config.getString("key");
int value1 = config.getInt("key1");
String[] value2 = config.getStringArray("key2");
List<Object> value3 = config.getList("key3");
And thats it. Your config object will hold all the config values and you can just pass that object to as many classes as you want. With so many available helpful methods you can extract whichever type of key you want.
It will be only one time cost if you are putting them in a property file and reading the file at the start of your application and initialize all the parameters as system parameters(System.setProperty) and then define constants in your code like
public static final String MY_CONST = System.getProperty("my.const");
But ensure the initialization at start of your application before any other class is loaded.
There are different types of configuration.
Usually some sort of bootstrapping configuration, for example to connect to a database or service, is needed to be able to start the application. The J2EE way to specify database connection parameters is via a 'datasource' specified in your container's JNDI registry (Glassfish, JBoss, Websphere, ...). This datasource is then looked up by name in your persistence.xml. In non-J2EE applications it is more common to specify these in a Spring context or even a .properties file. In any case, you usually need something to connect your application to some sort of data store.
After bootstrapping to a data store an option is to manage config values inside this datastore. For example if you have a database you can use a separate table (represented by e.g. a JPA Entity in your application) for configuration values. If you don't want/need this flexibility you can use simple .properties file for this instead. There is good support for .properties files in Java (ResourceBundle) and in frameworks like Spring. The vanilla ResourceBundle just loads the properties once, the Spring helper offers configurable caching and reloading (this helps with the performance aspect which you mentioned). Note: you can also use Properties backed by a data store instead of a file.
Often both approaches coexist in an application. Values that never change within a deployed application (like the application name) can be read from a properties file. Values that might need to be changed by an application maintainer at runtime without redeployment (e.g. the session timeout interval) might better be kept in a reloadable .properties file or in a database. Values that can be changed by users of the application should be kept in the application's data store and usually have an in-application screen to edit them.
So my advise is to separate your configuration settings into categories (e.g. bootstrap, deployment, runtime and application) and select an appropriate mechanism to manage them. This also depends on the scope of your application, i.e. is it a J2EE web app, a desktop app, command-line utility, a batch process?
What kind of configuration file do you have in mind? If it is a properties file, this might suit you:
public class Configuration {
// the configuration file is stored in the root of the class path as a .properties file
private static final String CONFIGURATION_FILE = "/configuration.properties";
private static final Properties properties;
// use static initializer to read the configuration file when the class is loaded
static {
properties = new Properties();
try (InputStream inputStream = Configuration.class.getResourceAsStream(CONFIGURATION_FILE)) {
properties.load(inputStream);
} catch (IOException e) {
throw new RuntimeException("Failed to read file " + CONFIGURATION_FILE, e);
}
}
public static Map<String, String> getConfiguration() {
// ugly workaround to get String as generics
Map temp = properties;
Map<String, String> map = new HashMap<String, String>(temp);
// prevent the returned configuration from being modified
return Collections.unmodifiableMap(map);
}
public static String getConfigurationValue(String key) {
return properties.getProperty(key);
}
// private constructor to prevent initialization
private Configuration() {
}
}
You could also return the Properties object immediately from the getConfiguration() method, but then it could potentially be modified by the code that access it. The Collections.unmodifiableMap() does not make the configuration constant (since the Properties instance gets its values by the load() method after it was created), however since it is wrapped in an unmodifiable map, the configuration cannot be changed by other classes.
Well this is a great problem which is faced in every one's life once in a will. Now coming to the problem, this can be solved by creating a singleton class which has instance variables same as in configuration file with default values. Secondly this class should have a method like getInstance() which reads the properties once and every times returns the same object if it exists. For reading file we can use Environmental variable to get path or something like System.getenv("Config_path");. Reading the properties (readProperties() method) should read each item from config file and set the value to the instance variables of singleton object. So now a single object contains all the configuration parameter's value and also if the parameter is empty than default value is considered.
One more way is to define a class and read the properties file in that class.
This class needs to be at the Application level and can be marked as Singleton.
Marking the class as Singleton will avoid multiple instances to be created.
Putting configuration keys directly to classes is bad: configuration keys will be scattered over the code. Best practice is separation of application code and configuration code. Usually dependency injection framework like spring is used. It loads a configuration file and constructs the objects using configuration values. If you need some configuration value in your class you should create a setter for this value. Spring will set this value during context initialization.
I recommend using JAXB or a similar binding framework that works with text based files. Since a JAXB implementation is part of the JRE, it's pretty easy to use. As Denis I advise against configuration keys.
Here is a simple example for an easy to use and still pretty mighty way to configure you application with XML and JAXB. When you use a DI framework you can just add a similar config object to the DI context.
#XmlRootElement
#XmlAccessorType(XmlAccessType.FIELD)
public class ApplicationConfig {
private static final JAXBContext CONTEXT;
public static final ApplicationConfig INSTANCE;
// configuration properties with defaults
private int number = 0;
private String text = "default";
#XmlElementWrapper
#XmlElement(name = "text")
private List<String> texts = new ArrayList<>(Arrays.asList("default1", "default2"));
ApplicationConfig() {
}
static {
try {
CONTEXT = JAXBContext.newInstance(ApplicationConfig.class);
} catch (JAXBException ex) {
throw new IllegalStateException("JAXB context for " + ApplicationConfig.class + " unavailable.", ex);
}
File applicationConfigFile = new File(System.getProperty("config", new File(System.getProperty("user.dir"), "config.xml").toString()));
if (applicationConfigFile.exists()) {
INSTANCE = loadConfig(applicationConfigFile);
} else {
INSTANCE = new ApplicationConfig();
}
}
public int getNumber() {
return number;
}
public String getText() {
return text;
}
public List<String> getTexts() {
return Collections.unmodifiableList(texts);
}
public static ApplicationConfig loadConfig(File file) {
try {
return (ApplicationConfig) CONTEXT.createUnmarshaller().unmarshal(file);
} catch (JAXBException ex) {
throw new IllegalArgumentException("Could not load configuration from " + file + ".", ex);
}
}
// usage
public static void main(String[] args) {
System.out.println(ApplicationConfig.INSTANCE.getNumber());
System.out.println(ApplicationConfig.INSTANCE.getText());
System.out.println(ApplicationConfig.INSTANCE.getTexts());
}
}
The configuration file looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<applicationConfig>
<number>12</number>
<text>Test</text>
<texts>
<text>Test 1</text>
<text>Test 2</text>
</texts>
</applicationConfig>
protected java.util.Properties loadParams() throws IOException {
// Loads a ResourceBundle and creates Properties from it
Properties prop = new Properties();
URL propertiesFileURL = this.getClass().getResource("/conf/config.properties");
prop.load(new FileInputStream(new File(propertiesFileURL.getPath())));
return prop;
}
Properties prop = loadParams();
String prop1=(String) prop.get("x.y.z");
Given the prevalence of YML to express configuration, I'd recommend creating a YML file with the configuration inside it and then loading that once, at startup, into a POJO, then accessing the fields of that POJO to get the configuration:
user: someuser
password: somepassword
url: jdbc://mysql:3306/MyDatabase
With Java Class
public class Config {
private String user;
private String password;
private String url;
// getters/setters
Jackson can be used to load YML as can SnakeYml directly.
On top of this, you could use the OS project I've been working on - https://github.com/webcompere/lightweight-config - which allows you to wrap this up, and even express placeholders in your file to interpolate environment variables:
user: ${USER}
password: ${PASSWORD}
url: jdbc://${DB_HOST}:3306/MyDatabase
then
Config config = ConfigLoader.loadYmlConfigFromResource("config.yml", Config.class);
So I have a class like so:
public class HBaseUtil {
private final String fileName = "hbase.properties";
private Configuration config;
private HBaseUtil() {
try {
config = new PropertiesConfiguration(fileName);
} catch (ConfigurationException e) {
// some exception handling logging
}
}
// now some getters pulling data out of the config object
public static String getProperty(String fieldKeyName) {...}
public static String getColumnFamily(String fieldName) {...}
// ... some more getters
// NO setters (thus making this a read-only class)
}
Thus, basically I have for myself a Singleton class, that the very first time that it is put to use, sets up a configuration object, and then simply keeps listening for get calls. There are a number of problems with this class:
Unit testing the static methods within class HBaseUtil becomes difficult because of a tight-knit coupling between the Singleton and the configurations file.
What I really want is me being able to supply the filename/filename+path to the class so that it can go in there, read the configuration properties from that file and offer them to incoming read requests. One important note here though: I need this flexibility in specifying the properties file ONLY ONCE per JVM launch. So I certainly don't need to maintain state.
Here is what I was able to come up with:
Instead of a Singleton, I have a normal class with all static methods and no explicit constructor defined.
public class HBaseUtil {
// directly start with getters
public static String getProperty(Configuration config, String fieldKeyName) {...}
public static String getColumnFamily(Configuration config, String fieldKeyName) {...}
// ...and so on
}
And then, instead of using the class in my other code like such:
HBaseUtil.getProperty(String fieldKeyName)
I'd use it like so:
Configuration externalConfig = new PropertiesConfiguration("my-custom-hbase.properties");
HbaseUtil.getProperty(externalConfig, fieldKeyName)
My questions:
Am I even thinking in the right direction? My requirement is to have the flexibility in the class only ONCE per JVM. All that needs to be configurable in my project for this, is the location/contents of the HBase .properties file. I was thinking having a Singleton is overkill for this requirement.
What other better approaches are there for my requirement (stated in above point)?
Thanks!
Note: I've read this StackOverflow discussion, but now it's gotten me even more confused.
You should avoid all static methods and instead design a class which does not mandate its lifecycle: it can be a typical immutable POJO with a public constructor.
Then, when you need it as a singleton, use it as a singleton. For testing, use it in some other way.
Usually, dependency injection is the preferred avenue to solve these problems: instead of hard-coding a pulling mechanism for your configuration object, you have the object delivered to any class which needs it. Then you can decide late what bean you will deliver.
Since you are probably not using Spring (otherwise dependency injection would be your default), consider using Guice, which is a very lightweight and non-intrusive approach to dependency injection.
The situation before:
A Resource exists. Class Root is created, and during it's creation it creates Child classes (many implementations of one base). Those can, but don't have to, posses children. Basically, this is a typical tree structure, that builds itself.
One of the Child classes "NeedyChild" requires access to a Resource that was opened during creation of Root. There was only one Resource possible, so I made it a singleton and accesed it from NeedyChild by Resource.getInstance();
The situation now:
I have to build many trees. Those are independent of each other and take quite some time, so I made it concurrent. Unfortunatelly, each tree has it's own Resource.
Question: how to access the Resource specific to the current runnable now?
My solutions:
Make Child base class require Resource parameter in it's constructor. Pros: it's quick. It's fairly efficient (passing a reference is no big deal), it's simple. Cons: it requires me to change constructor calls and signatures throughout the application, possibly >100 places.
Reflection to make base constructor find it's caller, save reference, then just give Resource to Root, and NeedyChild will go up "creators" hierarchy to get Root and then Resource. But I won't do that, it's evil.
Didn't research it, but maybe I can access the parent Runnable the code is currently executing inside? Cast it back to my RunnableClass which could hold reference to Resource? The problems are: I don't know if it's possible, it seems a bit "wrong" too...
(not really working) Make the tree traversable both ways, give Resource to Root and from NeedyChild getParent() long enough until you're at Root. It would be great, but the NeedyChild needs Resource during it's creation, and only after it is created, it is appended to the parent. I thought of delaying filling Resource-dependent fields, but it gets ugly very quick.
TBH I probably will have to modify the base constructor and either pass "this" (so I can traverse back to the Root), or pass Resource. The runnable-specific context seems also quite decent, but I'm already doing the parallel procesing to make things faster, I don't really want to negate it by some very slow reflection lookups...
So, any ideas?
Addendum 1: Sample Code
abstract class BaseElement{
List<BaseElement> children = new ArrayList<>();
public void addChild(BaseElement child){
children.add(child);
}
}
class Loader{
private PackageElement resultPackage;
private Resource resource;
public Loader(Resource resource){
this.resource = resource;
}
public callMeFromOutside(SourceTree source){
resultPackage = new Package(source.pack());
}
}
class PackageElement extends BaseElement {
public PackageElement(PackageTree pack){
this.addChild(new Element(pack.element()));
}
}
class Element extends BaseElement{
public Element(ElementTree element){
this.addChild(new NeedyChild(element.needyChild()));
}
}
class NeedyChild extends BaseElement{
public NeedyChild(NeedyTree needy){
this.setSomethingImportant(resource.loadSomethingBasedOn(needy));
}
}
Since there is a one-to-one correspondence between a tree and thread, modifying your singleton pattern to be thread-local should do the trick. For concreteness, here is a code sample:
public class Resource
{
private static final ThreadLocal<Resource> threadLocal = new ThreadLocal<Resource>();
public static Resource getInstance()
{
return threadLocal.get();
}
public static void setInstance(final Resource instance)
{
threadLocal.set(instance);
}
...
}
Just make sure that you call Resource.setInstance on the appropriate thread before instantiating NeedyChildobjects.
This might be an age old problem and I am sure everyone has their own ways.
Suppose I have some properties defined such as
secret.user.id=user
secret.password=password
website.url=http://stackoverflow.com
Suppose I have 100 different classes and places where I need to use these properties.
Which one is good
(1) I create a Util class that will load all properties and serve them using a key constant
Such as :
Util is a singleton that loads all properties and keeps up on getInstance() call.
Util myUtil = Util.getInstance();
String user = myUtil.getConfigByKey(Constants.SECRET_USER_ID);
String password = myUtil.getConfigByKey(Constants.SECRET_PASSWORD);
..
//getConfigByKey() - inturns invokes properties.get(..)
doSomething(user, password)
So wherever I need these properties, I can do steps above.
(2) I create a meaningful Class to represent these properties; say,
ApplicationConfig and provide getters to get specific properties.
So above code may look like:
ApplicationConfig config = ApplicationConfig.getInstance();
doSomething(config.getSecretUserId(), config.getPassword());
//ApplicationConfig would have instance variables that are initialized during
// getInstance() after loading from properties file.
Note: The properties file as such will have only minor changes in the future.
My personal choice is (2) - let me hear some comments?
Do it the most straightforward way (a class with static values):
package com.domain.packagename
public class Properties {
private static String hostName;
public static getHostName() { return hostName; }
private static int port;
public static int getPort() { return port; }
public static void load() {
//do IO stuff, probably
hostName = ??;
port = ??;
//etc
}
}
I find the first approach to be more verbose than necessary. (Especially if the properties are not expected to change very much.) Also, by using the second approach you can handle casting/type issues when the properties are loaded instead of when they are used.
Your option (2) to keep application specific getters sounds better and clean.
public static final keys from an interface had been a bad design in Java for ages.
I guess my first question is why you want to create an instance of something you're saying is a singleton (you mentioned using code like Util.getInstance()). A singleton only has 1 instance so you shouldn't try to instantiate multiple copies in your code.
If the data is static (like this appears to be) I'd create a singleton and retrieve the values from it.
I don't think there is any significant advantage of one method over the other and I don't think the solution (1) is more secure, just because it provides a property key instead of a java getter for getting passwords.
If I had to chose one though I would take option (2).