I am developing a framework that needs a lot of stuff to get working. I have several folders inside of my Eclipse project that are needed
[root]
- config
- src
- lib
- serialized
Also there are important files like the log4j.properties and the META-INF dir inside the src directory.
I wonder if there is a way to distribute one JAR containing all essential files so my gui will just have to import one jar. I guess that I have to exclude the config folder in order to make the framework configurable.
I also wonder, if there is a way to move for example the log4j.properties to the config dir so that I have one config folder containg all needed configurations?
Thanks for help and advise on this matter!
Marco
Yes, but not really. You can take all your dependencies, unpack them and simply merge them into a bigger jar. This is what the maven jar plugin does if you make a jar with dependencies. The only problem is that this might result in conflicting files (suppose two of your dependencies contain a log4j.properties). This is one of the problems when doing the above with some of the spring libraries for instance.
I think someone actually wrote a classloader that allows you to bundle the whole jar inside of your jar and use it as is. I'm not sure how mature that is though and can't at the moment recall the name.
I think you're better off distributing all your dependencies separately. Setting up the classpath is a bit of a pain but surely java programmers are used to it by now. You can add dependencies to the Class-Path header in your manifest file, in simple cases. Bigger libraries have to rely on the classpath being set up for them though.
As to the second part of your question, probably dropping the conf/ directory under META-INF is enough for its contents to be picked up. I'm not sure about this. I'm fairly sure it will always be picked up if you put its contents at the top level of the jar. In any case, this is a distribution problem. You can easily have a conf/ directory inside your source tree and have your build scripts (whatever you might be using) copy the files in it to wherever is most convenient.
As to your users configuring. Try to establish some conventions so they have to configure as little as possible. For things that must be configured, it's best to have a basic default configuration and then allow the user to override and add options through his/her own configuration file.
In terms of the resources, it is possible except that if you do that you are not going to be able to load resources (non class files) from the filesystem (via a file path).
It's likely that you're currently loading these resources from the file system. Once in the jar you need to load them as class path resources via the class.getResourceAsStream or similar.
As for the dependent jars you may have, it's common practice for these to be placed as extra jars on the classpath. I know it's complicates things but developers are used to doing this. The nature of the java landscape is that this is inevitable. What the spring framework for example does is supply a bundled zip file with the core jar and the jar dependencies included.
Is your library going to be used in an EE context or an SE context? If it is an EE context then you really don't have to worry about configuration and class path issues as the container takes care of that. In an SE context it is a lot more tricky as that work has to be done manually.
Related
We all know "inputable" resources are by convention in src/main/resources and src/test/resources, but what about the runtime outputted ones? Is it better to use target/ or target/{classes,test-classes}or simply give up and try to use external path even if it complicates things for security reasons? I've been brainstorming a bit regarding that decission as shown following, but need the help of more experienced users that can shed more light.
PROS of target/{classes, test-classes}
If maven engineers architected the convention by moving resources to target/classes instead of target/resources I assume they had a good reason in mind for preferring it
It organizes input and output resources on the same base folder
It makes test and main outputs independent, so no conflict can appear if names are equal
IT makes much conventional and secure to define the relative route of the resource ( by ClassLoadeR().getResource() or etProtectionDomain().getCodeSource().getLocation() [no file globs]
It makes much easy to centralize output behavior, in case our idea is having a function for UPSERTing resources, we need to use resolution for sufolder as they are not in the same path (so prepend /classes/ to the relative route but / for an original resource)
I think due to the previous ones, there can be a bit of confusion when using parent poms, because there is a target for both parent and module, but only one classloader URL -> /target/classes
It works flawlessly if using the classes directory as the base classpath of the app, when executing directly from console.
PROS of target/
When packaging the app as a library, you don't need to deal with an output folder inside the jar.
I have been told that maven doesn't like too much anything strictly outside of target/, but have no further info
A central folder can be used when executing tests in case production code generates files and tests works with them also
Perhaps a single target at parnet pom's level makes files easier to share among different modules
So how do you usually handle these sort if things?
I am sure some of you have a more authoritative question
So typically if you are allowing resources to be pulled in from outside of your Jar you need to either hard-code that relative path or read it from fixed config file. You would then document this when you release your application.
For example, Eclipse has a folder specially for plugins. You can drop new plugins here and know that Eclipse will pick them up and know what to do with them.
In my applications, I usually define a conf directory that sits at the same level as the executable Jar. I'll put any log4j and other such post-compile config files there.
You mention security issues, which is a good thing to think about. When you are pulling in data, always try to do some sort of sanity checking (make sure a directory exists, a zip file isn't corrupt, etc). Since you can't control what comes in, make sure you do as many checks as you can on the program side.
I have a maven project with several dependencies and use log4j.properties to control output. In some cases the same class may be referenced in different property files with different parameters. Is there a defined protocol for "overriding" properties or does it depend on the order in which packages are loaded?
(I am locating all log4j.properties directly under src/main/resources - is this the correct place?)
UPDATE:
I have accepted #Assen's answer as it makes sense though it doesn't make the solution easy. Essentially he recommends excluding log4j.properties from the jar. In principle I agree, but it puts the burden on the user to control the output and most of my users don't know what Java is, let alone properties files.
Maybe there is a way of renaming the properties files in each jar and using a switch (maybe with -D) to activates the properties.
I often have similar discussions on projects. I thing log4j.properties is typically something you want to keep out of the application, and not pack it in a war and deliver it together with the code. Logging configuration:
is environment specific. When you write the application, you simply can't define the appenders that will be desired, file locations etc.
its lifecycle is totally different than the application's. After an application is deployed, logging properties can be changed several times a day. Redeploying the application shouldn't override your last logging settings.
Why package logging configuration together with your code then? I usually keep somewhere a configuration folder, with soubfolders like 'dev', 'test-server-01', 'macbook-john' etc. Each subfolder contains list own copy of log4j.properties. None of them is included in the build artifact - jar or war.
When deploying, one of thuse subfolders is delivered separately. For the test server 1, this would be the content of test-server-01 subfolder. Dependng on the application server used, thers is a different trick tu put some files on the classpath.
When developing, I take care to set one of those subfolders on the path. When John develops on his macbook, he might want to put 'macbook-jihn' on the classpath, or create a new one. He can change logging settings and commit without conflicts.
I've developed a utility library that will be used in many of our enterprise Java applications. This library has numerous additional dependencies that also need to be on the classpath. I'd like to avoid forcing our developers to add a zillion entries to their MANIFEST.MF files, and let them instead just include my library. Is there any way that my library's MANIFEST.MF file can reference its dependencies and have them picked up by the enterprise applications that will be using my library?
I've tried referencing them in my library's MANIFEST.MF file using the full path to the dependencies on the filesystem. That didn't work. I end up with ClassNotFoundException errors for all of my dependencies. Is there something else I should be trying?
When you create a web application, you'd normally put it in a WAR file. The idea is that you bundle the required dependencies in that WAR file, by adding the jars to the /WEB-INF/lib folder inside the WAR. Web containers (like in a Java EE application server) know of this structure and will include those jars on the classpath.
If your library has additional dependencies, just tell the users about it and either redistribute them with your library if the license allows it, or tell them where to obtain them. When using a decent tool for creating a web app like an IDE, Ant with Ivy, or Maven (or a combination of these), then handling and bundling dependencies should be no problem.
Alternatively, this works so long as you stick to the format very carefully, i.e. stick to exactly two spaces before each "file:" etc:
Manifest-Version: 1.0
Main-Class: package.TestClass
Class-Path: file:/D:/WebServer/Tomcat/shared/lib/BlueCove.jar
file:/D:/WebServer/Tomcat/shared/lib/classes12.jar
file:/D:/WebServer/Tomcat/shared/lib/comm.jar
file:/D:/WebServer/Tomcat/shared/lib/FTP.jar
file:/D:/WebServer/Tomcat/shared/lib/FTP2.jar
file:/D:/WebServer/Tomcat/shared/lib/iText.jar
file:/D:/WebServer/Tomcat/shared/lib/j2ee.jar
file:/D:/WebServer/Tomcat/shared/lib/jmxremote.jar
file:/D:/WebServer/Tomcat/shared/lib/jmxri.jar
file:/D:/WebServer/Tomcat/shared/lib/jmxtools.jar
file:/D:/WebServer/Tomcat/shared/lib/jpos15.jar
file:/D:/WebServer/Tomcat/shared/lib/js.jar
file:/D:/WebServer/Tomcat/shared/lib/mail.jar
...
file:/C:/WebServer/Tomcat/shared/lib/soap.jar
file:/C:/WebServer/Tomcat/shared/lib/sqljdbc.jar
file:/C:/WebServer/Tomcat/shared/lib/tools.jar
I've done this with a number of tools. It is a truly horrible hack but seems to work reliably.
Give them a special manifest to use. Something like:
Manifest-Version: 1.0
Main-Class: com.xxx.yyy.zzz.YourSpecialClassThatHacksTheClassPath
Real-Main-Class: com.ppp.qqq.TheirMainClass
In your special class, screw around with the classpath (not easy), read the manifest "Real-Main-Class" entry (a bit easier) and launch their main from that (not really difficult at all).
Obviously this will not work with a .war file.
Even I had the same problem. As mentioned above, the solution was to have exact two space after file:/ and one space after .jar file and at the end, press enter key.
I know this is not a neat solution, but it works. enjoy.
Some projects, as part of their build process, clean up the classes folder in WEB-INF, while others keep some of their config files in the classes folder. The latter group might use a temporary folder somewhere else to build their project and then copy the generated class files from this temporary folder to the WEB-INF/classes, thus keeping their config files safe.
Is there any best practice regarding web application builds about this? I use ant.
User-accessible config files are better stored outside of /classes, e.g. in /WEB-INF/config. I differentiate config files that site admins can touch (they are placed under WEB-INF) and those, kind of 'static' ones, that are meant for developers/deployment configurators only (stuff like sql scripts, XML/XSLT templates, i18n etc).
It is preferred practice to clean up WEB-INF/classes during builds - some classes get deleted/renamed, so are resource files.
Other config files, under WEB-INF, but not in /classes or /lib, have to be treated as upgradable resources: either replace old ones only when there is a new one, or use specifically designed upgrade classes to add missing config tags or lines.
There are tricky situations, like log4j.properties is sometimes stored into /classes root. It is a bit of a different story how to properly handle it, but in most instances it falls under "delete all classes and copy everything anew".
i have a few batch java command-line applications which are planned to be deployed as:
batch_apps
app_1
batch1.jar
run_batch1.sh
app_2
batch2.jar
run_batch3.sh
{...etc...}
what would be the best practice on organizing a shared library pool - for example log4j:
batch_apps
app_1
batch1.jar
run_batch1.sh
app_2
batch2.jar
run_batch3.sh
libs
log4j.jar
ojdbc.jar
?
and include individual log4j.xml's in each app's own jar file?
i understand i would need to add 'libs' to the classpath either in manifests or in run_batchX.sh
(which way is preferable?)
I am mostly wondering what would be the most efficient setup performance-wise.
thanks
Having a shared libs directory at the root of your install dir is definitely the way to go. Since libs will be loaded in memory once, when the JVM launches, there is no impact on performance whatever solution you choose.
I would not put the classpath in the jar files, as this would force you to change your jars if you need to relocate your lib dir. Editing a script is much simpler.
I would not include the log4j conf file in your jar files either, for the same reason.
It appears your applications don't share a single JVM instance. (i.e. They are individually started via 'java -jar batch1.jar' or some such.) Therefore, sharing library .jar files only saves you DISK space not RAM.
If the apps are not sharing a single JVM then ease-of-deployment should take precedence over disk space. (Your time is worth more than a few "wasted" MB.)
In this instance I would recommend making each application self contained, in either a single .jar file including all its libraries, or a single folder of .jar files. (i.e. Put the libs for each app in a folder for that app.)
If the apps were sharing a single JVM then I would recommend the shared library folder option.
You can use the java extension mechanism. Place them in JAVA_HOME/lib/ext and they will be accessible by all apps. Of course, this may not be the best for all deployments, but its certainly easier.
This doesn't directly answer your question, but I have already tried the approach that you propose but would now create a single jar per application (see how to do it with Ant). That way, no need to include anything in the classpath:
java -jar myApp.jar
is all you need. I find it cleaner but everything is debatable.
It doesn't make any difference from a performance point-of-view since each application is run inside its own JVM.
The only downside is that some libraries will be present in each jar file. It only costs more to store on the HD, but these days, MB are pretty cheap :-) I trade simplicity (no external lib folder) and no jar hell (not placing your jars inside the Java ext folder) over storage price any time. If your application doesn't include terrabyte of libraries, I think it's fine.
For the Log4j configuration file, I would place one default file inside the jar but provide a sample config file (log4j-custom.xml.sample) that someone can modify and specify in the command line:
java -Dlog4j.configuration=log4j-custom.xml -jar myApp.jar