In one project (http://jodd.org) we have some classses generated by python scripts. Those are some big classes that has utility methods with similar behavior for all primitives and Object; we just wanted to simplify life and avoid potential errors made by repeating similar code.
Also note that this scripts are invoked manually, so we do NOT need any automation process etc.
We are mavenizing the project, and these scripts fall in the src/main/resources folder. Therefore, they become part of distributed jar with other resources, which is what we want to avoid.
What would be the standard maven location for these files; so they not become part of the distro jar? Is it src/main/scripts or src/main/python, or we should simply put them outside, in e.g. etc folder?
I think a good place to put python scripts is in src/main/python as you described. Doing so, they won't be copied to the final jar. I use similar approach with some native codes written in C in my projects.
Related
After developing a java app in Eclipse, I would like to deploy it by packaging it into a runnable .jar with only the minimum necessary by the main method and its dependencies.
I have several packages in my workspace I work with too, but do not need to be in the resulting .jar file.
When I try to export, it clearly says that the required libs will be there, but also the other independent packages are inside too. (With the Export option happen exactly the same)
I choose to export only the Main class of the com.project... package, but also the test.project... has been packaged.
In the project I have both com... and test... packages obviously.
How could I force it to truly package only the required ones?
Thanks in advance.
TEMPORARY ANSWER (2019-07-03):
Seems that, for now, there is no way to achieve this automatically, thus the answer from #arnonuem seems a good workaround.
If better news, please feel free to improve this thread.
Thank you all.
I would create an ANT file for this specific task. There you can freely customize which packages should be compiled into the jar and which not.
Please inspire yourself reading this example.
For a general overview what i am talking about you could take a quick look into this.
https://howtodoinjava.com/ant/ant-build-jar-file-example/
Please focus on
<javac srcdir="${src.dir}" destdir="${build.dir}" classpathref="classpath"
includes="src/path/to/TheClassToBeIncluded.java" />
For more detailed information on how to include or exclude files or packages you should refer to the documentation over here:
https://ant.apache.org/manual/Tasks/javac.html
At the bottom of this page there is a list with valid build parameters.
First of all, we have to distinguish build and export.
Build transfers .java source code into .class byte code and mostly copies other resources. While doing that it usually merges all source folders into a single bin folder.
Export runs a build (or relies on Build Automatically) and then modifies the resulting .class files and other resources. Usually it packages them into one or more .jar files.
Therefore, our solution includes two steps:
build everything that is unwanted for export into a separate folder (or more)
export from a specific folder (or more)
For step 1, refer to this answer on Stack Overflow.
In a nutshell: Go to Project Properties > Java Build Path > Source tab > Allow output folder for source folders. This enables you to configure a specific output folder for each source folder in the centered viewer.
For step 2, we need to understand that eclipse's Runnable Jar File Export relies on a Launch Configuration.
So before exporting, go to Run > Run Configurations..., select the Classpath tab, remove the default User Entries and hit Advanced.... Now you can Add Folders containing your built classes.
You might want to use separate Run Configurations for internal testing and exporting.
I have struggled with this problem on and off for years, supposing it was just me who was failing to find the right solution. Possible solutions always seem to involve detailed manual configuration e.g. configuration of the build path, or selection of the folders from which classes are exported into a jar, or learning ant, but which still requires manual configuration. The problem is that the inter-dependencies between classes (and packages) are complex - imagine drawing a network diagram from import statements. Manual configuration is time-consuming, error-prone and, I think, infeasible except in simple cases. I am a bit stunned. If there is no automatic solution for selecting necessary classes, I suppose people are regularly exporting their entire code base and that the world is full of bloated jars ... (and, incidentally, without obfuscation, the entire source code base is thereby made available through reverse engineering).
We all know "inputable" resources are by convention in src/main/resources and src/test/resources, but what about the runtime outputted ones? Is it better to use target/ or target/{classes,test-classes}or simply give up and try to use external path even if it complicates things for security reasons? I've been brainstorming a bit regarding that decission as shown following, but need the help of more experienced users that can shed more light.
PROS of target/{classes, test-classes}
If maven engineers architected the convention by moving resources to target/classes instead of target/resources I assume they had a good reason in mind for preferring it
It organizes input and output resources on the same base folder
It makes test and main outputs independent, so no conflict can appear if names are equal
IT makes much conventional and secure to define the relative route of the resource ( by ClassLoadeR().getResource() or etProtectionDomain().getCodeSource().getLocation() [no file globs]
It makes much easy to centralize output behavior, in case our idea is having a function for UPSERTing resources, we need to use resolution for sufolder as they are not in the same path (so prepend /classes/ to the relative route but / for an original resource)
I think due to the previous ones, there can be a bit of confusion when using parent poms, because there is a target for both parent and module, but only one classloader URL -> /target/classes
It works flawlessly if using the classes directory as the base classpath of the app, when executing directly from console.
PROS of target/
When packaging the app as a library, you don't need to deal with an output folder inside the jar.
I have been told that maven doesn't like too much anything strictly outside of target/, but have no further info
A central folder can be used when executing tests in case production code generates files and tests works with them also
Perhaps a single target at parnet pom's level makes files easier to share among different modules
So how do you usually handle these sort if things?
I am sure some of you have a more authoritative question
So typically if you are allowing resources to be pulled in from outside of your Jar you need to either hard-code that relative path or read it from fixed config file. You would then document this when you release your application.
For example, Eclipse has a folder specially for plugins. You can drop new plugins here and know that Eclipse will pick them up and know what to do with them.
In my applications, I usually define a conf directory that sits at the same level as the executable Jar. I'll put any log4j and other such post-compile config files there.
You mention security issues, which is a good thing to think about. When you are pulling in data, always try to do some sort of sanity checking (make sure a directory exists, a zip file isn't corrupt, etc). Since you can't control what comes in, make sure you do as many checks as you can on the program side.
I'm writing a custom tool to build jar files from our build tree. We like to build "minimal" jar files that contain only the .class files actually referenced from the "root" class (where main() lives) -- and so on, recursively following dependencies.
Historically we have done this by getting javac to follow source dependencies, but that means recompiling common files many many times. (We build 60 or 70 distinct application jars from a single source tree.) I'm writing a new build system that compiles each source file only once, but that means we need to follow dependencies by parsing .class files.
The good news is, I've got working code that does what I want. But I need to be absolutely sure I didn't screw it up, i.e. I want to ensure that I'm building internally consistent jar files, where "consistent" means that all unresolved references can be resolved with one of our known third-party jars.
So ideally I want a MagicTool that I can run like
MagicTool \
--classpath commons-lang.jar:commons-collections.jar:[...etc...] \
myapp.jar
that will examine every unresolved reference inside myapp.jar and make sure that it can be resolved by one of the third-party jars passed to --classpath. If not, barf.
Better hope nothing in that path includes any sort of reflection, forName, etc.
I use depfind (if I'm not using my own Java spelunking tools) which may or may not provide output in a way that's helpful for you. jdepend is another option, although I've never used it for anything other than package-level dependencies.
Other tools like ProGuard will strip out unused classes (amongst other things), with the same reflective caveats.
I'm very wary of trying too hard to create minimal jar files; there's a point of diminishing returns/increased risk.
I am pretty sure that ProGuard is your magic tool, even if I don't now the exact call syntax.
At one stage some years ago, trying to index a JAR file with jar -i made it throw an exception if there were unresolved dependencies. I can't quickly test the current state of things.
What is the simplest way to manage dependencies of Java classes to data files present in the classpath?
More specifically:
How should data dependencies be annotated? Perhaps using Java annotations (e.g., #Data)? Or rather some build entries in a build script or a properties file? Is there build tool that integrates and evaluates such information (Ant, Scons, ...)? Do you have examples?
Consider the following scenario:
A few lines of Ant create a Jar from my sources that includes everything found on the classpath. Then jarjar is used to remove all .class files that are not necessary to execute, say, class Foo. The problem is that all the data files that class Bar depends upon are still there in the Jar. The ideal deployment script, however, would recognize that the data files on which only class Bar depends can be removed while data files on which class Foo depends must be retained.
Any hints?
This is one of the many problems Maven has already solved with it's build, dependency, and resource management. Any maven project follows a standard directory layout which dictates where you should put your Data files: in the 'resources' directories. The conventional Maven directory structure is as follows...
/
/src/
/src/main/java/
/src/main/java/App.java
/src/main/resources/
/src/main/resources/my.prod.data.or.cfg.or.whatever
/src/test/java/
/src/test/java/AppTest.java
/src/test/resources/
/src/test/resources/my.test.data.or.cfg.or.whatever
/pom.xml
The benefit of this is that all files which are contained in the 'main' (prod) resources directories are available to your application at run-time from the Classpath. All of the 'test/resources' files are available to your code during build & unit test time but are NOT included in your final artifact.
I don't think a generic solution exists for the system you describe, however, I just had a stab at reading annotations on classes using ASM, since that is used by jarjar as well. It is not hard to read the annotation data that way (pass in a ClassVisitor to the accept() method on ClassReader, and do something useful on the visitAnnotation callback). This means you can either try and include your intended behavior to jarjar or you could add it as a custom step to your build process.
Can't you refactor your project so that you have submodules that each contain the relevant files for the project itself ; Bar class and Bar related files will be packaged in their bundle while Foo ones will packed into another?
Another possibility would be to use some package naming convention to be able to filter the files you want to see i your bundles.
Say I have developed a game, and placed it in the package structure:
com.dxmio.games.breakout
Where then is the 'best practice' place to put resources like audio and images that the game uses?
You can always adopt a standard Maven approach to your project and put all application source files in:
{home}/src/main/java/com/dmxio/games/breakout
and then your resources live in:
{home}/src/main/resources/com/dmxio/games/breakout
and your tests then live in:
{home}/src/test/java/com/dmxio/games/breakout
This structure is a standard convention to structuring projects in the Maven world. It brings a lot of advantages with it, which you may want to look at. You can check out the resources section here: Apache Maven Guides - How do I add resources to my JAR
Alternatively :) the other answer's approach here is just fine...
I have seen this handled in a number of different ways:
Place your resources directly in a subdirectory under com/dmxio/games/breakout (e.g. /com/dmxio/games/breakout/images/foo.gif and /com/dmxio/games/breakout/images/bar.gif)
Place your resources in a jar along with your class files (e.g. foo.gif and bar.gif bundled in breakout.jar)
Place your resources in a separate 'resources jar' in a subdirectory under com/dmxio/games/breakout (e.g. foo.gif and bar.gif bundled in /com/dmxio/games/breakout/images/images.jar)
I tend to favor the last option.
You can then use the java.lang.Class.getResource() method to retrieve your resources.