It is time to sub-divide a platform I'm developing and I'm looking for advice on how to handle cross-component dependencies. I spose there a many cases, so I'll give an example.
I have an Address class that I want to make visible to developers. It is also referenced by classes in my.Contacts, my.Appointments, and my.Location packages - each of which I want to be separately compiled, jar-d, and delivered. Of course I want Address to be a single class - an Address works across these platform components transparently.
How should Address be packaged, built, and delivered?
Thanks!
Two thoughts:
Address sounds like a common component that can be used in different deliverables and so should be available in some common or core library
It may make sense for your components to talk to an Address interface, and the implementation can be provided separately (e.g. provide an Address interface and an AddressImpl implementation). This will reduce the amount of binding between the core library and the library your developers will develop.
In this case Address is a part of a library which deserves its own jar. If you create a class named Address in my.Contacts, my.Appointments, and my.Location and you want to use all theses jar in a same application, you'll have a conflict for your Address class.
I suggest you don't "Deliver" these jars separately. Java has very subtle versioning issues that you don't want to run into. Build everything together and package it into one or two jars and always deliver both jars, or build them together and deliver a subset of jars (but never combine new and old jars--don't just try to send a single jar as an update).
If you must build them separately be very aware that final constants are compiled in and not referenced--so if you change one and deliver a new jar, any references from an older jar will not be updated.
Also method signatures that change will have strange, unpredictable results.
It sounds like you want a developer interface as well--that may be a set of interfaces and classes that reside in a separate jar. If you make that one jar well enough that you never have to rev it (and, of course, with no references to external constants) you can probably get away with not updating it which will keep your customer's extensions from getting crusty.
Related
Just as shown in the picture, one app (Java) referenced two third-party package jars (packageA and packageB), and they referenced packageC-0.1 and packageC-0.2 respectively. It would work well if packageC-0.2 was compatible with packageC-0.1. However sometimes packageA used something that could not be supported in packageC-0.2 and Maven can only use the latest version of a jar. This issue is also known as "Jar Hell".
It would be difficult in practice to rewrite package A or force its developers to update packageC to 0.2.
How do you tackle with these problems? This often happens in large-scale companies.
I have to declare that this problem is mostly occurred in BIG companies due to the fact that big company has a lot of departments and it would be very expensive to let the whole company update one dependency each time certain developers use new features of new version of some dependency jars. And this is not big deal in small companies.
Any response will be highly appreciated.
Let me throw away a brick in order to get a gem first.
Alibaba is one of the largest E-Commerces in the world. And we tackle with these problems by creating an isolation container named Pandora. Its principle is simple: packaging those middle-wares together and load them with different ClassLoaders so that they can work well together even they referenced same packages with different versions. But this need a runtime environment provided by Pandora which is running as a tomcat process. I have to admit that this is a heavy plan. Pandora is developed based on a fact that JVM identifies one class by class-loader plus classname.
If you know someone maybe know the answers, share the link with him/her.
We are a large company and we have this problem a lot. We have large dependency trees that over several developer groups. What we do:
We manage versions by BOMs (lists of Maven dependencyManagement) of "recommended versions" that are published by the maintainers of the jars. This way, we make sure that recent versions of the artifacts are used.
We try to reduce the large dependency trees by separating the functionality that is used inside a developer group from the one that they offer to other groups.
But I admit that we are still trying to find better strategies. Let me also mention that using "microservices" is a strategy against this problem, but in many cases it is not a valid strategy for us (mainly because we could not have global transactions on databases any more).
This is a common problem in the java world.
Your best options are to regularly maintain and update dependencies of both packageA and packageB.
If you have control over those applications - make time to do it. If you don't have control, demand that the vendor or author make regular updates.
If both packageA and packageB are used internally, you can use the following practise: have all internal projects in your company refer to a parent in the maven pom.xml that defines "up to date" versions of commonly used third party libraries.
For example:
<framework.jersey>2.27</framework.jersey>
<framework.spring>4.3.18.RELEASE</framework.spring>
<framework.spring.security>4.2.7.RELEASE</framework.spring.security>
Therefore, if your project "A" uses spring, if they use the latest version of your company's "parent" pom, they should both use 4.3.18.RELEASE.
When a new version of spring is released and desirable, you update your company's parent pom, and force all other projects to use that latest version.
This will solve many of these dependency mismatch issues.
Don't worry, it's common in the java world, you're not alone. Just google "jar hell" and you can understand the issue in the broader context.
By the way mvn dependency:tree is your friend for isolating these dependency problems.
I agree with the answer of #JF Meier ，In Maven multi-module project, the dependency management node is usually defined in the parent POM file when doing unified version management. The content of dependencies node declared by the node class is about the resource version of unified definition. The resources in the directly defined dependencies node need not be introduced into the version phase. The contents of the customs are as follows:
in the parent pom
<dependencyManagement>
<dependencies >
<dependency >
<groupId>com.devzuz.mvnbook.proficio</groupId>
<artifactId>proficio-model</artifactId>
<version>${project.version}</version>
</dependency >
</dependencies >
</dependencyManagement>
in your module ,you do not need to set the version
<dependencies >
<dependency >
<groupId>com.devzuz.mvnbook.proficio</groupId>
<artifactId>proficio-model</artifactId>
</dependency >
</dependencies >
This will avoid the problem of inconsistency .
This question can't be answered in general.
In the past we usually just didn't use dependencies of different versions. If the version was changed, team-/company-wide refactoring was necessary. I doubt it is possible with most build tools.
But to answer your question..
Simple answer: Don't use two versions of one dependency within one compilation unit (usually a module)
But if you really have to do this, you could write a wrapper module that references to the legacy version of the library.
But my personal opinion is that within one module there should not be the need for these constructs because "one module" should be relatively small to be manageable. Otherwise it might be a strong indicator that the project could use some modularization refactoring. However, I know very well that some projects of "large-scale companies" can be a huge mess where no 'good' option is available. I guess you are talking about a situation where packageA is owned by a different team than packageB... and this is generally a very bad design decision due to the lack of separation and inherent dependency problems.
First of all, try to avoid the problem. As mentioned in #Henry's comment, don't use 3rd party libraries for trivial tasks.
However, we all use libraries. And sometimes we end up with the problem you describe, where we need two different versions of the same library. If library 'C' has removed and added some APIs between the two versions, and the removed APIs are needed by 'A', while 'B' needs the new ones, you have an issue.
In my company, we run our Java code inside an OSGi container. Using OSGi, you can modularize your code in "bundles", which are jar files with some special directives in their manifest file. Each bundle jar has its own classloader, so two bundles can use different versions of the same library. In your example, you could split your application code that uses 'packageA' into one bundle, and the code that uses 'packageB' in another. The two bundles can call each others APIs, and it will all work fine as long as your bundles do not use 'packageC' classes in the signature of the methods used by the other bundle (known as API leakage).
To get started with OSGi, you can e.g. take a look at OSGi enRoute.
Let me throw away a brick in order to get a gem first.
Alibaba is one of the largest E-Commerces in the world. And we tackle with these problems by creating an isolation container named Pandora. Its principle is simple: packaging those middle-wares together and load them with different ClassLoaders so that they can work well together even they referenced same packages with different versions. But this need a runtime environment provided by Pandora which is running as a tomcat process. I have to admit that this is a heavy plan.
Pandora is developed based on a fact that JVM identifies one class by class-loader plus classname.
How can I cache server-wide (with cache scope spanning multiple WARs on this server) instances of classes from a JAR which is contained binary-identical in several WARs on a web container (server, e. g. Tomcat)?
<EDIT> I want to cache application data across WARs because the data is common to them. (It's a portal project, where it can be useful to share common data across different "views" implemented as different portlets deployed as different WARs, and using a Java object cache is much faster and more simple than using a central data-holding service.) </EDIT>
Is that possible at all? Or is it required to put such a JAR on a path accessed by a common parent classloader, like in /lib/ext ?
See: Java, Classpath, Classloading => Multiple Versions of the same jar/project
See: How does class loading work when the same class exists in different applications on the same server?
See: cast across classloader?
See: What is a serialVersionUID and why should I use it?
Yes, the best option is to put the classes in a class loader that is a parent of the two applications. If by lib/ext you mean JAVA_HOME/lib/ext, then I would not recommend that. Instead, you should put them in CATALINA_HOME/lib directory. See the Shared Library Files section of the documentation, which links to the Class Loader HOW-TO documentation.
You can add common classes (jars) to the shared.loader property in conf/catalina.properties. Those classes are available to all web apps but not tomcat itself.
If you implement a cache around a static singleton, then you would be able to access the objects from different web apps. I don't know if that is best practice however. For example it makes it hard to scale because it makes it impossible to load balance the apps onto many servers.
The answer seems to be "it depends".
If the JAR(s) (or classes) in question do not have dependencies conflicting with other components also deployed on the server, both proposed solutions (CATALINA_HOME/lib/ext/ and CATALINA_HOME/conf/catalina.properties :: shared.loader) should plainly work. Thus both are "correct answers" and I cannot see which one is "more correct" than the other.
However I missed a crucial detail when I first asked the question (but this does not invalidate it): In my case the JAR in question required Spring 4.2.9.RELEASE (and other dependencies), but other relevant WARs deployed on the same server contain and require Spring 3.0.7. (The objects to be cached do not depend on Spring, but the JAR was not designed with this problem in mind, and it also contains other related code depending on Spring which now would be very difficult to separate.)
Generally it should be possible to put into CATALINA_HOME/lib/ext/ what ever you want as long as all already deployed WARs contain everything they need: The "module first / parent last" class loading policy should prevent conflicts, even if (as in this example) Spring 4.2.9 is available to the parent classloader and Spring 3.0.7 is available to the WAR classloader. But it looks somewhat "unclean" and messy to me to mix-up things that way.
Therefore I decided to use the "to-be-cached" object's classloader hash code as the key in a map, in which the cached data are the values. Then all cached data is selected "by classloader" which automatically and transparently ensures assignment compatibility. If there is also another WAR deployed on the server which can change and thus invalidate the cached data, it can remove the whole map from the cache, forcing the "read-access" WARs to reload data on next access.
However this approach DOES NOT allow cross-WAR cacheing: Effectively every WAR will get its own private cache segment.
Another approach would be to deliberately transform all data to cache to/from e. g. JSON so as to get a "naturally global" data type like java.lang.String for the cached data. If chosen from the beginning of the project, to me this seems to be the cleanest way, but if there is already a complex (and generally working) implementation in place, this may cause some work to do.
Comments on this self-answer are welcome!
I wrote two jars. Each one of them is responsible for sending different http/https request.
Each one of them uses, naturally, certain same classes. Like the ones that builds the requests or send them. The process might be a bit different, but still the general structure and classes names are the same.
Building different jars per request is a requirement from my managers! So using 1 jar for all my http requests is not acceptable.
Now, in my client program I need to send a request one time for JarA and one time from JarB. But compilation fails because, naturally, I am using very similar namings for the classes and methods.
For example, I have a UserData class in both jars. So when I try to use it in my client program, the compiler yells: "reference to SystemData is ambiguous".
I can start improvising specific classes names for each jar, but it is ugly...
How would you suggest to solve this problem?
If the classes are identical, pull them out into a third JAR and then have the client program reference the common JAR plus JarA or JarB.
If the classes are similar but not identical, then put them into different packages. You can have classes with the same names if they're in different packages.
Put common classes in a third jar and either bundle it in the two http jars or add it to the classpath at runtime (which is the best choice will depend on how you're deploying, etc.).
Firstly you have to decide which kind of architecture you are working with.
If managers asking you to have different jar's for sake of modularization - sure it's worth to make common jar which will contain all common classes.
I suppose you should have your project built with Maven, Gradle or another build system which will help you managing dependencies.
Another issue could be if you are supposed to do 'Microservices' architecture. Then code duplication is inevitable.
To overcome same class names when you have duplication - I would recommend to have for every module different package names then.
Use a build system like maven where one can have library dependencies, to a common third jar. It maintains a repository of versioned jars.
One solution is that - if you see a same class with same package in two different jars and both jars are required in your project,
Solution
you can download the source code of that duplicate class and creat keep the same in your project with package structure. So this way JVM loads your project classes first and give first preference to invoke your project class rather then other jar's class
I recently started using Eclipse IDE and have read at a number of places that one shouldn't use the default(src) package and create new packages.
I just wanted to know the reason behind this.
Using the default package may create namespace collisions. Imagine you're creating a library which contains a MyClass class. Someone uses your library in his project and also has a MyClass class in his default package. What should the compiler do? Package in Java is actually a namespace which fully identifies your project. So it's important to not use the default package in the real world projects.
Originally, it was intended as a means to ensure there were no clashes between different pieces of Java code.
Because Java was meant to be run anywhere, and over the net (meaning it might pick up bits from Sun, IBM or even Joe Bloggs and the Dodgy Software Company Pty Ltd), the fact that I owned paxdiablo.com (I don't actually but let's pretend I do for the sake of this answer) meant that it would be safe to call all my code com.paxdiablo.blah.blah.blah and that wouldn't interfere with anyone else, unless they were mentally deficient in some way and used my namespace :-)
From chapter 7, "Packages", of the Java Language Spec:
Programs are organized as sets of packages. Each package has its own set of names for types, which helps to prevent name conflicts.
I actually usually start by using the default package and only move it into a real package (something fairly easy to do with the Eclipse IDE) if it survives long enough to be released to the wild.
Java uses the package as a way to differentiate between classes. By using packages, you can have an org.example.Something class and an org.example.extended.Something class and be able to differentiate between them even though they are both named Something. Since their packages are different, you can use them both in the same project.
By declaring a package you define your own namespace (for classes). This way if you have two identical classes using a different package name (namespace) will differentiate between which one you want to use.
The main reasons I can think of are:
It keeps things organised, which will help you (and others!) know where to look for classes/functionality.
You can define classes with the same name if they are in different packages.
Classes/etc in the default package cannot be imported into named packages. This means that in order to use your classes, other people will have to put all their classes in the default package too. This exacerbates the problems which reasons 1 & 2 solve.
From a java point of view, there are two general dev/deploy lifecycles you can folllow, either using ant to build and deploy, or the maven lifecycle. Both of these lifecycles look for source code and resources in local directories, and in the case of maven, in defined repositories, either locally or on the net.
The point is, when you set up a project, for development and eventually deployment, you want to build a project structure that is portable, and not dependent on the IDE, ie. your project can be built and deployed using either of your build environments. If you use a heavy dependence on the Eclipse framework for providing class variables, compile paths, etc.. you may run into the problem that your project will only build and deploy using that configurationj, and it may not be portable to another developers environment, so to speak.
In Java web application, what is the exact meaning of the term "package structure" and "directory structure" ? Aren't they the same? I saw some articles have these two terms, but I am not sure about the exact meaning and difference.
Package is a collection of code that changes together, is used together and is shipped together. So a jar/war is a package.
Package Design Principles
I understand that you meant source package, which is more like directory structure. But I believe, a directory is a physical representation on hard drive.
EDIT: I had writtern original answer more than 3years back. But did not change as it was accepted. But changing it now so that any new visitor may benefit and also to avoid link rot. Some additional meaning of package may be extracted based on the discussion below. For example, is a jar a package?
Classes that get reused together should be packaged together so that the package can be treated as a sort of complete product available for you. And those which are reused together should be separated away from the ones those are not reused with. For example, your Logging utility classes are not necessarily used together with your file io classes. So package all logging them separately. But logging classes could be related to one another. So create a sort of complete product for logging, say, for the want of better name commons-logging package it in a (re)usable jar and another separate complete product for io utilities, again for the want of better name, say commons-io.jar. If you update say commons-io library to say support java nio, then you may not necessarily want to make any changes to the logging library. So separating them is better.
Now, let's say you wanted your logging utility classes to support structured logging for say some sort of log analysis by tools like splunk. Some clients of your logging utility may want to update to your newer version; some others may not. So when you release a new version, package all classes which are needed and reused together for migration. So some clients of your utility classes can safely delete your old commons-logging jar and move to commons-logging-new jar. Some other clients are still ok with older jar. However no clients are needed to have both these jars (new and old) just because you forced them to use some classes for older packaged jar.
Avoid cyclic dependencies. a depend on b; b on c; c on d; but d depends on a. The scenario is obviously deterring as it will be very difficult to define layers or modules, etc and you cannot vary them independly relative to each other.
Also, you could package your classes such that if a layer or module changes, other module or layers do not have to change necessarily. So, for example, if you decide to go from old MVC framework to a rest APIs upgrade, then only view and controller may need changes; your model does not.
In most Java applications, the package structure should be matched by the directory structure for the .java and .class files. However these directories are part of a larger directory structure, including other data than the source and/or the bytecode.
Depending on the context, the "package structure" might also refer to delivery packages, each containing an application or a library.