I'm using the Websphere Application Server 8.5.5.6 and 8.5.5.8 and from time to time run into problems when some jar or the other in my application conflicts with one that is already present on the WAS. It's easy to fix of course, simply mark the dependency as "provided" in maven and there you go, but since IBM seemingly choose to write the AS with the most obscure error messages possible it takes ages to find something like that out.
My question which google hasn't been able to answer so far:
Is there a complete list somewhere which libraries in which versions are provided with Websphere?
Assuming you're referring primarily to Open Source packages, the official list is here: https://www.ibm.com/support/knowledgecenter/en/SSAW57_8.5.5/com.ibm.websphere.nd.multiplatform.doc/ae/opensourcesoftwareapis.html
Beyond that, most of the stuff visible to apps should be Java EE/SE APIs, which I assume you were already expecting, and IBM-specific implementations (things in com.ibm.* packages), which are hopefully at low risk of collision.
At least if you are on Windows: take Process Monitor (not Process Explorer), and fire it up filtering on Path contains .jar. Then start WebSphere. At some point it will starting loading jars from various directories. Process Monitor will show you which are those jars, and where they are being loaded from.
This should provide you with first hand information without reading IBM documents.
Besides, probably you are aware of that, but in any case: you should be careful with marking a dependency as "provided", since the version of the library used by your application might differ from the version used by WebSphere.
Related
This is regarding vulnerability reported with CVE-2021-44228 against the log4j-core jar and has been fixed in Log4J v2.15.0.
We use Logback API via slf4j. This is confirmed with below code.
final StaticLoggerBinder binder = StaticLoggerBinder.getSingleton();
System.out.println(binder.getLoggerFactory());
System.out.println(binder.getLoggerFactoryClassStr());
//output:
//ch.qos.logback.classic.LoggerContext[default]
//ch.qos.logback.classic.util.ContextSelectorStaticBinder
mvn dependency:tree shows log4j-core API (version <2.15) in classpath (both direct & transitive dependency).
Is the application still vulnerable due to maintaining log4j-core in classpath? Thank you!
In order for a vulnerability to be a risk to you, several things need to come together:
the corresponding library exists in your environment
the corresponding library calls do happen in your environment at runtime
3rd party users figure a way to get their (unchecked) input to that library call
Nobody here can tell you whether "2." and ".3" are applicable in your environment.
But: when you eliminate 1., you know that "2." and "3." are no longer possible. Or the other way round, as long as you 100% convinced that there is no path how a user can enter data into your system that makes it to the corresponding API, then you should be fine even with leaving the library sitting in your environment. But as said, having the library is the mandatory first element of the chain. So while that is present, it is possible that somebody writes code tomorrow that gets you to "2" and "3"!
Thus, keep in mind the perspective of higher management: most likely, the business decision might be: reduce the risk to 0, so make sure you don't even have the corresponding JAR sitting on your machines.
In my bigcorp environment, orders were pretty simple: don't waste any time analysing whether your code uses the corresponding interfaces. When your projects contain the vulnerable JAR, upgrade it immediately. Period.
Just as shown in the picture, one app (Java) referenced two third-party package jars (packageA and packageB), and they referenced packageC-0.1 and packageC-0.2 respectively. It would work well if packageC-0.2 was compatible with packageC-0.1. However sometimes packageA used something that could not be supported in packageC-0.2 and Maven can only use the latest version of a jar. This issue is also known as "Jar Hell".
It would be difficult in practice to rewrite package A or force its developers to update packageC to 0.2.
How do you tackle with these problems? This often happens in large-scale companies.
I have to declare that this problem is mostly occurred in BIG companies due to the fact that big company has a lot of departments and it would be very expensive to let the whole company update one dependency each time certain developers use new features of new version of some dependency jars. And this is not big deal in small companies.
Any response will be highly appreciated.
Let me throw away a brick in order to get a gem first.
Alibaba is one of the largest E-Commerces in the world. And we tackle with these problems by creating an isolation container named Pandora. Its principle is simple: packaging those middle-wares together and load them with different ClassLoaders so that they can work well together even they referenced same packages with different versions. But this need a runtime environment provided by Pandora which is running as a tomcat process. I have to admit that this is a heavy plan. Pandora is developed based on a fact that JVM identifies one class by class-loader plus classname.
If you know someone maybe know the answers, share the link with him/her.
We are a large company and we have this problem a lot. We have large dependency trees that over several developer groups. What we do:
We manage versions by BOMs (lists of Maven dependencyManagement) of "recommended versions" that are published by the maintainers of the jars. This way, we make sure that recent versions of the artifacts are used.
We try to reduce the large dependency trees by separating the functionality that is used inside a developer group from the one that they offer to other groups.
But I admit that we are still trying to find better strategies. Let me also mention that using "microservices" is a strategy against this problem, but in many cases it is not a valid strategy for us (mainly because we could not have global transactions on databases any more).
This is a common problem in the java world.
Your best options are to regularly maintain and update dependencies of both packageA and packageB.
If you have control over those applications - make time to do it. If you don't have control, demand that the vendor or author make regular updates.
If both packageA and packageB are used internally, you can use the following practise: have all internal projects in your company refer to a parent in the maven pom.xml that defines "up to date" versions of commonly used third party libraries.
For example:
<framework.jersey>2.27</framework.jersey>
<framework.spring>4.3.18.RELEASE</framework.spring>
<framework.spring.security>4.2.7.RELEASE</framework.spring.security>
Therefore, if your project "A" uses spring, if they use the latest version of your company's "parent" pom, they should both use 4.3.18.RELEASE.
When a new version of spring is released and desirable, you update your company's parent pom, and force all other projects to use that latest version.
This will solve many of these dependency mismatch issues.
Don't worry, it's common in the java world, you're not alone. Just google "jar hell" and you can understand the issue in the broader context.
By the way mvn dependency:tree is your friend for isolating these dependency problems.
I agree with the answer of #JF Meier ,In Maven multi-module project, the dependency management node is usually defined in the parent POM file when doing unified version management. The content of dependencies node declared by the node class is about the resource version of unified definition. The resources in the directly defined dependencies node need not be introduced into the version phase. The contents of the customs are as follows:
in the parent pom
<dependencyManagement>
<dependencies >
<dependency >
<groupId>com.devzuz.mvnbook.proficio</groupId>
<artifactId>proficio-model</artifactId>
<version>${project.version}</version>
</dependency >
</dependencies >
</dependencyManagement>
in your module ,you do not need to set the version
<dependencies >
<dependency >
<groupId>com.devzuz.mvnbook.proficio</groupId>
<artifactId>proficio-model</artifactId>
</dependency >
</dependencies >
This will avoid the problem of inconsistency .
This question can't be answered in general.
In the past we usually just didn't use dependencies of different versions. If the version was changed, team-/company-wide refactoring was necessary. I doubt it is possible with most build tools.
But to answer your question..
Simple answer: Don't use two versions of one dependency within one compilation unit (usually a module)
But if you really have to do this, you could write a wrapper module that references to the legacy version of the library.
But my personal opinion is that within one module there should not be the need for these constructs because "one module" should be relatively small to be manageable. Otherwise it might be a strong indicator that the project could use some modularization refactoring. However, I know very well that some projects of "large-scale companies" can be a huge mess where no 'good' option is available. I guess you are talking about a situation where packageA is owned by a different team than packageB... and this is generally a very bad design decision due to the lack of separation and inherent dependency problems.
First of all, try to avoid the problem. As mentioned in #Henry's comment, don't use 3rd party libraries for trivial tasks.
However, we all use libraries. And sometimes we end up with the problem you describe, where we need two different versions of the same library. If library 'C' has removed and added some APIs between the two versions, and the removed APIs are needed by 'A', while 'B' needs the new ones, you have an issue.
In my company, we run our Java code inside an OSGi container. Using OSGi, you can modularize your code in "bundles", which are jar files with some special directives in their manifest file. Each bundle jar has its own classloader, so two bundles can use different versions of the same library. In your example, you could split your application code that uses 'packageA' into one bundle, and the code that uses 'packageB' in another. The two bundles can call each others APIs, and it will all work fine as long as your bundles do not use 'packageC' classes in the signature of the methods used by the other bundle (known as API leakage).
To get started with OSGi, you can e.g. take a look at OSGi enRoute.
Let me throw away a brick in order to get a gem first.
Alibaba is one of the largest E-Commerces in the world. And we tackle with these problems by creating an isolation container named Pandora. Its principle is simple: packaging those middle-wares together and load them with different ClassLoaders so that they can work well together even they referenced same packages with different versions. But this need a runtime environment provided by Pandora which is running as a tomcat process. I have to admit that this is a heavy plan.
Pandora is developed based on a fact that JVM identifies one class by class-loader plus classname.
I've seen a question or two on Stack overflow regarding this error but I'm still unable to solve it, so I thought I would pose my own question.
Here's my issue:
I'm using Spring and Spring's JMSTemplate to do some messaging and queue work. I'm trying to read from a queue. I'm not 100% positive if my logic is correct in my code, but anytime I try to run my app I am greeted with this exception (I've included only the last section):
Caused by: com.ibm.msg.client.commonservices.CSIException: JMSCS0002
at com.ibm.msg.client.commonservices.workqueue.PIWorkQueueManager.enqueueItem(PIWorkQueueManager.java:67)
at com.ibm.msg.client.commonservices.workqueue.WorkQueueManager.enqueue(WorkQueueManager.java:225)
at com.ibm.msg.client.commonservices.workqueue.WorkQueueManager.enqueue(WorkQueueManager.java:194)
at com.ibm.msg.client.wmq.common.internal.WMQThreadPool.enqueue(WMQThreadPool.java:91)
Now I'm fairly certain this has nothing to do with my code because no matter how much I change my logic, if I try calling any of the methods made available by JMSTemplate, I receive this exception. After doing some research (based on the other stack overflow answers) I assume it has something to do with the way my classpath is setup. Here is a link to those questions:
One and Two
In addition to this, here's some info I found on IBM's site:
To compile and run WebSphere MQ classes for JMS applications, use the
CLASSPATH setting for your platform as shown in Table 1.
CLASSPATH=MQ_INSTALLATION_PATH\java\lib\com.ibm.mqjms.jar;
MQ_INSTALLATION_PATH\tools\jms;
I have tried this however and It still seems to be failing me. Here's what I have added in my .bat file for my application that I run:
c:\java\jre6\bin\javaw -cp "C:\ussco\wmsflgint\mqs\mqjms-7.5.0.0.jar; C:\ussco\wmsflgint\mqs\mq-7.5.0.0.jar; C:\ussco\wmsflgint\mqs\headers-1.4.2.jar; C:\ussco\wmsflgint\mqs\jmqi-7.5.0.0.jar;" -Xmx256M .... (there's more on the end but I don't feel it's relevant)
Am I not adding this correctly?
Thanks
I've just ran into the same issue with queue listeners. The solution was to place a file compinfo.properties under the directory src/main/resources/META-INF of the Spring project. The file should set values for two properties:
CompList: comp1
comp1_CompClass: com.ibm.msg.client.commonservices.j2se.J2SEComponent
Or you can change the property (comp1_CompClass) value right in the jar com.ibm.msg.client.commonservices.j2se.jar. It has the same effect though I doubt it's legal due to copyright.
Hope it would be helpful and save a couple of hours for someone.
The problem here is that you have been copying and renaming IBM MQ jar files and, as a result, do not have the full set on the Java class path at runtime. This can lead to all kinds problems and unexpected exceptions, such as the one you are experiencing.
Please note that copying MQ jar files, renaming them and/or bundling them into applications is not permitted by IBM Support and invalidates the MQ terms and conditions. (The rules are subtlety different for bundling into apps for the V8 and V9 redistributable client and allclient; but your not using that here).
If you perform a proper install of the MQ client onto your system (which you should do) and then use the instructions that you have already found in the Knowledge Center to reference the com.ibm.mq.jar file for classes for Java applications or the com.ibm.mqjms.jar for classes for JMS applications on the Java class path, your problem will be resolved.
This is a follow-up (somehow of my Third-party dependencies to an OSGi application) where it was suggested that some libraries e.g. log4j are already available as bundles.
In Eclipse Indigo I could not find a log4j bundle available to Import Package as part of my installation and so I created a Plugin Project from JAR archive to bundle log4j and also a Feature Project to bundle the log4j.xml configuration following this post.
To be honest I don't understand why the fragment project is needed but this process works.
So my question now is:
Since the log4j.xml is delivered in the export as part of the feature jar, it requires some "effort" for someone to find it and update the debug levels, so I was wondering is this indeed the correct process?
I had in mind that the final exported product would deliver the log4j configuration in an easy to find location, but now (although the logging works) I am concerned whether what I do, is indeed correct.
Any help here?
If you really need to expose the file, you could put it anywhere you want, and then make sure your program calls one of these methods at startup:
org.apache.log4j.xml.DOMConfigurator.configure(String filename)
org.apache.log4j.PropertyConfigurator.configure(String
configFilename)
Or use the "configureAndWatch"-variants if you would like to make changes to the config without restarting your application.
Edit: I write "If you really need to", because I have experienced that I never need to turn on debug-logging after deployment, because it is always turned on! This is OK for applications where I have normal (but not extreme) requirements on response-time and throughput. Logging to an UDP-appender is fast (and does not fill up the disk). Or using rolling file appender is quite safe, and fast enough for my use. Always having the debug-log available is a life-saver when nailing down those hard-to-reproduce bugs.
I suggest take a look at Pax-Logging this will give you all kinds of logging frameworks for usage in a OSGi environment. And you're able to use an external configuration file (no extender needed) to configure your logging.
The fragment is one option to extend the log4j bundles classpath to include the required configuration file. It is probably the simplest way of configuring application wide properties.
This is not meant to be altered after deployment though as it will be embedded within a jar file. You will have to come up with a different approach if you expect to make it configurable after deployment.
NOTE:
I am afraid you misunderstood the answer about the jars that are already available as bundles. This does not mean that they are part of your OSGi platform of choice (Indigo), only that they are ready to be deployed to an OSGi platform as is. Your creation of a plugin project was unnecessary, you simply needed to add the jar to your target platform to resolve your missing imports.
I'd like to use something like the Filtering Classloader to prevent specific packages from creeping into the application context and becoming visible to Spring.
Changing the classloader order causes all sorts of nasty problems so I´d like to try this route.
Is it possible to achieve this with Websphere 6? If not, can I replace my own application classolader and implement the filter myself?
There is no such filtering mechanism in WebSphere, and there is no way to replace the application class loader. You'll have to use PARENT_LAST to override classes, sorry.
bkail's answer is right, WAS doesn't have such feature even in its latest public version (8.5.5).
I just created a RFE requesting such feature so whoever is interested in this, please vote for it which may increase the possibility of this being implemented:
http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=43936
(IBM ID required)
In the meantime, you may use isolated shared libraries to override any particular classes (the above mentioned class loading order control - like parent_last - is too rough as it affects the class loading order of the whole application or module)
Create a shared library with desired jars on the classpath, configure it as isolated shared library, reference it from the deployed application (or module).
See here for complete documentation
http://pic.dhe.ibm.com/infocenter/wasinfo/v8r5/topic/com.ibm.websphere.base.doc/ae/tcws_sharedlib.html
I was just about to post the same question. But the answer was quite unsatisfying. I however checked the request from Petr H at the IBM developerworks and IBM did implement this feature (Huge thanks Petr!):
"WebSphere Application Server V8.5.5.7 (=Fixpack 7) gained the ability to prevent packages from the server classloader being visible to applications. This was delivered in the document "ISOLATE DEPLOYED ARTIFACTS FROM OSS PACKAGES" and is documented in 'Isolating open source software packages'.
The supplied links describe the mechanism by configuring always-protected packages. You basically have to do the following:
Under Server Infrastructure on the server settings page in the administrative console, click Java and process management > Process definition.
Select Java virtual machine.
Define the following system properties in the JVM generic arguments section as follows:
-Dcom.ibm.ws.classloader.server.alwaysProtectedPackages=org.bouncycastle.
Please not that the final dot "." is really important otherwise everything will be ignored! Several packages can be added by comma ","
Click Apply, OK and save the changes. Make sure that a file synchronization is done before you restart the servers. Restart WebSphere Application Server for the changes to take effect.
Examine the native_stdout.log and find the system properties that are previously defined. For example, when you specify always-protected package org.bouncycastle., statements such as the following might appear:
ProtectionMetaData.clinit: system property: com.ibm.ws.classloader.server.alwaysProtectedPackages=org.bouncycastle.