I have an older widget app, it is using JAXB and now being migrated to Java 11.
I am aware that java.xml.bind/JAXB has been removed in this version, so we are trying to replace it with Jakarta.
We have jakarta.activation.jar and jakarta.xml.bind-api.jar and it works fine for compilation, but not for runtime
when the app starts I am getting this:
javax.xml.bind.JAXBException: Implementation of JAXB-API has not been found on module path or classpath.
- with linked exception:
[java.lang.ClassNotFoundException: com.sun.xml.internal.bind.v2.ContextFactory]
at java.xml.bind/javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:232)
... ...
The com.sun.xml.internal.* classes were part of rt.jar, which i believe was removed in Java 9,
but the latest jakarta source still refers to them... how is that supposed to work?
I saw some posts with a similar problem, and typical solution is Add Maven dependency.
I'm not sure about details, but in any case we don't use Maven or Gradle and don't have pom.xml.
Is there anything i can do to make it work?
Turned out it is not as bad as I thought.
rt. jar was broken down to multiple modules and I started looking for the ones I needed.
jaxb-runtime.jar looked right to me and I ran strings / grep on it - it indeed had all the com.sun.xml.bind classes in it!
When I added this jar to the ones I had the original error changed to "some-other-class not found" and I had to add more jars until the app was happy.
In the end I have this:
jakarta.activation.jar, jakarta.xml.bind-api.jar, jaxb-runtime.jar,
istack-commons-runtime.jar, stax-ex.jar, FastInfoset.jar, txw2.jar
No changes to manifest, makefile or anything else related to build, purely packaging issue.
Big sigh...
Related
I am having issues trying to get Spark to load, read and query a parquet file. The infrastructure seems to be set up (Spark standalone 3.0) and can be seen and will pick up jobs.
The issue I am having is when this line is called
Dataset<Row> parquetFileDF = sparkSession.read().parquet(parquePath);
the following error is thrown
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.10.0 requires Jackson Databind version >= 2.10.0 and < 2.11.0
at com.fasterxml.jackson.module.scala.JacksonModule.setupModule(JacksonModule.scala:61)
I looked into JacksonModule.setupModule and when it gets to context.getMapperVersion the version that is being passed is 2.9.10. It appears to me that the DefaultScalaModule is pulling some older version.
I'm using Gradle to build and have the dependencies set up as such
implementation 'com.fasterxml.jackson.core:jackson-core:2.10.0'
implementation 'com.fasterxml.jackson.core:jackson-databind:2.10.0'
implementation 'org.apache.spark:spark-core_2.12:3.0.0'
implementation 'org.apache.spark:spark-sql_2.12:3.0.0'
implementation 'org.apache.spark:spark-launcher_2.12:3.0.0'
implementation 'org.apache.spark:spark-catalyst_2.12:3.0.0'
implementation 'org.apache.spark:spark-streaming_2.12:3.0.0'
That didn't work, so I tried forcing databind
implementation ('com.fasterxml.jackson.core:jackson-databind') {
version {
strictly '2.10.0'
}
}
I've tried a few different versions and still keep hitting this issue. Maybe I'm missing something super simple, but right now, I can't seem to get past this error.
Any help would be appreciated.
I was able to figure out the issue. I was pulling in jar file from another project. The functionality in the jar file wasn't being used at all, so it wasn't suspect. Unfortunately, that project hadn't been updated and there were some older Spark libraries that were some how being picked up by my current running app. Once I removed that, the error went away. What's interesting is the dependency graph didn't show anything about the libraries the other jar file was using.
I suppose if you run into a similar issue, double check any jar files being imported.
We have maven apps that until recently were on JDK8. We just upgraded them to JDK11 and are trying to take advantage of the JPMS from JDK9 by making our utility libraries into modules.
We originally had this kind of path:
utils/some-library1/src/main/java/com/company/team/utils/lib1/Util1.java
There, java is the "source root".
So a colleague placed the module-info.java file in the lib1 folder and declared it thus:
module utils.lib1 {
exports com.company.team.utils.lib1;
}
From the command line that builds and works, so he assumes everything is all module-y goodness.
But when I opened in Intellij, it had an ugly red line and the message said I should move it to source root. It then moved it to the "java" folder above. Fair enough.
That caused me to dig around trying to find out more about this JPMS that my colleague had implemented. After a lot of searching and experiments, I also determined that the "java" folder, as "source root", should be renamed to the name of the module ("utils.lib1"). So now I have these two files:
utils/some-library1/src/main/utils.lib1/module-info.java
utils/some-library1/src/main/utils.lib1/com/company/team/utils/lib1/Util1.java
And even Intellij is happy. Hooray! So I refactor all the other libraries. Suddenly I hit a major snag in let's call it lib2 with this line:
module utils.lib2 {
exports com.company.team.utils.lib2;
requires java.ws.rs;
}
Intellij flags the module with the red error squiggle again, this time saying:
Module 'utils.lib2' reads package 'javax.activation' from both 'jakarta.activation' and 'jakarta.activation'
I did some digging and found out the following:
java.ws.rs pulls in one of the following (it depends on which app):
javax.ws.rs-api-2.1.1.jar
jakarta.ws.rs-api-2.1.6.jar
Their module-info.java files contain this line:
requires transitive java.xml.bind;
Which pulls in one of:
jakarta.xml.bind-api-2.3.2.jar
jakarta.xml.bind-api-2.3.3.jar
jaxb-api-2.4.0-b180830.0359.jar
Which all have this line:
requires transitive jakarta.activation;
And that's where I give up. Our libraries are big hefty things that are hard to parse completely, so to simplify I created a maven app with just one class and all it does is import javax.ws.rs.core.Link.
And IntelliJ still gives that crazy error that I can't figure out and Google has been adamant in refusing to tell me.
Is it really broken or is Intellij just as confused as I?
I gave the long story both to show what we've done and to let you know that I'm very new to modules. So if it's a simple thing, please excuse me. I am just stumped though.
Additionally, are there any obvious tests one can perform at the command line to validate module configuration?
I've had inconsistent luck with jdeps, javac, and actually running as indicators of problems.
My suspicion is that things only work now because they're all in the unnamed module. But I need to get everything working if I'm going to convince anyone to change it.
EDIT
This question was reported as already answered, but that is incorrect. The suggested link deals with two different packages (A & B) importing package X. But in my case, the error is that the same package (A & A) imports package X. And this is done a few transitives down, so I have no control over the imports and can't see how to do an exclusion. Also, this problem can be repeated with just single requires statement in module-info.java!
Plus, there is a second question here that is also important that has not been addressed: how to validate the module configuration from command line (without involving the IDE at all).
I also determined that the "java" folder, as "source root", should be renamed to the name of the module
No, it should not. The java source root should be left as is but you must create a package name corresponding to your module name, so it should be /src/main/java/ - source root and then utils/lib1 directory - whidh would be the package.
I came across exact same warning in Intellij and it was genuine. In my case the collision was coming from three separate dependencies using same module name (i.e. 'jakarta.activation'):
'jakarta.activation:jakarta.activation-api:1.2.2'
'javax.activation:javax.activation-api:1.2.0'
'com.sun.activation:jakarta.activation:1.2.2'
It got it resolved for my project by applying explicit exclusions on dependencies which were pulling the last two.
Recently I've been working on report generation with Jasper. I have created a simple program to test it and when running it via IDE it did work fine.
Then I moved the (very short) class to WildFly sever application and despite having the exact same code and library generation fails with cannot find symbol. Those symbols it cannot find are JREvaluator, JRFillVariable as well as packages such as net.sf.jasperreports.engine
In so far I have confirmed that:
Project builds (meaning those classes are visible for javac, but not jvm)
jasperreports-6.13.0.jar is added to war (it's present in /WEB-INF/lib folder alongside other libraries, like gson and hibernate
jasperreports-6.13.0.jar contains the missing classes
It looks to me like the problem doesn't lie in library not being loaded or missing classes (because in testing environment it works), but like something was preventing JBoss class loader from loading those classes
Attempted (and failed) solutions
Clean and Build
Adding -Djava.awt.headless=true to VM options - this did not changed anything
Adding -Djava.awt.headless=false to VM options - also didn't change a thing, but once caused NullPointerException inside jasperreport library. For testing program - worked in both cases
Adding commons-beanutils-1.9.4.jar, commons-digester-2.1.jar, commons-collections4-4.4.jar and commons-loggin-1.2.jar - with no changes
Adding jasper-compiler-jdt-5.5.23.jar - this caused a different error, namely NoSuchMethodError for org.eclipse.jdt.internal.compiler.ICompilerRequestor and few others. This library however should not be necessary as - from what I understand - jasperreport-6.13.0.jar already contains it's compiler and separate library for compiler is not required since a long time.
What has not been attempted:
Forcing the classes to load (http://www.java2s.com/Code/Java/Reflection/Forcethegivenclasstobeloadedfully.htm)
Dynamically loading jar during Runtime or using custom class loader
Update: after looking at this answer and applying the suggestion the missing class was different. Which suggests that the dependencies inside jasperreport.jar are not being loaded properly
I have figured it out
For some reason in server project libraries used by jasperreport.jar were not loaded, but in the testing project they were (might be due to WildFly, might be due to differences between IntelliJ and NetBeans)
Here is the list of libraries, based on pom.xml file in jasperreport.jar that I have added. Some might not be necessary and the list might not be exhaustive (I basically stopped adding libraries once report started generating) but it's good enough base if someone else runs into this problem:
commons-beanutils-1.9.4.jar
itext-2.1.7.jar
poi-ooxml-4.1.1.jar
commons-collections4-4.4.jar
jcommon-1.0.23.jar
xalan-2.7.2.jar
commons-digester-2.1.jar
jfreechart-1.0.19.jar
xmpcore-5.1.3.jar
commons-logging-1.2.jar
poi-4.1.1.jar
I'm starting from the examples in the Spark distribution (v1.6.2). I added my own "Hello World" example and that worked fine. Trying to add something that uses a 3rd party dependency (com.google.cloud:gcloud-java-nio:0.2.5), here is what happens:
16/07/22 13:05:26 ERROR SparkUncaughtExceptionHandler: Uncaught
exception in thread Thread[Executor task launch worker-7,5,main]
java.lang.NoClassDefFoundError:
org/spark-project/guava/base/MoreObjects at
com.google.cloud.ServiceOptions.activeGoogleCloudConfig(ServiceOptions.java:282)
at
com.google.cloud.ServiceOptions.googleCloudProjectId(ServiceOptions.java:294)
at
com.google.cloud.ServiceOptions.defaultProject(ServiceOptions.java:270)
at com.google.cloud.ServiceOptions.(ServiceOptions.java:206)
at
com.google.cloud.HttpServiceOptions.(HttpServiceOptions.java:153)
at
com.google.cloud.storage.StorageOptions.(StorageOptions.java:69)
(...)
I ran my code as follows:
spark-1.6.2$ mvn -DskipTests clean package
(lots of time passes...)
spark-1.6.2$ ./bin/run-example JavaGcsTest
And to add the dependency I added those lines to examples/pom.xml:
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>gcloud-java-nio</artifactId>
<version>0.2.5</version>
</dependency>
It looks like the root cause is that both gcloud-java-nio and Spark depend on guava, and perhaps they depend on different versions of it.
I looked at related questions and the answers suggest making a fat jar. I'm not sure how to apply this here, though, as examples are already bundled into a fat jar (examples/target/scala-2.10/spark-examples-1.6.2-hadoop2.2.0.jar).
I tried changing the version of guava that was used, raising it from 14 to 19 (the latest), but of course then the compilation failed (SparkEnv.scala:84: method softValues in class MapMaker cannot be accessed in com.google.common.collect.MapMaker).
Hopefully someone has advice on how to get Spark to work with this 3rd party library!
A way to get out of this problem is to compile a shaded version of the third-party library and use that jar as the dependency.
In the case of gcloud-java-nio the project already includes a shaded jar target so after running mvn package it's in target/gcloud-java-nio-0.2.7-SNAPSHOT-shaded.jar.
Has someone tried to use MessagePack with an Android app?
Is it possible? I have tried to use the Jar from msgpack-java and received the following Exception:
Caused by: java.lang.ExceptionInInitializerError
at org.msgpack.Packer.pack(Packer.java:532)
at org.msgpack.MessagePack.pack(MessagePack.java:31)
... 15 more
Caused by: java.lang.ExceptionInInitializerError
at org.msgpack.template.TemplateRegistry.<clinit>(TemplateRegistry.java:38)
... 17 more
Caused by: java.lang.VerifyError: org.msgpack.template.BeansFieldEntryReader
at org.msgpack.template.builder.BeansTemplateBuilder.<init (BeansTemplateBuilder.java:42)
at org.msgpack.template.builder.BuilderSelectorRegistry.initForJava(BuilderSelectorRegistry.java:73)
at org.msgpack.template.builder.BuilderSelectorRegistry.<clinit>(BuilderSelectorRegistry.java:38)
... 18 more
The code that I use is very simple
PrintWriter out = new PrintWriter(socket.getOutputStream());
Message msg = new Message();
msg.body = "asdasdasd";
msg.from = "qwe";
msg.to = "ttt";
byte[] bytes = MessagePack.pack(msg);
out.print(bytes);
out.flush();
I have javassist.jar, msgpack-0.5.2.jar, slf4j-api-1.6.2.jar and slf4j-jdk14-1.6.2.jar in my lib directory.
In my server application this code works fine with the same libraries.
(Hopefully) FINAL UPDATE
msgpack : 0.6.8 works on Android without any problems
msgpack-rpc : 0.7.0 works on Android with one caveat.
Specifically, you need to add the following to onCreate for API Level 8 (Android 2.2.1), and possibly lower:
java.lang.System.setProperty("java.net.preferIPv4Stack", "true");
java.lang.System.setProperty("java.net.preferIPv6Addresses", "false");
due to this bug.
If you want to see a simple example, here's a pair of projects set up for this purpose:
https://github.com/mikkoz/msgpack-android-test-server/tree/master/msgpack-android-test-server
https://github.com/mikkoz/msgpack-android-test-client/tree/master/msgpack-android-test-client
Previous Versions
UPDATE: as of 0.6.7 msgpack should be compatible with Android (there is a small dependency exclusion issue). Check the text below for msgpack-rpc (which also might be adapted in the future).
NOTE: If you're also using msgpack-rpc, you need to do the following steps:
Download the msgpack-rpc source from git://github.com/msgpack/msgpack-rpc.git (specifically, the "java" folder).
Change the main msgpack artifact version to the one you've built.
In org.msgpack.rpc.loop.netty.NettyEventLoop, change the NioClientSocketChannelFactory to OioClientSocketChannelFactory(getWorkerExecutor()).
Build the MessagePack-RPC in the same way as in the case of the main MessagePack JAR (see Step 11 above).
The NettyEventLoop replacement is due to this issue:
http://markmail.org/message/ypa3nrr64kzsyfsa .
Important: I've only tested synchronous communication. Asynchronous might not work.
And here's the reason for msgpack not working with Android prior to 0.6.7:
The reason for the error is that MessagePack uses several java.beans classes that are not included in the Android SDK. You're probably using the MessagePackBeans annotation.
This is a similar problem to the one described here, for which the general solution is outlined here. Unfortunately, in our case it requires a rebuild of msgpack. Here's what I did (you can almost certainly skip Steps 5 and 8, but I haven't tried it that way) :
Download the MessagePack source from https://github.com/msgpack/msgpack-java.git.
Import the MessagePack source as a project in your IDE.
Download the Apache Harmony source for the relevant packages from http://svn.apache.org/repos/asf/harmony/enhanced/java/trunk/classlib/modules/beans/src/main/java .
Copy these packages into your MessagePack project's src/main/java folder:
java.beans
java.beans.beancontext
org.apache.harmony.beans
org.apache.harmony.beans.internal.nls
In your MessagePack project, remove the following classes:
PropertyChangeListener
IndexedPropertyChangeEvent
PropertyChangeEvent
PropertyChangeListenerProxy
PropertyChangeSupport
Rename the java.beans packages to something different, e.g. custom.beans .
Change all java.beans references to the renamed ID, so again e.g. custom.beans. This applies especially to BeansFieldEntryReader (this class is the reason for the original error).
Change the custom.beans references for the five classes you removed in Step 5 back to java.beans.
In the org.apache.harmony.beans.internal.nls.Messages class, comment out the method setLocale, and remove the imports associated with it.
Remove all classes that still have errors, except Encoder. In that class, comment out all references to the classes you've removed. You should now have an error-free project.
Build the MessagePack JAR:
If you're using Maven, change the version in the pom.xml to something unique, run Maven build with the install goal, then add the dependency in your Android project with that version.
If you're not using Maven, you have to run the jar goal for Ant with the included build.xml. Replace the msgpack JAR in your Android project with this one.
If you're publishing your app, remember to include the relevant legal notice for Apache Harmony. It's an Apache License, just like MessagePack.
That should do it. Using your example code, and my own data class, I was successfully able to pack and unpack data.
The entire renaming ritual is due to the fact that the DEX compiler complains about java.* package naming.
There is a critical msgpack bug saying data packed with msgpack will get corrupted on the Dalvik VM. http://jira.msgpack.org/browse/MSGPACK-51
There is an ongoing effort by #TheTerribleSwiftTomato and the MessagePack core team to get MessagePack working on Android, please see the related GitHub issue. The fix mentioned in #TheTerribleSwiftTomato's answer is to be found here.
Update
I've managed to get it at least running on Android by (painstakingly) adding all the necessary javassist Classes which are currently required for the build to succeed. An extra 600KB gain in size, yet at least it seems to work. All in all, it appears to be working to some extent on Android, eventually check out the lesser-known resources about Message Pack such as its User Group and its Wiki for more information.
On a side-note, be sure to use a HTTP Request Library (such as LoopJ's Android Async HTTP or Apache's HttpClient) which can handle binary data.
Last but not least you can ping me if there is interest in this jar which makes MessagePack seemingly work on Android – credits go out of course to #TheTerribleSwiftTomato who supplied the fix above!
I suggest you write this in the main proguard-rules file-
-dontwarn org.msgpack.**
-keep class org.msgpack.** { *; }