I have created a neo4j user-defined procedure. It also compiles and works in neo4j.
Recently though, I added a dependency to my procedure plugin that prevents neo4j from starting when I try to run the newly built jar. Concretely, I receive following exception at the bottom of the stack trace:
Caused by: java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
The only thing I changed is to add MapDB to my dependencies. So I suspect that it depends on some signed artifact. As it turns out neo4j plugins are "uber-jars" (i.e., shaded jars) which doesn't work very well with signed dependencies.
I figured I could try to exclude MapDB from the shading by changing the scope to provided and additionally adding the mapdb jar to plugins folder of neo4j. So the plugins folder of neo4j now includes both mapdb.jar and myprocedure.jar.
This doesn't seem to work though: neo4j starts, but when calling my procedure I receive a ClassNotFound Exception.
Any ideas on how I can solve this dilemma? I really depend on something like MapDB as my graph is very large and keeping everything in my procedure in-memory regularly leads to memory exceptions.
Many thanks in advance!
The important part of my pom.xml should it help (I started off with the procedure template so it still looks quite similar):
<dependencies>
<!-- other dependencies that don't make a difference ... -->
<dependency>
<groupId>org.mapdb</groupId>
<artifactId>mapdb</artifactId>
<version>3.0.7</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j</artifactId>
<version>${neo4j.version}</version>
<scope>provided</scope>
</dependency>
<!-- Test Dependencies -->
<dependency>
<groupId>org.neo4j.test</groupId>
<artifactId>neo4j-harness</artifactId>
<version>${neo4j.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.neo4j.driver</groupId>
<artifactId>neo4j-java-driver</artifactId>
<version>1.4.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<!-- Neo4j Procedures require Java 8 -->
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<!-- This generates a jar-file with our procedure code,
plus any dependencies marked as `compile` scope.
This should then be deployed in the `plugins` directory
of each Neo4j instance in your deployment.
After a restart, the procedure is available for calling. -->
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
So, one solution that worked was to use the maven-assembly-plugin instead of the maven-shade-plugin. The assembly plugin creates a standalone jar with all dependencies without re-packaging all dependencies.
Of course, I had to remove the <scope>provided</scope> for mapdb so it is also included in the assembled jar.
The rest is just replacing the shade plugin with following snippet (thanks to this question here):
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
Afterwards I received another exception which I thought was due to a wrong signature:
Caused by: java.lang.VerifyError ...
Turns out this was just because I had another procedure in neo4j that uses the same dependencies as mapdb. Quick fix was to remove that procedure library.
Of course, there are also other solutions, including removing the signature, or re-signing it. However, I explicitly did not want to remove any signatures.
I'm not aware that neo4j is checking any jar signatures by default. I suspect the repackaging of mapdb artifact being the culprit.
To validate that, remove the shade or assembly plugin in and build a jar that solely contains your code. Copy that jar and the mapdb.jar into the plugin folder and observe logs during a restart.
Related
I've been building a Java application, which runs as expected when executing it locally. Now I'm trying the next step, to move it into an external platform. I chose Heroku because of their free plans for a starter, and also because it lets me have a small PostGreSQL database, which I use along with the application.
Due to the way Heroku works, I was forced to integrate Java with Maven, which I know little to none about. But I was still able to work my way there until I was able to compile and deploy the application (source code is in GitHub), albeit with errors.
For once, I'm using a couple external libraries:
jsoup-1.13.1.jar – Jsoup, for easier parsing of HTML files
mail.jar – Java mail, for emailing reports
Whenever the application is deployed in Heroku, there's an Exception in thread "main" java.lang.NoClassDefFoundError: org/jsoup/Jsoup, ultimately triggered by the Jsoup.connect invocation. From what I understand, Maven only add the references to the program but does not include the actual libraries there for runtime, which I think may be the cause of the issue.
I've tried modifying my pom.xml file by adding these dependencies manually:
<dependencies>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.2.18</version>
</dependency>
<dependency>
<groupId>com.sun.mail</groupId>
<artifactId>javax.mail</artifactId>
<version>1.6.2</version>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.13.1</version>
</dependency>
</dependencies>
And also these plugins in the build section:
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<finalName>checker</finalName>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.4</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
</execution>
</executions>
</plugin>
What am I doing wrong? I would be very grateful for hints to help me get my application running, at least resolving the reason why Jsoup isn't working at all in Heroku. Thanks in advance!
I actually found a company (where I work) specific work-around this issue, but the problem is so bizarre, I wanted to see if anyone else has ran into this issue and how they solved it.
First I added two bouncy castle dependencies to my spark job:
<!-- https://mvnrepository.com/artifact/org.bouncycastle/bcprov-jdk15on -->
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
<version>1.52</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.bouncycastle/bcpg-jdk15on -->
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpg-jdk15on</artifactId>
<version>1.52</version>
</dependency>
I also have this plugin to create a jar with all dependencies:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4.1</version>
<configuration>
<!-- get all project dependencies -->
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<!-- MainClass in mainfest make a executable jar -->
<archive>
<manifest>
<mainClass>com.theclasspath.jobs.TestingJob</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<!-- bind to the packaging phase -->
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
So all the dependencies are getting installed. Now this will compile and create the jar. When I run spark-submit with the jar with dependencies, I get a no such method error for some methods in the org.bouncycastle.util package. The main one being:
java.lang.NoSuchMethodError: org.bouncycastle.util.Strings.newList()Lorg/bouncycastle/util/StringList;
when trying to just run Strings.newList(); in the main class. So I ran mvn dependency:tree to see if there were multiple versions of bouncy castle, and I only saw the dependencies I added to the pom.
Here is the really weird part: if I build the jar and run it with java -jar jar-name-jar-with-dependencies.jar (meaning not through spark-submit) the dependency gets added just fine, and I do not see any errors. So with that in mind, what would be the differences between the spark submit jar and just running the jar directly?
Why would one work and the other fail? I have tried running it on different machines, such as an emr job, and I still see the same error. (and to avoid this getting asked, I am using the jar-with-dependencies jar for the spark job)
One final note, I have seen this post Bouncy castle no such method error, and it doesn't apply, since I can get the jar to run when I just run the jar by itself without spark-submit.
Is there a specific recommended approach to the inclusion of the spring-boot parent pom into projects that already have a required parent POM?
What do you recommend for projects that need to extend from an organizational parent (this is extremely common and even something many/most projects published to Maven central depending on the feeder repos they come from). Most of the build stuff is related to creating executable JARs (e.g. running embedded Tomcat/Jetty). There are ways to structure things so that you can get all the dependencies without extending from a parent (similar to composition vs. inheritance). You can't get a build stuff that way though.
So is it preferable to include all of the spring-boot parent pom inside of the required parent POM or to simply have a POM dependency within the project POM file.
Other options?
TIA,
Scott
You can use the spring-boot-starter-parent like a "bom" (c.f. Spring and Jersey other projects that support this feature now), and include it only in the dependency management section with scope=import.That way you get a lot of the benefits of using it (i.e. dependency management) without replacing the settings in your actual parent.
The 2 main other things it does are
define a load of properties for quickly setting versions of dependencies that you want to override
configure some plugins with default configuration (principally the Spring Boot maven plugin). So those are the things you will have to do manually if you use your own parent.
Example provided in Spring Boot documentation:
<dependencyManagement>
<dependencies>
<dependency>
<!-- Import dependency management from Spring Boot -->
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>2.1.3.RELEASE</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Update 2022-05-29 with 1.5.9.RELEASE.
I have full code and runable example here https://github.com/surasint/surasint-examples/tree/master/spring-boot-jdbi/9_spring-boot-no-parent (see README.txt to see that you can try)
You need this as a basic
<dependencyManagement>
<dependencies>
<dependency>
<!-- Import dependency management from Spring Boot -->
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${springframework.boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
But that is not enough, you also need explicitly define goal for spring-boot-maven-plugin (If you use Spring Boot as parent, you do not have to explicitly define this)
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>${springframework.boot.version}</version>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
Otherwise you cannot build as executable jar or war.
Not yet, if you are using JSP, you need to have this:
<properties>
<failOnMissingWebXml>false</failOnMissingWebXml>
</properties>
Otherwise, you will get this error message:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.2:war (default-war) on project spring-boot-09: Error assembling WAR: webxml attribute is required (or pre-existing WEB-INF/web.xml if executi
ng in update mode) -> [Help 1]
NO NO , this is still not enough if you are using Maven Profile and Resource Filter with Spring Boot with "#" instead of "${}" (like this example https://www.surasint.com/spring-boot-maven-resource-filter/). Then you need to explicitly add this in
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
And this in
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>2.7</version>
<configuration>
<delimiters>
<delimiter>#</delimiter>
</delimiters>
<useDefaultDelimiters>false</useDefaultDelimiters>
</configuration>
</plugin>
See the example in the link https://www.surasint.com/spring-boot-with-no-parent-example/.
As per Surasin Tancharoen's answer, you may also want to define maven surefire plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
</plugin>
and possibly include fail-fast plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin.version}</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
I'm trying to generate java classes for OGC KML 2.2 as part of the maven generate-sources process using the org.codehaus.mojo xmlbeans-maven-plugin. The java code appears to be generated correctly, but I get tons of errors during compilation complaining that 'package org.apache.xmlbeans'. XMLBeans is clearly a dependency, it exists in my ~/.m2 repository, and I've been peek in the jar to make sure the classes are there. It looks like XMLBeans is successfully generating java files in target/generated-sources, but somehow its absent from the classpath during compilation.
I've tried changing the scope of the org.apache.xmlbeans dependency, but to no avail.
Here's the pom.xml
<modelVersion>4.0.0</modelVersion>
<groupId>net.opengis</groupId>
<artifactId>ogc-kml</artifactId>
<version>2.2.0</version>
<packaging>pom</packaging>
<name>ogc-kml</name>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>xmlbeans-maven-plugin</artifactId>
<version>2.3.3</version>
<executions>
<execution>
<goals>
<goal>xmlbeans</goal>
</goals>
</execution>
</executions>
<inherited>true</inherited>
<configuration>
<download>true</download>
<schemaDirectory>src/main/xsd</schemaDirectory>
</configuration>
</plugin>
</plugins>
</build>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.apache.xmlbeans</groupId>
<artifactId>xmlbeans</artifactId>
<version>2.6.0</version>
</dependency>
</dependencies>
</dependencyManagement>
The project consists of a single src/main/xsd folder containing the two xsds from http://schemas.opengis.net/kml/2.2.0/. The entire folder structure is at https://github.com/iancw/maven-xmlbeans-question.
I can compile the classes by hand if I put the xmlbeans jar from my ~/.m2 repo on the classpath, e.g.
xmlbeans$ javac -classpath ~/.m2/repository/org/apache/xmlbeans/xmlbeans/2.4.0/xmlbeans-2.4.0.jar org/w3/x2005/atom/*.java org/w3/x2005/atom/impl/*.java net/opengis/kml/x22/*.java x0/oasisNamesTcCiqXsdschemaXAL2/*.java x0/oasisNamesTcCiqXsdschemaXAL2/impl/*.java net/opengis/kml/x22/*.java net/opengis/kml/x22/impl/*.java
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
xmlbeans$
I've looked through a number of examples and it seems like I'm doing this right. I haven't seen anyone else complain of this issue. Any maven mavens have suggestions?
(A curious side note is that although i've tried both 2.4.0 and 2.6.0 of the xmlbeans dependency, maven hasn't ever seemed to download the 2.6.0 version into my repository)
From the POM file that you've included in your question you have only defined the xmlbeans dependency in the dependencyManagement section. You also need to define it in your dependencies section of your POM before it will be included in the classpath at build time.
So for example your POM would be:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>xmlbeans-maven-plugin</artifactId>
...
</plugin>
</plugins>
</build>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.apache.xmlbeans</groupId>
<artifactId>xmlbeans</artifactId>
<version>2.6.0</version>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<groupId>org.apache.xmlbeans</groupId>
<artifactId>xmlbeans</artifactId>
</dependencies>
One Additional issue that may look similar,
Check your java install jdk and ext folders for older beans jar.
The plugin puts the project dependencies at the end of the classpath.
I have a third party library coming with .jar and .so file.
I configured pom.xml as following:
<dependency>
<groupId>com.abc.def</groupId>
<artifactId>sdc</artifactId>
<version>1.1</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>com.abc.def</groupId>
<artifactId>sdc</artifactId>
<version>3</version>
<scope>compile</scope>
<type>so</type>
</dependency>
With this configure, I successfully tested through Intellij and apk file under target contains structure like lib/armeabi/sdc.so
However, after I do mvn clean package, the apk file generated did not contain sdc.so file, and after installing apk file on android device, lib folder is empty.
Searching through the internet, and did not find answer.
Btw, I do add <nativeLibrariesDirectory>${project.basedir}/libs</nativeLibrariesDirectory> into pluginManagement as mentioned Native libraries (.so files) are not added to an android project, but does not help.
Updates:
If someone is facing same problem as I am that SO file did not copy over, please try manually copy the so file over as following:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.1</version>
<executions>
<execution>
<id>copy</id>
<phase>prepare-package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>com.abc.def</groupId>
<artifactId>libdef</artifactId>
<version>2.1.3</version>
<type>so</type>
<destFileName>libdef.so</destFileName>
</artifactItem>
</artifactItems>
<outputDirectory>${project.build.directory}/libs/armeabi</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
I suggest you to use maven-android-plugin version 2.8.3 or more.
The scope of your native lib must be runtime (not sure it is required, but anyway it's a fact)
The artifactId must start with lib (i.e. it must be libsdc)
<dependency>
<groupId>com.abc.def</groupId>
<artifactId>libsdc</artifactId>
<version>3</version>
<scope>runtime</scope>
<type>so</type>
</dependency>
The artifactId will be the library name so load it with this line of code
System.loadLibrary("sdc");
reference
Note: I don't know if sdc if the real artifactId, but if it's the case you must consider re-publishing it with the lib prefix.
I would suggest looking into the Assembly plugin: http://maven.apache.org/plugins/maven-assembly-plugin/ I have not used it in any Android project yet, but I do use it in my regular java server projects which require non-maven items be in expected locations.