We Keep Coding

neo4j procedure depending on signed jar (mapdb)

I have created a neo4j user-defined procedure. It also compiles and works in neo4j.
Recently though, I added a dependency to my procedure plugin that prevents neo4j from starting when I try to run the newly built jar. Concretely, I receive following exception at the bottom of the stack trace:
Caused by: java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
The only thing I changed is to add MapDB to my dependencies. So I suspect that it depends on some signed artifact. As it turns out neo4j plugins are "uber-jars" (i.e., shaded jars) which doesn't work very well with signed dependencies.
I figured I could try to exclude MapDB from the shading by changing the scope to provided and additionally adding the mapdb jar to plugins folder of neo4j. So the plugins folder of neo4j now includes both mapdb.jar and myprocedure.jar.
This doesn't seem to work though: neo4j starts, but when calling my procedure I receive a ClassNotFound Exception.
Any ideas on how I can solve this dilemma? I really depend on something like MapDB as my graph is very large and keeping everything in my procedure in-memory regularly leads to memory exceptions.
Many thanks in advance!
The important part of my pom.xml should it help (I started off with the procedure template so it still looks quite similar):
<dependencies>
<!-- other dependencies that don't make a difference ... -->
<dependency>
<groupId>org.mapdb</groupId>
<artifactId>mapdb</artifactId>
<version>3.0.7</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j</artifactId>
<version>${neo4j.version}</version>
<scope>provided</scope>
</dependency>
<!-- Test Dependencies -->
<dependency>
<groupId>org.neo4j.test</groupId>
<artifactId>neo4j-harness</artifactId>
<version>${neo4j.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.neo4j.driver</groupId>
<artifactId>neo4j-java-driver</artifactId>
<version>1.4.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<!-- Neo4j Procedures require Java 8 -->
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<!-- This generates a jar-file with our procedure code,
plus any dependencies marked as `compile` scope.
This should then be deployed in the `plugins` directory
of each Neo4j instance in your deployment.
After a restart, the procedure is available for calling. -->
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

So, one solution that worked was to use the maven-assembly-plugin instead of the maven-shade-plugin. The assembly plugin creates a standalone jar with all dependencies without re-packaging all dependencies.
Of course, I had to remove the <scope>provided</scope> for mapdb so it is also included in the assembled jar.
The rest is just replacing the shade plugin with following snippet (thanks to this question here):
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
Afterwards I received another exception which I thought was due to a wrong signature:
Caused by: java.lang.VerifyError ...
Turns out this was just because I had another procedure in neo4j that uses the same dependencies as mapdb. Quick fix was to remove that procedure library.
Of course, there are also other solutions, including removing the signature, or re-signing it. However, I explicitly did not want to remove any signatures.

I'm not aware that neo4j is checking any jar signatures by default. I suspect the repackaging of mapdb artifact being the culprit.
To validate that, remove the shade or assembly plugin in and build a jar that solely contains your code. Copy that jar and the mapdb.jar into the plugin folder and observe logs during a restart.

Query DSL & Maven: Classes not generated, but only on command line (in eclipse it is working fine)

We are using a setup with Spring Boot, Hibernate, Query DSL and Maven with Java 1.8
Recently, I've added Query DSL to the project with the configuration listed below. To make it work, I had to configure the Java Compiler in the eclipse project settings to allow Annotation Processing and also add the Query DSL .jar file to the eclipse Annotation Factory Path.
This setup worked as expected. It generated the custom Q classes and I could use them in my code. When now running the mvn clean install on the command line, every class in my code throws the error cannot find symbol, because the class is missing. Is there anything else I need to configure - similar to the .jar file in the eclipse settings - to make the build process work?
EDIT: This question is not a duplicate of this question because I did not ask why this error (cannot find a symbol) occurs but rather how to configure QueryDSL to also work on the command line.
EDIT2: I have now tried to integrate the build-helper-maven-plugin to use multiple source paths as an input. This did not help either. I also tried to generate the files into a src folder. It did not help either.
When I first compile the library in eclipse, the mvn compile goes through on the command line, but mvn clean compile still fails, because it just uses the compiled files of eclipse again. The apt-maven-plugin is executed, which can be seen just before the build process fails:
[INFO] --- apt-maven-plugin:1.1.3:process (default) # project1 ---
[INFO]
[INFO] --- build-helper-maven-plugin:1.9.1:add-source (add-source) # project1 ---
[INFO] Source directory: C:\Users\user1\git\project1\src\main\generated added.
[INFO]
[INFO] --- maven-processor-plugin:2.2.4:process (process) # project1 ---
[ERROR] diagnostic: [...]
EDIT3: When I remove every import statement which is referring to the Q classes, the build process goes through (obviously). It is, however, remarkable, that the Q classes get compiled correctly in that case. They appear in the target folder as .class files as they should. Could it be, that the Q classes are compiled too late?
Here is an excerpt of the pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
[...]
<prerequisites>
<maven>3.0.0</maven>
</prerequisites>
<dependencies>
[...]
<dependency>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-apt</artifactId>
<version>4.1.3</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-jpa</artifactId>
<version>4.1.3</version>
</dependency>
</dependencies>
<build>
<defaultGoal>spring-boot:run</defaultGoal>
<plugins>
[...]
<plugin>
<groupId>com.mysema.maven</groupId>
<artifactId>apt-maven-plugin</artifactId>
<version>1.1.3</version>
<executions>
<execution>
<goals>
<goal>process</goal>
</goals>
<configuration>
<outputDirectory>target/generated-sources/java</outputDirectory>
<processor>com.querydsl.apt.jpa.JPAAnnotationProcessor</processor>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
[...]
</build>
</project>
This is the configuration of the eclipse project settings:
This is the error message which is displayed in the console:
[INFO] --- maven-processor-plugin:2.2.4:process (process) # project1 ---
[ERROR] diagnostic: C:\Users\user1\git\project1\src\main\java\com\project1\repository\UserRepositoryImpl.java:3: error: cannot find symbol
import static com.project1.domain.QUser.user;
^
symbol: class QUser
location: package com.project1.domain
[ERROR] diagnostic: C:\Users\user1\git\project1\src\main\java\com\project1\repository\UserRepositoryImpl.java:3: error: static import only from classes and interfaces
import static com.project.domain.QUser.user;
^

This is old question but this is how i find my solution, added classifier for jpa dependency:
<!-- BEGIN: 'querydsl-jpa' -->
<dependency>
<groupId>com.mysema.querydsl</groupId>
<artifactId>querydsl-jpa</artifactId>
<version>${querydsl-jpa.version}</version>
<classifier>apt</classifier>
</dependency>
<!-- END: 'querydsl-jpa' -->
My complete pom:
<!-- BEGIN: BUILD -->
<build>
<!-- BEGIN: PLUGINS -->
<plugins>
<!-- BEGIN: apt-maven-plugin -->
<plugin>
<groupId>com.mysema.maven</groupId>
<artifactId>apt-maven-plugin</artifactId>
<version>${apt.version}</version>
<executions>
<execution>
<goals>
<goal>process</goal>
</goals>
<configuration>
<outputDirectory>target/generated-sources/apt</outputDirectory>
<processor>com.mysema.query.apt.jpa.JPAAnnotationProcessor</processor>
</configuration>
</execution>
</executions>
</plugin>
<!-- END: apt-maven-plugin -->
</plugins>
<!-- END: PLUGINS -->
</build>
<!-- END: BUILD -->
<!-- BEGIN: DEPENDENCIES -->
<dependencies>
<!-- *********************************************** -->
<!-- BEGIN: 'QUERYDSL DEPENDENCIES' -->
<!-- BEGIN: 'querydsl-apt' -->
<dependency>
<groupId>com.mysema.querydsl</groupId>
<artifactId>querydsl-apt</artifactId>
<version>${querydsl-apt.version}</version>
</dependency>
<!-- END: 'querydsl-apt' -->
<!-- BEGIN: 'querydsl-jpa' -->
<dependency>
<groupId>com.mysema.querydsl</groupId>
<artifactId>querydsl-jpa</artifactId>
<version>${querydsl-jpa.version}</version>
<classifier>apt</classifier>
</dependency>
<!-- END: 'querydsl-jpa' -->
<!-- *********************************************** -->
<!-- END: 'QUERYDSL DEPENDENCIES' -->
</dependencies>

For me, it didn't work because it conflicted with maven-compiler-plugin with already set annotation processor. Just deleted the use of apt-maven-plugin and added its annotation processor in maven-compiler-plugin.
<build>
<plugins>
<!-- related to issues:-->
<!-- - https://github.com/querydsl/querydsl/issues/2654 -->
<!-- - https://github.com/querydsl/querydsl/issues/2242 -->
<!-- Using apt-maven-plugin conflicts with other annotation processors (like mapStruct) -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<annotationProcessorPaths>
<path>
<groupId>org.mapstruct</groupId>
<artifactId>mapstruct-processor</artifactId>
<version>${org.mapstruct.version}</version>
</path>
<path>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-apt</artifactId>
<version>5.0.0</version>
<classifier>jpa</classifier>
</path>
<path>
<groupId>jakarta.persistence</groupId>
<artifactId>jakarta.persistence-api</artifactId>
<version>2.2.3</version>
</path>
<path>
<groupId>javax.annotation</groupId>
<artifactId>javax.annotation-api</artifactId>
<version>1.3.2</version>
</path>
<!-- other annotation processors -->
</annotationProcessorPaths>
</configuration>
</plugin>
<!-- <plugin>-->
<!-- <groupId>com.mysema.maven</groupId>-->
<!-- <artifactId>apt-maven-plugin</artifactId>-->
<!-- <version>1.1.3</version>-->
<!-- <executions>-->
<!-- <execution>-->
<!-- <goals>-->
<!-- <goal>process</goal>-->
<!-- </goals>-->
<!-- <configuration>-->
<!-- <outputDirectory>target/generated-sources</outputDirectory>-->
<!-- <processor>com.mysema.query.apt.jpa.JPAAnnotationProcessor</processor>-->
<!-- </configuration>-->
<!-- </execution>-->
<!-- </executions>-->
<!-- </plugin>-->
</plugins>
</build>
But there is an issue with using querydsl annotation processor in maven-compiler-plugin. You have to add jakarta.persistence-api and javax.annotation-api.

I would rather use profile to generate these Qclasses only when db change occurs.
cons:
-your diff in pull requests is clean when you don't change db schema because for each generation these files tend to generate differently for some reason (atleast in my case).
-you can manage witch of tables present in your db will have Qclasses (sometimes it is a pain when you forget to regenerate them after changing db schema)
-well not that it is lots of time . but builds are faster if You don't change schema and profile is turned off.
Try something like this and turn on profile when You want to generate changed schema Qclasses :
<properties>
<whitelisted.tables>
user_accunt,
other
</whitelisted.tables>
</properties>
<profiles>
<profile>
<id>generate</id>
<build>
<plugins>
<plugin>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-maven-plugin</artifactId>
<version>${querydsl.version}</version>
<executions>
<execution>
<goals>
<goal>export</goal>
</goals>
</execution>
</executions>
<configuration>
<jdbcDriver>org.postgresql.Driver</jdbcDriver>
<jdbcUrl>jdbc:postgresql://localhost:port/dbname</jdbcUrl>
<packageName>your.package.name.for.q</packageName>
<jdbcUser>dbusername</jdbcUser>
<jdbcPassword>dbpassword</jdbcPassword>
<targetFolder>${project.basedir}/src/main/java/</targetFolder>
<spatial>true</spatial>
<tableNamePattern>${whitelisted.tables}</tableNamePattern>
</configuration>
<dependencies>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.1.1</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</profile>
</profiles>

The generated-source directory are not automatically included in the jar.
You need to use the Maven build helper plugin to fix this issue, for example:
https://github.com/alexec/javahelp-skeleton/blob/master/pom.xml

You can try out few things:
1.try to put <clearOutputDir>false</clearOutputDir> in your configuration tag
2. Sometimes classes might not be getting generated before the compile phase. So try to put phase in your plugin
<execution>
<phase>generate-sources</phase>
<goals>
<goal>...</goal>
</goals>
</execution>

By convention Maven assumes all source code is in 'src/main/java', compiles this and put all *.class file in target.
So if you have a class 'Alien.java' in <project-root>/alice/in/wonderland, your won't be able to access it (in src/main/java) because maven puts everything from src/main/java in classpath for your compiler and hence compiler is unaware of any source code (*.java) anywhere else.
In your case you are generating your source code in directory target/generated-sources/java, so you will have to tell maven about it. As mentioned in some other answers you may use build-helper-plugin for this, let maven know that your source resides on target/generated-sources/java by
<project>
...
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>target/generated-sources/java</source>
<source>alice/in/wonderland</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Edit: You have mentioned wrong path in build-helper-plugin
I see although you are using build helper plugin but you are using wrong path
--- build-helper-maven-plugin:1.9.1:add-source (add-source) # project1 ---
[INFO] Source directory: C:\Users\user1\git\project1\src\main\generated added.
Instead of src\main\generated you should use <source>target/generated-sources/java</source>

Lazerycode jmeter maven plugin Java Extension downloads excluded Dependencies

I am trying to use Java samplers in my tests.
I have a separate maven project where I create my extensions. After building the project I get a .jar lib. I include it in my maven plugin like this:
<dependencies>
<dependency>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>com.qiagen</groupId>
<artifactId>qa_toolkit</artifactId>
<version>RELEASE</version>
</dependency>
<dependency>
<groupId>com.qiagen</groupId>
<artifactId>JMeterExtensions</artifactId>
<version>jmeter3.2.3</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>2.2.0</version>
<executions>
<execution>
<id>jmeter-tests</id>
<phase>verify</phase>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
</executions>
<configuration>
<testFilesDirectory>${basedir}/src/test/jmeter/</testFilesDirectory>
<testFilesIncluded>
<jMeterTestFile>${jmxTest}</jMeterTestFile>
</testFilesIncluded>
<jmeterDirectory>${jmeter.home}</jmeterDirectory>
<jmeterExtensions>
<artifact>com.qiagen:JMeterExtensions:jmeter3.2.3</artifact>
</jmeterExtensions>
<propertiesUser>
<csvData>${basedir}/src/test/jmeter/${csvData}</csvData>
<threads>${threads}</threads>
<rampTime>${rampTime}</rampTime>
<loopCount>${loopCount}</loopCount>
<options>${options}</options>
<server>${server}</server>
<port>${port}</port>
<sleep>${sleep}</sleep>
<inputXmlFileDir>${inputXmlFileDir}</inputXmlFileDir>
<templatesCsv>${templatesCsv}</templatesCsv>
<xmlInputsCsv>${xmlInputsCsv}</xmlInputsCsv>
<reportScenariosCsv>${reportScenariosCsv}</reportScenariosCsv>
</propertiesUser>
<jMeterProcessJVMSettings>
<xms>2048</xms>
<xmx>2048</xmx>
<arguments>
<argument>-Xprof</argument>
<argument>-Xfuture</argument>
</arguments>
</jMeterProcessJVMSettings>
</configuration>
</plugin>
</plugins>
</build>
In my extensions i have some invalid transitive dependencies which i excluded from extensions pom.xml. I don't see them in the dependency tree.
When I run the tests, with the flag downloadExtensionDependencies on true, it looks like it tries to download all dependencies (also those excluded) and then the test fails because of that invalid dependency.
Failed to collect dependencies at org.springframework:spring-webmvc:jar:3.1.1.RELEASE -> jasperreports:jasperreports:jar:2.0.5 -> commons-collections:commons-collections:jar:3.2.1.redhat-7: Failed to read artifact descriptor for commons-collections:commons-collections:jar:3.2.1.redhat-7: Could not transfer artifact org.apache.commons:commons-parent:pom:22-redhat-2 from/to jaspersoft (http://www.jasperforge.org/maven2): www.jasperforge.org: Unknown host www.jasperforge.org -> [Help 1]
Do you have any ideas why is the plugin trying to download the excluded dependencies also?

Use version 2.6.0 of the plugin which has now better default values like not downloading optional dependencies.
And use this to exclude broken or excluded dependencies:
<excludedArtifacts>
<exclusion>commons-pool2:commons-pool2</exclusion>
<exclusion>commons-math3:commons-math3</exclusion>
<exclusion>com.sun.jdmk:jmxtools</exclusion>
<exclusion>com.sun.jmx:jmxri</exclusion>
</excludedArtifacts>

GWT project deployment to tomcat in STS(Eclipse)

I have a maven project in STS(Eclipse) and I try deploying it to Tomcat 6 in STS. For maven install I use gwt-maven-plugin. Installed war contains all folders (WEB-INF/, META-INF/, compiled GWT frontend folder). When I deploy this project to tomcat (in STS), there are only WEB-INF and META-INF folders in webapps/project folder. Folder with compiled GWT frontend is nowhere to find. In context.xml I even tried to set docBase to installed war, but no difference in result. Have anyone an idea where problem could be?, because now I have to manually copy this folder to unpacked folder in webapps. Thanks alot.

Might be a configuration problem, post your gwt-maven-plugin xml here.
Here's mine, just for reference
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>gwt-maven-plugin</artifactId>
<version>${gwt.version}</version>
<configuration>
<logLevel>DEBUG</logLevel>
<style>PRETTY</style>
<runTarget>/ApplicationScaffold.html</runTarget>
<hostedWebapp>${project.build.directory}/${project.build.finalName}</hostedWebapp>
<!-- <modules><module>${project.groupId}.Main</module></modules>-->
<copyWebapp>true</copyWebapp>
<debugPort>8001</debugPort>
<extraJvmArgs>-Xmx900m</extraJvmArgs>
<!-- instruct plugin not to require open browser in test mode -->
<mode>htmlunit</mode>
<!-- compiler speed up -->
<draftCompile>true</draftCompile>
<optimizationLevel>0</optimizationLevel>
<disableAggressiveOptimization>true</disableAggressiveOptimization>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>test</goal>
</goals>
</execution>
</executions>
<dependencies>
<!-- must override the plugin's default dependencies -->
<dependency>
<groupId>com.google.gwt</groupId>
<artifactId>gwt-user</artifactId>
<version>${gwt.version}</version>
</dependency>
<dependency>
<groupId>com.google.gwt</groupId>
<artifactId>gwt-dev</artifactId>
<version>${gwt.version}</version>
</dependency>
</dependencies>
</plugin>

Maven: How do I get .so libraries bundled in package

I have a third party library coming with .jar and .so file.
I configured pom.xml as following:
<dependency>
<groupId>com.abc.def</groupId>
<artifactId>sdc</artifactId>
<version>1.1</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>com.abc.def</groupId>
<artifactId>sdc</artifactId>
<version>3</version>
<scope>compile</scope>
<type>so</type>
</dependency>
With this configure, I successfully tested through Intellij and apk file under target contains structure like lib/armeabi/sdc.so
However, after I do mvn clean package, the apk file generated did not contain sdc.so file, and after installing apk file on android device, lib folder is empty.
Searching through the internet, and did not find answer.
Btw, I do add <nativeLibrariesDirectory>${project.basedir}/libs</nativeLibrariesDirectory> into pluginManagement as mentioned Native libraries (.so files) are not added to an android project, but does not help.
Updates:
If someone is facing same problem as I am that SO file did not copy over, please try manually copy the so file over as following:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.1</version>
<executions>
<execution>
<id>copy</id>
<phase>prepare-package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>com.abc.def</groupId>
<artifactId>libdef</artifactId>
<version>2.1.3</version>
<type>so</type>
<destFileName>libdef.so</destFileName>
</artifactItem>
</artifactItems>
<outputDirectory>${project.build.directory}/libs/armeabi</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

I suggest you to use maven-android-plugin version 2.8.3 or more.
The scope of your native lib must be runtime (not sure it is required, but anyway it's a fact)
The artifactId must start with lib (i.e. it must be libsdc)
<dependency>
<groupId>com.abc.def</groupId>
<artifactId>libsdc</artifactId>
<version>3</version>
<scope>runtime</scope>
<type>so</type>
</dependency>
The artifactId will be the library name so load it with this line of code
System.loadLibrary("sdc");
reference
Note: I don't know if sdc if the real artifactId, but if it's the case you must consider re-publishing it with the lib prefix.

I would suggest looking into the Assembly plugin: http://maven.apache.org/plugins/maven-assembly-plugin/ I have not used it in any Android project yet, but I do use it in my regular java server projects which require non-maven items be in expected locations.