Integrate Apache Mahout with ElasticSearch in Spring Boot - java

I have a Spring Boot application integrated with ElasticSearch.
When i am trying to add Apache Mahout as a maven dependency, the application is no longer running with this error:
Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: org/apache/lucene/util/Accountable
If i remove the mahout dependency the application is running.
pom.xml:
<dependencies>
<dependency>
<groupId>org.apache.mahout</groupId>
<artifactId>mahout-core</artifactId>
<version>0.9</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
The dependency is resolved by maven, the project compiles with no error but when i start it, the server stops.
Any idea?
Thank you

In order to use the latest versions of Elasticsearc v6.x and Spring-boot, you have to focus on the matrix versions. because while I was trying to integrate Elasticsearch v6.2.2 as dependencies into my spring-boot v1.5.X I faced several errors. thus spring-boot v2.x is the compatible one with ES v6.x
so the following link will help you to know the matrix between versions for your project :
https://www.elastic.co/support/matrix#matrix_compatibility
after doing that , use the last version of apache mahout .

Related

How to add properly Data REST dependency in Spring Boot Maven project?

I try to add Spring Data REST dependency using this dependency:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-rest</artifactId>
</dependency>
After adding this dependency I catch an exception - java.lang.NoSuchMethodError: org.springframework.plugin.core.PluginRegistry.of(Ljava/util/List;)Lorg/springframework/plugin/core/PluginRegistry;
And the Action suggestion: Correct the classpath of your application so that it contains a single, compatible version of org.springframework.plugin.core.PluginRegistry
I tried to add this dependency to fix the problem:
<dependency>
<groupId>org.springframework.plugin</groupId>
<artifactId>spring-plugin-core</artifactId>
<version>2.0.0.RELEASE</version>
</dependency>
But it didn't help me.
How could I fix this problem?
I use Spring Boot version - 2.2.7.RELEASE, Maven version - 3.3.9

Not able to add JPA dependency into spring-boot project

I am trying to add JPA Maven dependency to already created spring-boot project but I am getting following error:
Error: Could not find or load main class com.kame.demo.DemoApplication
When I remove it the error is gone.
pom.xml
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.4.196</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-jasper</artifactId>
<version>8.5.32</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
</dependencies>
application.properties
spring.mvc.view.prefix=/pages/
spring.mvc.view.suffix=.jsp
spring.h2.console.enabled=true
spring.datasource.platform=h2
spring.datasource.url=jdbc:h2:mem:navin
I tried to find answer online but none was solution for me.
Also tried to create > Spring starter project > an there immediately add JPA, Web and H2 but still same error.
I am using STS IDE, is something related to it maybe?
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
Is itself a demo project with an extending SpringBoot class as the main class of your project. But this dependency does the same :
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
So there is high chances that both will clash...
The solution is to import the correct dependency for jpa and not the spring boot starter jpa dependency.
EDIT
This one might do the trick instead :
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-jpa</artifactId>
</dependency>
But I recommend you read the official documents to get started properly : https://docs.spring.io/spring-data/jpa/docs/2.1.0.RC2/reference/html/
I faced the same issue while I tried to add JPA dependency manually in POM.xml after creating Spring boot with Web only.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>
Root cause of the problem:- JPA dependency jar was missing. verify:-
File->Project structure->Modules->Dependencies and you will not find any jar org.springframework.boot:spring-boot-starter-data-jpa:2.4.3 (I am using spring boot 2.4.3 but in your case version can be different)
Solution:-
Step 1
File -> Invalidate Caches / Restart... and choosing the Invalidate and Restart option fixed the issue. can be a stale cache issue
Step 2
Under IntelliJ Idea open POM.xml right click-> Maven-> Reload project.
Maven will download the JPA dependency jar and will add into the project. Verify if JPA jar is added or not.
Step 3
Selecting File -> Invalidate Caches / Restart... and choosing the Invalidate and Restart option fixed the issue.
NB:- If you are not using ItellIj id then please force Maven to download dependency (for step2)

Maven Spring Boot 2.0.4 - and elastic search 6.2.2 transport jar not in dependency tree

I am writing a SpringBoot 2.0.4 application, (Executable .jar) and I am trying to interface with an elastic search that I do not control. It is 6.2.2
Now I know Spring does not support that, and that's fine as I am not using any Spring classes, I am interfacing with elastic directly. However, The problem comes in trying to exclude elastic from spring in my pom. Here is my pom fragment:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-elasticsearch</artifactId>
</exclusion>
<exclusion>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>6.2.2</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>6.2.2</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>6.2.2</version>
</dependency>
Now Eclipse has a yellow line under the versions for elasticsearch and transport, saying that the version is being overridden. But when I look at the dependency tree, I see all the elastic jars with the correct version of 6.2.2 -- EXCEPT for transport. That jar is not there at all. I looked in my executable jar, and it's definitely not there. Shockingly, my app throws a class not found error for org/elasticsearch/common/transport/InetSocketTransportAddress
How do I get that jar into my dependencies?
EDIT
So I added:
<properties>
<elasticsearch.version>6.2.2</elasticsearch.version>
</properties>
to my pom, and overrode the spring version, but the transport jar is not showing up in my dependency tree! How do I get this file into my executable jar?

Maven - Unable to resolve dependency conflict b/w google-vision beta & aws-sdk sub-components

I'm trying to use google-vision to fetch text from an image (uploaded to AWS S3) and store it in AWS Dynamo DB. I'm encountering dependency conflicts on jackson-core as both google-api and aws-java-sdk are using two different versions.
Dependency Hierarchy
google-api-client: 1.22.0 uses jackson-core: 2.1.3
google-cloud-vision: 0.22.0-beta uses jackson-core: 2.1.3
aws-java-sdk: 1.11.106 uses jackson-core: 2.6.6
I tried "exclusions" and added explicit dependency in pom.xml to use jackson-core: 2.6.6. Google-vision api works fine with that change. However, AmazonDynamoDBClientBuilder fails with below error:
Exception in thread "main" java.lang.IllegalAccessError: tried to access method com.amazonaws.AmazonWebServiceClient.<init>(Lcom/amazonaws/client/AwsSyncClientParams;)V from class com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder.build(AmazonDynamoDBClientBuilder.java:60)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder.build(AmazonDynamoDBClientBuilder.java:26)
at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at com.oneglint.ImageProcessing.AddItem.main(AddItem.java:133)
Following error is displayed when there was version conflict
Exception in thread "main" java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
at com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:537)
at com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:448)
at com.amazonaws.partitions.PartitionsLoader.<clinit>(PartitionsLoader.java:51)
at com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30)
at com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:64)
at com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:52)
at com.amazonaws.regions.RegionUtils.getRegion(RegionUtils.java:105)
at com.amazonaws.client.builder.AwsClientBuilder.withRegion(AwsClientBuilder.java:239)
at com.oneglint.ImageProcessing.AddItem.main(AddItem.java:132)
What am I missing here? Thanks for the help..
BTW, I'm using example code from github to achieve this. Here are the links:
DynamoDB example: https://github.com/awsdocs/aws-doc-sdk-examples/tree/master/java/example_code/dynamodb
Google Vision DetectText example: https://github.com/GoogleCloudPlatform/java-docs-samples/blob/master/vision/cloud-client/src/main/java/com/example/vision/Detect.java
Additional Details
Both the examples are working fine if executed as independent projects. The problem occurs ONLY when both PutItem (AWS) & Detect (google-vision) classes are brought together in a single project, with appropriate code changes.
You can only have one version of jackson-core in your project. The easiest way to fix a version is to use <dependencyManagement> to set a version.
Your main problem is that jackson-core: 2.6.6 is not compatible with AmazonDynamoDBClientBuilder. The usual strategy is to try all versions from 2.1.3 to 2.6.6 until one of them works. If not, you can try to find versions of your amazon and google jars that require the same Jackson-core-version. In any case, this stupid and boring try-and-error.
If you do not come to any working solution, you can try to shade classes with maven-shade-plugin (I have not tried this, probably difficult) or you need to change your project in a way that not both dependencies are required.
After a lot of trial and error approach, the issue is finally solved.
It appears that I added multiple versions of aws-java-sdk jars during the process and an opennlp jar was associated with the project for some other module.
I had removed conflicting versions of aws-java-sdk and unnecessary libraries. Also, removed the exclusions and retained only the dependency addition in <dependencymanagement> for jackson-core.
Dependencies in my final pom listed below:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-bom</artifactId>
<version>1.11.106</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.6.6</version>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-vision</artifactId>
<version>v1-rev358-1.22.0</version>
</dependency>
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.22.0</version>
<exclusions>
<exclusion> <!-- exclude an old version of Guava -->
<groupId>com.google.guava</groupId>
<artifactId>guava-jdk5</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>20.0</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-vision</artifactId>
<version>0.22.0-beta</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-dynamodb</artifactId>
</dependency>
</dependencies>
Hope this helps others..

NoClassDefFoundError. How to set dependencies and deploy a java app?

My Java app requires org.objectweb.asm library. I specified 'asm' dependency in pom. That deploys the library together with the app. Still the app throws exception NoClassDefFoundError: org/objectweb/asm/ClassVisitor.
java.lang.NoClassDefFoundError: org/objectweb/asm/ClassVisitor
at com.sun.jersey.api.core.PackagesResourceConfig.init(PackagesResourceConfig.java:112)
at com.sun.jersey.api.core.PackagesResourceConfig.<init>(PackagesResourceConfig.java:78)
... many more
How can I fix the problem?
Details:
I am using Glassfish 2.1.1. The app requires jersey 1.1.4, jersey requires asm 3.1. I assume 1.1.4 version is required by glassfish 2.1.1
If I run Glassfish updatetool and install Jersey on the server then my app loads and runs with no problems. My client doesn't have Jersey installed on their server and they can not use updatetool.
Glassfish 2.1.1 updatetool installs jersey 1.1.4 and asm-3.1.jar in glassfish/lib directory.
When jersey is uninstalled, updatetool removes asm too.
If I include jersey and asm as dependencies and deploy my war file then jersey and asm jars go into local location, e.g. glassfish/domains/domain1/applications/j2ee-modules/MYAPPNAME/WEB-INF/lib/asm-3.1.jar.
Glassfish updatetool puts asm into lib folder: glassfish/lib directory the app start deploying and working correctly.
Here is my maven pom file dependency section:
<dependencies>
<dependency>
<groupId>asm</groupId>
<artifactId>asm</artifactId>
<version>3.1</version>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-bundle</artifactId>
<version>1.1.4</version>
</dependency>
<dependency>
<groupId>com.sun.jersey.jersey-test-framework</groupId>
<artifactId>jersey-test-framework-grizzly2</artifactId>
<version>1.17.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>javax.persistence</groupId>
<artifactId>persistence-api</artifactId>
<version>1.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.jboss.spec</groupId>
<artifactId>jboss-javaee-6.0</artifactId>
<version>1.0.0.Final</version>
<scope>provided</scope>
<type>pom</type>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
</dependencies>
I had a conflict with an older jersey library stored in glassfish/lib folder:
glassfish/lib/jersey-bundle-1.0.3.1.jar
It was loaded and used instead of jars placed in a folder local to the app:
glassfish/domains/domain1/applications/j2ee-modules/MYAPPNAME/WEB-INF/lib/
Jersey jar from 'lib' folder was looking for 'asm' library in the same 'lib' folder, hence my locally placed asm-3.1.jar was never found.
I expected that 'local' jars were loaded first. Appeared that the search order is different. Is that really the case? Please tell me if I am wrong.
I found the problem by verifying which library is used with the following code:
logger.debug(PackagesResourceConfig.class.getResource("PackagesResourceConfig.class"));
Note PackagesResourceConfig class listed in the exception thrown.
Hopefully this answer will help someone else to save their time.
Try something like that in your pom:
<dependency>
<groupId>asm</groupId>
<artifactId>asm</artifactId>
<version>3.3.1</version>
</dependency>

Categories