Maven dependency hell for spark mlib ALS algorithm [duplicate]

Maven dependency hell for spark mlib ALS algorithm [duplicate] - java

This question already has answers here:
Resolving dependency problems in Apache Spark
(7 answers)
Closed 4 years ago.
I have this small piece of java code to get apache spark recommendations:
public class Main {
public static class Rating implements Serializable {
private int userId;
private int movieId;
private float rating;
private long timestamp;
public Rating() {}
public Rating(int userId, int movieId, float rating, long timestamp) {
this.userId = userId;
this.movieId = movieId;
this.rating = rating;
this.timestamp = timestamp;
}
public int getUserId() {
return userId;
}
public int getMovieId() {
return movieId;
}
public float getRating() {
return rating;
}
public long getTimestamp() {
return timestamp;
}
public static Rating parseRating(String str) {
String[] fields = str.split(",");
if (fields.length != 4) {
throw new IllegalArgumentException("Each line must contain 4 fields");
}
int userId = Integer.parseInt(fields[0]);
int movieId = Integer.parseInt(fields[1]);
float rating = Float.parseFloat(fields[2]);
long timestamp = Long.parseLong(fields[3]);
return new Rating(userId, movieId, rating, timestamp);
}
}
static String parse(String str) {
Pattern pat = Pattern.compile("\\[[0-9.]*,[0-9.]*]");
Matcher matcher = pat.matcher(str);
int count = 0;
StringBuilder sb = new StringBuilder();
while (matcher.find()) {
count++;
String substring = str.substring(matcher.start(), matcher.end());
String itstr = substring.split(",")[0].substring(1);
sb.append(itstr + " ");
}
return sb.toString().trim();
}
static TreeMap<Long, String> res = new TreeMap<>();
public static void add(long k, String v) {
res.put(k, v);
}
public static void main(String[] args) throws IOException {
Logger.getLogger("org").setLevel(Level.OFF);
Logger.getLogger("akka").setLevel(Level.OFF);
SparkSession spark = SparkSession
.builder()
.appName("SomeAppName")
.config("spark.master", "local")
.getOrCreate();
JavaRDD<Rating> ratingsRDD = spark
.read().textFile(args[0]).javaRDD()
.map(Rating::parseRating);
Dataset<Row> ratings = spark.createDataFrame(ratingsRDD, Rating.class);
ALS als = new ALS()
.setMaxIter(1)
.setRegParam(0.01)
.setUserCol("userId")
.setItemCol("movieId")
.setRatingCol("rating");
ALSModel model = als.fit(ratings);
model.setColdStartStrategy("drop");
Dataset<Row> rowDataset = model.recommendForAllUsers(50);
rowDataset.foreach((ForeachFunction<Row>) row -> {
String str = row.toString();
long l = Long.parseLong(str.substring(1).split(",")[0]);
add(l, parse(str));
});
BufferedWriter bw = new BufferedWriter(new FileWriter(args[1]));
for (long l = 0; l < res.lastKey(); l++) {
if (!res.containsKey(l)) {
bw.write("\n");
continue;
}
String str = res.get(l);
bw.write(str);
}
bw.close();
}
}
I am trying different dependencies in my pom.xml to get it running, but all variants fail. This one:
<dependency>
<groupId>com.sparkjava</groupId>
<artifactId>spark-core</artifactId>
<version>2.8.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.12</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.6.4</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.6.4</version>
</dependency>
fails with java.lang.ClassNotFoundException: text.DefaultSource, to fix it I add
org.apache.spark
spark-sql-kafka-0-10_2.10
2.0.2
now it crashes with ClassNotFoundException: org.apache.spark.internal.Logging$class, to fix it I add another ones:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>2.2.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>2.2.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.10</artifactId>
<version>2.2.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka-0-8_2.11</artifactId>
<version>2.2.2</version>
</dependency>
now it fails with java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce to fix it I tried dozen of other combinations, all of them failed, the last one is
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>2.2.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.11</artifactId>
<version>2.2.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.2.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.2.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka-0-8_2.11</artifactId>
<version>2.2.2</version>
</dependency>
which again gives me ClassNotFoundException: text.DefaultSource, how can I fix it? Was there any logic behind implementing runtime linking in spark?
UPD: also tried
<dependencies>
<dependency> <!-- Spark dependency -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.0.1</version>
</dependency>
<dependency> <!-- Spark dependency -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.11</artifactId>
<version>2.0.1</version>
</dependency>
<dependency> <!-- Spark dependency -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.0.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>2.0.1</version>
</dependency>
<dependency>
<groupId>org.apache.bahir</groupId>
<artifactId>spark-streaming-twitter_2.11</artifactId>
<version>2.0.1</version>
</dependency>
</dependencies>
(this still gives me java.lang.ClassNotFoundException: text.DefaultSource))
I also tried dependencies published in this question, but they also fail: Resolving dependency problems in Apache Spark
Source code is available here, so you can try various maven settings yourself: https://github.com/stiv-yakovenko/sparkrec

Finally I was able to make it work:
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.11</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.11.8</version>
</dependency>
</dependencies>
You have to use these exact versions otherwise it will crash in multiple various ways.

Related

Getting "NoSuchMethodError: org.hamcrest.Matcher.describeMismatch" when running integration tests

I'm using JUnit 4.11 to run integration tests on my SpringBoot application. I have the following test case :
#Test
public void testClassementMarcheActeTranche() throws IOException, PlanDeClassementException {
NodeRef nodeRefRepertoireSasGda = servicePlanDeClassementMarche.getSpace(SpaceUtil.SAS_GED_MARCHE.name());
NodeRef nodeRefPiece = setUpCreateActe(nodeRefRepertoireSasGda, "1 (Tranche 1 Marché 2019-19M0059001) (Tranche 1 Marché 2019-19M58785858)", "Courrier de reconduction");
classementDocumentsActionJobExecuter.executeImpl(null, nodeRefPiece);
String siteName = serviceRegistry.getSiteService().getSite(nodeRefPiece).getShortName();
QName typePiece = serviceRegistry.getNodeService().getType(nodeRefPiece);
assertThat(typePiece, equalTo(TYPE_PIECE.getQName()));
NodeRef parentFolder = serviceRegistry.getNodeService().getPrimaryParent(nodeRefPiece).getParentRef();
String parentFolderName = serviceRegistry.getFileFolderService().getFileInfo(parentFolder).getName();
NodeRef marcheFolder = serviceRegistry.getNodeService().getPrimaryParent(parentFolder).getParentRef();
QName typeMarche = serviceRegistry.getNodeService().getType(marcheFolder);
assertThat(typeMarche, equalTo(TYPE_MARCHE.getQName()));
String marcheFolderName = serviceRegistry.getFileFolderService().getFileInfo(marcheFolder).getName();
NodeRef millesimeFolder = serviceRegistry.getNodeService().getPrimaryParent(marcheFolder).getParentRef();
String millesimeFolderName = serviceRegistry.getFileFolderService().getFileInfo(millesimeFolder).getName();
NodeRef cdrFolder = serviceRegistry.getNodeService().getPrimaryParent(millesimeFolder).getParentRef();
String cdrFolderName = serviceRegistry.getFileFolderService().getFileInfo(cdrFolder).getName();
String numeroMarche = (String)nodeService.getProperty(nodeRefPiece, PROP_NUMERO_DU_MARCHE.getQName());
String dateModificationnMarche = nodeService.getProperty(nodeRefPiece, PROP_DATE_DE_NOTIFICATION_DU_MARCHE.getQName()).toString();
String categorieMarche = (String)nodeService.getProperty(nodeRefPiece, PROP_CATEGORIE_DU_MARCHE.getQName());
String etatMarche = (String)nodeService.getProperty(nodeRefPiece, PROP_ETAT_DU_MARCHE.getQName());
List<String> codeTitulaire = (ArrayList<String>)nodeService.getProperty(nodeRefPiece, PROP_CODE_TITULAIRE.getQName());
List<String> titulaireMarche = (ArrayList<String>)nodeService.getProperty(nodeRefPiece, PROP_NOM_TITULAIRE.getQName());
List<String> codeSousTraitant = (ArrayList<String>)nodeService.getProperty(nodeRefPiece, PROP_CODE_SOUS_TRAITANTS.getQName());
List<String> nomSousTraitant = (ArrayList<String>)nodeService.getProperty(nodeRefPiece, PROP_NOM_SOUS_TRAITANTS.getQName());
String typeDeProcedureMarche = (String)nodeService.getProperty(nodeRefPiece, PROP_TYPE_DE_PROCEDURE.getQName());
String objetMarche = (String)nodeService.getProperty(nodeRefPiece, PROP_OBJET_DU_MARCHE.getQName());
List<String> axe = (ArrayList<String>)nodeService.getProperty(marcheFolder, PROP_AXE_OPERATION_ORYSON_MARCHE.getQName());
List<String> libEtablissement = (ArrayList<String>)nodeService.getProperty(marcheFolder, PROP_LIBELLE_ETABLISSEMENT_ENSEIGNEMENT_MARCHE.getQName());
List<String> codeEtablissement = (ArrayList<String>)nodeService.getProperty(marcheFolder, PROP_CODE_ETABLISSEMENT_ENSEIGNEMENT_MARCHE.getQName());
List<String> cdrMarche = (ArrayList<String>)nodeService.getProperty(marcheFolder, PROP_CDR_MARCHE.getQName());
assertThat(marcheFolderName, equalTo("2019-19M58785858 - Lot 1 - Chantier des collections");
nodeService.deleteNode(marcheFolder);
}
And the following error message :
java.lang.NoSuchMethodError: org.hamcrest.Matcher.describeMismatch(Ljava/lang/Object;Lorg/hamcrest/Description;)V
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
at fr.package.alfresco.marche.actions.ClassementActeActionJobIT.testClassementMarcheActeTranche(ClassementActeActionJobIT.java:203)
I have followed the answers in this question : Getting "NoSuchMethodError: org.hamcrest.Matcher.describeMismatch" when running test in IntelliJ 10.5
So I ended up having this pom file :
<dependencies>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-core</artifactId>
<version>1.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-library</artifactId>
<version>1.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<exclusions>
<exclusion>
<artifactId>hamcrest-core</artifactId>
<groupId>org.hamcrest</groupId>
</exclusion>
</exclusions>
<version>1.9.5</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<exclusions>
<exclusion>
<artifactId>hamcrest-core</artifactId>
<groupId>org.hamcrest</groupId>
</exclusion>
</exclusions>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jdbi</groupId>
<artifactId>jdbi</artifactId>
<version>2.78</version>
</dependency>
</dependencies>
Although this haven't fixed the problem, anyone has an idea of how this persisting problem would be fixed ?

Junit test does not replace placeHolders

I have this configuration class in a maven project:
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;
import lombok.Data;
#Data
#Configuration
public class SmsConfig {
#Value("${sms.domainId}")
private String domainId;
#Value("${sms.gateway.url}")
private String gatewayUrl;
#Value("${sms.cmd}")
private String cmd;
#Value("${sms.login}")
private String login;
#Value("${sms.passwd}")
private String passwd;
}
I have this service class in a Spring project:
Service("smsService")
public class AltiriaSMSRestServiceImpl implements SmsService {
private final SmsConfig smsConfig;
public AltiriaSMSRestServiceImpl(SmsConfig smsConfig) {
this.smsConfig = smsConfig;
}
#Override
public boolean sendSMS(String msg, String to) throws Exception {
...
}
...
}
and this Test:
#ContextConfiguration(classes = { SmsConfig.class })
#RunWith(SpringJUnit4ClassRunner.class)
public class AltiriaSMSRestServiceImplTest {
#Autowired
#Qualifier("smsService")
private AltiriaSMSRestServiceImpl smsService;
#Test
public void testSendSMS() throws Exception {
smsService.sendSMS("this is a test", "+34776498");
}
}
on IntelliJ IDEA it seems that the values on the config class are set correctly
but when I run the test Junit test does not replace placeHolders
5:25:40.009 [main] DEBUG org.springframework.web.client.RestTemplate - Writing [{ "credentials":{"domainId":"${sms.domainId}","login":"${sms.login}","passwd":"${sms.passwd}"},"destination":["32470855126"], "message":{"msg":"this is a test","concat":"true", "encoding":"unicode"} }] as "application/json"
and here my pom.xml:
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-web</artifactId>
<version>5.2.1.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-mail</artifactId>
<version>2.2.1.RELEASE</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.10</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.10.1</version>
</dependency>
<dependency>
<artifactId>lombok</artifactId>
<groupId>org.projectlombok</groupId>
<scope>provided</scope>
<version>1.18.10</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.10.19</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>5.2.1.RELEASE</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>3.14.0</version>
<scope>test</scope>
</dependency>
</dependencies>

#ContextConfiguration(classes = { SmsConfig.class, AltiriaSMSRestServiceImpl.class})
#RunWith(SpringJUnit4ClassRunner.class)
#TestPropertySource(locations="classpath:application.properties")
public class AltiriaSMSRestServiceImplTest {
..
}

SpringMVC Integration SpringBatch transaction have a problem

After springbatch integrates the mvc import interface, the transaction is abnormal, and the service layer is injected into the job to execute the transaction problem.
BatchController.class
#RestController
#RequestMapping(value = "/job")
public class BatchController {
private final JobLauncher jobLauncher;
private final Job batchGenerateJob;
#Autowired
public BatchController(JobLauncher jobLauncher, Job batchGenerateJob) {
this.jobLauncher = jobLauncher;
this.batchGenerateJob = batchGenerateJob;
}
#GetMapping(value = "/{cid}")
public AjaxObj batchgenbycid(#PathVariable String cid) {
if (StringUtils.isBlank(cid) || Integer.parseInt(cid) == 0) {
return ResultUtil.fail("请在左侧点击栏目");
}
JobParameters jobParameters = new JobParametersBuilder()
.addString("channelId", cid)
.addString("siteId", Integer.toString(1))
.addString("runtime", "最后执行时间：" + DateFormatUtils.format(new Date(), "yyyy_MM_dd_HH_mm_ss"))
.toJobParameters();
try {
jobLauncher.run(batchGenerateJob, jobParameters);
} catch (JobExecutionAlreadyRunningException e) {
e.printStackTrace();
} catch (JobRestartException e) {
e.printStackTrace();
} catch (JobInstanceAlreadyCompleteException e) {
e.printStackTrace();
} catch (JobParametersInvalidException e) {
e.printStackTrace();
}
return ResultUtil.success();
}
}
** BatchNewsConfig.class **
#Configuration
#EnableBatchProcessing
#EnableTransactionManagement
public class BatchNewsConfig implements StepExecutionListener {
public static final String SITE_ID = "siteId";
public static final String CHANNEL_ID = "channelId";
private final JobBuilderFactory jobBuilderFactory;
private final StepBuilderFactory stepBuilderFactory;
private final JobRepository jobRepository;
private Map<String, JobParameter> parameterMap;
#Autowired
public BatchNewsConfig(JobBuilderFactory jobBuilderFactory, StepBuilderFactory stepBuilderFactory, JobRepository jobRepository) {
this.jobBuilderFactory = jobBuilderFactory;
this.stepBuilderFactory = stepBuilderFactory;
this.jobRepository = jobRepository;
}
#Bean
public JobLauncher jobLauncher() {
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(jobRepository);
jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
return jobLauncher;
}
#Bean
public Job batchGenerateJob() {
return jobBuilderFactory.get("batchGenerateJob")
//获取站点信息
.start(getSiteParameterStep())
.build();
}
#Bean
public Step getSiteParameterStep() {
return stepBuilderFactory.get("getChannelIdParameterStep")
.listener(this)
.tasklet(new Tasklet() {
#Override
public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception {
String channelStr = parameterMap.get(CHANNEL_ID).toString();
Integer cid = Integer.valueOf(channelStr);
System.out.println(cid);
return RepeatStatus.FINISHED;
}
}).build();
}
#Override
public void beforeStep(StepExecution stepExecution) {
parameterMap = stepExecution.getJobParameters().getParameters();
}
#Override
public ExitStatus afterStep(StepExecution stepExecution) {
return null;
}
}
** Console output Can run**
16:37:42,840 INFO SimpleJobLauncher:133 - Job: [SimpleJob: [name=batchGenerateJob]] launched with the following parameters: [{channelId=368, siteId=1, runtime=最后执行时间：2018_12_20_16_37_42}]
16:37:42,913 INFO SimpleStepHandler:146 - Executing step: [getChannelIdParameterStep]
368
16:37:42,950 INFO SimpleJobLauncher:136 - Job: [SimpleJob: [name=batchGenerateJob]] completed with the following parameters: [{channelId=368, siteId=1, runtime：2018_12_20_16_37_42}] and the following status: [COMPLETED]
```
### When I injected the Service, I executed the error again.
** IChannelBatchService **
```java
import cn.dahe.model.Channel;
import java.util.Set;
public interface IChannelBatchService{
Channel get(int id);
Set<Integer> getAllChildrenByCid(int cid);
}
** ChannelBatchService **
import cn.dahe.dao.IChannelDao;
import cn.dahe.model.Channel;
import cn.dahe.service.IChannelBatchService;
import com.google.common.collect.Sets;
import org.springframework.stereotype.Service;
import javax.annotation.Resource;
import java.util.List;
import java.util.Set;
#Service("channelBatchService")
public class ChannelBatchService implements IChannelBatchService {
#Resource
private IChannelDao channelDao;
#Override
public Channel get(int id) {
return channelDao.get(id);
}
#Override
public Set<Integer> getAllChildrenByCid(int cid) {
return getChildrenCid(cid, Sets.newHashSet());
}
/**
* #param cid
* #param cidsSet
*/
private Set<Integer> getChildrenCid(int cid, Set<Integer> cidsSet) {
cidsSet.add(cid);
List<Channel> channels = channelDao.getChannelByPid(cid);
if (channels != null && !channels.isEmpty()) {
for (Channel channel : channels) {
cidsSet.add(channel.getId());
cidsSet = getChildrenCid(channel.getId(), cidsSet);
}
}
return cidsSet;
}
}
add service
#Configuration
#EnableBatchProcessing
#EnableTransactionManagement
public class BatchNewsConfig implements StepExecutionListener {
public static final String SITE_ID = "siteId";
public static final String CHANNEL_ID = "channelId";
private final JobBuilderFactory jobBuilderFactory;
private final StepBuilderFactory stepBuilderFactory;
private final JobRepository jobRepository;
private final IChannelBatchService iChannelBatchService;
private Map<String, JobParameter> parameterMap;
public BatchNewsConfig(JobBuilderFactory jobBuilderFactory, StepBuilderFactory stepBuilderFactory, JobRepository jobRepository, IChannelBatchService iChannelBatchService) {
this.jobBuilderFactory = jobBuilderFactory;
this.stepBuilderFactory = stepBuilderFactory;
this.jobRepository = jobRepository;
this.iChannelBatchService = iChannelBatchService;
}
#Bean
public JobLauncher jobLauncher() {
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(jobRepository);
jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
return jobLauncher;
}
#Bean
public Job batchGenerateJob() {
return jobBuilderFactory.get("batchGenerateJob")
.start(getSiteParameterStep())
.build();
}
#Bean
public Step getSiteParameterStep() {
return stepBuilderFactory.get("getChannelIdParameterStep")
.listener(this)
.tasklet(new Tasklet() {
#Override
public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception {
String channelStr = parameterMap.get(CHANNEL_ID).toString();
Integer cid = Integer.valueOf(channelStr);
System.out.println(cid);
Channel channel = iChannelBatchService.get(cid);
System.out.println(channel);
return RepeatStatus.FINISHED;
}
}).build();
}
/**
* 执行前获取JobParameters
*
* #param stepExecution
*/
#Override
public void beforeStep(StepExecution stepExecution) {
parameterMap = stepExecution.getJobParameters().getParameters();
}
#Override
public ExitStatus afterStep(StepExecution stepExecution) {
return null;
}
}
** Console output **
16:48:45,539 INFO JobRepositoryFactoryBean:183 - No database type set, using meta data indicating: MYSQL
16:48:45,674 INFO SimpleJobLauncher:195 - No TaskExecutor has been set, defaulting to synchronous executor.
16:48:45,735 INFO SimpleJobLauncher:133 - Job: [SimpleJob: [name=batchGenerateJob]] launched with the following parameters: [{channelId=368, siteId=1, runtime=2018_12_20_16_48_45}]
16:48:45,792 INFO SimpleStepHandler:146 - Executing step: [getChannelIdParameterStep]
368
16:48:45,811 ERROR AbstractStep:229 - Encountered an error executing step getChannelIdParameterStep in job batchGenerateJob
org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is java.lang.IllegalStateException: Already value [org.springframework.jdbc.datasource.ConnectionHolder#b675ab] for key [{
CreateTime:"2018-12-20 16:48:36",
ActiveCount:2,
PoolingCount:0,
CreateCount:2,
DestroyCount:0,
CloseCount:18,
ConnectCount:20,
Connections:[
]
}] bound to thread [SimpleAsyncTaskExecutor-1]
at org.springframework.orm.hibernate5.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:542)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:377)
at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:461)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:277)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy45.get(Unknown Source)
at cn.dahe.batch.BatchNewsConfig$1.execute(BatchNewsConfig.java:88)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:272)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:392)
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Already value [org.springframework.jdbc.datasource.ConnectionHolder#b675ab] for key [{
CreateTime:"2018-12-20 16:48:36",
ActiveCount:2,
PoolingCount:0,
CreateCount:2,
DestroyCount:0,
CloseCount:18,
ConnectCount:20,
Connections:[
]
}] bound to thread [SimpleAsyncTaskExecutor-1]
at org.springframework.transaction.support.TransactionSynchronizationManager.bindResource(TransactionSynchronizationManager.java:190)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:516)
... 26 more
16:48:45,827 INFO SimpleJobLauncher:136 - Job: [SimpleJob: [name=batchGenerateJob]] completed with the following parameters: [{channelId=368, siteId=1, runtime=2018_12_20_16_48_45}] and the following status: [FAILED]
** prom.xml **
<dependencies>
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-core</artifactId>
<version>3.0.9.RELEASE</version>
</dependency>
<!-- jstl -->
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
<version>1.2</version>
</dependency>
<dependency>
<groupId>taglibs</groupId>
<artifactId>standard</artifactId>
<version>1.1.2</version>
</dependency>
<!-- servlet -->
<dependency>
<groupId>javax.servlet.jsp</groupId>
<artifactId>jsp-api</artifactId>
<version>2.2.1-b03</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.21</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid</artifactId>
<version>1.0.6</version>
</dependency>
<!-- SPRING -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>4.3.16.RELEASE</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
<version>4.3.16.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>4.3.16.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context-support</artifactId>
<version>4.3.16.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-aop</artifactId>
<version>4.3.16.RELEASE</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-tx</artifactId>
<version>4.3.16.RELEASE</version>
</dependency>
<!-- spring orm -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-orm</artifactId>
<version>4.3.16.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jdbc</artifactId>
<version>4.3.16.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-web</artifactId>
<version>4.3.16.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc</artifactId>
<version>4.3.16.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-oxm</artifactId>
<version>4.3.16.RELEASE</version>
</dependency>
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz</artifactId>
<version>2.1.3</version>
</dependency>
<!-- AOP begin -->
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjrt</artifactId>
<version>1.7.4</version>
</dependency>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>1.7.4</version>
</dependency>
<dependency>
<groupId>cglib</groupId>
<artifactId>cglib</artifactId>
<version>3.1</version>
<scope>runtime</scope>
</dependency>
<!-- AOP end -->
<!-- Hibernate -->
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>5.1.11.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-ehcache</artifactId>
<version>5.1.11.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-validator</artifactId>
<version>6.0.0.Alpha2</version>
</dependency>
<dependency>
<groupId>org.javassist</groupId>
<artifactId>javassist</artifactId>
<version>3.18.1-GA</version>
</dependency>
<!-- LOGGING end -->
<!-- GENERAL UTILS begin -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.7</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.8</version>
</dependency>
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
<version>1.8.3</version>
<exclusions>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>commons-net</groupId>
<artifactId>commons-net</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>3.6.0</version>
</dependency>
<!-- google java library -->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>23.0</version>
</dependency>
<!-- gson -->
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.3</version>
</dependency>
<!-- TEST begin -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>4.3.16.RELEASE</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.4</version>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.11.2</version>
</dependency>
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.3</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20171018</version>
</dependency>
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-core-asl</artifactId>
<version>1.9.13</version>
</dependency>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-core-lgpl</artifactId>
<version>1.9.13</version>
</dependency>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-mapper-asl</artifactId>
<version>1.9.13</version>
</dependency>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-mapper-lgpl</artifactId>
<version>1.9.13</version>
</dependency>
<dependency>
<groupId>org.freemarker</groupId>
<artifactId>freemarker</artifactId>
<version>2.3.23</version>
</dependency>
<!-- json -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.7</version>
</dependency>
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>2.9.0</version>
</dependency>
<dependency>
<groupId>cn.jpush.api</groupId>
<artifactId>jpush-client</artifactId>
<version>3.2.9</version>
</dependency>
<dependency>
<groupId>com.ibeetl</groupId>
<artifactId>beetl</artifactId>
<version>2.7.27</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
<version>3.17</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>3.17</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-scratchpad</artifactId>
<version>3.17</version>
</dependency>
<dependency>
<groupId>com.google.zxing</groupId>
<artifactId>core</artifactId>
<version>3.0.0</version>
</dependency>
<dependency>
<groupId>net.sf.barcode4j</groupId>
<artifactId>barcode4j-light</artifactId>
<version>2.0</version>
</dependency>

Seems you have multiple transaction managers. In such a case, you can specify the transaction manager for tasklets explicitly.
<tasklet transaction-manager="transactionManager">
#EnableBatchProcessing automatically registers a transaction manager. You can change transaction manager that batch uses simply by implementing the BatchConfigurer interface.

Spring Data Neo4j - findById method throws an invalid query exception after upgrading to Spring Boot 2

I have been working on a micro-service created using Spring Boot with Neo4j as the database. The two relevant entities within it Product and AccessoryRelation are thus:
#NodeEntity
#QueryEntity
public class Product extends AbstractNeo4jEntity {
#Index(primary = true, unique = true)
private String productId;
#Transient
private String market;
#JsonIgnore
#Relationship(type = "HAS_ACCESSORY")
private Set<AccessoryRelation> relations = new HashSet<>();
public Product() {
}
public Product(final String productId) {
this.productId = productId;
}
public String getProductId() {
return productId;
}
public String getMarket() {
return market;
}
public void setMarket(final String market) {
this.market = market;
}
public Set<AccessoryRelation> getRelations() {
return new HashSet<>(relations);
}
public void addAccessory(final Product accessory) {
Assert.notNull(accessory, "Accessory product can not be null");
Assert.hasText(accessory.getProductId(),
"Accessory product should have a valid identifier");
Assert.hasText(accessory.getMarket(),
"Accessory product should be available atleast in one market");
Assert.isTrue(!this.equals(accessory),
"A product can't be an accessory of itself.");
final AccessoryRelation relation = this.relations.stream() //
.filter(rel -> rel.getAccessory().equals(accessory)) //
.findAny() //
.orElseGet(() -> {
final AccessoryRelation newRelation = new AccessoryRelation(this,
accessory, Sets.newHashSet());
this.relations.add(newRelation);
return newRelation;
});
relation.addMarket(accessory.getMarket());
}
public void removeAccessory(final Product accessory) {
Assert.notNull(accessory, "Accessory product can not be null");
Assert.hasText(accessory.getProductId(),
"Accessory product should have a valid identifier");
Assert.hasText(accessory.getMarket(),
"Accessory product should be available atleast in one market");
final Optional<AccessoryRelation> relation = this.relations.stream() //
.filter(rel -> rel.getAccessory().equals(accessory)) //
.findAny();
if (relation.isPresent()) {
relation.get().removeMarket(accessory.getMarket());
if (relation.get().getMarkets().isEmpty()) {
this.relations.remove(relation.get());
}
}
}
public void removeAccessories() {
this.relations.clear();
}
#Override
public boolean equals(final Object obj) {
if (obj == null) {
return false;
}
if (obj == this) {
return true;
}
if (!(obj instanceof Product)) {
return false;
}
final Product product = (Product) obj;
return new EqualsBuilder() //
.append(getProductId(), product.getProductId()) //
.isEquals();
}
#Override
public int hashCode() {
return new HashCodeBuilder() //
.append(getProductId()) //
.toHashCode();
}
#Override
public String toString() {
return new ToStringBuilder(this) //
.append("ProductId", getProductId()) //
.toString();
}
}
#RelationshipEntity(type = "HAS_ACCESSORY")
public class AccessoryRelation extends AbstractNeo4jEntity {
#StartNode
private Product product;
#EndNode
private Product accessory;
private Set<String> markets = new HashSet<>();
public AccessoryRelation() {
}
public AccessoryRelation(final Product product, final Product accessory,
final Set<String> markets) {
this.product = product;
this.accessory = accessory;
this.markets = markets;
}
public Product getProduct() {
return product;
}
public void setProduct(final Product product) {
this.product = product;
}
public Product getAccessory() {
return accessory;
}
public void setAccessory(final Product accessory) {
this.accessory = accessory;
}
public Set<String> getMarkets() {
return new HashSet<>(markets);
}
public void addMarket(final String market) {
this.markets.add(market);
}
public void removeMarket(final String market) {
this.markets.remove(market);
}
#Override
public boolean equals(final Object obj) {
if (obj == null) {
return false;
}
if (obj == this) {
return true;
}
if (!(obj instanceof AccessoryRelation)) {
return false;
}
final AccessoryRelation relation = (AccessoryRelation) obj;
return new EqualsBuilder() //
.append(getProduct(), relation.getProduct()) //
.append(getAccessory(), relation.getAccessory()) //
.isEquals();
}
#Override
public final int hashCode() {
return new HashCodeBuilder() //
.append(getProduct()) //
.append(getAccessory()) //
.toHashCode();
}
#Override
public String toString() {
return new ToStringBuilder(this) //
.append("Product", getProduct()) //
.append("Accessory", getAccessory()) //
.append("Markets", getMarkets()) //
.toString();
}
}
For JUnit test cases I am using nosqlunit-neo4j version 1.0.0-rc.5 and an embedded databsase.
Now I have the findById(productId) method which would return the Product along with the associated AccessoryRelation objects. However, after upgrading to Spring Boot 2, this method does not work. It throws an exception thus:
org.springframework.data.neo4j.exception.UncategorizedNeo4jException:
Error executing Cypher; Code: Neo.ClientError.Statement.InvalidSyntax;
Description: Invalid input '|': expected whitespace, comment, a
relationship pattern, '.', node labels, '[', "=~", IN, STARTS, ENDS,
CONTAINS, IS, '^', '*', '/', '%', '+', '-', '=', "<>", "!=", '<', '>',
"<=", ">=", AND, XOR, OR, ',' or ']' (line 1, column 113 (offset:
112))
"MATCH (n:`Product`) WHERE n.`productId` = { id } WITH n RETURN n,[ [
(n)-[r_h1:`HAS_ACCESSORY`]->(p1:`Product`) | [ r_h1, p1 ] ] ]"
^; nested exception is org.neo4j.ogm.exception.CypherException: Error
executing Cypher; Code: Neo.ClientError.Statement.InvalidSyntax;
Description: Invalid input '|': expected whitespace, comment, a
relationship pattern, '.', node labels, '[', "=~", IN, STARTS, ENDS,
CONTAINS, IS, '^', '*', '/', '%', '+', '-', '=', "<>", "!=", '<', '>',
"<=", ">=", AND, XOR, OR, ',' or ']' (line 1, column 113 (offset:
112))
"MATCH (n:`Product`) WHERE n.`productId` = { id } WITH n RETURN n,[ [
(n)-[r_h1:`HAS_ACCESSORY`]->(p1:`Product`) | [ r_h1, p1 ] ] ]"
By the looks of it, it is treating the '|' character as an invalid character within the query. However, when I run the same query in a neo4 3.4.9 instance on the browser, it works fine. No syntax errors on that one. The application has a default neo4j version of 2.3.3 which coming from the nosqlunit dependency. I have tried including neo4j 3.4.9 dependencies in my pom, but that creates a whole set of new problems.
The following is my pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.1.RELEASE</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-
8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
<project.build.itDirectory>src/it/java</project.build.itDirectory>
<neo4j-cypher-dsl.version>2.3-RELEASE</neo4j-cypher-dsl.version>
<querydsl.version>4.1.4</querydsl.version>
<querydsl-apt.version>1.1.3</querydsl-apt.version>
<cglib.version>2.2.2</cglib.version>
<spring-cloud.version>Finchley.RC2</spring-cloud.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>cglib</groupId>
<artifactId>cglib</artifactId>
<version>${cglib.version}</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>20.0</version>
</dependency>
<dependency>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-apt</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-lucene3</artifactId>
<version>${querydsl.version}</version>
<exclusions>
<exclusion>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>de.codecentric</groupId>
<artifactId>spring-boot-admin-starter-client</artifactId>
<version>2.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.5</version>
</dependency>
<!-- This jar is used by CypherQueryDSL at runtime. If its not present
in classpath then java.lang.ClassNotFoundException: org.apache.lucene.search.Query
error is thrown -->
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
<version>3.6.2</version>
</dependency>
<dependency>
<groupId>com.consol.citrus</groupId>
<artifactId>citrus-core</artifactId>
<version>2.7.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.consol.citrus</groupId>
<artifactId>citrus-http</artifactId>
<version>2.7.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.consol.citrus</groupId>
<artifactId>citrus-java-dsl</artifactId>
<version>2.7.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-cypher-dsl</artifactId>
<version>${neo4j-cypher-dsl.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-rest</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-neo4j</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-rest-hal-browser</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>
<!-- test dependencies -->
<dependency>
<groupId>com.lordofthejars</groupId>
<artifactId>nosqlunit-neo4j</artifactId>
<version>1.0.0-rc.5</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.openpojo</groupId>
<artifactId>openpojo</artifactId>
<version>0.8.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>nl.jqno.equalsverifier</groupId>
<artifactId>equalsverifier</artifactId>
<version>2.1.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-ogm-embedded-driver</artifactId>
<version>3.1.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.ow2.asm</groupId>
<artifactId>asm-all</artifactId>
<version>6.0_ALPHA</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<finalName>product-accessory-service</finalName>
<plugins>
<plugin>
<groupId>com.mysema.maven</groupId>
<artifactId>apt-maven-plugin</artifactId>
<version>${querydsl-apt.version}</version>
<dependencies>
<dependency>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-apt</artifactId>
<version>${querydsl.version}</version>
</dependency>
</dependencies>
<configuration>
<processor>com.querydsl.apt.QuerydslAnnotationProcessor</processor>
</configuration>
<executions>
<execution>
<id>add-sources</id>
<phase>generate-sources</phase>
<goals>
<goal>process</goal>
</goals>
<configuration>
<outputDirectory>target/generated-sources/querydsl</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.sonarsource.scanner.maven</groupId>
<artifactId>sonar-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.4.13</version>
<configuration>
Has anyone encountered a similar issue before? Could anyone please help me out in resolving this? Thanks in advance.

Lucene exception NoSuchMethodError with version 6.4.0

I am relatively new to Lucene and playing with the latest version 6.4.0.
I have written a cutom analyzer class for doing synonyms,
public class MySynonymAnalyzer extends Analyzer {
#Override
protected TokenStreamComponents createComponents(String fieldName) {
Tokenizer source = new ClassicTokenizer();
TokenStream filter = new StandardFilter(source);
filter = new LowerCaseFilter(filter);
filter = new SynonymGraphFilter(filter, getSynonymsMap(), false);
filter = new FlattenGraphFilter(filter);
return new TokenStreamComponents(source, filter);
}
private SynonymMap getSynonymsMap() {
try {
SynonymMap.Builder builder = new SynonymMap.Builder(true);
builder.add(new CharsRef("work"), new CharsRef("labor"), true);
builder.add(new CharsRef("work"), new CharsRef("effort"), true);
SynonymMap mySynonymMap = builder.build();
return mySynonymMap;
} catch (Exception ex) {
return null;
}
}
In the line where I call getSynonymsMap(), I get the following exception:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.lucene.util.UnicodeUtil.UTF16toUTF8WithHash([CIILorg/apache/lucene/util/BytesRef;)I
at org.apache.lucene.analysis.synonym.SynonymMap$Builder.add(SynonymMap.java:192)
at org.apache.lucene.analysis.synonym.SynonymMap$Builder.add(SynonymMap.java:239)
at m2_lab4.MySynonymAnalyzer.getSynonymsMap(MySynonymAnalyzer.java:37)
at m2_lab4.MySynonymAnalyzer.createComponents(MySynonymAnalyzer.java:28)
at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:162)
at org.apache.lucene.document.Field.tokenStream(Field.java:568)
Version 6.4.0 doesn't seem to have method UTF16toUTF8WithHash in class UnicodeUtil. I am using everything from lucene 6.4.0 and there doesn't seem to be any old versioned jar in my classpath. This is how my maven dependendies look like:
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<lucene.version>6.4.0</lucene.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
<version>${lucene.version}</version>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-analyzers-common</artifactId>
<version>${lucene.version}</version>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-queryparser</artifactId>
<version>${lucene.version}</version>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-queries</artifactId>
<version>${lucene.version}</version>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-analyzers</artifactId>
<version>3.6.2</version>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-facet</artifactId>
<version>${lucene.version}</version>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-spatial</artifactId>
<version>${lucene.version}</version>
</dependency>
<dependency>
<groupId>com.spatial4j</groupId>
<artifactId>spatial4j</artifactId>
<version>0.4.1</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20090211</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
<version>1.3.2</version>
</dependency>
</dependencies>
Any idea what's going on? I am specially puzzled by the [CIILorg/apache/lucene/util/BytesRef text inside the exception description.

We Keep Coding

Java is a programming language and computing platform first released by Sun Microsystems in 1995.

Maven dependency hell for spark mlib ALS algorithm [duplicate] - java

Related

Getting "NoSuchMethodError: org.hamcrest.Matcher.describeMismatch" when running integration tests

Junit test does not replace placeHolders

SpringMVC Integration SpringBatch transaction have a problem

Spring Data Neo4j - findById method throws an invalid query exception after upgrading to Spring Boot 2

Lucene exception NoSuchMethodError with version 6.4.0

Categories

Resources