I have a SpringBoot App, that works as expected when I run it with Eclipse. I create a runnable jar and test it. Everything works expect one thing: it does not save anything on the DB, there is no log or exception to check. I assume there has to be something with the export, because it does not store anything,
not even running it locally.
I have two projects, one depending from the other.
The project POM.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>BatchCare</groupId>
<artifactId>BatchCare</artifactId>
<version>0.0.1-SNAPSHOT</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.3.3.RELEASE</version>
</parent>
<build>
<sourceDirectory>src</sourceDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>1.4.2.RELEASE</version>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
<version>2.5</version>
</dependency>
<dependency>
<groupId>org.simpleframework</groupId>
<artifactId>simple-xml</artifactId>
<version>2.6.9</version>
</dependency>
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc6</artifactId>
<version>11.1.0.7.0</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
<version>2.5</version>
<!-- <version>1.3.2</version> -->
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
<version>3.9</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>3.9</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml-schemas</artifactId>
<version>3.9</version>
</dependency>
<dependency>
<groupId>jaxen</groupId>
<artifactId>jaxen</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>CareLibrary</groupId>
<artifactId>CareLibrary</artifactId>
<version>0.0.1-SNAPSHOT</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.3.156</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>spring-releases</id>
<url>https://repo.spring.io/libs-release</url>
</repository>
<repository>
<id>local-repo</id>
<name>Local Project Repo</name>
<url>file://${basedir}/lib/</url>
<layout>default</layout>
</repository>
</repositories>
Classes:
#ComponentScan({"ar.com.tr.latam.care.utils", "ar.com.tr.latam.care.unification.process", "ar.com.tr.latam.care.repository"})
#EnableJpaRepositories
//#Component
#Repository
public class RequestLoggerImpl implements RequestLogger {
/**
* Log.
*/
private static final Logger LOG = LoggerFactory.getLogger(RequestLoggerImpl.class);
#Autowired
private RequestLogDao requestLogDao;
#Override
public void log() {
RequestLog requestLog = new RequestLog();
requestLog.setDate(new Date());
requestLog.setSource("jar");
try {
requestLogDao.save(requestLog);
} catch (Exception ex) {
System.out.println("-------------ERROR-------------------");
ex.printStackTrace();
}
}
}
RequestLogDAO
public interface RequestLogDao extends CrudRepository<RequestLog, Integer> {
}
RquestLog
#Entity
#Table(name = "REQUEST_LOG")
public class RequestLog {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "RequestLogSeq")
#SequenceGenerator(name = "RequestLogSeq", sequenceName = "REQUEST_LOG_SEQ")
private Integer id;
#Column
#NotNull
private Date date;
#NotNull
private String source;
public void setDate(Date date) {
this.date = date;
}
public void setSource(String source) {
this.source = source;
}
}
}
My main Class:
#ComponentScan({"ar.com.tr.latam.care", "ar.com.tr.latam.care.repository", "ar.com.tr.latam.care.filtro"})
#EnableJpaRepositories
#SpringBootApplication
public class InitBatch implements CommandLineRunner {
#Autowired
private Batch batch;
#Override
public void run(String... args) throws Exception {
batch.processFiles();
}
public static void main(String[] args) throws Exception {
long startTime = System.nanoTime();
SpringApplication.run(InitBatch.class, args).close();
System.out.println(String.format("Total process time (%s) seconds",(System.nanoTime() - startTime) / 1000000000));
}
}
How do I export the project? First, I do a mvn clean install for the Library project . After this I go to my Project and do mvn clean package.
This creates a Runnable jar that works, expect for this problem that can't save anyhing on the DB.
Also, the RequestLogDao is made with JPA.
Any help? Thanks in advance!
Related
I have a question about Cucumber library, I was taking a course of selenium with cucumber and testNg, but I have run into some problems because some methods no longer exist
For Example:
I cant used "CucumberFeatureWrapper" what could be the relative to this? Image: https://i.stack.imgur.com/3JMcD.png
The same way happens when i need to used .provideFeatures(); in testNgCucumberRunner.provideFeatures Image: https://i.stack.imgur.com/L76xQ.png
POM:
e<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>groupId</groupId>
<artifactId>CRMFramework</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>10</source>
<target>10</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<testFailureIgnore>true</testFailureIgnore>
<suiteXmlFiles>
<suiteXmlFile>src/test/java/TestNg.xml</suiteXmlFile>
</suiteXmlFiles>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
<plugin>
<groupId>net.masterthought</groupId>
<artifactId>maven-cucumber-reporting</artifactId>
<version>5.4.0</version>
<executions>
<execution>
<id>execution</id>
<phase>verify</phase>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<projectName>ExecuteAutomation</projectName>
<!-- output directory for the generated report -->
<outputDirectory>${project.build.directory}/cucumber-reports</outputDirectory>
<inputDirectory>${project.build.directory}/cucumber-json-report.json</inputDirectory>
<jsonFiles>
<!-- supports wildcard or name pattern -->
<param>**/*.json</param>
</jsonFiles>
<mergeFeaturesWithRetest>true</mergeFeaturesWithRetest>
<mergeFeaturesById>true</mergeFeaturesById>
<checkBuildResult>false</checkBuildResult>
<skipEmptyJSONFiles>true</skipEmptyJSONFiles>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<properties>
<maven.compiler.source>15</maven.compiler.source>
<maven.compiler.target>15</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>3.141.59</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/net.sourceforge.jexcelapi/jxl -->
<dependency>
<groupId>net.sourceforge.jexcelapi</groupId>
<artifactId>jxl</artifactId>
<version>2.6.12</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>6.9.1</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-core</artifactId>
<version>6.9.1</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
<version>6.9.1</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-testng</artifactId>
<version>6.9.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.testng/testng -->
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.3.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/io.cucumber/datatable-dependencies -->
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>datatable-dependencies</artifactId>
<version>1.0.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.aventstack/extentreports -->
<dependency>
<groupId>com.aventstack</groupId>
<artifactId>extentreports</artifactId>
<version>5.0.6</version>
</dependency>
<dependency>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
<version>RELEASE</version>
<scope>compile</scope>
</dependency>
</dependencies>
TestRunner
import com.aventstack.extentreports.gherkin.model.Feature;
import com.crm.framework.utilities.ExtentReport;
import io.cucumber.testng.*;
import org.testng.annotations.Test;
import org.testng.ITestContext;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.DataProvider;
import java.util.List;
//json:target/cucumber-reports/cucumberTestReport.json
#CucumberOptions(
features = {"src/test/java/features/"},
glue = {"steps"},
plugin = {"json:target/cucumber-json-report.json",
"pretty", "html:target/cucumber-report-html"})
public class TestRunner {
private TestNGCucumberRunner testNGCucumberRunner;
#BeforeClass(alwaysRun = true)
public void setUpClass() {
testNGCucumberRunner = new TestNGCucumberRunner(this.getClass());
}
#Test(dataProvider = "features")
public void LoginTest(CucumberFeatureWrapper cucumberFeatureWrapper) throws ClassNotFoundException {
//Insert the Feature Name
//ExtentReport.startFeature("login").assignAuthor("DiegoHM").assignDevice("Chrome").assignCategory("Regression");
}
#DataProvider
public Object[] features(ITestContext context) {
// return testNGCucumberRunner.
return testNGCucumberRunner.provideFeatures();
}
#AfterClass(alwaysRun = true)
public void afterClass() {
testNGCucumberRunner.finish();
}
}
You're using classes that don't exist anymore. You can find out which classes are in a library by using ctrl/cmd + clicking on a package name in most modern IDEs.
So try using:
package io.cucumber.examples.testng;
import io.cucumber.testng.CucumberOptions;
import io.cucumber.testng.FeatureWrapper;
import io.cucumber.testng.PickleWrapper;
import io.cucumber.testng.TestNGCucumberRunner;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
#CucumberOptions(....)
public class RunCucumberByCompositionTest {
private TestNGCucumberRunner testNGCucumberRunner;
#BeforeClass(alwaysRun = true)
public void setUpClass() {
testNGCucumberRunner = new TestNGCucumberRunner(this.getClass());
}
#Test(groups = "cucumber", description = "Runs Cucumber Scenarios", dataProvider = "scenarios")
public void scenario(PickleWrapper pickle, FeatureWrapper cucumberFeature) {
testNGCucumberRunner.runScenario(pickle.getPickle());
}
#DataProvider
public Object[][] scenarios() {
return testNGCucumberRunner.provideScenarios();
}
#AfterClass(alwaysRun = true)
public void tearDownClass() {
testNGCucumberRunner.finish();
}
}
From:
https://github.com/cucumber/cucumber-jvm/blob/main/examples/java-calculator-testng/src/test/java/io/cucumber/examples/testng
I'm developing a web app with spring boot, I built the project with Maven.
In the project I use files .properties and I gave the path of the project like this:
Properties FileProperties = FileUtils.getProperties("src\main\resources\file.properties");
Running the project with IntelliJ, all work.
But at the moment I built with maven in Jar, and I open the web app, don't find the file properties and give NullPointerException.
This is my pom.xml
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.6.RELEASE</version>
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-freemarker</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-thymeleaf</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.9.9</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.6</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
This is the project map.
https://m.imgur.com/gallery/zZzTzID
Thanks so much
I tried this, but dont work.
ConfigProperties.java
package bperTube.transfer.Utils;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
#Configuration
#PropertySource("classpath:file.properties")
#ConfigurationProperties(prefix = "upload")
public class ConfigProperties {
private String dateFormat;
private String directoryPath;
private String videoDirectory;
private String imageDirectory;
public String getDateFormat() {
return dateFormat;
}
public void setDateFormat(String dateFormat) {
this.dateFormat = dateFormat;
}
public String getDirectoryPath() {
return directoryPath;
}
public void setDirectoryPath(String directoryPath) {
this.directoryPath = directoryPath;
}
public String getVideoDirectory() {
return videoDirectory;
}
public void setVideoDirectory(String videoDirectory) {
this.videoDirectory = videoDirectory;
}
public String getImageDirectory() {
return imageDirectory;
}
public void setImageDirectory(String imageDirectory) {
this.imageDirectory = imageDirectory;
}
}
file.properties
upload.dateFormat=dd-MM-yyyy
upload.directoryPath=/test/
upload.videoDirectory=swf/
upload.imageDirectory=poster/
I´m trying to use spring-boot inside my AWS lambda application to make calls to a SOAP web-service. But looks like it isn´t autowiring my SOAP component.
Here´s my code:
#SpringBootApplication(scanBasePackages={"com.fenix"})
#EnableAutoConfiguration
#ComponentScan
public class Aplicacao extends SpringBootServletInitializer {
public static void main(String[] args) {
SpringApplication.run(Aplicacao.class, args);
}
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder springApplicationBuilder) {
return springApplicationBuilder.sources(Aplicacao.class);
}
}
#Configuration
public class Beans {
#Bean
public Jaxb2Marshaller marshaller() {
Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
marshaller.setPackagesToScan("com.tranzaxis.schemas", "org.radixware.schemas", "org.xmlsoap.schemas", "com.compassplus.schemas");
return marshaller;
}
#Bean
public TranClient tranClient(Jaxb2Marshaller marshaller) {
TranClient client = new TranClient();
client.setDefaultUri("http://rhel72.tx:12301?wsdl");
client.setMarshaller(marshaller);
client.setUnmarshaller(marshaller);
return client;
}
#Bean(name = "Tran")
public TranClient getTranClient() {
return tranClient(marshaller());
}
}
public class PostMovimentacao extends Handler implements Service {
#Autowired
#Qualifier("Tran")
private TranClient tranClient;
#Inject
private PessoaCompassBO pessoaCompassBO;
private CompassConfig compassConfig;
private static final Logger LOGGER = Logger.getLogger(PostMovimentacao.class);
#Override
protected ResponseEntity execute(ApiRequest request, Context context) throws HttpException {
MovimentacaoRequest movimentacaoRequest = new MovimentacaoRequest();
movimentacaoRequest.setOrigem(669L);
movimentacaoRequest.setDestino(657L);
movimentacaoRequest.setValor(BigDecimal.valueOf(1L));
TranInvoke invoke = tranClient.movimentacaoFinanceira(movimentacaoRequest, compassConfig); -->NullPointer here
return ResponseEntity.of(Optional.of(invoke), Optional.empty(), HttpStatus.SC_OK);
}
#Override
public void setup() {
try {
compassConfig = CompassConfig.build();
} catch (InvalidConfigException e) {
LOGGER.error("Compass config error", e);
}
}
}
Here´s my pom.xml:
<build>
<finalName>integrador-compass</finalName>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>1.5.6.RELEASE</version>
<configuration>
<layout>MODULE</layout>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.0</version>
<configuration>
<source>${maven.compiler.source}</source>
<target>${maven.compiler.target}</target>
<encoding>${project.build.sourceEncoding}</encoding>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.3</version>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>1.1.11</version>
<configuration>
<targetClasses>
<param>com.fenix.*</param>
</targetClasses>
<excludedClasses>
<excludedClasse>com.fenix.handler.request*</excludedClasse>
<excludedClasse>com.fenix.handler.response*</excludedClasse>
<excludedClasse>com.fenix.model*</excludedClasse>
</excludedClasses>
<avoidCallsTo>
<avoidCallsTo>java.util.logging</avoidCallsTo>
<avoidCallsTo>org.apache.log4j</avoidCallsTo>
<avoidCallsTo>org.slf4j</avoidCallsTo>
<avoidCallsTo>org.apache.commons.logging</avoidCallsTo>
</avoidCallsTo>
<timestampedReports>false</timestampedReports>
<outputFormats>
<outputFormat>XML</outputFormat>
<outputFormat>HTML</outputFormat>
</outputFormats>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<includes>
<include>**/*Tests.java</include>
<include>**/*Test.java</include>
</includes>
</configuration>
</plugin>
</plugins>
<extensions>
<extension>
<groupId>org.springframework.build</groupId>
<artifactId>aws-maven</artifactId>
<version>5.0.0.RELEASE</version>
</extension>
</extensions>
</build>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.3.0.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.springframework.ws</groupId>
<artifactId>spring-ws-core</artifactId>
</dependency>
<dependency>
<groupId>com.fenix</groupId>
<artifactId>compass-api</artifactId>
<version>0.0.15</version>
</dependency>
<dependency>
<groupId>com.fenix</groupId>
<artifactId>lambda-commons</artifactId>
<version>1.6.2</version>
</dependency>
<dependency>
<groupId>commons-validator</groupId>
<artifactId>commons-validator</artifactId>
<version>1.5.1</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>2.6.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>3.6.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.mashape.unirest</groupId>
<artifactId>unirest-java</artifactId>
<version>1.4.9</version>
</dependency>
</dependencies>
Has anyone already used this configuration? I added spring-boot cause it was the only way to make my SOAP calls. If I try to make call just using code from WSDL, when it tries to connect to server, I got an error saying that request was empty. With spring-boot it doesn´t need to connect first, it just sends the request and it works fine.
Any help is welcome.
Thanks a lot.
Apparently, the Spring Boot application context is not being initialized in your code...If you really want to use Spring Boot in your Lambda Function you could try something like:
pom.xml
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.4.0.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
<version>1.11.181</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-core</artifactId>
<version>1.1.0</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-lambda</artifactId>
<version>1.11.181</version>
</dependency>
</dependencies>
Service Example:
#Component
public class MyService {
public void doSomething() {
System.out.println("Service is doing something");
}
}
Some Bean Example:
#Component
public class MyBean {
#Autowired
private MyService service;
public void executeService() {
service.doSomething();
}
}
A Lambda Handler Example:
#SpringBootApplication
public class LambdaHandler implements RequestHandler<Request, Response> {
private ApplicationContext getApplicationContext(String [] args) {
return new SpringApplicationBuilder(LambdaHandler.class)
.web(false)
.run(args);
}
public Response handleRequest(Request input, Context context) {
ApplicationContext ctx = getApplicationContext(new String[]{});
MyBean bean = ctx.getBean(MyBean.class);
bean.executeService();
return new Response();
}
}
Here is my Spark Code and pom.xml. The issue with this program is JavaStreamingContext is streaming only first batch of records, it's not streaming any further and i am getting error at jscc.start().
Can anyone give me some clue on why this is happening. Is there anything wrong with my spark dependencies?
17/04/11 10:32:20 ERROR StreamingContext: Error starting the context, marking it as stopped
java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute
at scala.Predef$.require(Predef.scala:224)
at org.apache.spark.streaming.DStreamGraph.validate(DStreamGraph.scala:163)
at org.apache.spark.streaming.StreamingContext.validate(StreamingContext.scala:513)
at org.apache.spark.streaming.StreamingContext.liftedTree1$1(StreamingContext.scala:573)
at org.apache.spark.streaming.StreamingContext.start(StreamingContext.scala:572)
at org.apache.spark.streaming.api.java.JavaStreamingContext.start(JavaStreamingContext.scala:554)
at com.comcast.emm.vodip.WholeTextLocal.WholeTextLocal.main(WholeTextLocal.java:72)
Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute
at scala.Predef$.require(Predef.scala:224)
at org.apache.spark.streaming.DStreamGraph.validate(DStreamGraph.scala:163)
at org.apache.spark.streaming.StreamingContext.validate(StreamingContext.scala:513)
at org.apache.spark.streaming.StreamingContext.liftedTree1$1(StreamingContext.scala:573)
at org.apache.spark.streaming.StreamingContext.start(StreamingContext.scala:572)
at org.apache.spark.streaming.api.java.JavaStreamingContext.start(JavaStreamingContext.scala:554)
at com.comcast.emm.vodip.WholeTextLocal.WholeTextLocal.main(WholeTextLocal.java:72)
Code: pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.abcd.emm.mpp</groupId>
<artifactId>WholeTextLocal</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>WholeTextLocal</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<build>
<plugins>
<!-- Maven Shade Plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<executions>
<!-- Run shade goal on package phase -->
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<excludeScope>system</excludeScope>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<excludeGroupIds>junit,org.mockito,org.hamcrest</excludeGroupIds>
<transformers>
<!-- add Main-Class to manifest file -->
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>com.abcd.emm.mpp.WholeTextLocal.WholeTextLocal</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
<configuration>
<finalName>${project.artifactId}</finalName>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.apache.commons/commons-compress -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
<version>1.5</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.4</version>
<exclusions>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId>
<version>2.4.4</version> </dependency> -->
<dependency>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
<version>1.8.0_101</version>
<scope>system</scope>
<systemPath>${JAVA_HOME}/lib/tools.jar</systemPath>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.4</version>
<exclusions>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- required at run time by hive -->
<dependency>
<groupId>org.datanucleus</groupId>
<artifactId>datanucleus-core</artifactId>
<version>3.2.2</version>
</dependency>
<dependency>
<groupId>org.datanucleus</groupId>
<artifactId>datanucleus-api-jdo</artifactId>
<version>3.2.1</version>
</dependency>
<dependency>
<groupId>org.datanucleus</groupId>
<artifactId>datanucleus-rdbms</artifactId>
<version>3.2.1</version>
</dependency>
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.sparkjava/spark-core -->
<!-- https://mvnrepository.com/artifact/com.sparkjava/spark-core -->
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.0.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming_2.11 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>2.0.2</version>
</dependency>
<!-- <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-streaming_2.10</artifactId>
<version>2.0.2</version> <scope>provided</scope> </dependency> -->
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.10 -->
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.11 -->
<!-- <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-sql_2.11</artifactId>
<version>2.0.2</version> </dependency> -->
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-hive_2.10 -->
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-hive_2.10 -->
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-hive_2.11 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.11</artifactId>
<version>2.0.2</version>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-metastore</artifactId>
<version>0.14.0</version>
</dependency>
<dependency>
<groupId>org.mariadb.jdbc</groupId>
<artifactId>mariadb-java-client</artifactId>
<version>1.2.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-hive-thriftserver_2.11 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive-thriftserver_2.11</artifactId>
<version>2.0.2</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.6.0</version>
<!-- <scope>provided</scope> -->
</dependency>
<!-- http://mvnrepository.com/artifact/org.apache.avro/avro-tools -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.6.0</version>
<!--<scope>provided</scope> -->
</dependency>
<!-- <dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId>
<version>1.2.17</version> </dependency> -->
<!-- <dependency> <groupId>org.scalatest</groupId> <artifactId>scalatest_2.10</artifactId>
<version>2.2.4</version> <scope>test</scope> </dependency> -->
</dependencies>
</project>
Java code:
package com.comcast.emm.vodip.WholeTextLocal;
import java.util.Arrays;
import java.util.Iterator;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaPairRDD;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.function.FlatMapFunction;
import org.apache.spark.api.java.function.Function;
import org.apache.spark.api.java.function.VoidFunction;
import org.apache.spark.streaming.Durations;
import org.apache.spark.streaming.api.java.JavaStreamingContext;
import scala.Tuple2;
public class WholeTextLocal
{
public static void main(String[] args) throws InterruptedException
{
SparkConf sparkConf = new SparkConf().setAppName("My app").setMaster("local")
.set("spark.driver.allowMultipleContexts", "true");
JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, Durations.seconds(15));
JavaPairRDD<String, String> WholeTextLocalFiles = jssc.sparkContext()
.wholeTextFiles("C:/Users/aaa/files/abcd7/simple/*.txt");
// Pick the content.
JavaRDD<String> myRDD = WholeTextLocalFiles.map(new Function<Tuple2<String, String>, String>()
{
private static final long serialVersionUID = -551872585218963131L;
public String call(Tuple2<String, String> v1) throws Exception
{
System.out.println("v1._2=" + v1._2);
return v1._2;
}
});
// Split the content in each file with /n character.
JavaRDD<String> myRDD2 = myRDD.flatMap(new FlatMapFunction<String, String>()
{
public Iterator<String> call(String t) throws Exception
{
return Arrays.asList(t.split("\\r?\\n")).iterator();
}
});
// Loop through and print.
myRDD2.foreachPartition(new VoidFunction<Iterator<String>>()
{
private static final long serialVersionUID = -4895942417886562330L;
public void call(Iterator<String> t) throws Exception
{
while (t.hasNext())
{
System.out.println(t.next());
}
}
});
jssc.start();
jssc.awaitTermination();
}
}
UPDATED CODE: this is streaming. but, sysout is not printing anything eventhough file is being recognized by spark.
public class WholeTextLocal
{
public static void main(String[] args) throws InterruptedException
{
SparkConf sparkConf = new SparkConf().setAppName("My app").setMaster("local")
.set("spark.driver.allowMultipleContexts", "true");
JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, Durations.seconds(10));
JavaPairInputDStream<Text, Text> dStream = jssc.fileStream("C:/Users/aaa/files/abcd7/simple/", Text.class, Text.class, WholeTextFileInputFormat.class);
// Pick the content.
JavaDStream<String> myRDD = dStream.map(new Function<Tuple2<Text, Text>, String>()
{
private static final long serialVersionUID = -551872585218963131L;
#Override
public String call(Tuple2<Text, Text> v1) throws Exception
{
System.out.println("v1._2=" + v1._1);
return v1._2.toString();
}
});
// Split the content in each file with /n character.
JavaDStream<String> myRDD2 = myRDD.flatMap(new FlatMapFunction<String, String>()
{
#Override
public Iterator<String> call(String t) throws Exception
{
return Arrays.asList(t.split("\\r?\\n")).iterator();
}
});
// Loop through and print.
myRDD2.foreachRDD(itr -> new VoidFunction<Iterator<String>>()
{
private static final long serialVersionUID = -4895942417886562330L;
#Override
public void call(Iterator<String> t) throws Exception
{
while (t.hasNext())
{
System.out.println(t.next());
}
}
});
myRDD2.count();
jssc.start();
jssc.awaitTermination();
}
}
I think that error message tells you everything:
No output operations registered, so nothing to execute at
You must add some action at the end, i.e. foreachRDD instead of foreachPartition:
myRDD2.foreachRDD(new VoidFunction<JavaRDD<String>>() {
private static final long serialVersionUID = -4895942417886562330L;
public void call(JavaRDD<String> rdd) throws Exception {
rdd.foreachPartition(new VoidFunction<Iterator<String>>() {
private static final long serialVersionUID = -1L;
public void call(Iterator<String> rdd) throws Exception {
while (t.hasNext()) {
System.out.println(t.next());
}
}
}
});
Note: you can use foreachPartition inside foreachRDD
Second:
you are reading files using JavaRDD.wholeTextFiles. It's not a streaming function. You should read using JavaStreamingContext.textFileStream and operate on DStreams
You can use also fileStream with input format like WholeTextFileInputFormat:
JavaPairInputDStream<Text, Text> dStream = jssc.fileStream("dir", Text.class, Text.class, WholeTextFileInputFormat.class);
This is my pom
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mkyong.common</groupId>
<artifactId>RESTfulExample</artifactId>
<packaging>war</packaging>
<version>1.0-SNAPSHOT</version>
<name>RESTfulExample Maven Webapp</name>
<url>http://maven.apache.org</url>
<repositories>
<repository>
<id>maven2-repository.java.net</id>
<name>Java.net Repository for Maven</name>
<url>http://download.java.net/maven/2/</url>
<layout>default</layout>
</repository>
</repositories>
<properties>
<project.build.java.target>1.6</project.build.java.target>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-server</artifactId>
<version>1.8</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>4.3.0.Final</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.28</version>
</dependency>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjrt</artifactId>
<version>1.6.5</version>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.9.5</version>
</dependency>
<dependency>
<groupId>com.sun.jersey.contribs.jersey-oauth</groupId>
<artifactId>oauth-server</artifactId>
<version>1.18</version>
</dependency>
<dependency>
<groupId>com.sun.jersey.contribs.jersey-oauth</groupId>
<artifactId>oauth-signature</artifactId>
<version>1.18</version>
</dependency>
<dependency>
<groupId>com.sun.jersey.contribs.jersey-oauth</groupId>
<artifactId>oauth-client</artifactId>
<version>1.18</version>
</dependency>
</dependencies>
<build>
<finalName>RESTfulExample</finalName>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>aspectj-maven-plugin</artifactId>
<version>1.3.1</version>
<configuration>
<complianceLevel>1.6</complianceLevel>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<!--<goal>test-compile</goal> -->
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
This is my oath filter class
public class OAuthAuthenticationFilter1 implements ContainerRequestFilter {
public ContainerRequest filter(ContainerRequest containerRequest) {
OAuthServerRequest request = new OAuthServerRequest(containerRequest);
OAuthParameters params = new OAuthParameters();
params.readRequest(request);
OAuthSecrets secrets = new OAuthSecrets();
OAuthParameters param=new OAuthParameters();
secrets.consumerSecret("OwnAccount");
param.consumerKey("OwnAccount");
try {
if(!OAuthSignature.verify(request, params, secrets)) {
throw new WebApplicationException(401);
}
} catch (OAuthSignatureException e) {
throw new WebApplicationException(e, 401);
}
return containerRequest;
}
}
This is my client
public class Sample1 {
public static final String HOSTNAME = "http://localhost:8080/RESTfulExample/rest/hello/planetext";
public static final String CONSUMER_KEY = "OwnAccount";
public static final String CONSUMER_SECRET = "OwnAccount";
/**
* #param args
*/
public static void main(String[] args) {
Client client = Client.create();
OAuthParameters params = new OAuthParameters().signatureMethod("HMAC-SHA1").consumerKey(CONSUMER_KEY);
OAuthSecrets secrets = new OAuthSecrets().consumerSecret(CONSUMER_SECRET);
OAuthClientFilter filter = new OAuthClientFilter(client.getProviders(), params, secrets);
WebResource res = client.resource(HOSTNAME );
res.addFilter(filter);
String responseString = res.get(String.class);
System.out.println(responseString);
}
}
I am using following oath and jersey pom referring the site
http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22com.sun.jersey.contribs.jersey-oauth%22
<dependency>
<groupId>com.sun.jersey.contribs.jersey-oauth</groupId>
<artifactId>oauth-server</artifactId>
<version>1.18</version>
</dependency>
<dependency>
<groupId>com.sun.jersey.contribs.jersey-oauth</groupId>
<artifactId>oauth-signature</artifactId>
<version>1.18</version>
</dependency>
<dependency>
<groupId>com.sun.jersey.contribs.jersey-oauth</groupId>
<artifactId>oauth-client</artifactId>
<version>1.18</version>
</dependency>
When following code on filter class is called
try {
// exception is thrown from here
if(!OAuthSignature.verify(request, params, secrets)) {
throw new WebApplicationException(401);
}
} catch (OAuthSignatureException e) {
throw new WebApplicationException(e, 401);
}
I am getting this exception
com.sun.jersey.oauth.signature.UnsupportedSignatureMethodException: HMAC-SHA1
at com.sun.jersey.oauth.signature.OAuthSignature.getSignatureMethod(OAuthSignature.java:257)
I am using tomcat 7.
I searched on the net but could not find the cause for tomcat .What i am missing ?Do any one has faced it for tomcat ?
I solved this problem when i updated jersey-server jar to 1.18.I was using jersey-server 1.8 jar before.