I have a Lambda function that takes around ~10 seconds to start up using Java-11. I did some googling and came across a number of posts that suggest that lowering the JAR size of the package may help with quicker start times (less redundant libraries loaded etc...).
I also read in some posts that using below may help and tried to add
<scope>provided</scope>
in the AWS related dependencies thinking that well...AWS Lambda would have AWS specific libraries present? Turns out that is not the case! adding scope provided does not work when trying to execute the function.
My current pom.xml is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>REMOVED</groupId>
<artifactId>REMOVED</artifactId>
<version>REMOVED</version>
<properties>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>30.1-jre</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.20</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.7.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.7.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20220320</version>
</dependency>
<dependency>
<groupId>software.amazon.kinesis</groupId>
<artifactId>amazon-kinesis-client</artifactId>
<version>2.4.1</version>
</dependency><dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-kinesis</artifactId>
<version>1.12.228</version>
</dependency>
<!-- <dependency><groupId>software.amazon.awssdk</groupId><artifactId>firehose</artifactId><version>2.17.198</version></dependency> -->
<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/kinesis -->
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>kinesis</artifactId>
<version>2.17.201</version>
</dependency>
<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/secretsmanager -->
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>secretsmanager</artifactId>
<version>2.17.204</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.0</version>
</plugin>
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>3.1.0</version>
</plugin>
<!-- default lifecycle, jar packaging: see https://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
</plugin>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
</plugin>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
<!-- site lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#site_Lifecycle -->
<plugin>
<artifactId>maven-site-plugin</artifactId>
<version>3.7.1</version>
</plugin>
<plugin>
<artifactId>maven-project-info-reports-plugin</artifactId>
<version>3.0.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
AWS Kinesis usage is just to create the KinesisClient and use PutRecordRequest and PutRecordResponse, with AWS Secret Manager , just to access specific secrets.
I am not that versed in AWS API, from my understanding I am already including a limited amount of libraries required to perform those tasks based on the dependencies.
The JAR file is about 65 MB large. Would I be able to optimise dependency loads further? And I guess would lowering the JAR file increase start up time of the Lambda function?
Thank you,
I think you've accidentally included three copies of the same dependency. You just want the latest 2.x.x version of the Kinesis library.
You certainly don't need a 1.x.x SDK and a 2.x.x SDK.
This article does not reduce the size of your Lambda (which I am also struggling with), but it helps with cold starts without any code changes.
In their example it changed the start time from about 5 seconds to 2 seconds. It worked for me.
Simply add an lambda environment variable:
Key: JAVA_TOOL_OPTIONS
Value: -XX:+TieredCompilation -XX:TieredStopAtLevel=1
There is also an article that describes how to remove Netty and Apache clients in favour of the built in JDK client. This reduces the size.
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>url-connection-client</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<exclusions>
<exclusion>
<groupId>software.amazon.awssdk</groupId>
<artifactId>netty-nio-client</artifactId>
</exclusion>
<exclusion>
<groupId>software.amazon.awssdk</groupId>
<artifactId>apache-client</artifactId>
</exclusion>
</exclusions>
</dependency>
Use the UrlConnectionHttpClient which wraps Java's HTTP client.
S3Client client = S3Client.builder()
.region(Region.US_WEST_2)
.credentialsProvider(EnvironmentVariableCredentialsProvider.create())
.httpClient(UrlConnectionHttpClient.builder().build())
.build();
Finally you can use Jackson-jr which is a +- 100kb version of Jackson.
Related
I set up Selenium version 4.7.1 in my project in IntelliJ using maven. In order to get past the error reported here, I added an explicit dependency for guava version 31.1 (which is what is included in the lib folder of the downloadable package on the Selenium website for that version) and I added an exclusion on the selenium dependency for guava.
Now the application works fine when I run it in IntelliJ, but I'm getting this error when I run the jar in my terminal:
Exception in thread "main" java.lang.NoClassDefFoundError: org/openqa/selenium/WebDriver
...
Caused by: java.lang.ClassNotFoundException: org.openqa.selenium.WebDriver
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
Other similar questions I've found, like this one, all focus on the guava issue, but it seems like at this point my problem must have something to do with my build configuration in IntelliJ and the jar that is getting created.
I did find some discussions that seemed more pertinent to my issue, but they were geared towards people manually adding the jars, and the advice given was usually to switch to use maven or gradle, but I'm already using maven. I have gone to Project Settings > Libraries and removed any external jars so that all dependencies should be getting pulled in by Maven.
Another discussion I found suggested using the maven-shaded-plugin, which I added and did mvn clean install, but still no luck.
How can I make sure the jar created by IntelliJ runs exactly the same as it runs in IntelliJ? Any help is appreciated.
Here is my current pom.xml, which includes a suggestion from the current answer, but at this point I'm still experiencing the same issue:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>groupId</groupId>
<artifactId>stock-analysis</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>com.github.sh0nk</groupId>
<artifactId>matplotlib4j</artifactId>
<version>0.5.0</version>
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-web</artifactId>
<version>5.3.6</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-openfeign-core</artifactId>
<version>3.0.2</version>
</dependency>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-mapper-asl</artifactId>
<version>1.9.13</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.13</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.4.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.4.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>3.8.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>RELEASE</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>31.1-jre</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>4.7.2</version>
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.6</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<addDefaultImplementationEntries>false</addDefaultImplementationEntries>
<packageName>main.java.termaat.stocks.strategy.processing</packageName>
<mainClass>StockReportWriter</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
<includeScope>compile</includeScope>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<properties>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
</properties>
Here is a screenshot of my project structure after the updates to my pom. Currently the stock-analysis-1.0-SNAPSHOT.jar that is built is much to small, so I'm trying to figure out if something is configured wrong with the maven-jar-plugin.
Probably you aren't building dependencies correctly. U need both:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.6</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<addDefaultImplementationEntries>false</addDefaultImplementationEntries>
<packageName>your package</packageName>
<mainClass>your class</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
<includeScope>compile</includeScope>
</configuration>
</execution>
</executions>
</plugin>
It's hard to help if you don't post your pom.xml though
I am trying to package my JavaFX application using Maven, running it using exec:java is no problem and running maven - package yields no errors. But when I want to execute the runnable jar it returns the error 'Error: Could not find or load main class .\FormulaTelemetryApp-1.0-SNAPSHOT.jar' where the second part is the jar I want to execute.
Beneath I included my POM, I already went looking for solutions, this is where I stumbled upon the maven-jar-plugin, but including that didn't solve the problem. I also tried cleaning the project and compiling/packaging again but that didn't change anything either.
As you might notice, I am fairly new to Maven (trying to self-learn it) so there might be some strange things in the POM.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.formulaElectric</groupId>
<artifactId>FormulaTelemetryApp</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>FormulaTelemetryApp</name>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<release>11</release>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
<configuration>
<mainClass>org.feb.telemetry.application.TelemetryApplication</mainClass>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.1.1</version>
<configuration>
<archive>
<index>true</index>
<manifest>
<mainClass>org.feb.telemetry.application.TelemetryApplication</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
<distributionManagement>
<!-- use the following if you're not using a snapshot version. -->
<repository>
<id>localSnap</id>
<name>RepositoryProxyRel</name>
<url>http://127.0.0.1:8080/nexus/content/repositories/releases/</url>
</repository>
<!-- use the following if you ARE using a snapshot version. -->
<snapshotRepository>
<id>MylocalSnap</id>
<name>RepositoryProxySnap</name>
<url>http://127.0.0.1:8080/nexus/content/repositories/snapshots/</url>
</snapshotRepository>
</distributionManagement>
<dependencies>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-controls</artifactId>
<version>11</version>
</dependency>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-fxml</artifactId>
<version>11</version>
</dependency>
<dependency>
<groupId>org.controlsfx</groupId>
<artifactId>controlsfx</artifactId>
<version>9.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>4.1</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.23</version>
</dependency>
<dependency>
<groupId>com.fazecast</groupId>
<artifactId>jSerialComm</artifactId>
<version>1.3.11</version>
</dependency>
<dependency>
<groupId>eu.hansolo</groupId>
<artifactId>tilesfx</artifactId>
<version>1.4.7</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>23.0-rc1</version>
</dependency>
<dependency>
<groupId>com.sun.mail</groupId>
<artifactId>javax.mail</artifactId>
<version>1.6.2</version>
</dependency>
<dependency>
<groupId>org.xerial</groupId>
<artifactId>sqlite-jdbc</artifactId>
<version>3.21.0</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.1</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>com.jcraft</groupId>
<artifactId>jsch</artifactId>
<version>0.1.55</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.2</version>
</dependency>
<dependency>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
<version>RELEASE</version>
<scope>compile</scope>
</dependency>
</dependencies>
</project>
As stated in the comments, the problem was two-fold: on one hand a fat jar was needed, maven-shade-plugin was used for that. Besides that a launcher class (non-application extending class) was needed as main class, as is stated in the answer to this question.
I generally use JEE and Glassfish to create web applications, however I've created a web application using Jetty which does not require a container. However each time I run the application I have to manually stop the currently running instance. This doesn't take long but it is a bit of a pain. I would like a smoother development cycle, I'm sure it can be automated at the command line and will do so, but I think it should be a pretty common issue.
I'm creating my project using NB 8.2.
File -> New Project -> Under Categories selecting "Maven", Under Projects "Java Application"
Each time the project is run, a new running instance is deployed, such that after 5 runs there are 5 running programs (well that would normally be the case for this type of application but because of a port conflict I must stop the previous instance).
Any of the following would be a viable solution:
Ideally, if their is a running instance of the program it is compiled and relaunched on save, don't know if this is possible outside of a container like Glassfish.
The application kills any previous deployment and compiles and runs a new instance when "Run Project" button is pressed (F6).
Anything else that meets this intention.
For pom based solutions here is what I'm working with:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.kenmcwilliams</groupId>
<artifactId>MpawServices</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<dependencies>
<!-- Web dependencies -->
<dependency>
<groupId>com.sparkjava</groupId>
<artifactId>spark-core</artifactId>
<version>2.5</version>
<type>jar</type>
</dependency>
<!-- JSON parsing -->
<dependency>
<groupId>io.fastjson</groupId>
<artifactId>boon</artifactId>
<version>0.34</version>
</dependency>
<!-- needed for DB access -->
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>5.2.10.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate.javax.persistence</groupId>
<artifactId>hibernate-jpa-2.1-api</artifactId>
<version>1.0.0.Final</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.mariadb.jdbc</groupId>
<artifactId>mariadb-java-client</artifactId>
<version>1.6.0</version>
</dependency>
<!-- needed for Dagger2 DI -->
<dependency>
<groupId>com.google.dagger</groupId>
<artifactId>dagger</artifactId>
<version>2.11</version>
</dependency>
<dependency>
<groupId>com.google.dagger</groupId>
<artifactId>dagger-compiler</artifactId>
<version>2.11</version>
<optional>true</optional>
</dependency>
<!-- to prevent: SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder” -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>
<!-- AspectJ -->
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjrt</artifactId>
<version>1.8.7</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>aspectj-maven-plugin</artifactId>
<version>1.8</version>
<configuration>
<complianceLevel>1.8</complianceLevel>
<source>1.8</source>
<target>1.8</target>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
<configuration>
<mainClass>com.kenmcwilliams.mpawservices.App</mainClass>
</configuration>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
I don't think there is such a feature in NetBeans at the moment, but you should be able to file a feature request at the NetBeans Jira Issue Tracker. In fact, it would be a nice feature, I also searched for something similar in the past.
When I'm using spark 1.6.1, everything is alright. When I switch to Spark 2.1.0, I come across the problem below:
Task 33 in stage3.0 failed 4times; aborting job
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 33 in stage 3.0 failed 4 times, most recent failure: Lost taks 33.3 in stage 3.0 (TID 310, 192.168.1.5, executor 3): java.io.invalidclassexception scala.tuple2; local class incompatible; local class incompatible: stream classdesc serialVersionUID = -4864544146559264103, local class serialVersionUID = 3356420310891166197
I know -4864544146559264103 is correspond to scala 2.10, while 3356420310891166197 is correspond to scala 2.11. Although I changed my configuration to
EDIT: the entire pom file shows below.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>test.spark</groupId>
<artifactId>spark</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>spark</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<build>
<plugins>
<plugin>
<!-- solve the problem of : java.lang.ClassNotFoundException: kafka.producer.ProducerConfig -->
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.11</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka-0-8_2.11</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-server</artifactId>
<version>1.1.2</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.1.2</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-common</artifactId>
<version>1.1.2</version>
</dependency>
<dependency>
<groupId>io.fastjson</groupId>
<artifactId>boon</artifactId>
<version>0.33</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.7</version>
</dependency>
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1.1</version>
</dependency>
</dependencies>
</project>
the problem is still exists. How to fix this problem? Any detail needed will be added. Thanks for any help!
Finally, I fixed this problem. It's my fault, the pom file is alright, and the project works well.
The problem is resulted from one detail, the code reads scala.Tuple2 object from HDFS, which is not mentioned in my question (I'm sorry to say this). The objects in HDFS are generated with scala 2.10 by another project, so the problem occurs.
Anyway, thanks for your help.
We are using GWT in our product with mojo gwt-maven-plugin 2.4 to compile the GWT code.
But we observed that when we run mvn install, the gwt maven plugin is running the permutations even when no code changes are made to GWT code.
After struggling on this issue for quite some time, we were able to reproduce it with a simple setup using 2 maven projects:
1st project(MyGwtMavenSampleSource) : Contains the GWT code along with the gwt module xml file
2nd project (MavenBuilderMod): Contains the MyGwtMavenSampleSource as maven dependency and will build the war
file.
Below are the poms for both the projects:
1st Project: MyGwtMavenSampleSource
<?xml version="1.0" encoding="UTF-8"?>
<project
xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<!-- POM file generated with GWT webAppCreator -->
<modelVersion>4.0.0</modelVersion>
<groupId>GWTMavenPlugin</groupId>
<artifactId>MyGwtMavenSampleSource</artifactId>
<packaging>jar</packaging>
<version>0.0.1-SNAPSHOT</version>
<name>GWT Maven Archetype</name>
<properties>
<!-- Convenience property to set the GWT version -->
<gwtVersion>2.4.0</gwtVersion>
<!-- GWT needs at least java 1.5 -->
<webappDirectory>${project.build.directory}/${project.build.finalName}</webappDirectory>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>com.google.gwt</groupId>
<artifactId>gwt-servlet</artifactId>
<version>${gwtVersion}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.google.gwt</groupId>
<artifactId>gwt-user</artifactId>
<version>${gwtVersion}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
<version>1.0.0.GA</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
<version>1.0.0.GA</version>
<classifier>sources</classifier>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<resources>
<resource>
<directory>src/main/java</directory>
<includes>
<include>**/*.*</include>
</includes>
</resource>
<resource>
<directory>src/main/resources</directory>
<includes>
<include>**/*.*</include>
</includes>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
2nd Project: MavenBuilderMod
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org /2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>GWTMavenPlugin</groupId>
<artifactId>MavenBuilderMod</artifactId>
<packaging>war</packaging>
<version>0.0.1-SNAPSHOT</version>
<properties>
<!-- Convenience property to set the GWT version -->
<gwtVersion>2.4.0</gwtVersion>
<!-- GWT needs at least java 1.5 -->
<webappDirectory>${project.build.directory}/${project.build.finalName}</webappDirectory>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>com.google.gwt</groupId>
<artifactId>gwt-servlet</artifactId>
<version>${gwtVersion}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.google.gwt</groupId>
<artifactId>gwt-user</artifactId>
<version>${gwtVersion}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
<version>1.0.0.GA</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
<version>1.0.0.GA</version>
<classifier>sources</classifier>
<scope>test</scope>
</dependency>
<dependency>
<groupId>GWTMavenPlugin</groupId>
<artifactId>MyGwtMavenSampleSource</artifactId>
<version>0.0.1-SNAPSHOT</version>
<scope>compile</scope>
</dependency>
</dependencies>
<build>
<outputDirectory>${webappDirectory}/WEB-INF/classes</outputDirectory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.8</version>
<executions>
<execution>
<id>src-dependencies</id>
<phase>package</phase>
<goals>
<goal>unpack-dependencies</goal>
</goals>
<configuration>
<classifier>sources</classifier>
<failOnMissingClassifierArtifact>false</failOnMissingClassifierArtifact>
<outputDirectory>${project.build.directory}/sources</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<!-- GWT Maven Plugin -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>gwt-maven-plugin</artifactId>
<version>2.4.0</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
<!-- Plugin configuration. There are many available options, see
gwt-maven-plugin documentation at codehaus.org -->
<configuration>
<runTarget>MyGwtMavenSampleSource.html</runTarget>
<hostedWebapp>${webappDirectory}</hostedWebapp>
<modules>
<module>com.ca.gwt.maven.sample.MyGwtMavenSampleSource</module>
</modules>
</configuration>
</plugin>
<!-- Copy static web files before executing gwt:run -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.1.1</version>
<executions>
<execution>
<phase>compile</phase>
</execution>
</executions>
<configuration>
<webappDirectory>${webappDirectory}</webappDirectory>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
So, the question is, how can we make gwt-maven-plugin / compiler to run the permutations only on code change in this kind of setup.
Note: This works as expected (i.e permutations are run only when the code is changed) if gwt module xml file is placed in the same maven module that contains the gwt-maven-plugin
The root issue is Maven's broken support for incremental builds, and possibly broken incremental-build support in the gwt-maven-plugin (particularly if you're still using a more-than-2-year-old version!).
The gwt-maven-plugin somehow sees that the gwt.xml file has changed (track the build of your first module, ideally it should really be a no-op if nothing changed; re-copying the resources and/or rebuilding the JAR would probably be enough to trigger the issue downwards), and particularly is more recent than the nocache.js file it produced.
I think a better setup is to run the gwt-maven-plugin in the first module and package everything in a ZIP or WAR, that you can then use as a WAR overlay. It unfortunately makes running DevMode a bit harder, but it's workable.