cannot import graphframes dependency in maven project - java

I have a maven project and i need import graphframe dependency to use spark grapx,this's my pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>demoGraphX</artifactId>
<version>1.0-SNAPSHOT</version>
<repositories>
<repository>
<id>SparkPackagesRepo</id>
<url>http://dl.bintray.com/spark-packages/maven</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-graphx_2.11</artifactId>
<version>2.2.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/graphframes/graphframes -->
<dependency>
<groupId>graphframes</groupId>
<artifactId>graphframes</artifactId>
<version>0.7.0-spark2.4-s_2.11</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.2.1</version>
</dependency>
</dependencies>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>
</project>
But when i import graphframe dependency, my IDE return:
Cannot resolve org.apache.parquet:parquet-jackson:1.8.2
Cannot resolve com.twitter:chill_2.11:0.8.0
Cannot resolve org.apache.parquet:parquet-hadoop:1.8.2
Cannot resolve io.dropwizard.metrics:metrics-core:3.1.2
Cannot resolve org.glassfish.hk2:hk2-utils:2.4.0-b34
Cannot resolve com.univocity:univocity-parsers:2.2.1
Cannot resolve org.apache.directory.api:api-asn1-api:1.0.0-M20
Cannot resolve org.glassfish.hk2:osgi-resource-locator:1.0.1
Cannot resolve org.objenesis:objenesis:2.1
Cannot resolve commons-cli:commons-cli:1.2
Cannot resolve org.apache.hadoop:hadoop-yarn-common:2.6.5
Cannot resolve org.mortbay.jetty:jetty-util:6.1.26
Cannot resolve com.google.code.gson:gson:2.2.4
Cannot resolve com.chuusai:shapeless_2.11:2.3.2
etc...
I guest something wrong in my Maven. I use:
Intellij lastest
java 1.8.0_292
Apache Maven 3.6.3
Can you help me solve this problem. Thank you!

The bintray service was shutdown starting from 1st of May. (Press release)
So Apache spark community has provided new repo to host all spark packages. You can add/replace below code snippet in your code and things should work.
<repositories>
<repository>
<id>bintray</id>
<name>Bintray Repository</name>
<url>https://repos.spark-packages.org</url>
</repository>
</repositories>

Related

How to include dependencies targeting JDK 1.8 in jenkins plugin?

I have followed the official tutorial for creating Jenkins plugins: I have run mvn -U org.jenkins-ci.tools:maven-hpi-plugin:create to generate initial POM
Then I added guava as a dependency.
However, when I perform mvn package I get:
[INFO] --- maven-enforcer-plugin:1.4.2.jenkins-1:enforce (display-info) # flaky-tests ---
[INFO] Restricted to JDK 1.7 yet com.google.guava:guava:jar:22.0:compile contains com/google/common/annotations/Beta.class targeted to JDK 1.8
...
[WARNING] Rule 2: org.apache.maven.plugins.enforcer.EnforceBytecodeVersion failed with message:
Found Banned Dependency: com.google.guava:guava:jar:22.0
Use 'mvn dependency:tree' to locate the source of the banned dependencies.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
How can I include dependencies targeting JDK 1.8 in my Jenkins plugin?
Here's my full POM:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>plugin</artifactId>
<version>2.32</version>
<relativePath/>
</parent>
<groupId>com.github.dzieciou.testing</groupId>
<artifactId>flaky-tests</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>hpi</packaging>
<properties>
<jenkins.version>2.68</jenkins.version>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<repositories>
<repository>
<id>repo.jenkins-ci.org</id>
<url>https://repo.jenkins-ci.org/public/</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>repo.jenkins-ci.org</id>
<url>https://repo.jenkins-ci.org/public/</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>22.0</version>
</dependency>
</dependencies>
</project>
It was enough to add the following property to POM:
<java.level>8</java.level>

How to inherit dependencies from other libraries in maven?

I created a custom my-commons library with maven. The commons pom contains eg the following dependency:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.4.RELEASE</version>
<relativePath/>
</parent>
<groupId>de.mydomain</groupId>
<artifactId>my-commons</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<properties>
<!-- workaround for http://jira.codehaus.org/browse/MASSEMBLY-603-->
<maven.build.timestamp>${maven.build.timestamp}</maven.build.timestamp>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>
</project>
I installed the custom library into my local repository.
In a different project, I reuse the library. It resolves correctly, so it exists in the repo:
<project ...>
<groupId>de.mydomain</groupId>
<artifactId>my-core</artifactId>
<version>1.0.0</version>
<dependencies>
<dependency>
<groupId>my.domain</groupId>
<artifactId>my-commons</artifactId>
<version>1.0.0</version>
</dependency>
</dependencies>
</project>
Now in this project I'd like to use a class org.springframework.web.bind.annotation.ResponseBody.
Problem: the class is not in classpath. So maven complains [ERROR] cannot find symbol.
But why?? I'd expect the dependency being inherited!
Sidenote: I'm using Intellij IDEA if that matters. The same problem applies to all "inherited" libraries. The spring-web lib is just an example.
Please check if .m2 folder contains the jar you want to use or not. You can use mvn install command in commons project to generate the jar. Also you need to add commons project in your project pom that you are running.
You need to create the artifact for your commons project, through its pom.xml.
Like in my below example i created a separate project where i am using the AWS and some other dependency and then final jar, which i am using in other project :-
My AWS-client pom.xml looks like below :-
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.kaamkaj</groupId>
<artifactId>aws-clients</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>aws-clients</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<aws.sdk.version>1.11.104</aws.sdk.version>
</properties>
<dependencies>
<!--AWS dependency-->
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-dynamodb -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-dynamodb</artifactId>
<version>${aws.sdk.version}</version>
</dependency>
<dependency>
<groupId>com.mercadopago</groupId>
<artifactId>sdk</artifactId>
<version>0.3.4</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>mercadopago</id>
<url>https://github.com/mercadopago/sdk-java/raw/master/releases</url>
</repository>
</repositories>
</project>
Also, run mvn clean install on the commons pom, in your case.
And then in all other project where i need to include the aws-client , i use the artifact-id of above project, like below :-
<dependency>
<groupId>com.kaamkaj</groupId>
<artifactId>aws-clients</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
Let me know if you have any questions:
Found it: removing the <maven.build.timestamp>... <property> solved the problem. I don't know why it prevented the inheritance to work properly, especially as I didn't get any errors.

Error parsing lifecycle processing instructions

Below is my pom.xml file. On the very first line, I am getting an error of
Error parsing lifecycle processing instructions.
I need help finding what caused the error.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>First</name>
<description>Demo project for Spring Boot</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.2.4.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
<repository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/milestone</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</pluginRepository>
<pluginRepository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/milestone</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</project>
This issue was driving me insane, I've finally managed to solve it by deleting the whole maven repository located in:
Windows: c:\Users\<username>\.m2\
After that I just updated the project (Alt+F5 in Eclipse). Issue gone!
Maybe a clash of different versions of Maven (I use several different versions of Eclipse + m2e plugins).
There is some dependency got corrupted into .m2 folder.
You need to delete that dependency from .m2 folder.
If you are not able to find which one is corrupted then delete all
the dependency which are declared into pom.xml file.
It is due to an older version of m2e-wtp integration plugin. To resolve the issue, in Eclipse(Neon) you can go to Help --> Install New Software.. --> Enter http://download.eclipse.org/m2e-wtp/milestones/neon/1.3/ in the Work with box and press enter. Tick all the checkboxes, install the plugins and restart the IDE. It should work. Similarly, for other versions, you might refer the following link https://www.eclipse.org/m2e-wtp/ and try the latest/previous builds depending on your Eclipse version.
Take a look at eclipse web
http://marketplace.eclipse.org/content/maven-integration-eclipse-wtp
The m2eclipse-wtp project has moved to the Eclipse Foundation. The m2eclipse-wtp plugin is now deprecated in favor of the newer m2e-wtp
Automatic installation from the marketplace has been suspended. m2eclipse-wtp 0.15.3 can be manually installed from http://download.jboss.org/jbosstools/updates/m2eclipse-wtp/
You need to first uninstall all m2e-wtp plugins in Eclipse, then install them from jboss url
No need to clean .m2 folder, I fixed this issue by adding corresponding maven to eclipse
Inspite of adding the new path and making changes to the user settings the error "Error parsing lifecycle processing instructions" persists. I restarted the server. Is there any other change required?
This same problem happened to me.
I'm using the new Eclipse Neon. Maybe it's an issue related to the Maven version included in this new release.
I have solved it by using Gradle instead of Maven.
Note: you can try to use Maven with the previous version of Eclipse.
Try this, it might be helpful.
<project xmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.demo</groupId>
<artifactId>test1</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>war</packaging>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.2.4.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<java-version>1.8</java-version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.0.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
<version>1.2</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<finalName>HelloWorld</finalName>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>${java-version}</source>
<target>${java-version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.4</version>
<configuration>
<warSourceDirectory>src/main/webapp</warSourceDirectory>
<warName>SpringWebExample</warName>
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
In my case the same issue was due to different version of maven. One which I installed manualy and other one which came Embedded with STS.
I resolved this issue using below steps:
1. Deleted .M2 folder (c:\Users\.m2)
Downloaded exact version of maven which is embedded with STS and configured same in env. variables and path variable.
STS Embedded maven- Image
Updated projects (Right click on project -> Maven -> Update Project...)
It took some time in update step as all dependencies downloaded again in .M2 folder. Finally issue is resolved.
I suffered the same problem in several projects of my workspace, the ones who belonged to the same parent. I have just one Maven version installed, and my local .m2 repository is working fine for a bunch of other maven projects in my wokspace.
In my case, the cause was a simple XML syntax error in the parent pom. I fixed it, refreshed the affected maven projects, and it went OK.
Before I made modifications to the .xml I made a copy in the same folder. After making my mods I began having issues.
Moving the backup out of the .m2 folder resolved the issue for me.

Trying to use maven with android eclipse project not working?

This is the first time I've used maven, I am trying to integrate zopim sdk and I followed the tutorial and converted my project to a maven project and added the repository and dependencies, but now eclipse does not compile and gives me the following error :
Description Resource Path Location Type
Missing artifact com.android.support:appcompat-v7:jar:23.0.0 pom.xml /4SaleApp line 2 Maven Dependency Problem
Missing artifact com.android.support:design:jar:23.0.0 pom.xml /4SaleApp line 2 Maven Dependency Problem
Missing artifact com.android.support:recyclerview-v7:jar:23.0.0 pom.xml /4SaleApp line 2 Maven Dependency Problem
What should I do ?
EDIT
my pom.xml file
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>4SaleApp</groupId>
<artifactId>4SaleApp</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<repositories>
<repository>
<id>chatsdk-repo</id>
<name>Chat SDK Repo</name>
<url>https://zendesk.artifactoryonline.com/zendesk/repo</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>com.zopim.android</groupId>
<artifactId>sdk</artifactId>
<version>1.1.0</version>
<type>aar</type>
</dependency>
</dependencies>
</project>
This seems like a conflicting issue when changing from Gradle to Maven. This questions provides a lot more info on the issue.
But it looks like you are missing the android support dependencies in the pom. Maybe try adding the missing support libraries to the pom. For example as shown here for appcompat-v7:
<dependency>
<groupId>android.support</groupId>
<artifactId>compatibility-v7-appcompat</artifactId>
<version>${compatibility.version}</version>
<type>apklib</type>
</dependency>
<dependency>
<groupId>android.support</groupId>
<artifactId>compatibility-v7-appcompat</artifactId>
<version>${compatibility.version}</version>
<type>jar</type>
</dependency>

Maven + ODL osgi.wiring.package javax

I am currently working on an application on top of OpenDaylight. I want to use an ObjectDB.
But I can not activate my bundle.
opendaylight-user#root>bundle:diag
ntf-impl (171)
--------------
Status: Installed
Unsatisfied Requirements:
osgi.wiring.package; resolution:="mandatory"; filter:="(&(osgi.wiring.package=javax.jdo.annotations)(&(version>=3.1.0)(!(version>=4.0.0))))"
osgi.wiring.package; resolution:="mandatory"; filter:="(&(osgi.wiring.package=javax.persistence))"
During mvn clean install
Unresolved constraint in bundle org.opendaylight.ntf.impl [205]: Unable to resolve 205.0: missing requirement [205.0] osgi.wiring.package; (&(osgi.wiring.package=javax.jdo.annotations)(version>=3.1.0)(!(version>=4.0.0)))
I will just post you my pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>org.opendaylight.controller</groupId>
<artifactId>config-parent</artifactId>
<version>0.3.0-SNAPSHOT</version>
<relativePath/>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.opendaylight.ntf</groupId>
<artifactId>ntf-impl</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>bundle</packaging>
<repositories>
<repository>
<id>objectdb</id>
<name>ObjectDB Repository</name>
<url>http://m2.objectdb.com</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>ntf-api</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>javax.jdo</groupId>
<artifactId>jdo-api</artifactId>
<version>3.1</version>
</dependency>
<dependency>
<groupId>javax.persistence</groupId>
<artifactId>persistence-api</artifactId>
<version>1.0.2</version>
</dependency>
<dependency>
<groupId>com.objectdb</groupId>
<artifactId>objectdb</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
I tried many different dependencies. Has anyone got suggestions how I could get this running?
Make sure to add the relevant bundles to features.xml inside your feature, such as features/src/main/features/features.xml.
I had a bundle dependency error with javax.jms which I fixed by adding the bundle dependencies in the features.xml as shown below.
<feature name='odl-odlmq' version='${project.version}' description='OpenDaylight :: odlmq'>
<feature version='${mdsal.version}'>odl-mdsal-broker</feature>
<feature version='${project.version}'>odl-odlmq-api</feature>
<bundle>mvn:org.opendaylight.odlmq/odlmq-impl/${project.version}</bundle>
<bundle dependency="true">mvn:org.apache.geronimo.specs/geronimo-jms_1.1_spec/1.1.1</bundle>
<bundle dependency="true">mvn:org.apache.geronimo.specs/geronimo-jta_1.1_spec/1.1.1</bundle>
</feature>
Similarly add the relevant bundles for javax.jdo into features.xml and build again. The build should succeed with the errors resolved.
Alternatively to just get this running, you may build with tests skipped.
$ mvn clean install -DskipTests
However, in that case, you will have to manually install the dependencies later from Karaf.

Categories