We Keep Coding

cannot import graphframes dependency in maven project

I have a maven project and i need import graphframe dependency to use spark grapx,this's my pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>demoGraphX</artifactId>
<version>1.0-SNAPSHOT</version>
<repositories>
<repository>
<id>SparkPackagesRepo</id>
<url>http://dl.bintray.com/spark-packages/maven</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-graphx_2.11</artifactId>
<version>2.2.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/graphframes/graphframes -->
<dependency>
<groupId>graphframes</groupId>
<artifactId>graphframes</artifactId>
<version>0.7.0-spark2.4-s_2.11</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.2.1</version>
</dependency>
</dependencies>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>
</project>
But when i import graphframe dependency, my IDE return:
Cannot resolve org.apache.parquet:parquet-jackson:1.8.2
Cannot resolve com.twitter:chill_2.11:0.8.0
Cannot resolve org.apache.parquet:parquet-hadoop:1.8.2
Cannot resolve io.dropwizard.metrics:metrics-core:3.1.2
Cannot resolve org.glassfish.hk2:hk2-utils:2.4.0-b34
Cannot resolve com.univocity:univocity-parsers:2.2.1
Cannot resolve org.apache.directory.api:api-asn1-api:1.0.0-M20
Cannot resolve org.glassfish.hk2:osgi-resource-locator:1.0.1
Cannot resolve org.objenesis:objenesis:2.1
Cannot resolve commons-cli:commons-cli:1.2
Cannot resolve org.apache.hadoop:hadoop-yarn-common:2.6.5
Cannot resolve org.mortbay.jetty:jetty-util:6.1.26
Cannot resolve com.google.code.gson:gson:2.2.4
Cannot resolve com.chuusai:shapeless_2.11:2.3.2
etc...
I guest something wrong in my Maven. I use:
Intellij lastest
java 1.8.0_292
Apache Maven 3.6.3
Can you help me solve this problem. Thank you!

The bintray service was shutdown starting from 1st of May. (Press release)
So Apache spark community has provided new repo to host all spark packages. You can add/replace below code snippet in your code and things should work.
<repositories>
<repository>
<id>bintray</id>
<name>Bintray Repository</name>
<url>https://repos.spark-packages.org</url>
</repository>
</repositories>

Cannot determine why dependency pom.xml is considered invalid

Working with proprietary source code. Some dependencies are stored in a local (e.g. on my filesystem) maven repo, which is exposed in my main project pom.xml like so:
<repository>
<!-- Embed mvn repo with non-public dependencies -->
<id>myproj.local</id>
<url>file:${basedir}/src/repo</url>
<name>MyProj Embedded Repo</name>
<releases>
<updatePolicy>always</updatePolicy>
</releases>
<snapshots>
<updatePolicy>always</updatePolicy>
</snapshots>
</repository>
This worked OK until today, when I added a new jar file into the embedded repo under src/repo/com/hidden/v2/some-api/1.5.0/some-api-1.5.0.jar and a new handcrafted pom at src/repo/com/hidden/v2/some-api/1.5.0/some-api-1.5.0.pom.
Now my code builds and runs OK, but I constantly get this error from maven and cannot figure out why:
[WARNING] The POM for com.hidden.v2:some-api:jar:1.5.0 is invalid, transitive dependencies (if any) will not be available: 2 problems were encountered while building the effective model for :some-api:1.5.0
[ERROR] 'groupId' is missing. # [unknown-group-id]:some-api:1.5.0
[ERROR] 'dependencies.dependency.version' for javax.servlet:servlet-api:jar is missing. # [unknown-group-id]:some-api:1.5.0
My custom pom file seems to have the fields that are supposedly missing:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.hidden.v2</groupId>
<artifactId>some-api</artifactId>
<version>1.5.0</version>
<dependencies>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<!-- <scope>provided</scope> -->
<version>2.5</version>
</dependency>
</dependencies>
</project>

Failed to execute goal org.apache.maven.plugins:maven-clean-plugin:3.1.0:clean (default-clean) on project spring-boot-keycloak

My application was working fine and i was able to do mvn clean install using <packaging>jar</packaging> and able to execute a jar file but i change it to war file and i was't able to do mvn clean install .
is there any deference between these too ? I looked for more solutions but with no luck
Error :
Failed to execute goal org.apache.maven.plugins:maven-clean-plugin:3.1.0:clean (default-clean) on project spring-boot-keycloak: Failed to clean project: Failed to delete C:\Users\user\Documents\New folder\E.Services\target
My POM File :
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.baeldung.keycloak</groupId>
<artifactId>spring-boot-keycloak</artifactId>
<version>0.0.1</version>
<name>spring-boot-keycloak</name>
<packaging>war</packaging>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.2.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.keycloak.bom</groupId>
<artifactId>keycloak-adapter-bom</artifactId>
<version>3.3.0.Final</version>
<type>pom</type>
<scope>provided</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<-- Many dependencies -->
</dependencies>
<repositories>
<repository>
<id>central</id>
<name>Maven Plugin Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
<releases>
<updatePolicy>never</updatePolicy>
</releases>
</repository>
</repositories>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<executable>true</executable>
</configuration>
</plugin>
</plugins>
</build>
</project>

unable to delete target -- it's a file system issue a pretty common problem actually -- you probably have a file open or a Windows Explorer open in your ${project.home}/target directory...

it's a file system issue a pretty common problem actually -- you probably have a file open or a Windows Explorer open in your ${project.home}/target directory...

There can be many causes for this error. One of the main reasons could be that the directory maven is trying to delete is locked. Try to access the directory that maven is trying to delete. If you get a prompt like "the file or directory is corrupted or unreadable", try fixing disk errors. It worked for me. The steps I followed are:
Launch CMD on your Windows PC via the Start Menu. Right click on it
to run it as administrator.
Type chkdsk /f x: in the Command Prompt window and press Enter to
execute the command. X here is the drive letter of the corrupted
partition that you want to check. Please replace it with your own
drive letter.

Install maven dependency from location other than Maven Central

I have this pom.xml file:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>interos</groupId>
<artifactId>jenkins</artifactId>
<version>1.0</version>
<packaging>jar</packaging>
<name>Maven Quick Start Archetype</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>org.jenkins-ci.main</groupId>
<artifactId>cli</artifactId>
<version>2.49</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.jenkins-ci.main/cli -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.2</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
I run
mvn install
and I get:
[ERROR] Failed to execute goal on project jenkins: Could not resolve
dependencies for project interos:jenkins:jar:1.0: Failure to find
org.jenkins-ci.main:cli:jar:2.49 in
https://repo.maven.apache.org/maven2 was cached in the local
repository, resolution will not be reattempted until the update
interval of central has elapsed or updates are forced -> [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
I tried doing this:
<repositories>
<repository>
<id>repository.maven</id>
<name>Spring Framework Maven Release Repository</name>
<url>https://mvnrepository.com</url>
</repository>
</repositories>
But then I got:
[ERROR] Failed to execute goal on project jenkins: Could not resolve
dependencies for project interos:jenkins:jar:1.0: Could not find
artifact org.jenkins-ci.main:cli:jar:2.49 in repository.maven
(https://mvnrepository.com) -> [Help 1]

You need to add jenkins repository
<repositories>
<repository>
<id>repo.jenkins-ci.org</id>
<url>https://repo.jenkins-ci.org/public/</url>
</repository>
</repositories>

There are 2 possible solutions (perhaps more):
Add also Maven central to the repositories section of your pom file.
Add repositories and mirrors to the Maven settings.xml file.
Option 1 and 2 can also be combined.
Another but maybe the best solution is to use your own instance of a Nexus repository manager and load all the dependencies from that Nexus instance (that Nexus should be defined as repository). That Nexus instance should synchronize with/proxy to external repositories like Maven central, Spring, ... and so on.

How to inherit dependencies from other libraries in maven?

I created a custom my-commons library with maven. The commons pom contains eg the following dependency:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.4.RELEASE</version>
<relativePath/>
</parent>
<groupId>de.mydomain</groupId>
<artifactId>my-commons</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<properties>
<!-- workaround for http://jira.codehaus.org/browse/MASSEMBLY-603-->
<maven.build.timestamp>${maven.build.timestamp}</maven.build.timestamp>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>
</project>
I installed the custom library into my local repository.
In a different project, I reuse the library. It resolves correctly, so it exists in the repo:
<project ...>
<groupId>de.mydomain</groupId>
<artifactId>my-core</artifactId>
<version>1.0.0</version>
<dependencies>
<dependency>
<groupId>my.domain</groupId>
<artifactId>my-commons</artifactId>
<version>1.0.0</version>
</dependency>
</dependencies>
</project>
Now in this project I'd like to use a class org.springframework.web.bind.annotation.ResponseBody.
Problem: the class is not in classpath. So maven complains [ERROR] cannot find symbol.
But why?? I'd expect the dependency being inherited!
Sidenote: I'm using Intellij IDEA if that matters. The same problem applies to all "inherited" libraries. The spring-web lib is just an example.

Please check if .m2 folder contains the jar you want to use or not. You can use mvn install command in commons project to generate the jar. Also you need to add commons project in your project pom that you are running.

You need to create the artifact for your commons project, through its pom.xml.
Like in my below example i created a separate project where i am using the AWS and some other dependency and then final jar, which i am using in other project :-
My AWS-client pom.xml looks like below :-
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.kaamkaj</groupId>
<artifactId>aws-clients</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>aws-clients</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<aws.sdk.version>1.11.104</aws.sdk.version>
</properties>
<dependencies>
<!--AWS dependency-->
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-dynamodb -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-dynamodb</artifactId>
<version>${aws.sdk.version}</version>
</dependency>
<dependency>
<groupId>com.mercadopago</groupId>
<artifactId>sdk</artifactId>
<version>0.3.4</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>mercadopago</id>
<url>https://github.com/mercadopago/sdk-java/raw/master/releases</url>
</repository>
</repositories>
</project>
Also, run mvn clean install on the commons pom, in your case.
And then in all other project where i need to include the aws-client , i use the artifact-id of above project, like below :-
<dependency>
<groupId>com.kaamkaj</groupId>
<artifactId>aws-clients</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
Let me know if you have any questions:

Found it: removing the <maven.build.timestamp>... <property> solved the problem. I don't know why it prevented the inheritance to work properly, especially as I didn't get any errors.