We Keep Coding

ERROR JacksonVersion - Version '2.9.4' of package 'jackson-databind' is not supported

i am using azure sdk(azure-storage-file-datalake, azure-core, azure-identity) for writing the avro data to azure datalake gen 2 from kafka(using connectors), but i am getting very weird errors as below
2022-05-12 10:46:10 ERROR JacksonVersion:506 - Version '2.9.4' of package 'jackson-annotations' is not supported (older than earliest supported version - 2.10.0), please upgrade.
2022-05-12 10:46:10 ERROR JacksonVersion:506 - Version '2.9.4' of package 'jackson-core' is not supported (older than earliest supported version - 2.10.0), please upgrade.
2022-05-12 10:46:10 ERROR JacksonVersion:506 - Version '2.9.4' of package 'jackson-databind' is not supported (older than earliest supported version - 2.10.0), please upgrade.
java.lang.LinkageError: Package versions: jackson-annotations=2.9.4, jackson-core=2.9.4, jackson-databind=2.9.4, jackson-dataformat-xml=unknown, jackson-datatype-jsr310=unknown, azure-core=1.28.0, Troubleshooting version conflicts: https://aka.ms/azsdk/java/dependency/troubleshoot\n\tat com.azure.core.implementation
I have tried excluding the jackson dependencies and tried adding explicitly, still i am getting the errors.
please some one help on this
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>2.13.2</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.13.2</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.13.2</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
<version>2.13.2</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
<version>2.13.2</version>
<scope>runtime</scope>
</dependency>
<dependencies>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>connect-api</artifactId>
<version>2.0.1</version>
<scope>provided</scope>
</dependency>
<!--<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>1.1.0</version>
<scope>provided</scope>
</dependency>-->
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<!-- Apache avro dependency -->
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
<version>1.8.2</version>
</dependency>
<!-- Newer versions of the storage account jar is not working. -->
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-storage-file-datalake</artifactId>
<version>12.9.0</version>
<exclusions>
<exclusion>
<groupId>com.azure</groupId>
<artifactId>azure-core</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- https://mvnrepository.com/artifact/com.azure/azure-core -->
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core</artifactId>
<version>1.28.0</version>
<exclusions>
<!-- Collision with Kafka Jackson dependencies -->
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
</exclusion>
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</exclusion>
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusion>
<exclusion>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
</exclusion>
<exclusion>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-identity</artifactId>
<version>1.5.1</version>
<exclusions>
<exclusion>
<groupId>com.azure</groupId>
<artifactId>azure-core</artifactId>
</exclusion>
<exclusion>
<groupId>com.microsoft.azure</groupId>
<artifactId>msal4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.linguafranca.pwdb</groupId>
<artifactId>KeePassJava2</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>msal4j</artifactId>
<version>1.9.1</version> <!-- {x-version-update;com.microsoft.azure:msal4j;external_dependency} -->
<exclusions>
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- Schema registry dependency -->
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-connect-avro-converter</artifactId>
<version>3.3.0</version>
<exclusions>
<exclusion>
<groupId>io.confluent</groupId>
<artifactId>kafka-schema-registry-client</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-schema-registry-client</artifactId>
<version>3.3.0</version>
<exclusions>
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>2.21.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.2</version>
<scope>test</scope>
</dependency>
</dependencies>

Dependencies with scope "runtime" are not used for compilation, but only for execution. A dependency with this scope is not put on the compile classpath, so you could remove the runtime scope.
However a better way to overwrite transitive dependency versions that you don't use yourself is to use depependency management like this:
<project>
[...]
<dependencyManagement>
<dependencies>
<dependency>
<groupId>group-a</groupId>
<artifactId>artifact-a</artifactId>
<version>1.0</version>
</dependency>
[...]
</dependencies>
</dependencyManagement>
</project>

This is a challenge with Azure-SDK (here is a similar issue). Azure-SDK has a dependency on Jackson. The specific version of Jackson depends on the azure-sdk version you are using.
The latest release supports Jackson 2.10-2.13. What it means is you may have other dependencies on Jackson, and the SDK will still work. Provided Jackson version fetched is from 2.10-2.13. Azure-SDK is compatible with all these versions.
If you wish to use Jackson 2.9 (or if that's coming as a dependency from other dependency), then there is no way around it. You would necessarily need to upgrade your dependencies so that it fetches jackson 2.10 and above.
In your case, there would be some dependency that is fetching 2.9.x series.
This is what I would do:
Based on all non-azure-sdk dependencies. Decide which jackson version to hold on to. Explicitly add this jackson dependency in the pom.xml
Use azure-sdk BOM
You dont have to specify exclusion. Specifying a version in pom overrides other dependency versions

Hosting environment had those 2.9.4 versioned jackson jars which were loading at run time and causing the issues, i replaced those with newer version jars , its working all fine now.
(we had installed kafka client on the host, which internally had those jars inside kafka/libs)
got the ans from - https://github.com/Azure/azure-sdk-for-java/issues/28902

Swagger-api with dropwizard.io 0.9.1

I try to configure swagger with dropwizard framework 0.9.1.
In my pom.xml :
<dependency>
<groupId>io.swagger</groupId>
<artifactId>swagger-jersey2-jaxrs</artifactId>
<version>1.5.4</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>com.sun.jersey</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
but a java.lang.NoClassDefFoundError:
jersey/repackaged/com/google/common/collect/ImmutableMap is thrown.
What is wrong?
The solution is to set the right exclusions in the dependency section:
<dependency>
<groupId>io.swagger</groupId>
<artifactId>swagger-jersey2-jaxrs</artifactId>
<version>1.5.4</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-servlet</artifactId>
</exclusion>
<exclusion>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-servlet-core</artifactId>
</exclusion>
</exclusions>
</dependency>

Apache spark maven tomcat:run

I want to run Apache Spark from a spring project with multiple maven module.
Spring project running correctly without apache spark dependency but when I added spark dependency I see the following:
java.lang.ClassCastException: org.springframework.web.servlet.DispatcherServlet cannot be cast to javax.servlet.Servlet
spark dependencies:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.2.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.10</artifactId>
<version>1.2.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.5.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-assembly-jar</artifactId>
<version>1.3.0</version>
</dependency>

Here is what works for me. Reason is spark engine has transitive dependencies to a particular version of servlet api and implementation. In the classpath it finds first smth different and complains about it.
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>${spark.version}</version>
<exclusions>
<exclusion>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.glassfish</groupId>
<artifactId>javax.servlet</artifactId>
</exclusion>
<exclusion>
<groupId>org.eclipse.jetty.orbit</groupId>
<artifactId>javax.servlet</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.10</artifactId>
<version>${spark.version}</version>
</dependency>
Also in order to use web service as a driver I believe it's required to use compile scope of the spark libraries.

Spring-Boot logging with log4j2?

I'm using spring-boot-starter, and would like to configure log4j2.xml to log asynchron + different content to different logfiles.
I created the log4j2 file, but Spring still uses the spring-boot default logging. How can I switch the logging?

I've a better way:
Exclude logback logger:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
Add log4j2 boot starter:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
Source: http://docs.spring.io/spring-boot/docs/1.5.1.RELEASE/reference/htmlsingle/#howto-configure-log4j-for-logging
Enjoy!

Try this:
Exclude spring-boot-starter-logging e.g.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
Add dependencies for your logging interface e.g. slf4j
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.0.2</version>
</dependency>
Add other logging implementations pointing to chosen logging interface e.g.
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jul-to-slf4j</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>log4j-over-slf4j</artifactId>
</dependency>
Add your target logging implementation e.g.
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.0.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.0.2</version>
</dependency>
And it should work.

This is what worked for me. Having two additional exclusions. Else Application was not picking up log4j and was had conflict
Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>1.0.0</version>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
<exclusion>
<artifactId>logback-classic</artifactId>
<groupId>ch.qos.logback</groupId>
</exclusion>
<exclusion>
<artifactId>log4j-over-slf4j</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>

Use Mina JAR dependencies instead of bundles?

Mina seems to be deploying their artifacts as *.bundle files these days, which, at the very least, is annoying and at best downright evil.
If I depend on Mina in Maven like this:
<dependency>
<groupId>org.apache.mina</groupId>
<artifactId>mina-core</artifactId>
<version>${mina.version}</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.mina</groupId>
<artifactId>mina-integration-beans</artifactId>
<version>${mina.version}</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.mina</groupId>
<artifactId>mina-integration-jmx</artifactId>
<version>${mina.version}</version>
<type>jar</type>
</dependency>
...then I get the following error:
Multiple annotations found at this line:
- Missing artifact org.apache.mina:mina-integration-ognl:bundle:2.0.4
- Missing artifact org.apache.mina:mina-core:bundle:2.0.4
- Missing artifact org.apache.mina:mina-integration-beans:bundle:2.0.4
I'm not sure why it's still looking for *.bundle files instead of *.jar files. Why is it not looking for and finding the JAR files?
This is especially frustrating because I have projects which depend on this project and they all fail without the Mina JARs. Whatever happened to good-old Java JARs?

First i have found out that using only the core it works without any problem, but if you start to use mina-integration-bean or mina-integration-jmx you will get the problem. Furthermore i have found the following issue in the JIRA tracker of Mina project which exactly states your problem.
On the other hand you can exclude those bundle dependencies in maven like this:
<dependencies>
<dependency>
<groupId>org.apache.mina</groupId>
<artifactId>mina-core</artifactId>
<version>2.0.4</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.apache.mina</groupId>
<artifactId>mina-integration-beans</artifactId>
<version>2.0.4</version>
<type>jar</type>
<exclusions>
<exclusion>
<groupId>org.apache.mina</groupId>
<artifactId>mina-core</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.mina</groupId>
<artifactId>mina-integration-ognl</artifactId>
<version>2.0.4</version>
<type>jar</type>
<exclusions>
<exclusion>
<groupId>org.apache.mina</groupId>
<artifactId>mina-core</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.mina</groupId>
<artifactId>mina-integration-beans</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.mina</groupId>
<artifactId>mina-integration-jmx</artifactId>
<version>2.0.4</version>
<type>jar</type>
<exclusions>
<exclusion>
<groupId>org.apache.mina</groupId>
<artifactId>mina-core</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.mina</groupId>
<artifactId>mina-integration-beans</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.mina</groupId>
<artifactId>mina-integration-ognl</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>