We Keep Coding

NiFI :- Failed to create Extension Definition for CONTROLLER_SERVICE org.apache.nifi.record.sink.lookup.RecordSinkServiceLookup

I'm trying to use DBCPConnectionPool service in my custom processor. So, how to use in-built controller services (which are already available on NiFi) in our custom processors.
Here are my properties in *-nar/->pom.xml
<dependencies>
<dependency>
<groupId>org.sample.com</groupId>
<artifactId>nifi-customprocessor-processors</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-standard-services-api-nar</artifactId>
<version>1.14.0</version>
<type>nar</type>
</dependency>
</dependencies>
Here are my dependencies in *processors/->pom.xml
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-utils</artifactId>
<version>1.14.0</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-mock</artifactId>
<version>1.14.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-dbcp-service</artifactId>
<version>1.14.0</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-dbcp-service-api</artifactId>
<version>1.14.0</version>
</dependency>
And here is my propertydescriptor for DBCPConnetionPool service :-
public static final PropertyDescriptor DBCPConnectionPool_SERVICE = new PropertyDescriptor.Builder()
.name("DBCPConnectionPoolLookup Service")
.description("The Controller Service to use in order to establish a connection")
.required(true)
.dynamic(true)
.identifiesControllerService(DBCPService.class)
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.build();
and in my OnTrigger method I didn't used it as :-
DBCPService DBCPService = context.getProperty(DBCPConnectionPool_SERVICE)
.asControllerService(DBCPService.class);
Connection con = DBCPService.getConnection();
and was trying to use this con object as in regular passion createStatement->executeQuery.
When I tried to package it using mvn clean install it started throwing error as : A required class was missing while executing org.apache.nifi:nifi-nar-maven-plugi
n:1.3.1:nar: org/apache/nifi/record/sink/RecordSinkService
then I added the following dependencies in processors/->pom.xml
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-record-sink-service</artifactId>
<version>1.14.0</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-record-sink-api</artifactId>
<version>1.14.0</version>
</dependency>
This time it was new error as :- Failed to create Extension Definition for CONTROLLER_SERVICE org.apache.nifi.record.sink.lookup.RecordSinkServiceLookup: Null
PointerException
I also checked by commenting out code which I have written, to check the way I'm using that property was wrong but, even that didn't worked.:-
DBCPService DBCPService = context.getProperty(DBCPConnectionPool_SERVICE)
.asControllerService(DBCPService.class);
Connection con = DBCPService.getConnection();
Can someone help me out in order to use DBCPConnectionPoolService controller service in my custom processor?

I had the same issues the other day and I solved it by using AbstractControllerService instead of DBCPConnectionPool therefore not needing org.apache.nifi.dbcp.DBCPService anymore. Therefore my code would look like this:
MyService DBCPService = context.getProperty(DBCPConnectionPool_SERVICE)
.asControllerService(MyService.class);
Connection con = DBCPService.getConnection();
MyService is the custom interface that extends ControllerService.(you have to use it when you also define the PropertyDescriptor DBCPConnectionPool_SERVICE)
And StandardMyService the class that extends AbstractControllerService and implements MyService and getConnection() from it.

Resteasy 2.3.10.Final ClientResponse.readEntity AbstractMethodError

I am currently trying to create a Resteasy Client, having to use Version 2.3.10.Final. Unfortunately, I should not use a newer Version. I am very new to this, so I hope I manage to provide all required Information to make this question answerable.
First, the Request is created and finally "PUT". The returned ClientResponse is to be evaluated.
I can successfully call
Response.getStatus()
showing me that the "PUT" has been successful, as the Status is equal to
Response.Status.CREATED.getStatusCode()
However, any attempt to call response.readEntity Returns the error
java.lang.AbstractMethodError: javax.ws.rs.core.Response.readEntity(Ljava/lang/Class;)Ljava/lang/Object;
at blabla.evaluateResponse(MyResteasyClient.java:80)´
This is for any attempt to call .readEntity, but for the sake of specificity, let's concentrate on
String entity = Response.readEntity(String.class);
When attempting response.getEntity(), it Returns the following error message:
java.lang.RuntimeException: RESTEASY001555: No type information to extract entity with, use other getEntity() methods
at org.jboss.resteasy.client.core.BaseClientResponse.getEntity(BaseClientResponse.java:337)
at blabla.evaluateResponse(MyResteasyClient.java:81)
Am I even on the right Track? My Goal is to achieve something along the lines of:
List<? extends MyResponse> myResponses = response.readEntity(new GenericType<List<MyResponse>>() {});
Where the class "MyResponse" is an implementation of the Interface
public interface MyPartialResponse {
boolean isSuccessful();
String getMessage();
}
The dependencies within pom.xml are:
<properties>
<version.jackson>2.6.5</version.jackson>
<version.resteasy-client>2.3.10.Final</version.resteasy-client>
</properties>
<dependencies>
(removed a company specific dependency)
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-jaxrs</artifactId>
<version>${version.resteasy-client}</version>
</dependency>
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-jackson-provider</artifactId>
<version>${version.resteasy-client}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>${version.jackson}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${version.jackson}</version>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.6</version>
</dependency>
</dependencies>

If anyone happens to land on this question with the same error, let me provide the answer, how I solved my issue:
By adding
BaseClientResponse<?> r = (BaseClientResponse<?>) response;
I could successfully call
r.getEntity(new GenericType<List<MyResponse>>() {});

I think there is a conflicts between Jar files in Maven. Please check your maven dependency hierarchy.

Hadoop 2.6.x and Amazon AWS SDK Library conflicts http-core conflict

My job writes each record to DynamoDB in the Hadoop's map.
I cannot make it run with Hadoop 2.6 which has httpclient-4.2.5.jar and httpcore-4.2.5.jar.
AWS which I am using was built using httpclient-4.5.2.jar and httpcore-4.4.4.jar.
When I am using classpath to include the new jar files, it gives the following exception.
java.lang.Exception: java.lang.NoSuchFieldError: INSTANCE
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NoSuchFieldError: INSTANCE
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.<clinit>(SSLConnectionSocketFactory.java:144)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.getPreferredSocketFactory(ApacheConnectionManagerFactory.java:87)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:65)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:58)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:49)
To me, it looks like Hadoop was built using the old libraries and something has changed in the syntax.
What's the reasonable solution than recompiling older sources of AWS?

As an update, I had to switch to Maven and play around with the versions a bit.
<!-- http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>2.6.0</version>
</dependency>
<!-- http://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.4</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-dynamodb</artifactId>
<version>1.9.13</version>
</dependency>
<dependency>
<groupId>org.netpreserve.commons</groupId>
<artifactId>webarchive-commons</artifactId>
<version>1.1.4</version>
</dependency>
Finally, it works

java.lang.NoClassDefFoundError: org/apache/http/conn/SchemePortResolver with AmazonHttpClient

All
I am running into this error in my project when I updated aws library to the latest 1.11.3.
Caused by:
java.lang.NoClassDefFoundError: org/apache/http/conn/SchemePortResolver
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.<init>(ApacheHttpClientFactory.java:40)
at com.amazonaws.http.AmazonHttpClient.<clinit>(AmazonHttpClient.java:97)
at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:145)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:393)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:373)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:355)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:339)
in my pom.xml
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
<version>1.11.3</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>1.11.3</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-kms</artifactId>
<version>1.11.3</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-ext-jdk15on</artifactId>
<version>1.54</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-encryption-sdk-java</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
Anyone know what I did wrong?
thanks

I had a similar issue with my grails application. In my case the ClassNotFoundException was being thrown from a deploy script. For me the reason SchemePortResolver wasn't being resolved implicitly was because it wasn't required at compile time, it was needed at runtime. Here's what I added to my BuildConfig.groovy to fix it:
runtime 'org.apache.httpcomponents:httpclient:4.5.2' //Required by BeanstalkDeploy.groovy at runtime
Since the OP's question was for Maven, here's the equivalent include:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.2</version>
<scope>runtime</scope>
</dependency>

If you add,
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.2</version>
</dependency>
,it should work ok as it contains the missing class definitions.

A common cause would be a Maven dependency conflict. In the example below (pom.xml as viewed in Eclipse's Dependency Hierarchy tab), the POM explicitly includes v4.1 of httpclient, which forces Maven to omit the v4.5.5 that aws-java-sdk-core requires (and thus, causes the similar error java.lang.NoClassDefFoundError: org/apache/http/conn/DnsResolver at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory...):

In my case I delete the .meta file in eclipse workplace then import the project again thereafter it work like a charm.
Could not know where the problem exactly.
Before delete .meta I add all files from
aws-java-sdk-1.11.606\third-party\lib

What dependencies needed for AndroidJUnit4.class

I am creating a test class to test an Android application and I would like to use the AndroidJUnit4 class as runner:
#RunWith(AndroidJUnit4.class)
However I can't figure out what dependencies do I need to add to my Maven project to get it. I tried these:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
<dependency>
<groupId>com.google.android</groupId>
<artifactId>android</artifactId>
<version>4.1.1.4</version>
</dependency>
and:
<dependency>
<groupId>com.uphyca</groupId>
<artifactId>android-junit4</artifactId>
<version>0.5</version>
</dependency>
The last one seemed the most promising but it only added the following classes:
AndroidJUnit4TestAdapter
AndroidJUnit38ClassRunner
AndroidJUnit3Builder
which is not what I need.

You can either add this directly:
<dependency>
<groupId>com.uphyca</groupId>
<artifactId>android-junit4</artifactId>
<version>0.5</version>
</dependency>
Or if you are just looking for the dependencies, try to look this page:
http://mvnrepository.com/artifact/com.uphyca/android-junit4/0.5