We Keep Coding

Cannot deserialize instance of `java.lang.String` in node.js server to kafka connect connection

I use curl to submit to our Kafka Connect service a JSON request message with information about the connector, it is working successfully.
$ curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{ \"name\": \"inventory-connector\", \"config\": { \"connector.class\": \"io.debezium.connector.mysql.MySqlConnector\", \"tasks.max\": \"1\", \"database.hostname\": \"mysql\", \"database.port\": \"3306\", \"database.user\": \"debezium\", \"database.password\": \"dbz\", \"database.server.id\": \"184054\", \"database.server.name\": \"dbserver1\", \"database.whitelist\": \"inventory\", \"database.history.kafka.bootstrap.servers\": \"kafka:9092\", \"database.history.kafka.topic\": \"dbhistory.inventory\" } }'
now I am using node.js server to send data to kafka connect server.
var body = {
"name": "abc",
"config": {
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"tasks.max": "1",
"database.hostname": "mysql",
"database.port": "3306",
"database.user": "debezium",
"database.password": "dbz",
"database.server.id": "184054",
"database.server.name": "dbserver1",
"database.whitelist": "inventory",
"database.history.kafka.bootstrap.servers": "kafka:9092",
"database.history.kafka.topic": "schema-changes.inventory"
}
};
var options = {
method: 'PUT',
uri: 'http://localhost/connectors/abc/config',
headers: {
'User-Agent': 'Request-Promise'
},
json: true ,
body: body
};
rp(options)
.then(function (data) {
return res.status(200).json({ 'data': data});
})
.catch(function (err) {
console.log(err);
return res.status(500).json({ error: err});
});
however the code throw out an error: saying
{ StatusCodeError: 500 - {"error_code":500,"message":"Cannot deserialize instance of `java.lang.String` out of START_OBJECT token\n at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 42] (through reference chain: java.util.LinkedHashMap[\"config\"])"}
The API description is from https://docs.confluent.io/current/connect/references/restapi.html

If I read the confluent doc correctly, you did mix up two different API endpoints.
In your code, you use the endpoint /connectors/abc/config, which according to the documentation takes a single config object as toplevel, so like this:
{
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"tasks.max": "1",
"database.hostname": "mysql",
"database.port": "3306",
"database.user": "debezium",
"database.password": "dbz",
"database.server.id": "184054",
"database.server.name": "dbserver1",
"database.whitelist": "inventory",
"database.history.kafka.bootstrap.servers": "kafka:9092",
"database.history.kafka.topic": "schema-changes.inventory"
}
But your JSON object looks like it was meant for the /connectors endpoint.
Changing either the endpoint or your JSON object to match the endpoint you have chosen may fix the problem.

Swagger Codegen not generating API Key example in it's documentation

I am new to using Swagger Codegen so I assume I am doing something completely wrong. I have a remote swagger.json that I am using to generate a java client. Everything works and looks good but the example usages in the readme are not referencing any API key.
I don't have access to the remote API that I am using just trying to create a nice Java SDK for interfacing with it.
This is what I am using to generate the code.
java -jar C:/Development/codegen/swagger-codegen-cli.jar generate
-a "client_id:XXXXXXXXXX,client_secret:YYYYYYYYYYYYY"
-i https://BLAH/swagger/v1/swagger.json -l java
-o C:/Development/workspace/JavaClient
-c C:/Development/workspace/JavaClient/java-genconfig.json
Then the example that get's generated in the read me looks something like this...
public static void main(String[] args) {
BusinessGroupApi apiInstance = new BusinessGroupApi();
Integer businessGroupId = 56; // Integer | The ID of the business group to fetch.
String apiVersion = "1.0"; // String | The requested API version
try {
BusinessGroup result = apiInstance.getBusinessGroupAsync(businessGroupId, apiVersion);
System.out.println(result);
} catch (ApiException e) {
System.err.println("Exception when calling BusinessGroupApi#getBusinessGroupAsync");
e.printStackTrace();
}
}
Side Note
When I try running the example, I get an error saying
Failed to connect to localhost/0:0:0:0:0:0:0:1:443
Is there some setting that I need to set so it looks for the service at a remote location instead of locally?
EDIT
I realized that I needed to modify their swagger as they might not have everything fleshed out.
Swagger Codegen Version = 2.4.5
swagger.json (Excluded the paths and definitions) I also downloaded thiers locally and added some more info to make it generate more info. I added host, basePath, schemes, consumes, produces, security and securityDefinitions.
{
"swagger": "2.0",
"info": {
"version": "1.0",
"title": "removed"
},
"host": "apitest.removed.com",
"basePath": "/removed/api",
"schemes": [
"https"
],
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"security": [
{
"clientId": []
},
{
"clientSecret": []
}
],
"securityDefinitions": {
"clientId": {
"type": "apiKey",
"in": "header",
"name": "client_id"
},
"clientSecret": {
"type": "apiKey",
"in": "header",
"name": "client_secret"
}
}
}
This actually updated the readme to look how I would expect it to look
ApiClient defaultClient = Configuration.getDefaultApiClient();
// Configure API key authorization: client_id
ApiKeyAuth client_id = (ApiKeyAuth) defaultClient.getAuthentication("client_id");
client_id.setApiKey("YOUR API KEY");
// Uncomment the following line to set a prefix for the API key, e.g. "Token" (defaults to null)
//client_id.setApiKeyPrefix("Token");
// Configure API key authorization: client_secret
ApiKeyAuth client_secret = (ApiKeyAuth) defaultClient.getAuthentication("client_secret");
client_secret.setApiKey("YOUR API KEY");
// Uncomment the following line to set a prefix for the API key, e.g. "Token" (defaults to null)
//client_secret.setApiKeyPrefix("Token");
BusinessGroupApi apiInstance = new BusinessGroupApi();
Integer businessGroupId = 56; // Integer | The ID of the business group to fetch.
String apiVersion = "1.0"; // String | The requested API version
try {
BusinessGroup result = apiInstance.getBusinessGroupAsync(businessGroupId, apiVersion);
System.out.println(result);
} catch (ApiException e) {
System.err.println("Exception when calling BusinessGroupApi#getBusinessGroupAsync");
e.printStackTrace();
}
Yet when I plug in my client_id and client_secret and try to run it I get a Exception in thread "main" java.lang.NullPointerException
at removed.Tester.main(Tester.java:18). I think I might have the security definitions not set up correctly.
It's not OAuth but they require a client_id and client_secret to be passed in the header parameters.
EDIT 2
Exception when calling BusinessGroupApi#getBusinessGroupAsync
package.client.ApiException:
at package.client.ApiClient.handleResponse(ApiClient.java:927)
at package.client.ApiClient.execute(ApiClient.java:843)
at package.client.api.BusinessGroupApi.getBusinessGroupAsyncWithHttpInfo(BusinessGroupApi.java:148)
at package.client.api.BusinessGroupApi.getBusinessGroupAsync(BusinessGroupApi.java:133)
at Tester.main(Tester.java:30)
Line 30 -> BusinessGroup result = apiInstance.getBusinessGroupAsync(businessGroupId, apiVersion);
EDIT 3
I enabled debugging with
defaultClient.setDebugging(true);
It looks like it's working but for some reason doesn't like my client_id and client_secret despite them both being the ones I used in postman.
--> GET <removed>/api/api/businessGroups/56?api-version=1.0 HTTP/1.1
Accept: application/json
client_secret: <removed>
client_id: <removed>
User-Agent: Swagger-Codegen/1.0-SNAPSHOT/java
--> END GET
<-- HTTP/1.1 403 (393ms)
Content-Length: 80
Content-Type: application/json
Date: Mon, 10 Jun 2019 20:05:07 GMT
QL-LB-Appliance: <removed>
QL-LB-Pool: <removed>
QL-LB-Server: <removed>
OkHttp-Sent-Millis: <removed>
OkHttp-Received-Millis: <removed>
{ "error": "invalid_client", "description": "wrong client_id or client_secret" }
<-- END HTTP (80-byte body)
Exception when calling BusinessGroupApi#getBusinessGroupAsync
<removed>.client.ApiException:
at <removed>.client.ApiClient.handleResponse(ApiClient.java:927)
at <removed>.client.ApiClient.execute(ApiClient.java:843)
at <removed>.client.api.BusinessGroupApi.getBusinessGroupAsyncWithHttpInfo(BusinessGroupApi.java:148)
at <removed>.client.api.BusinessGroupApi.getBusinessGroupAsync(BusinessGroupApi.java:133)
at Tester.main(Tester.java:32)

Spring Cloud Gateway for composite API calls?

I am starting to build a Microservice API Gateway, and I am considering Spring Cloud to help me with the routing. But some calls to the Gateway API will need multiple requests to different services.
Lets say I have 2 services: Order Details Service and Delivery Service. I want to have a Gateway endpoint GET /orders/{orderId} that makes a call to Order Details service and then Delivery Service and combine the two to return full Order details with delivery. Is this possible with the routing of Spring cloud or should I make these by hand using something like RestTemplate to make the calls?

There is an enhancement proposal posted on GitHub to have routes support multiple URIs. So far, there aren't any plans to implement this yet, at least, not according to one of the contributors.

As posted in the Spring Cloud Gateway Github issue mentioned by g00glen00b, until the library develops a Filter for this, I resolved it using the ModifyResponseBodyGatewayFilterFactory in my own custom Filter.
Just in case it's useful for anyone else, I provide the base implementation here (it may need some rework, but it should be enough to make the point).
Simply put, I have a "base" service retrieving something like this:
[
{
"targetEntryId": "624a448cbc728123b47d08c4",
"sections": [
{
"title": "sadasa",
"description": "asda"
}
],
"id": "624a448c45459c4d757869f1"
},
{
"targetEntryId": "624a44e5bc728123b47d08c5",
"sections": [
{
"title": "asda",
"description": null
}
],
"id": "624a44e645459c4d757869f2"
}
]
And I want to enrich these entries with the actual targetEntry data (of course, identified by targetEntryId).
So, I created my Filter based on the ModifyResponseBody one:
/**
* <p>
* Filter to compose a response body with associated data from a second API.
* </p>
*
* #author rozagerardo
*/
#Component
public class ComposeFieldApiGatewayFilterFactory extends
AbstractGatewayFilterFactory<ComposeFieldApiGatewayFilterFactory.Config> {
public ComposeFieldApiGatewayFilterFactory() {
super(Config.class);
}
#Autowired
ModifyResponseBodyGatewayFilterFactory modifyResponseBodyFilter;
ParameterizedTypeReference<List<Map<String, Object>>> jsonType =
new ParameterizedTypeReference<List<Map<String, Object>>>() {
};
#Value("${server.port:9080}")
int aPort;
#Override
public GatewayFilter apply(final Config config) {
return modifyResponseBodyFilter.apply((c) -> {
c.setRewriteFunction(List.class, List.class, (filterExchange, input) -> {
List<Map<String, Object>> castedInput = (List<Map<String, Object>>) input;
// extract base field values (usually ids) and join them in a "," separated string
String baseFieldValues = castedInput.stream()
.map(bodyMap -> (String) bodyMap.get(config.getOriginBaseField()))
.collect(Collectors.joining(","));
// Request to a path managed by the Gateway
WebClient client = WebClient.create();
return client.get()
.uri(UriComponentsBuilder.fromUriString("http://localhost").port(aPort)
.path(config.getTargetGatewayPath())
.queryParam(config.getTargetQueryParam(), baseFieldValues).build().toUri())
.exchangeToMono(response -> response.bodyToMono(jsonType)
.map(targetEntries -> {
// create a Map using the base field values as keys fo easy access
Map<String, Map> targetEntriesMap = targetEntries.stream().collect(
Collectors.toMap(pr -> (String) pr.get("id"), pr -> pr));
// compose the origin body using the requested target entries
return castedInput.stream().map(originEntries -> {
originEntries.put(config.getComposeField(),
targetEntriesMap.get(originEntries.get(config.getOriginBaseField())));
return originEntries;
}).collect(Collectors.toList());
})
);
});
});
}
;
#Override
public List<String> shortcutFieldOrder() {
return Arrays.asList("originBaseField", "targetGatewayPath", "targetQueryParam",
"composeField");
}
/**
* <p>
* Config class to use for AbstractGatewayFilterFactory.
* </p>
*/
public static class Config {
private String originBaseField;
private String targetGatewayPath;
private String targetQueryParam;
private String composeField;
public Config() {
}
// Getters and Setters...
}
}
For completeness, this is the corresponding route setup using my Filter:
spring:
cloud:
gateway:
routes:
# TARGET ENTRIES ROUTES
- id: targetentries_route
uri: ${configs.api.tagetentries.baseURL}
predicates:
- Path=/api/target/entries
- Method=GET
filters:
- RewritePath=/api/target/entries(?<segment>.*), /target-entries-service$\{segment}
# ORIGIN ENTRIES
- id: originentries_route
uri: ${configs.api.originentries.baseURL}
predicates:
- Path=/api/origin/entries**
filters:
- RewritePath=/api/origin/entries(?<segment>.*), /origin-entries-service$\{segment}
- ComposeFieldApi=targetEntryId,/api/target/entries,ids,targetEntry
And with this, my resulting response looks as follows:
[
{
"targetEntryId": "624a448cbc728123b47d08c4",
"sections": [
{
"title": "sadasa",
"description": "asda"
}
],
"id": "624a448c45459c4d757869f1",
"targetEntry": {
"id": "624a448cbc728123b47d08c4",
"targetEntityField": "whatever"
}
},
{
"targetEntryId": "624a44e5bc728123b47d08c5",
"sections": [
{
"title": "asda",
"description": null
}
],
"id": "624a44e645459c4d757869f2",
"targetEntry": {
"id": "624a44e5bc728123b47d08c5",
"targetEntityField": "somethingelse"
}
}
]

How to get a attribute variable with Json response using Wiremock

I need a dynamic response with different values using traffic parrot and wiremock.
I have an integration test with patterns in json files to get a response when I call the API's.
I need to call a product service to retrieve a specific product, my request json:
request.json
{
"request" {
"urlPathPattern": "/urlResponse/product/1000",
"method": "GET"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "response.json",
"transformers": [
"response-template"
]
}
}
And I need something like this:
response.json
{
"id": {{request.path.[2]}},
"type": VARIABLE or DYNAMIC CONTENT,
"other attr"....
}
I Want to pass a variable content depending of the request, for example, I have an Object
public CustomObject {
private int id = 1000;
private String type = "product";
}
When I call the API of product with ID 1000, I want a response.json with type set to "product". I see documentation of Traffic parrot example, but I do not know how to apply.
Edit 1: It is possible to define in request.json a map to define the variable type for the response? Something like this:
If I have an ID with 1000
objMap = 1000; type = "product",
objMap = 2000; type = "Balloon",
etc...

Get all the output from Watson conversation using java

If I have a IBM bluemix Watson conversation dialog output JSON like:
"output": {
"text": {
"values": [
"What is your name?",
"Name of the person?",
"Please specify the name of the person."
],
"selection_policy": "random",
"append": true
}
}
How can I get all the suggestions from the output response?

You can use context variables for saves what the user says using <? input.text ?>. Try to follow this simple example:
Create one child node in this Node above, and add:
{
"context": {
"userTypes": "<? input.text ?>"
},
"output": {
"text": {
"values": [
"All you said here: $userTypes."
],
"selection_policy": "sequential"
}
}
}
So, in your Java example, you can get the value of this context variable with this follow method:
private Map<String, Object> context = new HashMap<>();
ConversationService service = new ConversationService(ConversationService.VERSION_DATE_2016_09_20);
service.setUsernameAndPassword('Your Watson service UserName', 'Your watson service PassWord');
MessageRequest newMessage = new MessageRequest.Builder().inputText(inputmessage).context(context).build();
MessageResponse response = service.message('Your Workspace Id', newMessage).execute();
//Passing Context of last conversation
if(response.getContext() !=null)
{
context.clear();
context = response.getContext();
}
See more about Accessing and Evaluate objects.
See more about methods inside Watson Conversation.
See the official API Reference using Java.
See one full example from IBM Developer's in Watson Developer Cloud.