I’m trying to use a OpenFeign client to hit an API, get some JSON, and convert it to a POJO array.
Previously I was simply getting a string of JSON and using Gson to convert it to the array like so
FeignInterface {
String get(Request req);
}
String json = feignClient.get(request);
POJO[] pojoArray = new Gson().fromJson(json, POJO[].class);
This was working. I would like to eliminate the extra step and have feign auto decode the JSON and return a POJO directly though, so I am trying this
FeignInterface {
POJO[] get(Request req);
}
POJO[] pojoArray = feignClient.getJsonPojo(request);`
I am running into this error
feign.codec.DecodeException: java.lang.IllegalStateException: Expected BEGIN_ARRAY but was STRING at line 1 column 2 path $
Both methods used the same builder
feignClient = Feign.builder()
.encoder(new GsonEncoder())
.decoder(new GsonDecoder())
.target(FeignInterface.class, apiUrl);
Anyone have any ideas?
You have broken JSON payload. Before serialising you need to remove all unsupported characters. Feign allows this:
If you need to pre-process the response before give it to the Decoder,
you can use the mapAndDecode builder method. An example use case is
dealing with an API that only serves jsonp, you will maybe need to
unwrap the jsonp before send it to the Json decoder of your choice:
public class Example {
public static void main(String[] args) {
JsonpApi jsonpApi = Feign.builder()
.mapAndDecode((response, type) -> jsopUnwrap(response, type), new GsonDecoder())
.target(FeignInterface.class, apiUrl);
}
}
So, you need to do the same in your configuration and:
trim response and remove all whitespaces at the beginning and end of payload.
remove all new_line characters like: \r\n, \r, \n
Use online tool to be sure your JSON payload is valid and ready to be deserialised .
Related
I am currently running into
JsonParseException: Illegal character ((CTRL-CHAR, code 0)): only regular white space (\r, \n, \t) is allowed between tokens
at [Source: (String)" ){"BusinessDate"
It is because of the ')' before business date and there is four NUL before that, but they don't show up here as they are imaginary characters. That is the beginning of my json string I'm trying to map to my object. I'm just sending a ByteArray serialized as Avro, and trying to Deserialize using a ByteArray Deserializer (which I'm assuming is the problem). How do I get rid of those imaginary characters so it maps correctly to my object, or change my SCS consumer config to use Avro deserialization, only on that one consumer.
// External topic listener
#StreamListener(ChannelsScheduler.SCHEDULER_IN_FROM_EXTERNAL_EVENT)
public void consumeMessage(#Payload GenericMessage<String> message) throws IOException
{
logger.info("Consumed message from external topic: {}", message);
I have tried
GPTMStatus gptmStatus = mapper.readValue(message.getPayload(), GPTMStatus.class);
GPTMStatus gptmStatus = mapper.readValue(message.getPayload().trim(), GPTMStatus.class);
GPTMStatus gptmStatus = mapper.readValue(message.getPayload().replace(")", ""), GPTMStatus.class);
String json = StringUtils.newStringUtf8(message.getPayload().getBytes(StandardCharsets.UTF_8));
GPTMStatus gptmStatus = mapper.readValue(json, GPTMStatus.class);
All three are receiving the same error as above.
Full Payload:
" ){"BusinessDate":"2020-03-05","ContentUri":"20180712_EOB/1583443159984/0.xml","DFReference":"80712_EOB","DFRevision":0,"DFVersion":1583443159984}"
This is how I'm sending the message:
kafka
.send(
destination,
formatter.transform(df.getCanonicalPayload()).getBytes(StandardCharsets.UTF_8))
.get();
This line is slightly misleading (at least to me)
String json = StringUtils.newStringUtf8(message.getPayload().getBytes(StandardCharsets.UTF_8));
JSON CAN contain the NULL control character but only if it's escaped and within a string field. I'd suggest starting with something simple, json.replace("\0", ""); and checking if your code gets any further.
While processing the DialogFlow Response object, I get the below given string as textPayload. If this is a Json string, I can easily convert it to a JSONObject and then extract the values. However, could not convert this to a Json Object. How do I get the values for the keys in this string? What is a good way to parse this string in Java?
String to be processed
Dialogflow Response : id: "XXXXXXXXXXXX"
lang: "en"
session_id: "XXXXX"
timestamp: "2020-04-26T16:38:26.162Z"
result {
source: "agent"
resolved_query: "Yes"
score: 1.0
parameters {
}
contexts {
name: "enaccaccountblocked-followup"
lifespan: 1
parameters {
}
}
metadata {
intent_id: "XXXXXXXXXXXX"
intent_name: "EN : ACC : Freezing Process - Yes"
end_conversation: true
webhook_used: "false"
webhook_for_slot_filling_used: "false"
is_fallback_intent: "false"
}
fulfillment {
speech: "Since you have been permanently blocked, please request to unblock your account"
messages {
lang: "en"
type {
number_value: 0.0
}
speech {
string_value: "Since you have been permanently blocked, please request to unblock your account."
}
}
}
}
status {
code: 200
error_type: "success"
}
Convert it to valid json, then map using one of the many libraries out there.
You'll only need to:
replace "Dialogflow Response :" with {
add } to the end
add commas between attributes, ie
at the end of every line with a ":"
after "}", except when the next non-whitespace is also "}"
Jackson (at least) can be configured to allow quotes around attribute names as optional.
Deserializing to a Map<String, Object> works for all valid json (except an array, which this isn't).
If I understand you correctly the issue here is that the keys do not have quotations marks, hence, a JSON parser will reject this.
Since the keys all start on a new line with some white-space and all end with a colon : you can fix this easily with a regular expression.
See How to Fix JSON Key Values without double-quotes?
You can then parse it to a Map via
Map<String, Object> map
= objectMapper.readValue(json, new TypeReference<Map<String,Object>>(){});
(but I assume you are aware of this).
Create a class for TextPayload object like this.
public class TextPayload {
private int session_id;
private String lang;
private String timestamp;
private String[] metadata ;
//Other attributes
//getters setters
}
Then using an ObjectMapper extract the values from textpayload like this:
ObjectMapper mapper = new ObjectMapper();
TextPayload textPayload = mapper.readValue(output, User.class);
To utilize ObjectMapper and hands on with it follow this
you can use the nodejs package parse-dialogflow-log to parse the textResponse string.
replace "Dialogflow Response :" with "{"
add "}" to the end
run the package on the result and you'll get a nice json.
I am trying to automate twitter API. when tried to print "js.get("text") using
System.out.println(js.get("text")); I am getting error as
"The method println(boolean) is ambiguous for the type PrintStream"
I downloaded jars and passed in Build path as well "scribejava-apis-2.5.3" and "scribejava-core-4.2.0"
Below code is not allowing me use println for ------>js.get("text")
public class Basicfunc {
String Consumerkeys= "**************";
String Consumersecretkeys="*******************";
String Token="*******************";
String Tokensecret="***************************";
#Test
public void getLatestTweet(){
RestAssured.baseURI = "https://api.twitter.com/1.1/statuses";
Response res = given().auth().oauth(Consumerkeys, Consumersecretkeys, Token, Tokensecret).
queryParam("count","1").
when().get("/home_timeline.json").then().extract().response();
String response = res.asString();
System.out.println(response);
JsonPath js = new JsonPath(response);
System.out.println(js.get("text"));
}
}
Use System.out.println(js.getString("text")); instead of System.out.println(js.get("text"));, because get returns any primitive value.
I think your problem is that your twitter response is actually a list.
Try to use System.out.println(js.getList()[0].get("text")); and be aware that you are only using the first [0] entry and ignoring the rest.
I have a method in my Spring controller, in which I am returning an object containing a spring attribute with a value "\HelloWorld". To store it into Java String object I have to put escape character, then the string becomes "\\HelloWorld". When I print, that works totally fine and prints "\HelloWorld". but when I return it in JSon response, it's returning "\\HelloWorld".
But I want it to return "\HelloWorld".
Bellow is the snippet:
#RequestMapping("")
#ResponseBody
public MyDataObject greeting() {
MyDataObject f = new MyDataObject();
f.setMessage("\\HelloWorld");
return f;
}
It's Json response is "message":"\\HelloWorld", but I want it "\HelloWorld".
Note: I don't want to unescape manually specific to that string.
You can use a library such as Jakson and it will internally handle such complexities.
MyDataObject f = new MyDataObject();
f.setMessage("\\HelloWorld");
String payload = new ObjectMapper().writeValueAsString(params);
I have a malformed json array string which I get from an API call as follows:
[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Processed successfully\"}]\"}]
There is a double quote before open square bracket in the value of Response Msg property.
Is there a way to convert this into Java object ?
What I have tried so far:
I have used Jackson to parse it as follows but it gives error
ObjectMapper mapper = new ObjectMapper();
mapper.setPropertyNamingStrategy(new ResponseNameStrategy());
Response[] response = mapper.readValue(strOutput1, Response[].class);
Error: Can not deserialize instance of java.util.ArrayList out of VALUE_STRING token
I have also tried using Gson to parse it but it also gives error
Gson gson = new GsonBuilder()
.setFieldNamingPolicy(FieldNamingPolicy.UPPER_CAMEL_CASE)
.create();
Response[] response = gson.fromJson(strOutput1, Response[].class);
Error: Expected BEGIN_ARRAY but was STRING at line 1 column 35 path $[0].ResponseMsg
I have gone through the following links on StackOverflow but none of them has addressed my issue:
How to Convert String Array JSON in a Java Object
Convert a JSON string to object in Java ME?
JSON Array to Java objects
Convert json String to array of Objects
converting 'malformed' java json object to javascript
I think the answer is in the comments, you appear to be trying to solve the issue on the wrong place.
You are receiving json which you wish to parse into java objects, unfortunately the json is malformed so will not parse.
As a general rule you should never be trying to solve the symptom, but should look for the root cause and fix that, it may sound trivial but fixing symptoms leads to messy, unpredictable, and unmaintainable systems.
So the answer is fix the json where it is being broken. If this is something or of your control, while you wait for the fix, you could put a hack in to fix the json before you parse it.
This way you won't compromise your parsing, and only have a small piece of string replacement to remove when the third party has fixed the issue. But do not go live with the hack, it should only be used during development.
As i mentioned in the comment, you should prepare your service response in order to parse it.
I implemented an example:
public class JsonTest {
public static void main(String args[]) throws JsonProcessingException, IOException{
String rawJson =
"[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Processed successfully\"}]\"}]";
String goodJson = "{"+rawJson.split("[{{.}]")[2]+"}";
ObjectMapper mapper = new ObjectMapper();
final ObjectNode node = mapper.readValue(goodJson, ObjectNode.class);
System.out.println("Pretty Print: " + mapper.writerWithDefaultPrettyPrinter().writeValueAsString(node));
System.out.println("Just code: " + node.get("Code"));
}
}
Which returns:
This is how I finally solved my issue:
String inputJsonStr = "[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Claim has been added successfully.\"}"
+ "]\"}]";
int indexOfRes = inputJsonStr.indexOf("ResponseMsg");
if(inputJsonStr.substring(indexOfRes+13,indexOfRes+14).equals("\""))
{
inputJsonStr = inputJsonStr.substring(0,indexOfRes+13) + inputJsonStr.substring(indexOfRes+14);
}
int indexOfFirstClosingSquare = inputJsonStr.indexOf("]");
if(inputJsonStr.substring(indexOfFirstClosingSquare+1, indexOfFirstClosingSquare+2).equals("\"")) {
inputJsonStr = inputJsonStr.substring(0, indexOfFirstClosingSquare+1)+inputJsonStr.substring(indexOfFirstClosingSquare+2);
}
Now inputJsonStr contains a valid json array which can be parsed into Java custom object array easily with gson as given in this SO link:
Convert json String to array of Objects