We Keep Coding

How to extract values from a String that cannot be converted to Json

While processing the DialogFlow Response object, I get the below given string as textPayload. If this is a Json string, I can easily convert it to a JSONObject and then extract the values. However, could not convert this to a Json Object. How do I get the values for the keys in this string? What is a good way to parse this string in Java?
String to be processed
Dialogflow Response : id: "XXXXXXXXXXXX"
lang: "en"
session_id: "XXXXX"
timestamp: "2020-04-26T16:38:26.162Z"
result {
source: "agent"
resolved_query: "Yes"
score: 1.0
parameters {
}
contexts {
name: "enaccaccountblocked-followup"
lifespan: 1
parameters {
}
}
metadata {
intent_id: "XXXXXXXXXXXX"
intent_name: "EN : ACC : Freezing Process - Yes"
end_conversation: true
webhook_used: "false"
webhook_for_slot_filling_used: "false"
is_fallback_intent: "false"
}
fulfillment {
speech: "Since you have been permanently blocked, please request to unblock your account"
messages {
lang: "en"
type {
number_value: 0.0
}
speech {
string_value: "Since you have been permanently blocked, please request to unblock your account."
}
}
}
}
status {
code: 200
error_type: "success"
}

Convert it to valid json, then map using one of the many libraries out there.
You'll only need to:
replace "Dialogflow Response :" with {
add } to the end
add commas between attributes, ie
at the end of every line with a ":"
after "}", except when the next non-whitespace is also "}"
Jackson (at least) can be configured to allow quotes around attribute names as optional.
Deserializing to a Map<String, Object> works for all valid json (except an array, which this isn't).

If I understand you correctly the issue here is that the keys do not have quotations marks, hence, a JSON parser will reject this.
Since the keys all start on a new line with some white-space and all end with a colon : you can fix this easily with a regular expression.
See How to Fix JSON Key Values without double-quotes?
You can then parse it to a Map via
Map<String, Object> map
= objectMapper.readValue(json, new TypeReference<Map<String,Object>>(){});
(but I assume you are aware of this).

Create a class for TextPayload object like this.
public class TextPayload {
private int session_id;
private String lang;
private String timestamp;
private String[] metadata ;
//Other attributes
//getters setters
}
Then using an ObjectMapper extract the values from textpayload like this:
ObjectMapper mapper = new ObjectMapper();
TextPayload textPayload = mapper.readValue(output, User.class);
To utilize ObjectMapper and hands on with it follow this

you can use the nodejs package parse-dialogflow-log to parse the textResponse string.
replace "Dialogflow Response :" with "{"
add "}" to the end
run the package on the result and you'll get a nice json.

How to use OpenFeign to get a pojo array?

I’m trying to use a OpenFeign client to hit an API, get some JSON, and convert it to a POJO array.
Previously I was simply getting a string of JSON and using Gson to convert it to the array like so
FeignInterface {
String get(Request req);
}
String json = feignClient.get(request);
POJO[] pojoArray = new Gson().fromJson(json, POJO[].class);
This was working. I would like to eliminate the extra step and have feign auto decode the JSON and return a POJO directly though, so I am trying this
FeignInterface {
POJO[] get(Request req);
}
POJO[] pojoArray = feignClient.getJsonPojo(request);`
I am running into this error
feign.codec.DecodeException: java.lang.IllegalStateException: Expected BEGIN_ARRAY but was STRING at line 1 column 2 path $
Both methods used the same builder
feignClient = Feign.builder()
.encoder(new GsonEncoder())
.decoder(new GsonDecoder())
.target(FeignInterface.class, apiUrl);
Anyone have any ideas?

You have broken JSON payload. Before serialising you need to remove all unsupported characters. Feign allows this:
If you need to pre-process the response before give it to the Decoder,
you can use the mapAndDecode builder method. An example use case is
dealing with an API that only serves jsonp, you will maybe need to
unwrap the jsonp before send it to the Json decoder of your choice:
public class Example {
public static void main(String[] args) {
JsonpApi jsonpApi = Feign.builder()
.mapAndDecode((response, type) -> jsopUnwrap(response, type), new GsonDecoder())
.target(FeignInterface.class, apiUrl);
}
}
So, you need to do the same in your configuration and:
trim response and remove all whitespaces at the beginning and end of payload.
remove all new_line characters like: \r\n, \r, \n
Use online tool to be sure your JSON payload is valid and ready to be deserialised .

Converting malformed json array string to Java object

I have a malformed json array string which I get from an API call as follows:
[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Processed successfully\"}]\"}]
There is a double quote before open square bracket in the value of Response Msg property.
Is there a way to convert this into Java object ?
What I have tried so far:
I have used Jackson to parse it as follows but it gives error
ObjectMapper mapper = new ObjectMapper();
mapper.setPropertyNamingStrategy(new ResponseNameStrategy());
Response[] response = mapper.readValue(strOutput1, Response[].class);
Error: Can not deserialize instance of java.util.ArrayList out of VALUE_STRING token
I have also tried using Gson to parse it but it also gives error
Gson gson = new GsonBuilder()
.setFieldNamingPolicy(FieldNamingPolicy.UPPER_CAMEL_CASE)
.create();
Response[] response = gson.fromJson(strOutput1, Response[].class);
Error: Expected BEGIN_ARRAY but was STRING at line 1 column 35 path $[0].ResponseMsg
I have gone through the following links on StackOverflow but none of them has addressed my issue:
How to Convert String Array JSON in a Java Object
Convert a JSON string to object in Java ME?
JSON Array to Java objects
Convert json String to array of Objects
converting 'malformed' java json object to javascript

I think the answer is in the comments, you appear to be trying to solve the issue on the wrong place.
You are receiving json which you wish to parse into java objects, unfortunately the json is malformed so will not parse.
As a general rule you should never be trying to solve the symptom, but should look for the root cause and fix that, it may sound trivial but fixing symptoms leads to messy, unpredictable, and unmaintainable systems.
So the answer is fix the json where it is being broken. If this is something or of your control, while you wait for the fix, you could put a hack in to fix the json before you parse it.
This way you won't compromise your parsing, and only have a small piece of string replacement to remove when the third party has fixed the issue. But do not go live with the hack, it should only be used during development.

As i mentioned in the comment, you should prepare your service response in order to parse it.
I implemented an example:
public class JsonTest {
public static void main(String args[]) throws JsonProcessingException, IOException{
String rawJson =
"[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Processed successfully\"}]\"}]";
String goodJson = "{"+rawJson.split("[{{.}]")[2]+"}";
ObjectMapper mapper = new ObjectMapper();
final ObjectNode node = mapper.readValue(goodJson, ObjectNode.class);
System.out.println("Pretty Print: " + mapper.writerWithDefaultPrettyPrinter().writeValueAsString(node));
System.out.println("Just code: " + node.get("Code"));
}
}
Which returns:

This is how I finally solved my issue:
String inputJsonStr = "[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Claim has been added successfully.\"}"
+ "]\"}]";
int indexOfRes = inputJsonStr.indexOf("ResponseMsg");
if(inputJsonStr.substring(indexOfRes+13,indexOfRes+14).equals("\""))
{
inputJsonStr = inputJsonStr.substring(0,indexOfRes+13) + inputJsonStr.substring(indexOfRes+14);
}
int indexOfFirstClosingSquare = inputJsonStr.indexOf("]");
if(inputJsonStr.substring(indexOfFirstClosingSquare+1, indexOfFirstClosingSquare+2).equals("\"")) {
inputJsonStr = inputJsonStr.substring(0, indexOfFirstClosingSquare+1)+inputJsonStr.substring(indexOfFirstClosingSquare+2);
}
Now inputJsonStr contains a valid json array which can be parsed into Java custom object array easily with gson as given in this SO link:
Convert json String to array of Objects

Java REST response value with missing spelling char

I have REST written in Java and the JSON response message is not valid.
I have messages defined in single file messages.properties. I expect that response should be something like that:
NOT_FOUND_PERSON = Person doesn't exist
However I got response with missing spelling:
['errorMsg': 'Person doesnt exist.']
Where is the problem? Cannot be due to wrong setup ResourceBundleMessageSource in config? I noticed there is missing UTF8 coding.
Is there problem with some hide escape function or whatever?

I found the solution. The problem was with quotation mark in messages.properties.
In default, spring uses message bundle to represent messages from properties file. Any occurences of quotation mark must be escaped by single quote otherwise it won't be displayed properly.
This
test.message2={0}'s message
must be replaced by this
test.message2={0}''s message
Resource:
https://www.mscharhag.com/java/resource-bundle-single-quote-escaping

I used double quotes in the JSON string.
{
"objectDetail": {
"id": "1",
"anotherId": "1",
"errorText": "This element doesn't exist."
}
}
And I tried with your data.
['errorMsg': 'Person doesn't exist.']
This is my example, it returns an object which is converted to JSON String.
#RestController
#RequestMapping(method = RequestMethod.GET, produces = {MediaType.APPLICATION_JSON_VALUE})
public class JsonRestController {
#RequestMapping("/api/{id}")
public ResponseEntity<RestObject> getModel(#PathVariable Long id) {
RestObject restObject = restService.getModel(id);
return new ResponseEntity<>(restObject, HttpStatus.OK);
}
}
It escapes double quotes.
{
"timestamp": "2018-01-23 13:37:53",
"title": "Rest object. This is the error does\"t and don't\" aaa.",
"fullText": "This is the full text. ID: 3",
"id": 3,
"value": 0.449947838273684
}
https://stackoverflow.com/a/15637481/4587961
https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf

Android - decode unicode characters without StringEscapeUtils?

When I use Gson (JsonParser.parse) to decode the following:
{ "item": "Bread", "cost": {"currency": "\u0024", "amount": "3"}, "description": "This is bread\u2122. \u00A92015" }
The "currency" element is returned as a string of characters (and is not converted to a unicode character). Is there a setting or method in Gson that could help me?
If not, is there any way in Android to convert a string that contains one or more escaped character sequences (like "\u0024") to an output string with unicode characters (without writing my own and without using StringEscapeUtils from Apache)?
I'd like to avoid adding another library (for just one small feature).
Update
Looks like the server was double escaping the back slash in the unicode escape sequence. Thanks everyone for your help!

Is it only me or is it really more complicated than simply using TextView's setText() method? Anyhow, following is working just fine on my end with the given sample json (put the sample to assets and read it using loadJSONFromAsset()):
JsonParser parser = new JsonParser();
JsonElement element = parser.parse(loadJSONFromAsset());
JsonObject obj = element.getAsJsonObject();
JsonObject cost = obj.getAsJsonObject("cost");
JsonPrimitive sign = cost.get("currency").getAsJsonPrimitive();
TextView tv = (TextView)findViewById(R.id.dollar_sign);
tv.setText(sign.getAsString());

Gson returns "$". Something is wrong in your set up.
String s = "{ \"item\": \"Bread\", \"cost\": {\"currency\": "
+ "\"\\u0024\", \"amount\": \"3\"}, \"description\": "
+ "\"This is bread\\u2122. \\u00A92015\" }\n";
JsonElement v = new JsonParser().parse(s);
assertEquals("$", v.getAsJsonObject().get("cost").getAsJsonObject()
.get("currency").getAsString());

You can parse it as a hex number
char c = (char)Integer.parseInt(str.substring(2), 16);