We Keep Coding

How to extract values from a String that cannot be converted to Json

While processing the DialogFlow Response object, I get the below given string as textPayload. If this is a Json string, I can easily convert it to a JSONObject and then extract the values. However, could not convert this to a Json Object. How do I get the values for the keys in this string? What is a good way to parse this string in Java?
String to be processed
Dialogflow Response : id: "XXXXXXXXXXXX"
lang: "en"
session_id: "XXXXX"
timestamp: "2020-04-26T16:38:26.162Z"
result {
source: "agent"
resolved_query: "Yes"
score: 1.0
parameters {
}
contexts {
name: "enaccaccountblocked-followup"
lifespan: 1
parameters {
}
}
metadata {
intent_id: "XXXXXXXXXXXX"
intent_name: "EN : ACC : Freezing Process - Yes"
end_conversation: true
webhook_used: "false"
webhook_for_slot_filling_used: "false"
is_fallback_intent: "false"
}
fulfillment {
speech: "Since you have been permanently blocked, please request to unblock your account"
messages {
lang: "en"
type {
number_value: 0.0
}
speech {
string_value: "Since you have been permanently blocked, please request to unblock your account."
}
}
}
}
status {
code: 200
error_type: "success"
}

Convert it to valid json, then map using one of the many libraries out there.
You'll only need to:
replace "Dialogflow Response :" with {
add } to the end
add commas between attributes, ie
at the end of every line with a ":"
after "}", except when the next non-whitespace is also "}"
Jackson (at least) can be configured to allow quotes around attribute names as optional.
Deserializing to a Map<String, Object> works for all valid json (except an array, which this isn't).

If I understand you correctly the issue here is that the keys do not have quotations marks, hence, a JSON parser will reject this.
Since the keys all start on a new line with some white-space and all end with a colon : you can fix this easily with a regular expression.
See How to Fix JSON Key Values without double-quotes?
You can then parse it to a Map via
Map<String, Object> map
= objectMapper.readValue(json, new TypeReference<Map<String,Object>>(){});
(but I assume you are aware of this).

Create a class for TextPayload object like this.
public class TextPayload {
private int session_id;
private String lang;
private String timestamp;
private String[] metadata ;
//Other attributes
//getters setters
}
Then using an ObjectMapper extract the values from textpayload like this:
ObjectMapper mapper = new ObjectMapper();
TextPayload textPayload = mapper.readValue(output, User.class);
To utilize ObjectMapper and hands on with it follow this

you can use the nodejs package parse-dialogflow-log to parse the textResponse string.
replace "Dialogflow Response :" with "{"
add "}" to the end
run the package on the result and you'll get a nice json.

How to use OpenFeign to get a pojo array?

I’m trying to use a OpenFeign client to hit an API, get some JSON, and convert it to a POJO array.
Previously I was simply getting a string of JSON and using Gson to convert it to the array like so
FeignInterface {
String get(Request req);
}
String json = feignClient.get(request);
POJO[] pojoArray = new Gson().fromJson(json, POJO[].class);
This was working. I would like to eliminate the extra step and have feign auto decode the JSON and return a POJO directly though, so I am trying this
FeignInterface {
POJO[] get(Request req);
}
POJO[] pojoArray = feignClient.getJsonPojo(request);`
I am running into this error
feign.codec.DecodeException: java.lang.IllegalStateException: Expected BEGIN_ARRAY but was STRING at line 1 column 2 path $
Both methods used the same builder
feignClient = Feign.builder()
.encoder(new GsonEncoder())
.decoder(new GsonDecoder())
.target(FeignInterface.class, apiUrl);
Anyone have any ideas?

You have broken JSON payload. Before serialising you need to remove all unsupported characters. Feign allows this:
If you need to pre-process the response before give it to the Decoder,
you can use the mapAndDecode builder method. An example use case is
dealing with an API that only serves jsonp, you will maybe need to
unwrap the jsonp before send it to the Json decoder of your choice:
public class Example {
public static void main(String[] args) {
JsonpApi jsonpApi = Feign.builder()
.mapAndDecode((response, type) -> jsopUnwrap(response, type), new GsonDecoder())
.target(FeignInterface.class, apiUrl);
}
}
So, you need to do the same in your configuration and:
trim response and remove all whitespaces at the beginning and end of payload.
remove all new_line characters like: \r\n, \r, \n
Use online tool to be sure your JSON payload is valid and ready to be deserialised .

Converting malformed json array string to Java object

I have a malformed json array string which I get from an API call as follows:
[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Processed successfully\"}]\"}]
There is a double quote before open square bracket in the value of Response Msg property.
Is there a way to convert this into Java object ?
What I have tried so far:
I have used Jackson to parse it as follows but it gives error
ObjectMapper mapper = new ObjectMapper();
mapper.setPropertyNamingStrategy(new ResponseNameStrategy());
Response[] response = mapper.readValue(strOutput1, Response[].class);
Error: Can not deserialize instance of java.util.ArrayList out of VALUE_STRING token
I have also tried using Gson to parse it but it also gives error
Gson gson = new GsonBuilder()
.setFieldNamingPolicy(FieldNamingPolicy.UPPER_CAMEL_CASE)
.create();
Response[] response = gson.fromJson(strOutput1, Response[].class);
Error: Expected BEGIN_ARRAY but was STRING at line 1 column 35 path $[0].ResponseMsg
I have gone through the following links on StackOverflow but none of them has addressed my issue:
How to Convert String Array JSON in a Java Object
Convert a JSON string to object in Java ME?
JSON Array to Java objects
Convert json String to array of Objects
converting 'malformed' java json object to javascript

I think the answer is in the comments, you appear to be trying to solve the issue on the wrong place.
You are receiving json which you wish to parse into java objects, unfortunately the json is malformed so will not parse.
As a general rule you should never be trying to solve the symptom, but should look for the root cause and fix that, it may sound trivial but fixing symptoms leads to messy, unpredictable, and unmaintainable systems.
So the answer is fix the json where it is being broken. If this is something or of your control, while you wait for the fix, you could put a hack in to fix the json before you parse it.
This way you won't compromise your parsing, and only have a small piece of string replacement to remove when the third party has fixed the issue. But do not go live with the hack, it should only be used during development.

As i mentioned in the comment, you should prepare your service response in order to parse it.
I implemented an example:
public class JsonTest {
public static void main(String args[]) throws JsonProcessingException, IOException{
String rawJson =
"[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Processed successfully\"}]\"}]";
String goodJson = "{"+rawJson.split("[{{.}]")[2]+"}";
ObjectMapper mapper = new ObjectMapper();
final ObjectNode node = mapper.readValue(goodJson, ObjectNode.class);
System.out.println("Pretty Print: " + mapper.writerWithDefaultPrettyPrinter().writeValueAsString(node));
System.out.println("Just code: " + node.get("Code"));
}
}
Which returns:

This is how I finally solved my issue:
String inputJsonStr = "[{\"ResponseCode\":1,\"ResponseMsg\":\"[{\"Code\":\"CA2305181\",\"Message\":\"Claim has been added successfully.\"}"
+ "]\"}]";
int indexOfRes = inputJsonStr.indexOf("ResponseMsg");
if(inputJsonStr.substring(indexOfRes+13,indexOfRes+14).equals("\""))
{
inputJsonStr = inputJsonStr.substring(0,indexOfRes+13) + inputJsonStr.substring(indexOfRes+14);
}
int indexOfFirstClosingSquare = inputJsonStr.indexOf("]");
if(inputJsonStr.substring(indexOfFirstClosingSquare+1, indexOfFirstClosingSquare+2).equals("\"")) {
inputJsonStr = inputJsonStr.substring(0, indexOfFirstClosingSquare+1)+inputJsonStr.substring(indexOfFirstClosingSquare+2);
}
Now inputJsonStr contains a valid json array which can be parsed into Java custom object array easily with gson as given in this SO link:
Convert json String to array of Objects

Update JWT token field using numbus library

I have a Signed JWT token and I need to update an existing field, let's call it userName. I'm using NIMBUS + JOSE and. I figured out how to parse it and extract the claims:
SignedJWT.parse(token)
but parsing is not the only thing i need: I have update the field and reassemble its token back. Is there an easy way or any kind of idiomatic solution that will work without recreating the token from scratch.

I spend some time trying to figure out how to modify JWT token using the library.
And I used a quick and dirty solution:
// Split token into parts (parts are separated with '.'
final String[] tokenParts = token.split("\\.");
// decode payload part
final String decodedPayload =
new String(Base64.getDecoder().decode(tokenParts[1]), "UTF-8");
// enrich payload with additional userName field by adding it to the end of
// JSON. Remove the last character which is '}' and append data as String
final String updatedDecodedPayload =
decodedPayload.substring(0, decodedPayload.length() - 1)
+ ",\"userName\":\"" + "Richard" + "\"}";
// update payload with userId field and encode it back to base64
tokenParts[1] = Base64.getEncoder().encodeToString(
updatedDecodedPayload.getBytes()
);
final String updatedToken = String.join(".", tokenParts));

What is the error in the following HL7 encoding?

I am trying to encode an HL7 message of the type ORU_R01 using the HAPI 2.0 library for an OpenMRS module. I have followed the tutorials given in the HAPI documentation and according to that, I have populated the required fields of the ORU_R01 message. Now, I want to post this message using the following link:
http://localhost:8080/openmrs/remotecommunication/postHl7.form
I am using the following message for testing:
MSH|^~\&|||||20140713154042||ORU^R01|20140713154042|P|2.5|1
PID|||1
OBR|1||1234^SensorReading|88304
OBX|0|NM|1||45
OBX|1|NM|2||34
OBX|2|NM|3||23
I have properly ensured that all the parameters are correct. Once I have posted the HL7 message, I start the HL7 task from the scheduler. Then I go to the admin page and click on "Manage HL7 errors" in order to see if the message arrives there. I get the following stack trace:
ca.uhn.hl7v2.HL7Exception: HL7 encoding not supported
...
Caused by: ca.uhn.hl7v2.parser.EncodingNotSupportedException: Can't parse message beginning MSH|^~\
at ca.uhn.hl7v2.parser.Parser.parse(Parser.java:140)
The full stack trace is here: http://pastebin.com/ZnbFqfWC.
I have written the following code to encode the HL7 message (using the HAPI library):
public String createHL7Message(int p_id, int concept_id[], String val[])
throws HL7Exception {
ORU_R01 message = new ORU_R01();
SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMddHHmmss",
Locale.ENGLISH);
MSH msh = message.getMSH();
msh.getFieldSeparator().setValue("|");
msh.getEncodingCharacters().setValue("^~\\&");
msh.getProcessingID().getProcessingID().setValue("P");
msh.getSequenceNumber().setValue("1");
msh.getMessageType().getTriggerEvent().setValue("R01");
msh.getMessageType().getMessageCode().setValue("ORU");
msh.getVersionID().getVersionID().setValue("2.5");
msh.getMessageControlID().setValue(
sdf.format(Calendar.getInstance().getTime()));
msh.getDateTimeOfMessage().getTime()
.setValue(sdf.format(Calendar.getInstance().getTime()));
ORU_R01_ORDER_OBSERVATION orderObservation = message
.getPATIENT_RESULT().getORDER_OBSERVATION();
ca.uhn.hl7v2.model.v25.segment.PID pid = message.getPATIENT_RESULT()
.getPATIENT().getPID();
Patient patient = (Patient) Context.getPatientService()
.getPatient(p_id);
System.out.println(String.valueOf(p_id) + " " + patient.getGivenName()
+ " " + patient.getFamilyName());
pid.getPatientName(0).getFamilyName().getSurname()
.setValue(patient.getFamilyName());
pid.getPatientName(0).getGivenName().setValue(patient.getGivenName());
pid.getPatientIdentifierList(0).getIDNumber()
.setValue(String.valueOf(p_id));
System.out.println();
// Parser parser = new PipeParser();
// String encodedMessage = null;
// encodedMessage = parser.encode(message);
// System.out.println(encodedMessage);
// Populate the OBR
OBR obr = orderObservation.getOBR();
obr.getSetIDOBR().setValue("1");
obr.getFillerOrderNumber().getEntityIdentifier().setValue("1234");
obr.getFillerOrderNumber().getNamespaceID().setValue("SensorReading");
obr.getUniversalServiceIdentifier().getIdentifier().setValue("88304");
Varies value = null;
// Varies value[] = new Varies[4];
for (int i = 0; i < concept_id.length; i++) {
ORU_R01_OBSERVATION observation = orderObservation
.getOBSERVATION(i);
OBX obx2 = observation.getOBX();
obx2.getSetIDOBX().setValue(String.valueOf(i));
obx2.getObservationIdentifier().getIdentifier()
.setValue(String.valueOf(concept_id[i]));
obx2.getValueType().setValue("NM");
NM nm = new NM(message);
nm.setValue(val[i]);
value = obx2.getObservationValue(0);
value.setData(nm);
}
Parser parser = new PipeParser();
String encodedMessage = null;
encodedMessage = parser.encode(message);
return encodedMessage;
}
In all likelihood, something is wrong with the MSH segment of the message, but I cannot seem to figure out what it is. What can I do to correct this error?

Why do you declare the Encoding Characters using double backslashes?
msh.getEncodingCharacters().setValue("^~\\&");
Shouldn't it be:
msh.getEncodingCharacters().setValue("^~\&");
...and because your message is using the default encoding characters maybe you don't even need to declare them at all? Extract from HAPI MSH Class reference
getENCODINGCHARACTERS
public ST getENCODINGCHARACTERS()
Returns MSH-2: "ENCODING CHARACTERS" - creates it if necessary
Update
I have no previous experience with HAPI. A quick google found an ORU example. Could you try initializing your MSH with initQuickstart("ORU", "R01", "P");
According to the comments in the example-code the initQuickstart method populates all of the mandatory fields in the MSH segment of the message, including the message type, the timestamp, and the control ID. (...and hopefully the default encoding chars as well :-)