I've got a class that contains a HashMap in it and am struggling to deserialize it back into my class. Here's what the class looks like...
public class MyClass
{
public String message;
public String category;
public HashMap<String,String> customData = new HashMap<String, String>();
public ArrayList<Device> devices = new ArrayList<Device>();
}
If I don't register any kind of converter I get the following output...
{"MyClass": {
"message": "My Message",
"category": "My Category",
"customData": [
[
"Name1",
"Data1"
],
[
"Name2",
"Data2"
],
[
"Name3",
"Data3"
],
[
"Name4",
"Data4"
]
],
"devices": [
{
"id": 17,
"firstName": "John",
"lastName": "Smith",
"devMode": false
}
]
}}
This looks pretty good and what I am after. However when I try to then convert it back to MyClass I get a:
com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$DuplicateFieldException: Duplicate field customData
So its not liking the way my HashMap is encoded.
Therefore I've looked through various solutions to this and have tried the solution proposed here with this converter...
https://stackoverflow.com/a/44152959/1790578
and then register my converter...
theXStream.registerConverter (new XStreamMapConverter ());
The output I get when I encode to JSON is...
{"MyClass": {
"message": "My Message",
"category": "My Category",
"customData": [
"Data1",
"Data2",
"Data3",
"Data4"
],
"devices": [
{
"id": 17,
"firstName": "John",
"lastName": "Smith",
"devMode": false
}
]
}}
This obviously doesn't look right as it's dropping the first element of the HashMap (no Name's). This of course results in an exception when I try to convert it back from JSON to MyClass.
So I tried encoding it the first way above and then decoding it using the Converter. Still the same issue.
Any thoughts or insights here on how I could handle this?
UPDATE 1
Here's my new class with annotations...
public class MyClass
{
public String message;
public String category;
#XStreamImplicit(itemFieldName="name")
public HashMap<String,String> customData = new HashMap<String, String>();
public ArrayList<Device> devices = new ArrayList<Device>();
}
This did not change my resulting JSON at all. I tried it with the NamedMapConverter and that too did not help. But I'm not sure if I have it setup properly?
NamedMapConverter namedMapConverter = new NamedMapConverter(theXStream.getMapper(),"customData","name",String.class,"value",String.class);
theXStream.registerConverter (namedMapConverter);
Ok even though this has already been marked down -2 I still feel it relevant to specify what the solution is here.
The problem is that I was mixing the JSON drivers. I'm a long time xstream user at least 8 years or more. There was a time where two different drivers were required for encoding and decoding the json. JsonHierarchicalStreamDriver and JettisonMappedXmlDriver. I can't remember the specific reason for this way back when but it appears the Jettison driver can be used both ways now. It may very well have always been the case but when it was explained to me I was supposed to use the JsonHierarchicalStreamDriver for encoding to json and the Jettison driver for decoding. Once I changed to the Jettison driver for both everything worked. The HashMap was properly encoded.
So the solution, don't mix the json drivers when creating your xstream. The JsonHierarchicalStreamDriver was the problem here.
Related
I have a json array to work with, like so:
[
{
"id": "12345",
"eauthId": "123451234512345123451234512345",
"firstName": "Jane",
"middieInitial": "M",
"lastName": "Doe",
"email": "janedoe#usda.gov",
"roles": [
{
"id": "CTIS_ROLE_ID",
"name": "A test role for CTIS",
"treatmentName": "Fumigation"
}
]
},
{
"id": "67890",
"eauthId": "678906789067890678906789067890",
"firstName": "John",
"middieInitial": "Q",
"lastName": "Admin",
"email": "johnadmin#usda.gov",
"roles": [
{
"id": "CTIS_ADMIN",
"name": "An admin role for CTIS",
"treatmentName": "System Administration"
}
]
}
]
My task is to find out the user's "roles" --> "name", once match, get that user's email address and sign in using that email address. It seems like a simple task, but it has been really kicking my bottom, since digging into API is new to me. I've tried different libraries (Jackson, RestAssured, Json Simple) and finally GSon. I don't have time to sit and study everything from the scratch. I just needed a quick solution. But it definitely hasn't been quick. Is anyone kind enough to help me out with this. I'd really appreciate it.
closeableHttpResponse = restClient.get(ConfigurationReader.get("base_url") + ConfigurationReader.get("user_endpoint"));
//Status code
int statusCode = closeableHttpResponse.getStatusLine().getStatusCode();
System.out.println("statusCode = " + statusCode);
String responseString = EntityUtils.toString(closeableHttpResponse.getEntity(), "UTF-8");
Type userListType = new TypeToken<List<Users>>(){}.getType();
List<Users> users = (List<Users>) new Gson().fromJson(responseString, userListType);
Roles roles = new Gson().fromJson(responseString, Roles.class);
it gives me this error
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was BEGIN_ARRAY at line 1 column 2 path $
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:226)
at com.google.gson.Gson.fromJson(Gson.java:932)
The problem with your code is
Type userListType = new TypeToken<List>(){}.getType();
List users = (List) new Gson().fromJson(responseString, userListType);
You are not receiving just a list, you are actually deserializing an array of lists.
So try this:
List[] users = (List[]) new Gson().fromJson(responseString, List[].class);
I want to convert jsonobjcts into csv files. Wy (working) attempt so far is to load the json file as a JSONObject (from the googlecode.josn-simple library), then converting them with jsonPath into a string array which is then used to build the csv rows. However I am facing a problem with jsonPath. From the given example json...
{
"issues": [
{
"key": "abc",
"fields": {
"issuetype": {
"name": "Bug",
"id": "1",
"subtask": false
},
"priority": {
"name": "Major",
"id": "3"
},
"created": "2020-5-11",
"status": {
"name": "OPEN"
}
}
},
{
"key": "def",
"fields": {
"issuetype": {
"name": "Info",
"id": "5",
"subtask": false
},
"priority": {
"name": "Minor",
"id": "2"
},
"created": "2020-5-8",
"status": {
"name": "DONE"
}
}
}
]}
I want to select the following:
[
"abc",
"Bug",
"Major",
"2020-5-11",
"OPEN",
"def",
"Info",
"Minor",
"2020-5-8",
"DONE"
]
The csv should look like that:
abc,Bug,Major,2020-5-11,OPEN
def,Info,Minor,2020-5-8,DONE
I tried $.issues.[*].[key,fields] and I get
"abc",
{
"issuetype": {
"name": "Bug",
"id": "1",
"subtask": false
},
"priority": {
"name": "Major",
"id": "3"
},
"created": "2020-5-11",
"status": {
"name": "OPEN"
}
},
"def",
{
"issuetype": {
"name": "Info",
"id": "5",
"subtask": false
},
"priority": {
"name": "Minor",
"id": "2"
},
"created": "2020-5-8",
"status": {
"name": "DONE"
}
}
]
But when I want to select e.g. only "created" $.issues.[*].[key,fields.[created]
[
"2020-5-11",
"2020-5-8"
]
This is the result.
But I just do not get how to select "key" and e.g. "name" in the field issuetype.
How do I do that with jsonPath or is there a better way to filter a jsonfile and then convert it into a csv?
I recommend what I believe is a better way - which is to create a set of Java classes which represent the structure of your JSON data. When you read the JSON into these classes, you can manipulate the data using standard Java.
I also recommend a different JSON parser - in this case Jackson, but there are others. Why? Mainly, familiarity - see later on for more notes on that.
Starting with the end result: Assuming I have a class called Container which contains all the issues listed in the JSON file, I can then populate it with the following:
//import com.fasterxml.jackson.databind.ObjectMapper;
String jsonString = "{...}" // your JSON data as a string, for this demo.
ObjectMapper objectMapper = new ObjectMapper();
Container container = objectMapper.readValue(jsonString, Container.class);
Now I can print out all the issues in the CSV format you want as follows:
container.getIssues().forEach((issue) -> {
printCsvRow(issue);
});
Here, the printCsvRow() method looks like this:
private void printCsvRow(Issue issue) {
String key = issue.getKey();
Fields fields = issue.getFields();
String type = fields.getIssuetype().getName();
String priority = fields.getPriority().getName();
String created = fields.getCreated();
String status = fields.getStatus().getName();
System.out.println(String.join(",", key, type, priority, created, status));
}
In reality, I would use a CSV library to ensure records are formatted correctly - the above is just for illustration, to show how the JSON data can be accessed.
The following is printed:
abc,Bug,Major,2020-5-11,OPEN
def,Info,Minor,2020-5-8,DONE
And to filter only OPEN records, I can do something like this:
container.getIssues()
.stream()
.filter(issue -> issue.getFields().getStatus().getName().equals("OPEN"))
.forEach((issue) -> {
printCsvRow(issue);
});
The following is printed:
abc,Bug,Major,2020-5-11,OPEN
To enable Jackson, I use Maven with the following dependency:
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.10.3</version>
</dependency>
In case you don't use Maven, this gives me 3 JARs: jackson-databind, jackson-annotations, and jackson-core.
To create the nested Java classes I need (to mirror the structure of the JSON), I use a tool which generates them for me using your sample JSON.
In my case, I used this tool, but there are others.
I chose "Container" as the name of the root Java class; a source type of JSON; and selected Jackson 2.x annotations. I also requested getters and setters.
I added the generated classes (Fields, Issue, Issuetype, Priority, Status, and Container) to my project.
WARNING: The completeness of these Java classes is only as good as the sample JSON. But you can, of course, enhance these classes to more accurately reflect the actual JSON you need to handle.
The Jackson ObjectMapper takes care of loading the JSON into the class structure.
I chose to use Jackson instead of JsonPath, simply because of familiarity. JsonPath appears to have very similar object mapping capabilities - but I have never used those features of JsonPath.
Final note: You can use xpath style predicates in JsonPath to access individual data items and groups of items - as you describe in your question. But (in my experience) it is almost always worth the extra effort to create Java classes, if you want to process all your data in more flexible ways - especially if that involves transforming the JSON input into different output structures.
I have a Java Spring MVC Web Application. I am trying to implement a form submission where the user will be submitting the form and an email would be sent to the user with the form data. I will receive the form data as JSON string. But the email sent with this data doesn't look readable. I have tried the following code to make the data more readable, but doesn't seem to make much difference:
Gson gson = new GsonBuilder().setPrettyPrinting().create();
JsonParser jp = new JsonParser();
JsonElement je = jp.parse(jsonData);
String message = gson.toJson(je);
My JSON String looks something like below:
[ { "name": "FirstName", "value": "Abcd" }, { "name": "LastName", "value": "Efgh" }, { "name": "Email", "value": "test.mail#test.com" }, { "name": "PhoneNumber", "value": "9876543210" }, { "name": "Selected Locations", "value": " loc1, loc2, loc3" }, { "name": "Message", "value": "Hi" }, { "name": "g-recaptcha-response", "value": "03AMPJSYULxjYiDRiNxcOSVYxXR9F8dX5pqIYAZ6_F2igAwGvanS4Vh1Lm47ByS2qGELQ9W1h" } ]
I am looking for an option at least to bring name-value pair into separate lines so that it is in some readable manner when received through an email. Is there any way to do this. The form can be generic, SO I may not be able to map this string to any Java object.
I have already tried the following link and the solution does not work for me:
Pretty-Print JSON in Java
I also don't see many helpful details online regarding this. I am trying to put each name-value pair in a new line of the generated string to send the data as email.
I would like to trim the below json object. That is a json object I built on top of what mongoDB responded. What I want to do is to remove just $oid because they are redundant attributes and keep the value inside (_id or $id ) without Curley braces and simply call the attribute id.
so what I need is just "id": "2283cef627ff2cc33ad5990"
Could you please help me I am struggling with json.
{
"_id": {
"$oid": "22383cef627ff2cc33ad5990"
},
"name": "data1",
"users": [
{
"$ref": "user",
"$id": {
"$oid": "16a5fbcee4b0c2c2da3017ef"
}
},
{
"$ref": "user",
"$id": {
"$oid": "1795ff86e4b09fc66416cd2f"
}
},
],
},
a) You can use a mapper to convert your JSON to an object and then call the desired attribute, like Jackson:
ObjectMapper mapper = new ObjectMapper();
String jsonInString = YOUR_STRING;
//from String to MyClass
MyClass object = mapper.readValue(jsonInString, MyClass.class);
In this example you have to define a class MyClass with all the attributes you need (e.g. _id, name, users, etc).
b) If you want to implement a quicker solution you can manipulate directly the string; if you know that the oid is always 24 characters you can do something like
String c = str.substring(str.indexOf("\"", str.indexOf("$oid")+6)+1, str.indexOf("\"", str.indexOf("$oid")+6)+25);
but I highly recommend to take a look to Jackson and give it a try. A solution like this is very fragile and every change in the JSON will lead to a wrong result.
I am using Java API for CRUD operation on elasticsearch.
I have an typewith a nested field and I want to update this field.
Here is my mapping for the type:
"enduser": {
"properties": {
"location": {
"type": "nested",
"properties":{
"point":{"type":"geo_point"}
}
}
}
}
Of course my enduser type will have other parameters.
Now I want to add this document in my nested field:
"location":{
"name": "London",
"point": "44.5, 5.2"
}
I was searching in documentation on how to update nested document but I couldn't find anything. For example I have in a string the previous JSON obect (let's call this string json). I tried the following code but seems to not working:
params.put("location", json);
client.prepareUpdate(index, ElasticSearchConstants.TYPE_END_USER,id).setScript("ctx._source.location = location").setScriptParams(params).execute().actionGet();
I have got a parsing error from elasticsearch. Anyone knows what I am doing wrong ?
You don't need the script, just update it.
UpdateRequestBuilder br = client.prepareUpdate("index", "enduser", "1");
br.setDoc("{\"location\":{ \"name\": \"london\", \"point\": \"44.5,5.2\" }}".getBytes());
br.execute();
I tried to recreate your situation and i solved it by using an other way the .setScript method.
Your updating request now would looks like :
client.prepareUpdate(index, ElasticSearchConstants.TYPE_END_USER,id).setScript("ctx._source.location =" + json).execute().actionGet()
Hope it will help you.
I am not sure which ES version you were using, but the below solution worked perfectly for me on 2.2.0. I had to store information about named entities for news articles. I guess if you wish to have multiple locations in your case, it would also suit you.
This is the nested object I wanted to update:
"entities" : [
{
"disambiguated" : {
"entitySubTypes" : [],
"disambiguatedName" : "NameX"
},
"frequency" : 1,
"entityType" : "Organization",
"quotations" : ["...", "..."],
"name" : "entityX"
},
{
"disambiguated" : {
"entitySubType" : ["a", "b" ],
"disambiguatedName" : "NameQ"
},
"frequency" : 5,
"entityType" : "secondTypeTest",
"quotations" : [ "...", "..."],
"name" : "entityY"
}
],
and this is the code:
UpdateRequest updateRequest = new UpdateRequest();
updateRequest.index(indexName);
updateRequest.type(mappingName);
updateRequest.id(url); // docID is a url
XContentBuilder jb = XContentFactory.jsonBuilder();
jb.startObject(); // article
jb.startArray("entities"); // multiple entities
for ( /*each namedEntity*/) {
jb.startObject() // entity
.field("name", name)
.field("frequency",n)
.field("entityType", entityType)
.startObject("disambiguated") // disambiguation
.field("disambiguatedName", disambiguatedNameStr)
.field("entitySubTypes", entitySubTypeArray) // multi value field
.endObject() // disambiguation
.field("quotations", quotationsArray) // multi value field
.endObject(); // entity
}
jb.endArray(); // array of nested objects
b.endObject(); // article
updateRequest.doc(jb);
Blblblblblblbl's answer couldn't work for me atm, because scripts are not enabled in our server. I didn't try Bask's answer yet - Alcanzar's gave me a hard time, because I supposedly couldn't formulate the json string correctly that setDoc receives. I was constantly getting errors that either I am using objects instead of fields or vice versa. I also tried wrapping the json string with doc{} as indicated here, but I didn't manage to make it work. As you mentioned it is difficult to understand how to formulate a curl statement at ES's java API.
A simple way to update the arraylist and object value using Java API.
UpdateResponse update = client.prepareUpdate("indexname","type",""+id)
.addScriptParam("param1", arrayvalue)
.addScriptParam("param2", objectvalue)
.setScript("ctx._source.field1=param1;ctx._source.field2=param2").execute()
.actionGet();
arrayvalue-[
{
"text": "stackoverflow",
"datetime": "2010-07-27T05:41:52.763Z",
"obj1": {
"id": 1,
"email": "sa#gmail.com",
"name": "bass"
},
"id": 1,
}
object value -
"obj1": {
"id": 1,
"email": "sa#gmail.com",
"name": "bass"
}