I receive a List<org.apache.avro.generic.GenericRecord> with the data contents as shown below (JSON notation used for clarity). How can I best hold these record types using Java?
Record 1:
[
{
"serial_no" : "x",
"data1" : "d"
},
{
"serial_no" : "y",
"data2" : "d2"
},
............................MANY MORE
]
Record 2:
[
{
"id":"x",
"type":"A"
},
{
"id" : "x",
"type" : "B"
},
{
"id" : "y",
"type" : "A",
},
{
"id" : "y",
"type" : "B"
}
]
As you see here, each serial number has two records in record2. serial_no in record1 is same as id in record2.
My Goal is:
Fatsest way to find these two records.
Solution I think:
Create a map like
map.put("x", [map.put("A",List), map.put("B",List)]);
But I feel like, its a complex structure. Because map contains list of maps[each map is Map<String,List<Map<String,String>>>].
Any suggestions?
EDIT
Each entries in records are avro GenericRecord
It looks as if you are trying to parse JSON using Java. Why not use a specific library for that?
Like the basic http://www.json.org/java/ or Google's https://github.com/google/gson
Otherwise, I do not think that the complex structure you are proposing is especially slow. You might want to design your own object class to hold the data if you think it is more efficient or easier to get to the data.
EDIT
Based on your question I assumed JSON was the format you received it in, sorry.
I would just create a wrapper for GenericRecord, or subclass it. Then add the methods that you need to extract the data, or make it Comparable for sorting.
Something along the lines of
public class MyRecord extends GenericRecord implements Comparable<MyRecord>
{
// Determine the type
public int getType()
{
if ( this.get( "id") != null )
return 2;
return 1;
}
// Add methods that allow you to retrieve the serial field from any of the two record types
public String getId()
{
if ( this.get( "id") != null )
return (String)this.get("id");
return (String)this.get("serial_no");
}
// add comparator methods that will allow you to sort the list, compare based on Id, etc
#Override
public int compareTo(MyRecord another)
{
// Just a simple example
return this.getId().compareTo( another.getId() );
}
}
Define classes for repeated entries:
class SerialNoData {
String serialNo;
Object data;
}
and
class IdType {
String id;
String type;
}
; once parsed put the instances into arrays or Lists to get the desired format.
How complex the map is doesn't really make a difference for the speed. Depending on the type of Map you use getting a list of records will be constant time (with a reasonably small overhead). Finding something in the sublists will then be O(n), since you need to iterate through the list and look at all the Maps.
Define following classes
class Serial{
String serial-no;
String data;
List<IdType> idTypes;
}
class IdType{
String id;
String type;
}
After that you can use jackson or any kind of JSON processing library.
Related
{
"ssn1": {
"name": "person1",
"address": "address1"
"drivingLicense": "dl1"
},
"ssn2": {
"name": "person2",
"address": "address2"
"drivingLicense": "dl2"
}
}
I have a json configuration data as mentioned above.
class Citizen {
Map<String, Person> personMap;
}
class Person {
String name;
String address;
String drivingLicense;
}
The pojos is like the one mentioned above. I could easily deserialize the json configuration into a java Map<String, Person> and search using SSN.
Question: How can I deserialize to get two maps - one keyed on SSN and other with driving license? I could use the SSN->person map to create another map DrivingLicense->Person. Can I create the DrivingLicense map in an elegant way to that I don't have to write
ssnMap.entrySet().stream().collect(Collectors.toMap(
e -> e.getValue().getDrivingLicense(),
e -> e.getValue()));
again and again for other primary keys like passport number?
Assumptions
Driving license is unique for each person
Query by driving license does not require SSN as output
There could be other uniquely identifying keys in the Person class
You can't do that with any kind of content-agnostic deserialization, which operates only on structure. You can do something trivial in postprocessing like
var byDl = bySsn.values().stream().collect(toMap(Person::drivingLicense, identity()));
I have an enum as below:
#AllArgsConstructor
public enum EnumExample {
VAL1("val1 description", 100),
VAL2("val2 description", 200);
String description;
int value;
}
Now I want to return all enum values with attributes as a list of the map as below:
[
{
"name": "VAL1",
"description": "val1 description",
"value": 100
},
{
"name": "VAL2",
"description": "val2 description",
"value": 200
}
]
I am able to achieve this using the below code:
Arrays.stream(EnumExample.values())
.map(enumExample ->
ImmutableMap.of("name", enumExample.name(),
"description", enumExample.description,
"value", enumExample.value))
.collect(Collectors.toList())
But I want to know if there any best way to achieve the same without explicitly converting EnumExample to Map. If any new attribute gets added then it should be coming in the resulting map as a new K, V pair.
I tried the below ways but both return only enum values [VAL1, VAL2].
com.google.common.collect.Lists.newArrayList(EnumExample.values())
Arrays.stream(EnumExample.values()).collect(Collectors.toList())
Tried to convert to map too but returns {"VAL2":"VAL2","VAL1":"VAL1"}.
Arrays.stream(EnumExample.values())
.collect(Collectors.toMap(o -> o, Function.identity()))
Any leads or better ways that doesn't require a manual map creation is appreciated.
My requirement:
In a webservice, return all the Enum values along with attributes to the client. The client has the logic to parse all the attributes coming. Like today there is a description attribute and tomorrow if new attribute like boolean manadatoryField, then it only needs to be handled by client. But from the server end, I am unable to return the Enum values with attributes without manually creating a map out of each enum and returning the map.
Found a simple and another way of doing using Jackson:
Add annotations to the enum.
#Getter
#JsonFormat(shape = JsonFormat.Shape.OBJECT)
Add an explicit getter for name
public String getName() {
return this.name();
}
new ObjectMapper().writeValueAsString(EnumExample.values()) returns a valid JSON which can be converted to Map. In my case I return, this to client!
Answering my own question to help others. If this is the only way, then do upvote.
Arrays.stream(EnumExample.values())
.map(enumExample ->
ImmutableMap.of("name", enumExample.name(),
"description", enumExample.description,
"value", enumExample.value))
.collect(Collectors.toList())
Any best way to achieve the same without explicitly converting EnumExample to Map is greatly appreciated. For example, If any new attribute gets added then it should be coming in the resulting map as a new K, V pair.
I have JSON with objects in specific order:
{
"Aaa": {
"Langs": {
"Val": [
"Test"
],
"Pro": [
"Test2"
]
}
},
"Bbb": {
"Langs": {
"Val": [
"Test"
],
"Pro": [
"Test2"
]
}
},
"Ddd": {
"Langs": {
"Val": [
"Test"
],
"Pro": [
]
}
},
}
And I would like to add new object Ccc between Bbb and Ddd. I tried to configure object mapper like this:
ObjectMapper mapper = new ObjectMapper()
.enable(SerializationFeature.INDENT_OUTPUT)
.configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
.configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true);
and then print with this code, but Ccc ends at the end of file.
DefaultPrettyPrinter prettyPrinter = new DefaultPrettyPrinter();
prettyPrinter.indentArraysWith(DefaultIndenter.SYSTEM_LINEFEED_INSTANCE);
//Write whole JSON in FILE
String finalJson = mapper.writer(prettyPrinter).writeValueAsString(rootFlores);
finalJson = finalJson.replaceAll("\\[ ]", "[" + System.lineSeparator() + " ]");
finalJson = finalJson.replaceAll("/", "\\\\/");
Files.write(Paths.get("DictionaryFlores_new.json"), Collections.singleton(finalJson));
Is here a way how to print JSON ordered?
Jackson deserialization/serialization does not sort properties
According to this answer, the Jackson SORT_PROPERTIES_ALPHABETICALLY only applies to POJO properties, not Maps. In JSON there is no difference between a Map and an Object, so you need to set the order in the Map first by using a LinkedHashMap or TreeMap
By definition, the keys of an object are unordered. I guess some libraries could offer an option to control the order of the keys when stringifying, but I wouldn't count on it.
When you need a certain order in json, you need to use an array. Of course, then you'd have to move the keys to a property in the child objects, and then the resulting array could only be indexed by number (not by the key). So then you might have to do additional processing to covert the data structure in the JSON to the data structure you really want to process.
Since you seems ready to use regex to update a JSON, I would suggest a "safer" approach. Don't try to create a pattern that would unsure that you don't update a value somewhere.
Iterate you values, on object at the time. Stringify the object and append the String yourself. That way, you are in charge of the object order. Example :
StringBuilder sb = new StringBuilder("{");
List<JsonPOJO> list = new ArrayList<>();
//populate the list
for(JsonPOJO pojo : list){
sb.append(pojo.stringify()).append(",");
}
sb.setLength(sb.length() - 1); //remove the last commma
sb.append("}");
Here, you are only managing the comma between each JSON object, not create the "complex" part about the JSON. And you are in full control of the order of the value in the String representation, it will only depend on the way you populate the List.
Note: sorry for the "draft" code, I don't really have access to my system here so just write this snippet to give you a basic idea on how to "manage" a JSON without having to recreating an API completely.
Note2: I would note suggest this unless this looks really necessary. As you mention in a comment, you are have only the problem with one key where you already have a JSON with 80000 keys, so I guess this is a "bad luck" scenario asking for last resort solution ;)
I am getting response like this from server,
[
{
"id":"b2",
"type":"ball"
},
{
"id":"a1",
"type":"apple",
},
{
"id":"d4",
"type":"dog",
},
{
"id":"c3",
"type":"cat",
}
]
but I need to sort the above response data based on alphabets wise, based on "type". And then display the list, I am using model to parse the data .
What data structure are you parsing the JSON into? If, for example, you have an Item class similar to:
class Item {
Item(String id, String type) { ... }
String getType() { ... }
String getId() { ... }
}
And you parse that JSON into some kind of List<Item>, then the following should work for sorting by type (API level 24+), see Comparator.comparing:
List<Item> items = ...; // however you parse the JSON
Comparator<Item> byType = Comparator.comparing(Item::getType);
Collection.sort(items, byType); // items will now be sorted
For a more general approach that will work with older API levels, you can define the byType comparator like this:
Comparator<Item> byType = new Comparator<Item> {
public int compare(Item a, Item b) {
return a.getType().compareTo(b.getType());
}
};
Can't you make your class Comparable and use :
type.compareTo()
In your own compareTo method in oder to be able to sort them by type?
Json is an object is an unordered set of name/value pairs.
See http://json.org.
Convert to Object to sort.
Convert it to an Object and implement Comparable<> Interface. The use:
CompareTo()
I'm a beginner Java and Gson user and have been able to apply it to my needs. I now have some JSON data that I need to parse into a spinner as follows:
{
"lang":[
"arabic",
"bengali",
"dutch-utf8",
"eng_root",
"english",
"english-utf8",
...
],
"themes":{
"blue":{
"chinese_ibm500":1,
"spanish":1,
"bengali":1,
"japanese":1,
"english":1,
"russian":1,
"french-utf8":1,
"eng_root":1,
"arabic":1,
"spanish-utf8":1,
"portuguese":1,
...
},
"green":{
"eng_root":1,
"engmonsoon":1,
"english":1
...
},
"red":{
"chinese_ibm500":1,
"spanish":1,
"bengali":1,
...
}
}
}
So from this JSON I need 2 things:
1) the array under lang is dynamic as for its the languages installed on the server. How could I get all the entries?
I have a class as follows but im stuck as to what I should do after I return lang
public class ListData {
private List<Language> lang;
public List<Language> getLang {
return lang;
}
public static class Language {
???
}
}
2) after understanding 1 I might be able to figure this one out. Under themes are colors which again can be more or less {purple, orange, whatever}. I just need a list of those themes, as far as I'm concerned I don't need to know the languages for each.
Feel like this question is turning into a book. I have searched SO extensively and hate asking questions but I'm pretty stumped. Thanks in advance.
1) In order to get the "lang" array, just modify
private List<Language> lang;
for
private List<String> lang;
Since the elements inside "lang" array are all strings, you don't need any class Language to store those values, they'll be parsed correctly as strings. And it doesn't matter how many strings the array contains...
2) In order to parse "themes", you have to notice that it's not an array [ ], but an object { }, so you do need to parse it with some object, and the most suitable class here is a Map like this:
private Map<String, Object> themes;
Note: as you said that you don't need the data under "blue", "green", etc... you can just Object as the value type in the map, otherwise you'd need some class...
Using a Map here allows you to have an arbitrary number of themes in your JSON response.
So in summary, you just need a class like:
public class ListData {
private List<String> lang;
private Map<String, Object> themes;
//getters & setters
}
and parse your JSON with:
Gson gson = new Gson();
ListData data = gson.fromJson(yourJsonString, ListData.class);
Your list of langs will be under:
data.getLang();
and your list of themes will be under:
data.getThemes().keySet();
I suggest you to take a look at Gson documentation. It's quite short and clear and you'll understand everything much better...