We Keep Coding

GSON wont rename fields inside HashMap

I was trying to serialize/deserialize JSON using GSON. The payload in question is ApiGatewayAuthorizerContext. Inside it, there is a HashMap<String, String>. But when doing from/to json, the field naming strategy is not applied to the Keys.
#JsonIgnoreProperties(ignoreUnknown = true)
public class ApiGatewayAuthorizerContext {
//-------------------------------------------------------------
// Variables - Private
//-------------------------------------------------------------
private Map<String, String> contextProperties = new HashMap<>();
private String principalId;
private CognitoAuthorizerClaims claims;
}
Same with MultiValuedTreeMap<String, String> in AwsProxyRequest class too, which is a MultivaluedMap<Key, Value>.
My field naming strategy is simple, replace - with _, for example, the payload below is not a valid JSON for many downstream components I use, and want to replace all '-', with '_'.
"MultiValueHeaders": {
"Accept": [
"application/json, text/plain, */*"
],
"Authorization": [
"Bearer ey...b9w"
],
"Content-Type": [
"application/json;charset=utf-8"
],
"Host": [
"aws-us-east-1-dev-dws-api.xxxxxxxx.com"
],
"User-Agent": [
"axios/0.20.0"
],
"X-Amzn-Trace-Id": [
"Root=1-xxxxxxxx-xxxxxxxxxxxxxxxx"
],
"X-Forwarded-For": [
"127.0.232.171"
],
"X-Forwarded-Port": [
"443"
],
"X-Forwarded-Proto": [
"https"
]
},
Any idea?
EDIT: Adding Field Naming Strategy.
public class ApiEventNamingStrategy implements FieldNamingStrategy {
/**
* Translates the field name into its {#link FieldNamingPolicy.UPPER_CAMEL_CASE} representation.
*
* #param field the field object that we are translating
* #return the translated field name.
*/
public String translateName(Field field) {
String fieldName = FieldNamingPolicy.UPPER_CAMEL_CASE.translateName(field);
if (fieldName.contains("-")) {
fieldName = fieldName.replace('-', '_');
}
return fieldName;
}
}
which is used to setFieldNamingStrategy as shown below,
private static Gson gson =
(new GsonBuilder()).setFieldNamingStrategy(new ApiEventNamingStrategy()).create();
The result is, all the member variables other than the ones inside the Map gets checked, and renamed. Seems setFieldNamingStrategy wont look inside a Map and rename the Keys.
Now I'm looking at the registering a TypeAdapter by utilizing registerTypeAdapterFactory. Seems the the answer by #linfaxin here gson-wont-properly-serialise-a-class-that-extends-hashmap would come to rescue! But the problem is, where/how to and/or the right place to introduce the field naming strategy in the RetainFieldMapFactory class, becasue I see a lot of avenues to hack it in.
Any ideas are most welcome!
btw, the values are populated by AWS APIGateway AND a custom authorization lambda. No way I think I could change the behavior of the APIGateway.

GSON will not get inside map and consider what you want to do. Jackson either.
Considering that you already have your content in a map, I think it is much much easier to just convert the map with 3 lines of code instead of trying to hack how libraries serialize and deserialize objects.
Map<String, String> contextPropertiesNormalized= contextProperties.keySet()
.stream()
.collect(Collectors.toMap(k-> k.contains("-") ? k.replace("-","_"): k, v -> contextProperties::get));

Convert Enum with attributes to map in Java

I have an enum as below:
#AllArgsConstructor
public enum EnumExample {
VAL1("val1 description", 100),
VAL2("val2 description", 200);
String description;
int value;
}
Now I want to return all enum values with attributes as a list of the map as below:
[
{
"name": "VAL1",
"description": "val1 description",
"value": 100
},
{
"name": "VAL2",
"description": "val2 description",
"value": 200
}
]
I am able to achieve this using the below code:
Arrays.stream(EnumExample.values())
.map(enumExample ->
ImmutableMap.of("name", enumExample.name(),
"description", enumExample.description,
"value", enumExample.value))
.collect(Collectors.toList())
But I want to know if there any best way to achieve the same without explicitly converting EnumExample to Map. If any new attribute gets added then it should be coming in the resulting map as a new K, V pair.
I tried the below ways but both return only enum values [VAL1, VAL2].
com.google.common.collect.Lists.newArrayList(EnumExample.values())
Arrays.stream(EnumExample.values()).collect(Collectors.toList())
Tried to convert to map too but returns {"VAL2":"VAL2","VAL1":"VAL1"}.
Arrays.stream(EnumExample.values())
.collect(Collectors.toMap(o -> o, Function.identity()))
Any leads or better ways that doesn't require a manual map creation is appreciated.
My requirement:
In a webservice, return all the Enum values along with attributes to the client. The client has the logic to parse all the attributes coming. Like today there is a description attribute and tomorrow if new attribute like boolean manadatoryField, then it only needs to be handled by client. But from the server end, I am unable to return the Enum values with attributes without manually creating a map out of each enum and returning the map.

Found a simple and another way of doing using Jackson:
Add annotations to the enum.
#Getter
#JsonFormat(shape = JsonFormat.Shape.OBJECT)
Add an explicit getter for name
public String getName() {
return this.name();
}
new ObjectMapper().writeValueAsString(EnumExample.values()) returns a valid JSON which can be converted to Map. In my case I return, this to client!

Answering my own question to help others. If this is the only way, then do upvote.
Arrays.stream(EnumExample.values())
.map(enumExample ->
ImmutableMap.of("name", enumExample.name(),
"description", enumExample.description,
"value", enumExample.value))
.collect(Collectors.toList())
Any best way to achieve the same without explicitly converting EnumExample to Map is greatly appreciated. For example, If any new attribute gets added then it should be coming in the resulting map as a new K, V pair.

How to get the size of an array from a JSON string?

I have a String which looks like this
[
{
"Slot": 1,
"Ping": 40,
"Kills": 0,
"Deaths": 0,
"Score": 0,
"Team": "Reb",
"Name": "ascasdasd",
"KeyHash": "",
"IsBanned": false,
"TotalVisits": 34,
"GroupName": "",
"RemoteAddressStr": "",
"DatabaseId": 1412
},
{
"Slot": 3,
"Ping": 116,
"Kills": 0,
"Deaths": 0,
"Score": 0,
"Team": "Reb",
"Name": "",
"KeyHash": "",
"IsBanned": false,
"TotalVisits": 1,
"GroupName": "",
"RemoteAddressStr": "",
"DatabaseId": 8226
}
]
How can I get the size/length of it? In this case it should be 2
I'm not sure if I should parse it to JSONObject/JSONArray first? I've been trying to parse it using GSON Library with no success.
Update
Turns out it was simpler than I thought. I didn't even needed GSON:
int sizeOfJSONArrayString = new JSONArray(jsonString).length();
Thank you for your help!

Using Gson as you suggested:
Gson gson = new Gson();
MyPojo[] myPojoArray = gson.fromJson(json, MyPojo[].class);
System.out.println(myPojoArray.length); // prints 2
MyPojo is a class that represents your json:
class MyPojo {
#SerializedName("Slot")
#Expose
private Integer slot;
#SerializedName("Ping")
#Expose
private Integer ping;
// other properties
// getters and setters
}
To create a Pojo from your JSON you can check this.

There is no need for any POJO if you need only the size:
int length = new Gson().fromJson(JSON, Object[].class).length;
You could also use List but because it is generic type you would get complaints about raw type and maybe wanted to use type token which makes using array more convenient.

If you just want to know the length of the JSON array, you can parse it with just about any JSON parser, and then look at the result object. In this case the org.json parser would do fine.
Depending on the exact nature of the JSON, it might be possible to implement a hacky solution that (say) counts the number of { characters, or something a bit more sophisticated using regexes. This approach is not recommended. The problem is JSON syntax (in general) is too complicated for that to be practical and reliable. You are liable to be tripped up by edge cases; e.g. a nested object or a { character in a JSON string.

Don't deserialize to "POJO"s or Object[] as it was suggested at least twice:
it's a way to waste the heap (why have objects if you only need to count the number of the top array elements?);
and it's a way to sacrifice performance (deserializers do a more complex job than "simple" parsers do to some extent).
If you're stuck to Gson, it might be implemented in Gson like this:
public static int getTopElementCount(final JsonReader jsonReader)
throws IOException {
jsonReader.beginArray();
int length = 0;
for ( ; jsonReader.hasNext(); length++ ) {
jsonReader.skipValue();
}
jsonReader.endArray();
return length;
}
And it can be used for strings like this (despite intermediate JSON strings are a bad idea in general because of the cost to build such a string):
final int length = JsonUtils.getTopElementCount(new JsonReader(new StringReader(json)));
Or, if there's a file or a network input stream, then:
try ( final JsonReader jsonReader = new JsonReader(new InputStreamReader(new FileInputStream(...), StandardCharsets.UTF_8)) ) {
final int length = JsonUtils.getTopElementCount(jsonReader);
}

Print JSON with ordered properties

I have JSON with objects in specific order:
{
"Aaa": {
"Langs": {
"Val": [
"Test"
],
"Pro": [
"Test2"
]
}
},
"Bbb": {
"Langs": {
"Val": [
"Test"
],
"Pro": [
"Test2"
]
}
},
"Ddd": {
"Langs": {
"Val": [
"Test"
],
"Pro": [
]
}
},
}
And I would like to add new object Ccc between Bbb and Ddd. I tried to configure object mapper like this:
ObjectMapper mapper = new ObjectMapper()
.enable(SerializationFeature.INDENT_OUTPUT)
.configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true)
.configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true);
and then print with this code, but Ccc ends at the end of file.
DefaultPrettyPrinter prettyPrinter = new DefaultPrettyPrinter();
prettyPrinter.indentArraysWith(DefaultIndenter.SYSTEM_LINEFEED_INSTANCE);
//Write whole JSON in FILE
String finalJson = mapper.writer(prettyPrinter).writeValueAsString(rootFlores);
finalJson = finalJson.replaceAll("\\[ ]", "[" + System.lineSeparator() + " ]");
finalJson = finalJson.replaceAll("/", "\\\\/");
Files.write(Paths.get("DictionaryFlores_new.json"), Collections.singleton(finalJson));
Is here a way how to print JSON ordered?

Jackson deserialization/serialization does not sort properties
According to this answer, the Jackson SORT_PROPERTIES_ALPHABETICALLY only applies to POJO properties, not Maps. In JSON there is no difference between a Map and an Object, so you need to set the order in the Map first by using a LinkedHashMap or TreeMap

By definition, the keys of an object are unordered. I guess some libraries could offer an option to control the order of the keys when stringifying, but I wouldn't count on it.
When you need a certain order in json, you need to use an array. Of course, then you'd have to move the keys to a property in the child objects, and then the resulting array could only be indexed by number (not by the key). So then you might have to do additional processing to covert the data structure in the JSON to the data structure you really want to process.

Since you seems ready to use regex to update a JSON, I would suggest a "safer" approach. Don't try to create a pattern that would unsure that you don't update a value somewhere.
Iterate you values, on object at the time. Stringify the object and append the String yourself. That way, you are in charge of the object order. Example :
StringBuilder sb = new StringBuilder("{");
List<JsonPOJO> list = new ArrayList<>();
//populate the list
for(JsonPOJO pojo : list){
sb.append(pojo.stringify()).append(",");
}
sb.setLength(sb.length() - 1); //remove the last commma
sb.append("}");
Here, you are only managing the comma between each JSON object, not create the "complex" part about the JSON. And you are in full control of the order of the value in the String representation, it will only depend on the way you populate the List.
Note: sorry for the "draft" code, I don't really have access to my system here so just write this snippet to give you a basic idea on how to "manage" a JSON without having to recreating an API completely.
Note2: I would note suggest this unless this looks really necessary. As you mention in a comment, you are have only the problem with one key where you already have a JSON with 80000 keys, so I guess this is a "bad luck" scenario asking for last resort solution ;)

Associativity array in Java

I receive a List<org.apache.avro.generic.GenericRecord> with the data contents as shown below (JSON notation used for clarity). How can I best hold these record types using Java?
Record 1:
[
{
"serial_no" : "x",
"data1" : "d"
},
{
"serial_no" : "y",
"data2" : "d2"
},
............................MANY MORE
]
Record 2:
[
{
"id":"x",
"type":"A"
},
{
"id" : "x",
"type" : "B"
},
{
"id" : "y",
"type" : "A",
},
{
"id" : "y",
"type" : "B"
}
]
As you see here, each serial number has two records in record2. serial_no in record1 is same as id in record2.
My Goal is:
Fatsest way to find these two records.
Solution I think:
Create a map like
map.put("x", [map.put("A",List), map.put("B",List)]);
But I feel like, its a complex structure. Because map contains list of maps[each map is Map<String,List<Map<String,String>>>].
Any suggestions?
EDIT
Each entries in records are avro GenericRecord

It looks as if you are trying to parse JSON using Java. Why not use a specific library for that?
Like the basic http://www.json.org/java/ or Google's https://github.com/google/gson
Otherwise, I do not think that the complex structure you are proposing is especially slow. You might want to design your own object class to hold the data if you think it is more efficient or easier to get to the data.
EDIT
Based on your question I assumed JSON was the format you received it in, sorry.
I would just create a wrapper for GenericRecord, or subclass it. Then add the methods that you need to extract the data, or make it Comparable for sorting.
Something along the lines of
public class MyRecord extends GenericRecord implements Comparable<MyRecord>
{
// Determine the type
public int getType()
{
if ( this.get( "id") != null )
return 2;
return 1;
}
// Add methods that allow you to retrieve the serial field from any of the two record types
public String getId()
{
if ( this.get( "id") != null )
return (String)this.get("id");
return (String)this.get("serial_no");
}
// add comparator methods that will allow you to sort the list, compare based on Id, etc
#Override
public int compareTo(MyRecord another)
{
// Just a simple example
return this.getId().compareTo( another.getId() );
}
}

Define classes for repeated entries:
class SerialNoData {
String serialNo;
Object data;
}
and
class IdType {
String id;
String type;
}
; once parsed put the instances into arrays or Lists to get the desired format.

How complex the map is doesn't really make a difference for the speed. Depending on the type of Map you use getting a list of records will be constant time (with a reasonably small overhead). Finding something in the sublists will then be O(n), since you need to iterate through the list and look at all the Maps.

Define following classes
class Serial{
String serial-no;
String data;
List<IdType> idTypes;
}
class IdType{
String id;
String type;
}
After that you can use jackson or any kind of JSON processing library.