Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
i have a json
{
"content": [
{
"idnumber": "666",
"name": "mark",
"type": "band",
"tools": [
{
"idtools": "5657",
"blabla": null,
"blabla": false,
}
]
},
{
"idnumber": "666",
"name": "mark",
"type": "band",
"tools": [
{
"idtools": "5658",
"blabla": null,
"blabla": false
}
]
}
]
}
inside content array, i have 2 json. i want to change my json into this, because they have same id number.
{
"content": [
{
"idnumber": "666",
"name": "mark",
"type": "band",
"tools": [
{
"idtools": "5657",
"blabla": null,
"blabla": false,
},
{
"idtools": "5658",
"blabla": null,
"blabla": false
}
]
}
]
}
how to do that using distinct or filter?
i tried to distinct it and map it but still have error.
Assuming the following objects that match your JSON structure (for sake of brevity, I use Lombok):
#Data
#AllArgsConstructor
#NoArgsConstructor
class Content {
int idNumber;
String name;
String type;
List<Tool> tools;
}
#Data
#AllArgsConstructor
#NoArgsConstructor
class Tool {
int idTools;
String blabla;
}
You can use the Stream API with groupingBy by the id and reduce the values into a single one.
List<Content> mergedContents = contents.stream()
.collect(Collectors.groupingBy(Content::getIdNumber))
.values()
.stream()
.reduce(
new ArrayList<>(), // mutable List
(left, right) -> {
Content content = right.get(0); // they are same (by id)
List<Tool> tools = right.stream() // from each new list
.flatMap(c -> c.getTools().stream()) // .. flatmap the tools
.collect(Collectors.toList()); // .. and extract to a list
content.setTools(tools); // set the List<Tool>
left.add(content); // add the Content
return left; // return the left list
},
(left, right) -> left); // only for parallel Stream
The resulting structure comming from Collectors.groupingBy(Content::getIdNumber) is Map<Integer, List<Content>>. The subsequent mutable reduction on the map values (Collection<List<Content>>) merges each List<Content> with identical Content.id into a single Content with flatmapped List<Tools>. The List with a these modified Content is returned as a result of the reduction.
Sample data
List<Content> contents = new ArrayList<>();
contents.add(new Content(666, "Mark", "Band",
Collections.singletonList(new Tool(5657, null))));
contents.add(new Content(666, "Mark", "Band",
Collections.singletonList(new Tool(5658, null))));
List<Content> mergedContents = /* solution */
mergedContents.forEach(System.out::println);
Main.Content(idNumber=666, name=Mark, type=Band, tools=[Main.Tool(idTools=5657, blabla=null), Main.Tool(idTools=5658, blabla=null)])
This is equal to what your JSON samples.
Related
I have issue when I use GroupBy an ArrayList object in Kotlin.
I have JSON like here:
{
"securityAccountList": [
{
"accountNumber": "001548900001",
"accountName": "Tjandraningrum",
"currency": "IDR",
"debtInstrument": "0",
"equityInstrument": "0"
},
{
"accountNumber": "001548900001",
"accountName": "Tjandraningrum",
"currency": "IDR",
"debtInstrument": "0",
"equityInstrument": "0"
},
{
"accountNumber": "001548900001",
"accountName": "Tjandraningrum",
"currency": "USD",
"debtInstrument": "22",
"equityInstrument": "66"
}
]
}
I generate object by GSON like this:
data class ContentModel(
#SerializedName("securityAccountList") var securityAccountList: ArrayList<SecurityAccountList> = arrayListOf()
)
data class SecurityAccountList(
#SerializedName("accountNumber") var accountNumber: String? = null,
#SerializedName("accountName") var accountName: String? = null,
#SerializedName("currency") var currency: String? = null,
#SerializedName("debtInstrument") var debtInstrument: String? = null,
#SerializedName("equityInstrument") var equityInstrument: String? = null
)
now I want to grouping by field currency and I want result like this
{
"securityAccountList": [
{
"accountNumber": "001548900001",
"accountName": "Tjandraningrum",
"currency": "IDR",
"debtInstrument": "0",
"equityInstrument": "0"
},
{
"accountNumber": "001548900001",
"accountName": "Tjandraningrum",
"currency": "USD",
"debtInstrument": "22",
"equityInstrument": "66"
}
]
}
but the result not changed, still 3 securityAccountList,
this my code for grouping:
val newSecurityAccountList= contentModel.securityAccountList
newSecurityAccountList.groupingBy { it.currency }
newSecurityAccountList.sortBy { it.currency }
Note: contentModel.securityAccountList is ArrayList of object JSON the newSecurityAccountList not changed value.
please your advice, sorry for my English.
private fun List<SecurityAccountList>.customFilter(): List<SecurityAccountList> =
groupBy { it.currency }.map {
// HERE you can do whaterver you need.
it.value
.filter { item-> item.currency >= 0 }
.maxByOrNull { item-> item.accountName.length }
}.filterNotNull()
Using in code
val returnlist = yourListBeforeFilter.customFilter()
For more details tell us what exactly you need.
You can as well use here sortBy
It sounds like you are expecting groupingBy to modify the list. In fact it won't, but will instead return a new Grouping object which can be used to gather the results into a new, different list.
It can be confusing, because sortBy does mutate the list, whereas the similar-looking groupBy and groupingBy functions return a new object or copy and don't mutate the original.
The best way to be sure whether a function is going to mutate the existing object or create a new object is to check its documentation. In Kotlin, most of the standard library functions are non-mutating, because Kotlin encourages immutability.
You could try replacing your code with something like:
val groupedAccounts = contentModel.securityAccountList.groupBy { it.currency }
Note how the variable stores the result of the call to groupBy, not the original securityAccountList.
I have this json, I need to get a list of currency objects using Jackson (the objects itself has a currency name field - "USD", "type" and "show") as a result. How can I do it in a simple and clear way?
Any help would be welcome
{
...
"result": "OK",
"currency": {
"currencies": {
"LB": {
"type": "A",
"setting": {
"show": true,
},
"access" : true
},
"USD": {
"type": "B",
"setting": {
"show": true,
},
"access" : true
},
"RUB": {
"type": "A",
"setting": {
"show": true,
},
"access" : true
},
....
// and more..
},
...
}
}
"currencies": {
"LB": {
"type": "A",
"setting": {
"show": true,
},
"access" : true
},
"USD": {
"type": "B",
"setting": {
"show": true,
},
"access" : true
},
"RUB": {
"type": "A",
"setting": {
"show": true,
},
"access" : true
},
....
// and more..
},
...
This looks like an Map, not a List. Does that help you? So you would need to create Currency class and have jackson parse this structure to Map<String, Currency>. Note that currency will not contain the identifier (like "USD"), only they key in the map will.
You can read your Json as a Map<String, Object>. Once you have the map you iterate through your map and build your List.
If you want to de-serialize (read) from Json to List of your custom objects you need to have your Json restructured to be Json List and not Json Object. Then you can read your Json into a List directly. In any case, there is an Open source Library that has a feature based on Jackson library really simplifies reading/writing objects from/to Json. If you have a json string all you have to do is:
Map<String, Object> map = JsonUtils.readObjectFromJsonString(jsonString, Map.class);
It throws IOException though, so you will need to handle it. Here is the Javadoc. The library is called MgntUtils and could be found as Maven artifact and on the Github (including javadoc and source code)
I was able to add object saving to the list, but in a rather inconvenient way, it seems to me...
ArrayList<Dto> currencyObjectList = new ArrayList<Dto>();
CurrencyStaticDto obj = null;
JsonNode currencies = mapper.readTree(response.getBody()).get("currencies");
Iterator<Map.Entry<String, JsonNode>> fields = currencies.fields();
while(fields.hasNext()) {
Map.Entry<String, JsonNode> field = fields.next();
String fieldName = field.getKey();
JsonNode fieldValue = field.getValue();
obj = mapper.readValue(fieldValue.toString(),CurrencyStaticDto.class);
obj.setCurrency(fieldName);
currencyObjectList.add(obj);
}
I have a JSON like this below:
My aimed POJO is
[{
"id": "1",
"teams": [{
"name": "barca"
},
{
"name": "real"
}
]
},
{
"id": "2"
},
{
"id": "3",
"teams": [{
"name": "atletico"
},
{
"name": "cz"
}
]
}
]
My aimed POJO is
class Team
int id;
String name;
Meaning, for each "team" I want to create a new object. Like;
new Team(1,barca)
new Team(1,real)
new Team(2,null)
new Team(3,atletico)
...
Which I believe I did with custom deserializer like below:
JsonNode rootArray = jsonParser.readValueAsTree();
for (JsonNode root : rootArray) {
String id = root.get("id").toString();
JsonNode teamsNodeArray = root.get("teams");
if (teamsNodeArray != null) {
for (JsonNode teamNode: teamsNodeArray ) {
String nameString = teamNode.get("name").toString();
teamList.add(new Team(id, nameString));
}
} else {
teamList.add(new Team(id, null));
}
}
Condidering I am getting 750k records... having 2 fors is I believe making the code way slower than it should be. It takes ~7min.
My question is, could you please enlighten me if there is any better way to do this?
PS: I have checked many stackoverflow threads for this, could not find anything that fits so far.
Thank you in advance.
Do not parse the data yourself, use automatic de/serialization whenever possible.
Using jackson it could be as simple as:
MyData myData = new ObjectMapper().readValue(rawData, MyData.class);
For you specific example, we generate a really big instance (10M rows):
$ head big.json
[{"id": 1593, "group": "6141", "teams": [{"id": 10502, "name": "10680"}, {"id": 16435, "name": "18351"}]}
,{"id": 28478, "group": "3142", "teams": [{"id": 30951, "name": "3839"}, {"id": 25310, "name": "19839"}]}
,{"id": 29810, "group": "8889", "teams": [{"id": 5586, "name": "8825"}, {"id": 27202, "name": "7335"}]}
...
$ wc -l big.json
10000000 big.json
Then, define classes matching your data model (e.g.):
public static class Team {
public int id;
public String name;
}
public static class Group {
public int id;
public String group;
public List<Team> teams;
}
Now you can read directly the data by simply:
List<Group> xs = new ObjectMapper()
.readValue(
new File(".../big.json"),
new TypeReference<List<Group>>() {});
A complete code could be:
public static void main(String... args) throws IOException {
long t0 = System.currentTimeMillis();
List<Group> xs = new ObjectMapper().readValue(new File("/home/josejuan/tmp/1/big.json"), new TypeReference<List<Group>>() {});
long t1 = System.currentTimeMillis();
// test: add all group id
long groupIds = xs.stream().mapToLong(x -> x.id).sum();
long t2 = System.currentTimeMillis();
System.out.printf("Group id sum := %d, Read time := %d mS, Sum time = %d mS%n", groupIds, t1 - t0, t2 - t1);
}
With output:
Group id sum := 163827035542, Read time := 10710 mS, Sum time = 74 mS
Only 11 seconds to parse 10M rows.
To check data and compare performance, we can read directly from disk:
$ perl -n -e 'print "$1\n" if /"id": ([0-9]+), "group/' big.json | time awk '{s+=$1}END{print s}'
163827035542
4.96user
Using 5 seconds (the Java code is only half as slow).
The non-performance problem of processing the data can be solved in many ways depending on how you want to use the information. For example, grouping all the teams can be done:
List<Team> teams = xs.stream()
.flatMap(x -> x.teams.stream())
.collect(toList());
Map<Integer, Team> uniqTeams = xs.stream()
.flatMap(x -> x.teams.stream())
.collect(toMap(
x -> x.id,
x -> x,
(a, b) -> a));
I want to move the java JSON element(JSON array) into Java set. How do I do that in a simpler manner?
JsonElement newdcdc = npsdfs.seget();
sample JSON element
[
{
"expand": "desc",
"key":"qwe1"
},
{
"expand": "descrip",
"key":"qwe2"
},
{
"expand": "description",
"key":"qwe3"
}
]
i need to create some set with all keys so how i do that
I have a table called Group and it will have records like:
{
"id": "UniqueID1",
"name": "Ranjeeth",
"emailIdMappings": [
{
"emailId": "r.pt#r.com",
"userId": 324
},
{
"emailId": "r1.pt#r.com",
"userId": 325
}
]
},
{
"id": "UniqueID2",
"name": "Ranjeeth",
"emailIdMappings": [
{
"emailId": "r1.pt#r.com",
"userId": 325
},
{
"emailId": "r2.pt#r.com",
"userId": 326
}
]
}
I need to query and get result if emailId contains the input string.
I have reached so far and I am not able to get the result
AttributeValue attributeValue = new AttributeValue("r.pt#r.com");
Condition containsCondition = new Condition()
.withComparisonOperator(ComparisonOperator.CONTAINS)
.withAttributeValueList(attributeValue);
Map<String, Condition> conditions = newHashMap();
conditions.put("emailIdMappings.emailId", containsCondition);
ScanRequest scanRequest = new ScanRequest()
.withTableName("Group")
.withScanFilter(conditions);
amazonDynamoDB.scan(scanRequest)
dynamoDBMapper.marshallIntoObjects(Group.class, scanResult.getItems());
For the above code I am expecting record with id UniqueID1, but it's empty. If you pass "r1.pt#r.com" then you should get both records.
sdk used is com.amazonaws:aws-java-sdk-dynamodb:1.11.155
I tried posting the question in aws forum which didn't help much.
As you have List of Objects which has two attributes in a object (i.e. emailId and userId), you need to provide both values in order to match the item.
The CONTAINS function will not be able to match the item if the object has two attributes and only one attribute value mentioned in the filter expression.
Otherwise, you need to provide the occurrence (i.e. index) of the list to match the item.
Example:-
emailIdMappings[0].emailId = :emailIdVal