We Keep Coding

How to get the count of element with non-empty-array-field when group in mongodb aggregate using Spring Data Mongo?

I have the following documents in one collection named as mail_test. Some of them have a tags field which is an array:
/* 1 */
{
"_id" : ObjectId("601a7c3a57c6eb4c1efb84ff"),
"email" : "aaaa#bbb.com",
"content" : "11111"
}
/* 2 */
{
"_id" : ObjectId("601a7c5057c6eb4c1efb8590"),
"email" : "aaaa#bbb.com",
"content" : "22222"
}
/* 3 */
{
"_id" : ObjectId("601a7c6d57c6eb4c1efb8675"),
"email" : "aaaa#bbb.com",
"content" : "33333",
"tags" : [
"x"
]
}
/* 4 */
{
"_id" : ObjectId("601a7c8157c6eb4c1efb86f4"),
"email" : "aaaa#bbb.com",
"content" : "4444",
"tags" : [
"yyy",
"zzz"
]
}
There are two documents with non-empty-tags, so I want the result to be 2.
I use the the following statement to aggregate and get the correct tag_count:
db.getCollection('mail_test').aggregate([{$group:{
"_id":null,
"all_count":{$sum:1},
"tag_count":{"$sum":{$cond: [ { $ne: ["$tags", undefined] }, 1, 0]}}
//if replace `undefined` with `null`, I got the tag_count as 4, that is not what I want
//I also have tried `$exists`, but it cannot be used here.
}}])
and the result is:
{
"_id" : null,
"all_count" : 4.0,
"tag_count" : 2.0
}
and I use spring data mongo in java to do this:
private void test(){
Aggregation agg = Aggregation.newAggregation(
Aggregation.match(new Criteria()),//some condition here
Aggregation.group(Fields.fields()).sum(ConditionalOperators.when(Criteria.where("tags").ne(null)).then(1).otherwise(0)).as("tag_count")
//I need an `undefined` instead of `null`,or is there are any other solution?
);
AggregationResults<MailTestGroupResult> results = mongoTemplate.aggregate(agg, MailTest.class, MailTestGroupResult.class);
List<MailTestGroupResult> mappedResults = results.getMappedResults();
int tag_count = mappedResults.get(0).getTag_count();
System.out.println(tag_count);//get 4,wrong
}
I need an undefined instead of null but I don't know how to do this,or is there are any other solution?

You can use Aggregation operators to check if the field tags exists or not with one of the following constructs in the $group stage of your query (to calculate the tag_count value):
"tag_count":{ "$sum": { $cond: [ { $gt: [ { $size: { $ifNull: ["$tags", [] ] }}, 0 ] }, 1, 0] }}
// - OR -
"tag_count":{ "$sum": { $cond: [ $eq: [ { $type: "$tags" }, "array" ] }, 1, 0] }
Both, return the same result (as you had posted).

How to extract item from nested list - Rest Assured

I've got this json object
{
"type" : "employee",
"columns" :
[
{
"id" : 1,
"human" :
[
{
"name" : "ANA",
"age" : "23"
},
{
"name" : "IULIA",
"age" : "22"
}
]
},
{
"id" : 2,
"human" :
[
{
"name" : "ADI",
"age" : "21"
},
{
"name" : "GELU",
"age" : "18"
}
]
}
]
}
and I need to extract the first name from each human list.
I've tried .body("columns.human.name[0]", everyItem(containsString("A"))) but it doesn't work. Any ideas?

Using JsonPath you can get all columns and all humans.
Each of JSON Object is represented as HashMap<>. If it contains only fields it's HashMap<String, String> but if contains arrays or nested JSON Objects then it is HashMap<String, Object> where Object is either another JSON Object or array.
Given the above you can use following code to get all columns and name of first human in each column:
JsonPath path = response.jsonPath();
List<HashMap<String, Object>> columns = path.getList("columns");
for (HashMap<String, Object> singleColumn : columns) {
List<HashMap<String, Object>> humans = (List<HashMap<String, Object>>) singleColumn.get("human");
System.out.println(humans.get(0).get("name"));
}
The above code will print ANA and ADI in the console.
You can store the results in List<String> for further processing

you can get all "name" from humans with jsonPath : $.columns[*].human[*].name it will give below result :
[
"ANA",
"IULIA",
"ADI",
"GELU"
]
And if you want only first "name" then you need to use josn : $.columns[*].human[0].name this will give you below result:
[
"ANA",
"ADI"
]

Get count of unique ObjectId from array MongoDB

I'm new to working with MongoDb and do not know a lot of things.
I need to write an aggregation request.
Here is the JSON document structure.
{
"_id" : ObjectId("5a72f7a75ef7d430e8c462d2"),
"crawler_id" : ObjectId("5a71cbb746e0fb0007adc6c2"),
"skill" : "stack",
"created_date" : ISODate("2018-02-01T13:19:03.522+0000"),
"modified_date" : ISODate("2018-02-01T13:22:23.078+0000"),
"connects" : [
{
"subskill" : "we’re",
"weight" : NumberInt(1),
"parser_id" : [
ObjectId("5a71d88d5ef7d41964fbec11")
]
},
{
"subskill" : "b1",
"weight" : NumberInt(2),
"parser_id" : [
ObjectId("5a71d88d5ef7d41964fbec11"),
ObjectId("5a71d88d5ef7d41964fbec1b")
]
},
{
"subskill" : "making",
"weight" : NumberInt(2),
"parser_id" : [
ObjectId("5a71d88d5ef7d41964fbec1b"),
ObjectId("5a71d88d5ef7d41964fbec1c")
]
},
{
"subskill" : "delivery",
"weight" : NumberInt(2),
"parser_id" : [
ObjectId("5a71d88d5ef7d41964fbec1c"),
ObjectId("5a71d88d5ef7d41964fbec1e")
]
}
]
}
I need the result return the name of skill and the number of unique parser_id.
In this case, the result should be:
[
{
"skill": "stack",
"quantity": 4
}
]
where "stack" - skill name,
and "quantity" - count of unique parser_id.
ObjectId("5a71d88d5ef7d41964fbec11")
ObjectId("5a71d88d5ef7d41964fbec1b")
ObjectId("5a71d88d5ef7d41964fbec1c")
ObjectId("5a71d88d5ef7d41964fbec1e")
Can some one help me with this request ???

Given the document supplied in your question, this command ...
db.collection.aggregate([
{ $unwind: "$connects" },
// count all occurrences
{ "$group": { "_id": {skill: "$skill", parser_id: "$connects.parser_id"}, "count": { "$sum": 1 } }},
// sum all occurrences and count distinct
{ "$group": { "_id": "$_id.skill", "quantity": { "$sum": 1 } }},
// (optional) rename the '_id' attribute to 'skill'
{ $project: { 'skill': '$_id', 'quantity': 1, _id: 0 } }
])
... will return:
{
"quantity" : 4,
"skill" : "stack"
}
The above command groups by skill and connects.parser_id and then gets a distinct count of those groups.
Your command includes the java tag so I suspect you are looking to execute the same command using the MongoDB Java driver. The code below (using MongoDB Java driver v3.x) will return the same result:
MongoClient mongoClient = ...;
MongoCollection<Document> collection = mongoClient.getDatabase("...").getCollection("...");
List<Document> documents = collection.aggregate(Arrays.asList(
Aggregates.unwind("$connects"),
new Document("$group", new Document("_id", new Document("skill", "$skill").append("parser_id", "$connects.parser_id"))
.append("count", new Document("$sum", 1))),
new Document("$group", new Document("_id", "$_id.skill").append("quantity", new Document("$sum", 1))),
new Document("$project", new Document("skill", "$_id").append("quantity", 1).append("_id", 0))
)).into(new ArrayList<>());
for (Document document : documents) {
logger.info("{}", document.toJson());
}
Note: this code deliberately uses the form new Document(<pipeline aggregator>, ...) instead of the Aggregators utilities to make it easier to see the translation between the shell command and its Java equivalent.

try $project with $reduce
$setUnion is used to keep only the distinct ids and finally $size used to get the distinct array count
db.col.aggregate(
[
{$project : {
_id : 0,
skill : 1,
quantity : {$size :{$reduce : {input : "$connects.parser_id", initialValue : [] , in : {$setUnion : ["$$value", "$$this"]}}}}
}
}
]
).pretty()
result
{ "skill" : "stack", "quantity" : 4 }

Json reader with different type in an array

I would like to write a Json reader for such Json
{
"headers": [
{
"id": "time:monthly",
"type": "a"
},
{
"id": "Value1",
"type": "b"
},
{
"id": "Value2",
"type": "b"
}
],
"rows": [
[
"2013-01",
4,
5
],
[
"2013-02",
3,
6
]
]
}
I know (thanks to the header) that in the elements of rows the first element is of a type a, the second and the third will be of type b. My goal is to create an object row (List[a],List[b]) (
the number of element of type a and b varies that's why I use List).
My question is how can I parse rows or how can I read a Json array with different type of object and without an id ?

I'd be tempted to setup the model with cases classes and mix the play framework macro base json reader with a custom one for your rows like this.
import play.api.libs.json._
case class Header(id: String, `type`: String)
case class Row(a: String, b: Int, c: Int)
case class Data(headers: Seq[Header], rows: Seq[Row])
object RowReads extends Reads[Row] {
def reads(js: JsValue) = js match {
case JsArray(Seq(a,b,c)) =>
(a,b,c) match {
case (a: JsString, b: JsNumber, c: JsNumber) =>
JsSuccess(Row(a.value,b.value.toInt,c.value.toInt))
case _ => JsError("nope")
}
case _ => JsError("nope")
}
}
object Formats {
implicit val headerReads = Json.reads[Header]
implicit val rowReads = RowReads
implicit val dataReads = Json.reads[Data]
def readIt(js: JsValue) = {
Json.fromJson[Data](js: JsValue)
}
}
For more details.
https://playframework.com/documentation/2.4.x/ScalaJson

rest assured check the name is exist in the json response

Am new to rest assured.Using rest assured am trying to verify data detail is found or not.Here two data details present.Some times it will be 2 or 3 or 5
Am getting response as follows and using java
{
"queryPath": "/api/",
"nId": "f084f5ad24fcfaa9e9faea0",
"statusCode": 707
"statusMessage": "Success",
"results": {
"data": [
{
"id": "10248522500798",
"capabilities": [
"record",
"HDt"
],
"name": "errt2"
},
{
"id": "418143778",
"capabilities": [
"1record",
"HDy"
],
"name": "Livin"
}
]
}
}
code using
JsonPath jsonResponse = new JsonPath(response.asString());
ArrayList<String> list = new ArrayList<String>();
list = jsonResponse.get("results.data"); //
if (list.size() < 1 ) {
SpiceCheck.fail("data not found! " + list.size());
}
Rather than this i wwant to check the data name is null or not also.How can i do that rest assured.

Just so you know you are missing a comma after 707.
To verify that none of the names is null I would parse out the names as a list, then iterate over the names one by one and check that they aren't null.
List<String> names = from(response.asString()).getList("results.data.name");
for(String name : names){
if(name == null){
// error handling code here
}
}