We Keep Coding

MongoDB - Update parts of object

I have the collection that stores documents per some execution Flow.
Every Process includes "processes" and each process includes steps.
So I end up with a 'flows' collection that has documents that look like this:
{
"name" : "flow1",
"description" : "flow 1 description",
"processes" : [
{
"processId" : "firstProcessId",
"name" : "firstProcessName",
"startedAt" : null,
"finishedAt" : null,
"status" : "PENDING",
"steps" : [
{
"stepId" : "foo", ​
​"status" : "PENDING",
​"startedAt" : null,
​"finishedAt" : null
},
{
"stepId" : "bar",​
​"status" : "PENDING",
​"startedAt" : null,
​"finishedAt" : null
}
...
​]
},
{
"processId" : "secondProcessId",
"name" : "secondProcessName",
"startedAt" : null,
"finishedAt" : null,
"status" : "PENDING",
"steps" : [
{
"stepId" : "foo", ​
​"status" : "PENDING",
​"startedAt" : null,
​"finishedAt" : null
},
{
"stepId" : "xyz",​
​"status" : "PENDING",
​"startedAt" : null,
​"finishedAt" : null
}
...
​]
}
}
A couple of notes here:
Each flow contains many processes
Each process contains at least one step, it is possible that in different processes the steps with the same id might appear (id is something that the programmer specifies),
It can be something like "step of bringing me something from the DB", so this is a kind of reusable component in my system.
Now, when the application runs I would like to call DAO's method like
"startProcess", "startStep".
So I would like to know what is the correct query for starting step given processId and steps.
I can successfully update the process description to "running" given the flow Id and the process Id:
db.getCollection('flows').updateOne({"name" : "flow1", "processes" : {$elemMatch : {"processId" : "firstProcessId"}}}, {$set: {"processes.$.status" : "RUNNING"}})
However I don't know how to update the step status given the flowId, process Id and step Id, it looks like it doesn't allow multiple "$" signs in the path:
So, this doesn't work:
db.getCollection('flows').updateOne({"name" : "flow1", "processes" : {$elemMatch : {"processId" : "firstProcessId"}}, "processes.steps.stepId" : {$elemMatch : {"stepId" : "foo"}}}, {$set: {"processes.$.steps.$.status" : "RUNNING"}})
What is the best way to implement such an update?

To update the document in multi-level nested array, you need $[<identifier>] filtered positional operator and arrayFilters.
And the processes and processes.steps.stepId filter in the match operator can be removed as the filter is performed in arrayFilters.
db.collection.update({
"name": "flow1"
},
{
$set: {
"processes.$[process].steps.$[step].status": "RUNNING"
}
},
{
arrayFilters: [
{
"process.processId": "firstProcessId"
},
{
"step.stepId": "foo"
}
]
})
Sample Mongo Playground
Reference
Update Nested Arrays in Conjunction with $[]

As you mentioned it does not work with multiple arrays, straight from the docs:
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
I recommend you use arrayFilters instead, it's behavior is much clearer especially when working with nested structures:
db.collection.updateMany(
{
"name": "flow1",
"processes.processId": "firstProcessId",
"processes.steps.stepId": "foo"
},
{
$set: {
"processes.$[process].steps.$[step].status": "RUNNING"
}
},
{
arrayFilters: [
{
"process.processId": "firstProcessId"
},
{
"step.stepId": "foo"
}
]
})
Mongo Playground

Spring MongoTemplate: Possible to Get and Count all distinct values from Key value Object in each document

I have a large-ish dataset (+100,000) documents. In each document, there is a key value object called ProjectCategories for which I would like to be able to get distinct count on each of the values across the entire collection.
For example my document looks like this:
"_id" : ObjectId("60e5ae42fcc92f14c3a41208"),
"userId" : "xxxx",
"projectCreator" : {
"userId" : "xxx|xxxx"
},
"hashTags" : [
"Spring",
"Java"
],
"projectCategories" : {
"60d76ef0597444095b8ab4b2" : "Backend",
"60d76ef0597444095b8ab232" : "Infrastructure"
},
"createdDate" : ISODate("2021-07-07T13:38:10.655Z"),
"updatedAt" : ISODate("2021-07-08T11:48:36.200Z"),
"_class" : "xxxx.model.project.Project"
}
I would like to get back all the unique values and their counts. Something like this:
Backend : 1002
FrontEnd : 1232
Infrastructure: 902
Is this possible to do using Java and mongoTemplate
Thanks for reading

Indexing heavily nested json data in solr using DIH

I using DIH to import the data from a NoSQL data source . The format looks something like below
{
"id" : "123-145-app"
"name" : "apple",
"type" : "electronic",
"information": {
"category":["tablets","laptops","mobile"],
"stores": [
{
"name": "imagine",
"location" : "DLH"
},
{
"name": "abc",
"location" : "BLR"
}
],
"head_office" : "US"
}
}
when I try to index this using :
https://lucene.apache.org/solr/guide/6_6/transforming-and-indexing-custom-json.html
I am getting error stating : "Unknown operation for the an atomic update, operation ignored" , and the record with that data is getting skipped.
And in the documentation they have not mentioned about how to incorporate the split via configuration file or schema.xml
Can some one please help me with this ?

jackson annotation to hide json type

I am using spring boot and I use a composite key. I want to hide the type of the JSON object.I tried with the annotations like #jsonIgnore, #jsonIgnoreType and nothing works. I have this:
[ {
"consumoId" : {
"inc_acc_id" : "1069931929",
"inc_clu_cellular_number" : "2954316800",
"inc_description" : "Minutes"
},
"inc_frt_id" : null,
"inc_unit" : "Min.",
"inc_amount" : 0.0,
"inc_quantity" : 22.68
}]
and I expect this:
[ {"inc_acc_id" : "1069931929",
"inc_clu_cellular_number" : "2954316800",
"inc_description" : "Minutes" ,
"inc_frt_id" : null,
"inc_unit" : "Min.",
"inc_amount" : 0.0,
"inc_quantity" : 22.68
}]

It looks like you want to inline the properties under the consumoId parent?
#JsonIgnore would skip it altogether, but #JsonUnwrapped will unwrap (omit) the containing object.

What is wrong with this Cypher query?

I am trying to send the following to Neo4j using the REST interface, specifically the method given in the 2.2.9 manual tutorial on using REST from Java, sendTransactional Cypher query. However, as you can see below, I keep on getting an error that doesn't seem to have much to do with the query itself.
Any tips on how I can debug this?
CREATE (p:player { props }), "parameters" {
"props" : {
"screen_name" : "testone",
"email" : "test.one#gmail.com",
"rank" : "-12",
"password" : "testonepass",
"details" : "test one details",
"latitude" : "0.0",
"longitude" : "0.0",
"available" : "true",
"publish" : "true" }}
{"results":[],"errors":[{"code":"Neo.ClientError.Request.InvalidFormat","message":"Unable to deserialize request: Unexpected character ('p' (code 112)): was expecting comma to separate OBJECT entries\n at [Source: HttpInputOverHTTP#10401de; line: 1, column: 66]"}]}

Form the snippet you posted, it looks like the payload to the transactional endpoint is incomplete. You could try this statement in the browser. I just copied your statement and formatted it so it could be posted to the browser. Then you can at least see it work. Clearly the data is being posted and it seems it is just comes down to formatting.
:POST /db/data/transaction/commit {
"statements": [
{
"statement": "CREATE (p:player { props })",
"parameters":
{
"props" : {
"screen_name" : "testone",
"email" : "test.one#gmail.com",
"rank" : "-12",
"password" : "testonepass",
"details" : "test one details",
"latitude" : "0.0",
"longitude" : "0.0",
"available" : "true",
"publish" : "true" }
}
}
]
}