We Keep Coding

logback with MDC: Using multiple custom blocks instead of one mdc block for custom key values

I'm using Java and logback in my application, as well as MDC to log custom key-values. i way, to have my logged key-values pairs to be unders two blocks, instead of having them all unders MDC block ?
Example:
{
"timestamp" : "2022-12-19 10:17:05.460",
"level" : "INFO",
"thread" : "XNIO-1 task-1",
"logger" : "ca.bnc.cybercriminalite.fraud.service.check.FraudCheckService",
"message" : "Calling IFM to evaluate request: localhost:9999",
"server": {
"url" : "http://example.com",
"port" : "2310",
},
"transactionInfo": {
"request_Id" : "abcd-1234",
"EndToEndId" : "1234567"
},
"context" : "fraud_check.ifm.request"
}
instead of:
{
"timestamp" : "2022-12-19 10:17:05.460",
"level" : "INFO",
"thread" : "XNIO-1 task-1",
"logger" : "ca.bnc.cybercriminalite.fraud.service.check.FraudCheckService",
"message" : "Calling IFM to evaluate request: localhost:9999",
"mdc": {
"url" : "http://example.com",
"port" : "2310",
"request_Id" : "abcd-1234",
"EndToEndId" : "1234567"
},
"context" : "fraud_check.ifm.request"
}

MongoDB - Update parts of object

I have the collection that stores documents per some execution Flow.
Every Process includes "processes" and each process includes steps.
So I end up with a 'flows' collection that has documents that look like this:
{
"name" : "flow1",
"description" : "flow 1 description",
"processes" : [
{
"processId" : "firstProcessId",
"name" : "firstProcessName",
"startedAt" : null,
"finishedAt" : null,
"status" : "PENDING",
"steps" : [
{
"stepId" : "foo", ​
​"status" : "PENDING",
​"startedAt" : null,
​"finishedAt" : null
},
{
"stepId" : "bar",​
​"status" : "PENDING",
​"startedAt" : null,
​"finishedAt" : null
}
...
​]
},
{
"processId" : "secondProcessId",
"name" : "secondProcessName",
"startedAt" : null,
"finishedAt" : null,
"status" : "PENDING",
"steps" : [
{
"stepId" : "foo", ​
​"status" : "PENDING",
​"startedAt" : null,
​"finishedAt" : null
},
{
"stepId" : "xyz",​
​"status" : "PENDING",
​"startedAt" : null,
​"finishedAt" : null
}
...
​]
}
}
A couple of notes here:
Each flow contains many processes
Each process contains at least one step, it is possible that in different processes the steps with the same id might appear (id is something that the programmer specifies),
It can be something like "step of bringing me something from the DB", so this is a kind of reusable component in my system.
Now, when the application runs I would like to call DAO's method like
"startProcess", "startStep".
So I would like to know what is the correct query for starting step given processId and steps.
I can successfully update the process description to "running" given the flow Id and the process Id:
db.getCollection('flows').updateOne({"name" : "flow1", "processes" : {$elemMatch : {"processId" : "firstProcessId"}}}, {$set: {"processes.$.status" : "RUNNING"}})
However I don't know how to update the step status given the flowId, process Id and step Id, it looks like it doesn't allow multiple "$" signs in the path:
So, this doesn't work:
db.getCollection('flows').updateOne({"name" : "flow1", "processes" : {$elemMatch : {"processId" : "firstProcessId"}}, "processes.steps.stepId" : {$elemMatch : {"stepId" : "foo"}}}, {$set: {"processes.$.steps.$.status" : "RUNNING"}})
What is the best way to implement such an update?

To update the document in multi-level nested array, you need $[<identifier>] filtered positional operator and arrayFilters.
And the processes and processes.steps.stepId filter in the match operator can be removed as the filter is performed in arrayFilters.
db.collection.update({
"name": "flow1"
},
{
$set: {
"processes.$[process].steps.$[step].status": "RUNNING"
}
},
{
arrayFilters: [
{
"process.processId": "firstProcessId"
},
{
"step.stepId": "foo"
}
]
})
Sample Mongo Playground
Reference
Update Nested Arrays in Conjunction with $[]

As you mentioned it does not work with multiple arrays, straight from the docs:
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
I recommend you use arrayFilters instead, it's behavior is much clearer especially when working with nested structures:
db.collection.updateMany(
{
"name": "flow1",
"processes.processId": "firstProcessId",
"processes.steps.stepId": "foo"
},
{
$set: {
"processes.$[process].steps.$[step].status": "RUNNING"
}
},
{
arrayFilters: [
{
"process.processId": "firstProcessId"
},
{
"step.stepId": "foo"
}
]
})
Mongo Playground

Indexing heavily nested json data in solr using DIH

I using DIH to import the data from a NoSQL data source . The format looks something like below
{
"id" : "123-145-app"
"name" : "apple",
"type" : "electronic",
"information": {
"category":["tablets","laptops","mobile"],
"stores": [
{
"name": "imagine",
"location" : "DLH"
},
{
"name": "abc",
"location" : "BLR"
}
],
"head_office" : "US"
}
}
when I try to index this using :
https://lucene.apache.org/solr/guide/6_6/transforming-and-indexing-custom-json.html
I am getting error stating : "Unknown operation for the an atomic update, operation ignored" , and the record with that data is getting skipped.
And in the documentation they have not mentioned about how to incorporate the split via configuration file or schema.xml
Can some one please help me with this ?

Spring Mongo criteria on list of list

Hi I am using spring data mongo, I need to fetch data based on multiple where condition. The problem I have when I want to apply a where clause to a list in a list.
For example
{
"_id" : ObjectId("5982bf9339f3c92b84be4737"),
"_class" : "com.paladion.payment.model.GroupQuestionMapping",
"saqID" : "SAQ A",
"saqVersion" : "3",
"questionTab" : {
"Secure Network" : [
{
"number" : "2.1 (a)",
"question" : "Are vendor-supplied"
"description" : "<ul><li>Review"
},
{
"number" : "2.1 (b)",
"question" : "Are unnecessary"
"description" : "<ul><li>Review policies
}
],
"Access Control" : [
{
"number" : "2.1 (a)",
"question" : "Are vendor-supplied"
"description" : "<ul><li>Review"
},
{
"number" : "2.1 (b)",
"question" : "Are unnecessary"
"description" : "<ul><li>Review policies
}
]
}
}
Over here I need to fetch data where saqId is SAQ A and saq Version is 3 and questionTab is secure network.
I have problem in applying criteria on questionTab.
my code:
Query query = new Query();
query.addCriteria(Criteria.where("saqtype").is(saqType));
query.addCriteria(Criteria.where("saqversion").is(saqVersion));
query.addCriteria(/* criteria on questionTab */);

query.addCriteria(Criteria.where("questionTab.Secure Network").exists(true));
Thing to note is that it will bring you the full document based on criteria, so you would have to filter out other type of questionTab from document.
Other way is aggregation but then I think processing on application layer might be preferable.

How to construct query to update nested array document in mongo?

I am having following document in mongo,
{
"_id" : ObjectId("506e9e54a4e8f51423679428"),
"description" : "ffffffffffffffff",
"menus" : [
{
"_id" : ObjectId("506e9e5aa4e8f51423679429"),
"description" : "ffffffffffffffffffff",
"items" : [
{
"name" : "xcvxc",
"description" : "vxvxcvxc",
"text" : "vxcvxcvx",
"menuKey" : "0",
"onSelect" : "1",
"_id" : ObjectId("506e9f07a4e8f5142367942f")
} ,
{
"name" : "abcd",
"description" : "qqq",
"text" : "qqq",
"menuKey" : "0",
"onSelect" : "3",
"_id" : ObjectId("507e9f07a4e8f5142367942f")
}
]
},
{
"_id" : ObjectId("506e9e5aa4e8f51423679429"),
"description" : "rrrrr",
"items" : [ {
"name" : "xcc",
"description" : "vx",
"text" : "vxc",
"menuKey" : "0",
"onSelect" : "2",
"_id" : ObjectId("506e9f07a4e8f5142367942f")
} ]
}
]
}
Now , i want to update the following document :
{
"name" : "abcd",
"description" : "qqq",
"text" : "qqq",
"menuKey" : "0",
"onSelect" : "3",
"_id" : ObjectId("507e9f07a4e8f5142367942f")
}
I am having main documnet id: "_id" : ObjectId("506e9e54a4e8f51423679428") and menus id
"_id" : ObjectId("506e9e54a4e8f51423679428") as well as items id "_id" : ObjectId("507e9f07a4e8f5142367942f") which is to be updated.
I have tried using the following query:
db.collection.update({ "_id" : { "$oid" : "506e9e54a4e8f51423679428"} , "menus._id" : { "$oid" : "506e9e5aa4e8f51423679429"}},{ "$set" : { "menus.$.items" : { "_id" : { "$oid" : "506e9f07a4e8f5142367942f"}} , "menus.$.items.$.name" : "xcvxc66666", ...}},false,false);
but its not working...

The positional operator does not work on the number of levels you are trying to get it to work on ( https://jira.mongodb.org/browse/SERVER-831?focusedCommentId=22438&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel ) with menus.$.items.$.name and even if it did MongoDB query parser would have no idea what the other $ is from the find of the update.
You will need to pull out the items from the schema, update that seprately and then update the root document.
One good way of judging when queries should be done separately is to think that each menu sounds like a separate entity (or table in a relational database) as such you should probably work on updating those entites (or tables in a relational model) separately to the parent entity (table).
So first you would get out the main root document. Scroll across it's menus in client side and then $set that particular menu to the entire item you build on client side.
Edit
The way I imagine this work client side is (in pseudo code since my Java is a little rusty) by first getting that document in an active record fashion:
doc = db.col.find({ "_id" : { "$oid" : "506e9e54a4e8f51423679428"} ,
"menus._id" : { "$oid" : "506e9e5aa4e8f51423679429"}});
Then you would iterate through the document assigning your values:
foreach(doc.menus as menu_key => menu){
foreach(menu['items'] as key => item){
if(item._id == { "$oid" : "506e9f07a4e8f5142367942f"}){
doc.menus[menu_key][key][name] = "xcvxc66666"
}
}
}
And then simple save the doc after all changes are commited:
db.col.save(doc);
This is of course just one way of doing it and this way uses the activen record paradigm which I personally like. In this idea you would combine the find with everything else you need to modify on the document, building it up client side and then sending it all down as one single query to your DB.