This is a follow up to an earlier question. Thanks #harshank for the quick help. I am posting this question separately since there was an additional check required which I had not asked in the earlier question. So, the solution posted earlier works fine for the stated requirement.
I am adding the additional requirement here:
The decision if gp and parent match depends on an additional attribute eye_colour (please see sample payload below).
Also in the response additional fields need to be populated (which was not mentioned in the earlier question). Things like name and eye_colour.
So, as an example this is the payload:
[
{
"gp": "T1",
"gp_eye_colour": "blue",
"gp_name" : "John",
"parent": "T1",
"parent_eye_colour" : "black",
"parent_name" : "Sam",
"child": "C1",
"child_eye_colour" : "brown",
"child_name" : "C1"
},
{
"gp": "T1",
"gp_eye_colour": "blue",
"gp_name" : "John",
"parent": "T1",
"parent_eye_colour" : "black",
"parent_name" : "Sam",
"child": "C1",
"child_eye_colour" : "brown",
"child_name" : "C1"
}
]
And now using the earlier solution, I have tried to build on top of it as:
%dw 2.0
output application/xml
// hierarchy's structure. If, lets say you add a "grand child" it should work. (haven't tested this though)
var hierarchy = {
gp: {
parent: {
child: null
}
}
}
fun getDirectGeanologies(records, hierarchyStructure,isGp,isParent,isChild) = do {
var topLevel = keysOf(hierarchyStructure)[0] as String
var secondLevel = keysOf(hierarchyStructure[topLevel])[0]
---
(records groupBy $[topLevel])
mapObject ((level1Childs, code) ->
genealogy: {
code: code,
eye_colour: if(isGp) (level1Childs.gp_eye_colour) else if(isParent) (level1Childs.parent_eye_colour) else (level1Childs.child_eye_colour),
name: if(isGp) (level1Childs.gp_name) else if(isParent) (level1Childs.parent_name) else (level1Childs.child_name),
hierarchy:
if(secondLevel != null) // If there are no more childs, do nothing.
(level1Childs groupBy ((item) -> item[secondLevel] ~= code) // This is to handle when parent = gp
mapObject ((value, hasSameParent) ->
// If parent = gp, we want to skip parent and link directly to child
if(hasSameParent as Boolean) getDirectGeanologies(value, hierarchyStructure[topLevel][secondLevel],false,false,true)
// recurrsively call the function with child records and going down the hierarchyStructure by one
else getDirectGeanologies(value, hierarchyStructure[topLevel],false,true,false)
))
else {}
}
)
}
---
list: getDirectGeanologies(payload,hierarchy,true,false,false)
However, it is generating duplicate elements for eye_colour and name. Ideally, it should print the values only once with the value of either gp / parent / child.
This is the expected response:
<?xml version='1.0' encoding='UTF-8'?>
<list>
<genealogy>
<code>T1</code>
<eye_colour>blue</eye_colour>
<name>John</name>
<hierarchy>
<genealogy>
<code>C1</code>
<eye_colour>brown</eye_colour>
<name>C1</name>
<hierarchy/>
</genealogy>
</hierarchy>
</genealogy>
</list>
I was also not able to figure out how to check if gp and parent have same values based on two fields (gp & gp_eye_colour vs parent & parent_eye_colour)
Thank you #sudhish_s for the response, adding an example to elaborate regarding decision of when to skip parent (when gp & gp_eye_colour of gp are same as parent & parent_eye_colour of parent).
Sample input: (Here parent Sam should not be skipped since though gp and parent are same their eye colours are different. Similarly, in case of Don, this parent should be skipped since gp and parent and their eye colours are the same).
[
{
"gp": "T1",
"gp_eye_colour": "blue",
"gp_name" : "John",
"parent": "T1",
"parent_eye_colour" : "black",
"parent_name" : "Sam",
"child": "C1",
"child_eye_colour" : "brown",
"child_name" : "C1"
},
{
"gp": "T1",
"gp_eye_colour": "blue",
"gp_name" : "John",
"parent": "T1",
"parent_eye_colour" : "blue",
"parent_name" : "Don",
"child": "C1",
"child_eye_colour" : "brown",
"child_name" : "C1"
}
]
Here is the expected response:
<?xml version='1.0' encoding='UTF-8'?>
<list>
<genealogy>
<code>T1</code>
<eye_colour>blue</eye_colour>
<name>John</name>
<hierarchy>
<genealogy>
<code>T1</code>
<eye_colour>black</eye_colour>
<name>Sam</name>
<hierarchy>
<genealogy>
<code>C1</code>
<eye_colour>brown</eye_colour>
<name>C1</name>
<hierarchy/>
</genealogy>
</hierarchy>
</genealogy>
<genealogy>
<code>C1</code>
<eye_colour>brown</eye_colour>
<name>C1</name>
<hierarchy/>
</genealogy>
</hierarchy>
</genealogy>
</list>
Slightly different version. Used arrays for reference.
%dw 2.0
output application/xml
var hierarchy = ["gp", "parent", "child"]
fun getDirectGeanologies(records, level) = do {
var hLevel = hierarchy[level]
---
records groupBy $[hLevel] mapObject ((children, code) ->
genealogy: {
code: code,
eye_colour: children[0][hLevel ++ "_eye_colour"],
name: children[0][hLevel ++ "_name"],
hierarchy:
if (level == sizeOf(hierarchy) - 1) {}
else do {
var nextLevel = level + 1
var nextGen = children groupBy ($[hierarchy[nextLevel]])
---
nextGen mapObject ((nextGenChildren, nextGenCode) ->
if (nextGenCode == code)
getDirectGeanologies (nextGenChildren, nextLevel + 1 )
else
getDirectGeanologies (nextGenChildren, nextLevel)
)
}
}
)
}
---
list: getDirectGeanologies(payload,0)
The issue with elements appear multiple times is due to the level1Childs is Array. Change the output to json, it will be more clear. If you want to use your code, change the eye_colour and name mapping to below
eye_colour: if(isGp) (level1Childs[0].gp_eye_colour) else if(isParent) (level1Childs[0].parent_eye_colour) else (level1Childs[0].child_eye_colour),
name: if(isGp) (level1Childs[0].gp_name) else if(isParent) (level1Childs[0].parent_name) else (level1Childs[0].child_name),
Related
I have the collection that stores documents per some execution Flow.
Every Process includes "processes" and each process includes steps.
So I end up with a 'flows' collection that has documents that look like this:
{
"name" : "flow1",
"description" : "flow 1 description",
"processes" : [
{
"processId" : "firstProcessId",
"name" : "firstProcessName",
"startedAt" : null,
"finishedAt" : null,
"status" : "PENDING",
"steps" : [
{
"stepId" : "foo",
"status" : "PENDING",
"startedAt" : null,
"finishedAt" : null
},
{
"stepId" : "bar",
"status" : "PENDING",
"startedAt" : null,
"finishedAt" : null
}
...
]
},
{
"processId" : "secondProcessId",
"name" : "secondProcessName",
"startedAt" : null,
"finishedAt" : null,
"status" : "PENDING",
"steps" : [
{
"stepId" : "foo",
"status" : "PENDING",
"startedAt" : null,
"finishedAt" : null
},
{
"stepId" : "xyz",
"status" : "PENDING",
"startedAt" : null,
"finishedAt" : null
}
...
]
}
}
A couple of notes here:
Each flow contains many processes
Each process contains at least one step, it is possible that in different processes the steps with the same id might appear (id is something that the programmer specifies),
It can be something like "step of bringing me something from the DB", so this is a kind of reusable component in my system.
Now, when the application runs I would like to call DAO's method like
"startProcess", "startStep".
So I would like to know what is the correct query for starting step given processId and steps.
I can successfully update the process description to "running" given the flow Id and the process Id:
db.getCollection('flows').updateOne({"name" : "flow1", "processes" : {$elemMatch : {"processId" : "firstProcessId"}}}, {$set: {"processes.$.status" : "RUNNING"}})
However I don't know how to update the step status given the flowId, process Id and step Id, it looks like it doesn't allow multiple "$" signs in the path:
So, this doesn't work:
db.getCollection('flows').updateOne({"name" : "flow1", "processes" : {$elemMatch : {"processId" : "firstProcessId"}}, "processes.steps.stepId" : {$elemMatch : {"stepId" : "foo"}}}, {$set: {"processes.$.steps.$.status" : "RUNNING"}})
What is the best way to implement such an update?
To update the document in multi-level nested array, you need $[<identifier>] filtered positional operator and arrayFilters.
And the processes and processes.steps.stepId filter in the match operator can be removed as the filter is performed in arrayFilters.
db.collection.update({
"name": "flow1"
},
{
$set: {
"processes.$[process].steps.$[step].status": "RUNNING"
}
},
{
arrayFilters: [
{
"process.processId": "firstProcessId"
},
{
"step.stepId": "foo"
}
]
})
Sample Mongo Playground
Reference
Update Nested Arrays in Conjunction with $[]
As you mentioned it does not work with multiple arrays, straight from the docs:
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
I recommend you use arrayFilters instead, it's behavior is much clearer especially when working with nested structures:
db.collection.updateMany(
{
"name": "flow1",
"processes.processId": "firstProcessId",
"processes.steps.stepId": "foo"
},
{
$set: {
"processes.$[process].steps.$[step].status": "RUNNING"
}
},
{
arrayFilters: [
{
"process.processId": "firstProcessId"
},
{
"step.stepId": "foo"
}
]
})
Mongo Playground
I have a list of java POJO that's coming from a third party mysql stored procedure that includes location information mainly region,city,building and floor and the incident count for each location grouped by floors. I need to aggregate the incident count for each region,city and building as well. I also need to collect the children under each parent in a POJO as a location tree:
how can I do this with java stream api ? Thanks in advance!
input data :
output json :
[{
label : "EMEA",
type : "REGION",
incidentCount : 78,
children : [
{
label : "PAIS",
type : "CITY",
incidentCount : 37,
children : [
{
label: "F1",
TYPE : "FLOOR",
incidentCount: 18
},
{
label: "F2",
TYPE : "FLOOR",
incidentCount: 19
},
...
]
},
...
]
},
...
]
I am dealing with a dataset where both underscores and hyphens are being used between tokens in property names:
{
"id" : "116",
"priority" : 3,
"table_id" : 0,
"hard-timeout" : 0,
"match" : {
"ethernet-match" : {
"ethernet-type" : {
"type" : 2048
}
},
"ipv4-destination" : "10.0.0.25/32"
},
"strict" : false,
"flow-name" : "port_X_to_8_ip",
"instructions" : {
"instruction" : [ {
"order" : 0,
"apply-actions" : {
"action" : [ {
"order" : 1,
"output-action" : {
"max-length" : 60,
"output-node-connector" : "8"
}
} ]
}
} ]
},
Notice most elements have hyphens, but a few use underscores, such as table_id.
On the Java side, I am using this code to create my mapper:
import com.fasterxml.jackson.databind.json.JsonMapper;
...
JsonMapper jsonMapper = JsonMapper.builder().configure(SerializationFeature.INDENT_OUTPUT, true).build();
jsonMapper.setPropertyNamingStrategy(PropertyNamingStrategy.SNAKE_CASE); // <-- combine this
jsonMapper.setPropertyNamingStrategy(PropertyNamingStrategy.KEBAB_CASE); // <-- with this?
The last two lines are my current crux. I want the naming strategy to work for either SNAKE_CASE or KEBAB_CASE however I don't see a way to or the properties and can't find much on google.
You pick the naming strategy that fits for most properties, then use the #JsonProperty() annotation to name the ones that don't follow the standard.
In your case, that would be PropertyNamingStrategy.KEBAB_CASE and #JsonProperty("table_id").
I am creating a small program that will make an SVG file based on options I select. The problem I am having is that JSVGCanvas (from Batik) will not seem to render gradients that are made (using the DOM api). The weird part is, if I export the file and open it in Inkscape or with the test class here, it (the gradient) renders! Here is all the info I think may be important:
Batik Version: 1.7
Gradient Build Code:
public Element parse(Document doc, String id) { //This is inside a gradient object.
Element defs = doc.createElementNS(Main.svgNS, "defs"); //Since multiple defs work in an SVG (or at least Inkscape), I add a defs tag per set of gradients.
defs.setAttribute("id", id); //Sets the id of the defs tag.
for (JsonElement je : json.getAsJsonArray("Linears")) { //I use GSON to store data, I am iterating through all objects in the Linears array.
if (je.isJsonObject()) { //Just a type check.
JsonObject jo = je.getAsJsonObject();
Element lin = doc.createElementNS(Main.svgNS, "linearGradient"); //Creates the linearGradient element.
for (Entry<String, JsonElement> entry : jo.entrySet()) { //For all elements other than Stops (it is an array) we want to set the attribute.
if (!entry.getKey().equals("Stops")) {
lin.setAttribute(entry.getKey(), entry.getValue().getAsJsonPrimitive().getAsString());
}
}
for (JsonElement stop : jo.getAsJsonArray("Stops")) { //For all objects in stops we want to add a stop.
Element el = doc.createElementNS(Main.svgNS, "stop"); //Creates a stop element.
for (Entry<String, JsonElement> entry : stop.getAsJsonObject().entrySet()) {
if (entry.getValue().isJsonObject()) {
el.setAttribute(entry.getKey(), parseObject(entry.getValue().getAsJsonObject())); //The parseObject method makes a string based on the JsonObject (this is used to get "variables" such as colors).
} else {
el.setAttribute(entry.getKey(), entry.getValue().getAsString());
}
}
lin.appendChild(el); //Add stop to the linear gradient.
}
defs.appendChild(lin); //Add the linear gradient to defs.
}
}
for (JsonElement je : json.getAsJsonArray("Radials")) { //Loops through all the radial gradients.
if (je.isJsonObject()) { //Type check.
JsonObject jo = je.getAsJsonObject();
Element rad = doc.createElementNS(Main.svgNS, "radialGradient"); //Creates the radialGradient element.
for (Entry<String, JsonElement> entry : jo.entrySet()) {
if (entry.getValue().isJsonPrimitive()) {
rad.setAttribute(entry.getKey(), entry.getValue().getAsJsonPrimitive().getAsString());
} else {
rad.setAttribute(entry.getKey(), parseObject(entry.getValue().getAsJsonObject()));
}
}
defs.appendChild(rad); //Adds the radial gradient to defs.
}
}
cache = defs;
return cache; //We return the defs to be added in the main build code.
}
Document Creation:
Constructor:
impl = SVGDOMImplementation.getDOMImplementation();
Build Method:
doc = (SVGDocument) impl.createDocument(svgNS, "svg", null);
JSVGCanvas Creation:
preview = new JSVGCanvas();
preview.setEnableImageZoomInteractor(true);
preview.setEnablePanInteractor(true);
Here is what the gradients part of my template files looks like (JSON):
"Gradients": {
"Radials": [
{
"gradientTransform": "matrix(0.96774313,-1.5022501e-7,3.5488612e-7,1.483871,-201.69415,-217.84922)",
"inkscape:collect": "always",
"xmlns:xlink": "http://www.w3.org/1999/xlink",
"id": "radialGradient5607",
"gradientUnits": "userSpaceOnUse",
"xlink:show": "other",
"xlink:type": "simple",
"r": "15.5",
"cx":{
pre:"",
var:"X",
add:327,
post:""
},
"fx":{
pre:"",
var:"Y",
add:327,
post:""
},
"cy": {
pre:"",
var:"X",
add:209,
post:""
},
"fy": {
pre:"",
var:"Y",
add:209,
post:""
},
"xlink:href": "#linearGradient4114",
"xlink:actuate": "onLoad"
}
],
"Linears": [
{
"xlink:actuate": "onLoad",
"xlink:type": "simple",
"id": "linearGradient4114",
"xlink:show": "other",
"xmlns:xlink": "http://www.w3.org/1999/xlink",
"Stops": [
{
"offset": "0",
"style": {
"pre": "stop-color:",
"var": "Eye",
"post": ";stop-opacity:1;"
},
"id": "stop4116"
},
{
"style": {
"pre": "stop-color:",
"var": "Eye2",
"post": ";stop-opacity:1;"
},
"offset": "0.5",
"id": "stop4122"
},
{
"offset": "1",
"style": {
"pre": "stop-color:",
"var": "Eye3",
"post": ";stop-opacity:1;"
},
"id": "stop4118"
}
]
}
]
}
That part of the config file then gets turned into:
<defs id="defs0">
<linearGradient
xmlns:xlink="http://www.w3.org/1999/xlink"
xlink:type="simple"
xlink:actuate="onLoad"
id="linearGradient4114"
xlink:show="other">
<stop
style="stop-color:#d80a00;stop-opacity:1;"
offset="0"
id="stop4116"/>
<stop
style="stop-color:#ffffff;stop-opacity:1;"
offset="0.5"
id="stop4122"/>
<stop
style="stop-color:#ffffff;stop-opacity:1;"
offset="1"
id="stop4118"/>
</linearGradient>
<radialGradient
gradientTransform="matrix(0.96774313,-1.5022501e-7,3.5488612e-7,1.483871,-201.69415,-217.84922)"
xmlns:xlink="http://www.w3.org/1999/xlink"
xlink:type="simple"
xlink:href="#linearGradient4114"
id="radialGradient5607"
xlink:show="other"
gradientUnits="userSpaceOnUse"
r="15.5"
xlink:actuate="onLoad"
cx="392.0"
fx="427.0"
cy="274.0"
fy="309.0"
inkscape:collect="always"/>
I assume this is a namespace thing (I don't quite get XML/SVG). I assume this because I have had an error like this before, where it wouldn't work in the preview but it would work after export (and loading the svg file). Note that this is a quick home application, so the code isn't the finest (nor the design), but it works for what I am doing. Also note that I do not type these config files by hand (that would be insane), I have a converter that converts an svg file to these configs (yes it sounds like going back and forth but I am combining multiple of these configs to make an image).
I figured it out after messing around yesterday and today. The xlink:href attribute requires that you set its namespace to be xlink's. For my project that meant just doing this:
if (entry.getKey().equals("xlink:href")) {
rad.setAttributeNS(XMLConstants.XLINK_NAMESPACE_URI, "xlink:href", entry.getValue().getAsJsonPrimitive().getAsString());
}
After adding that in, my gradients worked successfully.
I am using Java API for CRUD operation on elasticsearch.
I have an typewith a nested field and I want to update this field.
Here is my mapping for the type:
"enduser": {
"properties": {
"location": {
"type": "nested",
"properties":{
"point":{"type":"geo_point"}
}
}
}
}
Of course my enduser type will have other parameters.
Now I want to add this document in my nested field:
"location":{
"name": "London",
"point": "44.5, 5.2"
}
I was searching in documentation on how to update nested document but I couldn't find anything. For example I have in a string the previous JSON obect (let's call this string json). I tried the following code but seems to not working:
params.put("location", json);
client.prepareUpdate(index, ElasticSearchConstants.TYPE_END_USER,id).setScript("ctx._source.location = location").setScriptParams(params).execute().actionGet();
I have got a parsing error from elasticsearch. Anyone knows what I am doing wrong ?
You don't need the script, just update it.
UpdateRequestBuilder br = client.prepareUpdate("index", "enduser", "1");
br.setDoc("{\"location\":{ \"name\": \"london\", \"point\": \"44.5,5.2\" }}".getBytes());
br.execute();
I tried to recreate your situation and i solved it by using an other way the .setScript method.
Your updating request now would looks like :
client.prepareUpdate(index, ElasticSearchConstants.TYPE_END_USER,id).setScript("ctx._source.location =" + json).execute().actionGet()
Hope it will help you.
I am not sure which ES version you were using, but the below solution worked perfectly for me on 2.2.0. I had to store information about named entities for news articles. I guess if you wish to have multiple locations in your case, it would also suit you.
This is the nested object I wanted to update:
"entities" : [
{
"disambiguated" : {
"entitySubTypes" : [],
"disambiguatedName" : "NameX"
},
"frequency" : 1,
"entityType" : "Organization",
"quotations" : ["...", "..."],
"name" : "entityX"
},
{
"disambiguated" : {
"entitySubType" : ["a", "b" ],
"disambiguatedName" : "NameQ"
},
"frequency" : 5,
"entityType" : "secondTypeTest",
"quotations" : [ "...", "..."],
"name" : "entityY"
}
],
and this is the code:
UpdateRequest updateRequest = new UpdateRequest();
updateRequest.index(indexName);
updateRequest.type(mappingName);
updateRequest.id(url); // docID is a url
XContentBuilder jb = XContentFactory.jsonBuilder();
jb.startObject(); // article
jb.startArray("entities"); // multiple entities
for ( /*each namedEntity*/) {
jb.startObject() // entity
.field("name", name)
.field("frequency",n)
.field("entityType", entityType)
.startObject("disambiguated") // disambiguation
.field("disambiguatedName", disambiguatedNameStr)
.field("entitySubTypes", entitySubTypeArray) // multi value field
.endObject() // disambiguation
.field("quotations", quotationsArray) // multi value field
.endObject(); // entity
}
jb.endArray(); // array of nested objects
b.endObject(); // article
updateRequest.doc(jb);
Blblblblblblbl's answer couldn't work for me atm, because scripts are not enabled in our server. I didn't try Bask's answer yet - Alcanzar's gave me a hard time, because I supposedly couldn't formulate the json string correctly that setDoc receives. I was constantly getting errors that either I am using objects instead of fields or vice versa. I also tried wrapping the json string with doc{} as indicated here, but I didn't manage to make it work. As you mentioned it is difficult to understand how to formulate a curl statement at ES's java API.
A simple way to update the arraylist and object value using Java API.
UpdateResponse update = client.prepareUpdate("indexname","type",""+id)
.addScriptParam("param1", arrayvalue)
.addScriptParam("param2", objectvalue)
.setScript("ctx._source.field1=param1;ctx._source.field2=param2").execute()
.actionGet();
arrayvalue-[
{
"text": "stackoverflow",
"datetime": "2010-07-27T05:41:52.763Z",
"obj1": {
"id": 1,
"email": "sa#gmail.com",
"name": "bass"
},
"id": 1,
}
object value -
"obj1": {
"id": 1,
"email": "sa#gmail.com",
"name": "bass"
}