I am able to successfully create an index using java api and search the source field. My Problem is not able to get the nested key field using search.
For ex my sample json stucture:
{
"name": "John Smith",
"age": 42,
"confirmed": true,
"join_date": "2014-06-01",
"timestamp": "14331222299",
"home": {
"lat": 51.5,
"lon": 0.1
},
"accounts": [
{
"type": "facebook",
"id": "johnsmith"
},
{
"type": "twitter",
"id": "johnsmith"
}
]
}
i can able to index this json as source through java client api:-
IndexResponse response = client.prepareIndex(mohan+"-"+14-10-2014, "mohanelastic")
.setSource(jsonoutput.toString())
.execute()
.actionGet();
My java client search api:
QueryBuilder en = QueryBuilders.matchQuery("name", "John Smith");
FilterBuilder flb = FilterBuilders.andFilter(
FilterBuilders
. .rangeFilter("timestamp").from(starttimeinmilli).to(endtimeinmilli),
FilterBuilders.queryFilter(QueryBuilders.matchQuery("confirmed", "true"))
);
SearchResponse response = client.prepareSearch(mohan+"-"+14-10-2014)
.setTypes("mohanelastic")
.setSearchType(SearchType.QUERY_AND_FETCH)
.setPostFilter(flb)
.setQuery(en)
.setFrom(0).setSize(60)
.execute().actionGet();
In this i can able to get the total hits, key field values(name,age,join_date). But not able to get the key value for (home.lat) it shows null value. Nested values for any json shows null.
I am retrieving the source field json keys and it shows respective value:-
System.out.println("event type"+response.getHits().getAt(0).getSource().get("name"));
System.out.println("event type"+response.getHits().getAt(0).getSource().get("timestamp"));
But when i try home.lat it shows null value:
System.out.println("event type"+response.getHits().getAt(0).getSource().get("home.lat"));
You can't access home.lat value using dot notation in the Java API. Think of nested objects as maps (home) or a list containing maps (accounts). To get the lat value you would need to do the following:
Map<String, Object> source = response.getHits().getAt(0).getSource();
Map<String, Object> home = source.get('home');
Double latValue = (Double) home.get('lat');
Related
I have a Json body like the example below. I need to extract the value from a key that has another key with a specific value in an array. I am passing in a JsonNode with everything in the detail component of the message, I can easily extract from each level, however, I'm struggling with the array.
In this case, I need to extract the value of "value" (Police/Fire/Accident Report) from the object in the array which has a key/value pair of "name":"documentTitle". I understand this is a JSONArray, but I can't find a good example that shows me how to extract the values for an object in the array that contains a certain key/value pair, I don't think I can rely on getting the object in position [2] in the array as the same objects may not always be present in the additionalMetadata array.
Sample Json:
"sourceVersion": "1.0",
"eventId": "8d74b892-810a-47c3-882b-6e641fd509eb",
"clientRequestId": "b84f3a7b-03cc-4848-a1e8-3519106c6fcb",
"detail": {
"stack": "corona",
"visibilityIndicator": null,
"documentUid": "b84f3a7b-03cc-4848-a1e8-3519106c6fcb",
"additionalMetadata": [
{
"name": "lastModifiedDate",
"value": "2021-05-21T04:53:53Z"
},
{
"name": "documentName",
"value": "Police/Fire Report, 23850413, 2021-05-20 14:51:23"
},
{
"name": "documentTitle",
"value": "Police/Fire/Accident Report"
},
{
"name": "documentAuthor",
"value": "System Generated"
},
{
"name": "lastModifiedBy",
"value": "System Updated"
},
{
"name": "createdBy",
"value": "System Generated"
},
{
"name": "documentDescription",
"value": "Police/Fire Report received"
},
{
"name": "organizationCode",
"value": "Claims"
}
]
}
}```
Loop through the json array and extract the json object with name documentTitile. From that json object you can get the value
Well, either the JSON framework you're using supports this out of the box (check the documentation) or you could convert it manually to a map:
List<AdditionalMetadataEntry> additionalMetadata;
[...]
Map<String, String> additionalMetadataMap = additionalMetadata.stream().collect(Collectors.toMap(AdditionalMetadataEntry::getName, AdditionalMetadataEntry::getValue));
I was able to figure it out. I created a new node off the existing notificationBody JsonNode, then parsed through the metadata key/value pairs:
String docTitle = "";
JsonNode additionalMetadata = notificationBody.get("detail").get("additionalMetadata");
for (JsonNode node: additionalMetadata) {
String name = node.get("name").asText();
String value = node.get("value").asText();
if(name.equals("documentTitle")){
docTitle = value;
}
}
I make a request on api graphQL and have the responce of body:
{
"dataRequests": [
{
"status": "success",
"title": "token",
"values": {
"limit": 1,
"offset": 0,
"count": 1,
"total": 1,
"elements": [
{
"type": "DOMAIN",
"permission": "default",
"properties": [
{
"name": "property:id",
"value": 390
},
{
"name": "setting:crawler:token",
"value": "token(here's real token)"
}
],
"filters": []
}
]
}
}
]
}
I want to get value of the field "value" with token. But i have a problem to desirialize it.
My code can get the list of maps of 'dataRequests' field(using RestAssured):
GraphQLSteps graphQLSteps = new GraphQLSteps();
Response response = graphQLSteps.postProjectToken(id);
List<Map<String, String>> dataRequest = response.jsonPath().getList("dataRequests");
but, if i try to get the list of maps of 'value' field:
List<Map<String, String>> dataRequest = response.jsonPath().getList("value");
i get the "null" value. I think it's because i have to wrap it through the whole tree: values - elements - properties and only then value. But it's very complicated and i tried to get it in this way, but obtain only the same result "null".
I noticed that if i print an entrySet of an existing map from list:
List<Map<String, String>> dataRequest = response.jsonPath().getList("dataRequests");
for (Map<String, String> map : dataRequest) {
System.out.println(map.entrySet());
}
I get the result:
[status=success, title=token, values={limit=1, offset=0, count=1, total=1, elements=[{type=DOMAIN, permission=default, properties=[{name=property:id, value=390}, {name=setting:crawler:token, value=someToken}], filters=[]}]}]
And can see value with token there.
Can you prompt me, how can i get the "value" with token either from this List<Map<String, String>> or by other way with deserialitation api?
Your json is kind of complicated with many nested level. I would recommend to use lib jsonpath, instead of using jsonpath provided by rest-assured.
Example:
List<String> values = JsonPath.read(response.toString(), "$..value");
System.out.println(values);
//[390,"token(here's real token)"]
add jsonpath to pom.xml
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<version>2.6.0</version>
</dependency>
I'm looking for a java library that can translate json input into SQL dynamically.
For example, the library could expect the following json data:
{
"rules": [
{
"field": "firstname",
"value": [
"John",
"Doe"
],
"operator": "in"
},
{
"rules": [
{
"field": "age",
"value": 18,
"operator": "EQUALS"
}
],
"condition": "AND"
}
]
}
And should then be able of translating this into a dynamic sql query:
SELECT * FROM persons where firstname IN ("John", "Doe") AND age = 18;
Is there any existing framework that I could build this upon?
I am not sure about library to get the expected output. But you can use the following code to write Java method.
Assuming that input will be JSON object.
JSONObject jo = (JSONObject) obj;
// getting fieldname and operator
String field= (String) jo.get("field");
String operator= (String) jo.get("operator");
//getting values
JSONArray ja = (JSONArray) jo.get("value");
// iterating values
Iterator itr2 = ja.iterator();
Then you can use above to write dynamic SQL query.
I have a table called Group and it will have records like:
{
"id": "UniqueID1",
"name": "Ranjeeth",
"emailIdMappings": [
{
"emailId": "r.pt#r.com",
"userId": 324
},
{
"emailId": "r1.pt#r.com",
"userId": 325
}
]
},
{
"id": "UniqueID2",
"name": "Ranjeeth",
"emailIdMappings": [
{
"emailId": "r1.pt#r.com",
"userId": 325
},
{
"emailId": "r2.pt#r.com",
"userId": 326
}
]
}
I need to query and get result if emailId contains the input string.
I have reached so far and I am not able to get the result
AttributeValue attributeValue = new AttributeValue("r.pt#r.com");
Condition containsCondition = new Condition()
.withComparisonOperator(ComparisonOperator.CONTAINS)
.withAttributeValueList(attributeValue);
Map<String, Condition> conditions = newHashMap();
conditions.put("emailIdMappings.emailId", containsCondition);
ScanRequest scanRequest = new ScanRequest()
.withTableName("Group")
.withScanFilter(conditions);
amazonDynamoDB.scan(scanRequest)
dynamoDBMapper.marshallIntoObjects(Group.class, scanResult.getItems());
For the above code I am expecting record with id UniqueID1, but it's empty. If you pass "r1.pt#r.com" then you should get both records.
sdk used is com.amazonaws:aws-java-sdk-dynamodb:1.11.155
I tried posting the question in aws forum which didn't help much.
As you have List of Objects which has two attributes in a object (i.e. emailId and userId), you need to provide both values in order to match the item.
The CONTAINS function will not be able to match the item if the object has two attributes and only one attribute value mentioned in the filter expression.
Otherwise, you need to provide the occurrence (i.e. index) of the list to match the item.
Example:-
emailIdMappings[0].emailId = :emailIdVal
I am using Java API for CRUD operation on elasticsearch.
I have an typewith a nested field and I want to update this field.
Here is my mapping for the type:
"enduser": {
"properties": {
"location": {
"type": "nested",
"properties":{
"point":{"type":"geo_point"}
}
}
}
}
Of course my enduser type will have other parameters.
Now I want to add this document in my nested field:
"location":{
"name": "London",
"point": "44.5, 5.2"
}
I was searching in documentation on how to update nested document but I couldn't find anything. For example I have in a string the previous JSON obect (let's call this string json). I tried the following code but seems to not working:
params.put("location", json);
client.prepareUpdate(index, ElasticSearchConstants.TYPE_END_USER,id).setScript("ctx._source.location = location").setScriptParams(params).execute().actionGet();
I have got a parsing error from elasticsearch. Anyone knows what I am doing wrong ?
You don't need the script, just update it.
UpdateRequestBuilder br = client.prepareUpdate("index", "enduser", "1");
br.setDoc("{\"location\":{ \"name\": \"london\", \"point\": \"44.5,5.2\" }}".getBytes());
br.execute();
I tried to recreate your situation and i solved it by using an other way the .setScript method.
Your updating request now would looks like :
client.prepareUpdate(index, ElasticSearchConstants.TYPE_END_USER,id).setScript("ctx._source.location =" + json).execute().actionGet()
Hope it will help you.
I am not sure which ES version you were using, but the below solution worked perfectly for me on 2.2.0. I had to store information about named entities for news articles. I guess if you wish to have multiple locations in your case, it would also suit you.
This is the nested object I wanted to update:
"entities" : [
{
"disambiguated" : {
"entitySubTypes" : [],
"disambiguatedName" : "NameX"
},
"frequency" : 1,
"entityType" : "Organization",
"quotations" : ["...", "..."],
"name" : "entityX"
},
{
"disambiguated" : {
"entitySubType" : ["a", "b" ],
"disambiguatedName" : "NameQ"
},
"frequency" : 5,
"entityType" : "secondTypeTest",
"quotations" : [ "...", "..."],
"name" : "entityY"
}
],
and this is the code:
UpdateRequest updateRequest = new UpdateRequest();
updateRequest.index(indexName);
updateRequest.type(mappingName);
updateRequest.id(url); // docID is a url
XContentBuilder jb = XContentFactory.jsonBuilder();
jb.startObject(); // article
jb.startArray("entities"); // multiple entities
for ( /*each namedEntity*/) {
jb.startObject() // entity
.field("name", name)
.field("frequency",n)
.field("entityType", entityType)
.startObject("disambiguated") // disambiguation
.field("disambiguatedName", disambiguatedNameStr)
.field("entitySubTypes", entitySubTypeArray) // multi value field
.endObject() // disambiguation
.field("quotations", quotationsArray) // multi value field
.endObject(); // entity
}
jb.endArray(); // array of nested objects
b.endObject(); // article
updateRequest.doc(jb);
Blblblblblblbl's answer couldn't work for me atm, because scripts are not enabled in our server. I didn't try Bask's answer yet - Alcanzar's gave me a hard time, because I supposedly couldn't formulate the json string correctly that setDoc receives. I was constantly getting errors that either I am using objects instead of fields or vice versa. I also tried wrapping the json string with doc{} as indicated here, but I didn't manage to make it work. As you mentioned it is difficult to understand how to formulate a curl statement at ES's java API.
A simple way to update the arraylist and object value using Java API.
UpdateResponse update = client.prepareUpdate("indexname","type",""+id)
.addScriptParam("param1", arrayvalue)
.addScriptParam("param2", objectvalue)
.setScript("ctx._source.field1=param1;ctx._source.field2=param2").execute()
.actionGet();
arrayvalue-[
{
"text": "stackoverflow",
"datetime": "2010-07-27T05:41:52.763Z",
"obj1": {
"id": 1,
"email": "sa#gmail.com",
"name": "bass"
},
"id": 1,
}
object value -
"obj1": {
"id": 1,
"email": "sa#gmail.com",
"name": "bass"
}