I want to get the Content in the "hits Array" beginning with "target" until "text" from this String JsonData as bellow ,
I get this JSONData from elasticsearch server,
String holdedEntity =
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [
{
"_index" : "try1",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"target" : {
"br_id" : 0,
"wo_id" : 2,
"process" : [
"element 1",
"element 2"
]
},
"explanation" : {
"an_id" : 1311,
"pa_name" : "micha"
},
"text" : "hello world"
}
}
]
}
}
Result should look like this :
String result =
{
"target" : {
"br_id" : 0,
"wo_id" : 2,
"process" : [
"element 1",
"element 2"
]
},
"explanation" : {
"an_id" : 1311,
"pa_name" : "micha"
},
"text" : "hello world"
}
I tried this , but it is not giving the right result as above, please any Suggestion, i will be thankfull ,
JSONObject jsonObj = new JSONObject(holdedEntity);//convert the holdedEntity into JSONObject.
JSONObject jsonObjectContent = jsonObj.getJSONObject("target");//trying to get the content starting from "target" until the "text" .
String result = jsonObjectContent.toString(); //converting the jsonObjectContent toString.
But it could not recognise the Field "target" and throw me this Failure ,
JSONObject["target"] not found
Please any Advice.
thx
There is no target field at the top level; you have to get the hits field, then the hits field of that, then the appropriate element of that (since it is an array); finally, the source element of that will get you your desired result.
Related
How can i configure ElasticSearch to delete logs passed 1 month,
or if there is no sush conf, how can i call api delete for this purpose from java
Thank you
Have you tried using the Delete by query API?
This post also discusses how to go about doing so for 10 days. You could try it with 30 days instead.
Reading over these, you should get an idea of what to do. I do not know the exact answer but I hope this at least helps.
This is the sample index with 2 documents.
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "sample",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"created" : "06/18/2021"
}
},
{
"_index" : "sample",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"created" : "06/17/2021"
}
}
]
}
}
Now you can use delete by query
POST /sample/_delete_by_query
{
"query": {
"bool": {
"must": [
{
"range": {
"created": {
"lte": "now-30d/d",
"format": "MM/dd/yyyy"
}
}
}
]
}
}
}
Output:
{
"took" : 8,
"timed_out" : false,
"total" : 2,
"deleted" : 2,
"batches" : 1,
"version_conflicts" : 0,
"noops" : 0,
"retries" : {
"bulk" : 0,
"search" : 0
},
"throttled_millis" : 0,
"requests_per_second" : -1.0,
"throttled_until_millis" : 0,
"failures" : [ ]
}
You will see Total: 2 and Deleted: 2
Hope this helps #java_dev
I have an index where each of my objects has status field which can have some predefined values. I want to fetch all of them which has statusINITIATED, UPDATED, DELETED, any match with these and hence created this query by java which I got printing on console, using Querybuilder and nativeSearchQuery, executing by ElasticsearchOperations:
{
"bool" : {
"must" : [
{
"terms" : {
"status" : [
"INITIATED",
"UPDATED",
"DELETED"
],
"boost" : 1.0
}
}
],
"adjust_pure_negative" : true,
"boost" : 1.0
}
}
I have data in my index with 'INITIATED' status but not getting anyone with status mentioned in the query. How to fix this query, please?
If you need anything, please let me know.
Update: code added
NativeSearchQueryBuilder nativeSearchQueryBuilder=new NativeSearchQueryBuilder();
QueryBuildersingleQb=QueryBuilders.boolQuery().must(QueryBuilders.termsQuery("status",statusList));
Pageable pageable = PageRequest.of(0, 1, Sort.by(Defs.START_TIME).ascending());
FieldSortBuilder sort = SortBuilders.fieldSort(Defs.START_TIME).order(SortOrder.ASC);
nativeSearchQueryBuilder.withQuery(singleQb);
nativeSearchQueryBuilder.withSort(sort);
nativeSearchQueryBuilder.withPageable(pageable);
nativeSearchQueryBuilder.withIndices(Defs.SCHEDULED_MEETING_INDEX);
nativeSearchQueryBuilder.withTypes(Defs.SCHEDULED_MEETING_INDEX);
NativeSearchQuery searchQuery = nativeSearchQueryBuilder.build();
List<ScheduledMeetingEntity> scheduledList=elasticsearchTemplate.queryForList(searchQuery, ScheduledMeetingEntity.class);
Update 2: sample data:
I got this from kibana query on this index:
"hits" : [
{
"_index" : "index_name",
"_type" : "type_name",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"createTime" : "2021-03-03T13:09:59.198",
"createTimeInMs" : 1614755399198,
"createdBy" : "user1#domain.com",
"editTime" : "2021-03-03T13:09:59.198",
"editTimeInMs" : 1614755399198,
"editedBy" : "user1#domain.com",
"versionId" : 1,
"id" : "1",
"meetingId" : "47",
"userId" : "129",
"username" : "user1#domain.com",
"recipient" : [
"user1#domain.com"
],
"subject" : "subject",
"body" : "hi there",
"startTime" : "2021-03-04T07:26:00.000",
"endTime" : "2021-03-04T07:30:00.000",
"meetingName" : "name123",
"meetingPlace" : "placeName",
"description" : "sfsafsdafsdf",
"projectName" : "",
"status" : "INITIATED",
"failTry" : 0
}
}
]
Confirm your mapping:
GET /yourIndexName/_mapping
And see if it is valid
Your mapping needs to have keyword for TermsQuery to work.
{
"status": {
"type" "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
ES can automatically do the mapping for you (without you having to do it yourself) when you first push a document. However you probably have finer control if you do the mapping yourself.
Either way, you need to have keyword defined for your status field.
=====================
Alternative Solution: (Case Insensitive)
If you have a Field named (status), and the values you want to search for are (INITIATED or UPDATED, or DELETED).
Then you can do it like this:
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery()
.must(createStringSearchQuery());
public QueryBuilder createStringSearchQuery(){
QueryStringQueryBuilder queryBuilder = QueryBuilders.queryStringQuery(" INITIATED OR UPDATED OR DELETED ");
queryBuilder.defaultField("status");
return queryBuilder;
}
Printing the QueryBuilder:
{
"query_string" : {
"query" : "INITIATED OR UPDATED OR DELETED",
"default_field" : "status",
"fields" : [ ],
"type" : "best_fields",
"default_operator" : "or",
"max_determinized_states" : 10000,
"enable_position_increments" : true,
"fuzziness" : "AUTO",
"fuzzy_prefix_length" : 0,
"fuzzy_max_expansions" : 50,
"phrase_slop" : 0,
"escape" : false,
"auto_generate_synonyms_phrase_query" : true,
"fuzzy_transpositions" : true,
"boost" : 1.0
}
}
Below is my feature file.
Scenario 1: Getting all AppsCount
When we request for all apps then we get 4 apps and the application list is sent json form:
"[
{ '_id' : 'app1', 'Name' : 'App 1', 'Type' : 1, 'Rules' : [ { '_id' : 'Rule1', 'FilterCriteria' : 8 }, { '_id' : 'Rule2', 'FilterCriteria' : 4 } ], 'Email' : 'test#dell.com', 'IsSlackEnabled' : false, 'IsEmailEnabled' : true, 'IsSMSEnabled' : false, 'IsPhoneEnabled' : false, 'IsNonProdEnabled' : false, 'CreatedBy' : 'test#dell.com' },
{ '_id' : 'app2', 'Name' : 'App 2', 'Type' : 2, 'Rules' : [ { '_id' : 'Rule3', 'FilterCriteria' : 8 } ], 'Email' : 'test1#dell.com', 'IsSlackEnabled' : true, 'SlackChannel' : 'testChannel', 'IsEmailEnabled' : false, 'IsSMSEnabled' : false, 'IsPhoneEnabled' : true, 'IsNonProdEnabled' : false, 'CreatedBy' : 'test1#dell.com' },
{ '_id' : 'app3', 'Name' : 'App 3', 'Type' : 3, 'Rules' : [ { '_id' : 'Rule4', 'FilterCriteria' : 7 } ], 'Email' : 'test3#dell.com', 'IsSlackEnabled' : true, 'IsEmailEnabled' : false, 'SlackChannel' : 'testChannel2', 'IsSMSEnabled' : true, 'IsPhoneEnabled' : false, 'IsNonProdEnabled' : false, 'CreatedBy' : 'test3#dell.com' },
{ '_id' : 'app4', 'Name' : 'App 4', 'Type' : 1, 'Rules' : [], 'Email' : 'test4#dell.com', 'IsSlackEnabled' : false, 'IsEmailEnabled' : false, 'IsSMSEnabled' : false, 'IsPhoneEnabled' : true, 'IsNonProdEnabled' : false, 'CreatedBy' : 'test4#dell.com' }
]"
But this is giving issue, as we need double quotes when writing a json string
So I tried the below feature file.
Scenario 2: Getting all AppsCount
When we request for All Apps then we get 4 apps and the application list is sent json form:
"[
{ "_id" : "app1", "Name" : "App 1", "Type" : 1, "Rules" : [{ "_id" : "Rule1", "FilterCriteria" : 8 }, { "_id" : "Rule2", "FilterCriteria" : 4 } ], "Email" : "test#dell.com", "IsSlackEnabled" : false, "IsEmailEnabled" : true, "IsSMSEnabled" : false, "IsPhoneEnabled" : false, "IsNonProdEnabled" : false, "CreatedBy" : "test#dell.com" },
{ "_id" : "app2", "Name" : "App 2", "Type" : 2, "Rules" : [ { "_id" : "Rule3", "FilterCriteria" : 8 } ], "Email" : "test1#dell.com", "IsSlackEnabled" : true, "SlackChannel" : "testChannel", "IsEmailEnabled" : false, "IsSMSEnabled" : false, "IsPhoneEnabled" : true, "IsNonProdEnabled" : false, "CreatedBy" : "test1#dell.com" },
{ "_id" : "app3", "Name" : "App 3", "Type" : 3, "Rules" : [ { "_id" : "Rule4", "FilterCriteria" : 7 } ], "Email" : "test3#dell.com", "IsSlackEnabled" : true, "IsEmailEnabled" : false, "SlackChannel" : "testChannel2", "IsSMSEnabled" : true, "IsPhoneEnabled" : false, "IsNonProdEnabled" : false, "CreatedBy" : "test3#dell.com" },
{ "_id" : "app4", "Name" : "App 4", "Type" : 1, "Rules" : [], "Email" : "test4#dell.com", "IsSlackEnabled" : false, "IsEmailEnabled" : false, "IsSMSEnabled" : false, "IsPhoneEnabled" : true, "IsNonProdEnabled" : false, "CreatedBy" : "test4#dell.com" }
] "
But when creating a step definition, it comes out with each field's value coming as it's own argument.
I am aware of the
| field1 | field2 |
| value1 | value2 |
solution, but my data is too huge for that and was wondering, if there is a way to pass JSON string directly.
Create a file text.json ..and put json input in the file .. place this file in class path..
IN feature file place the file name in this case- text.json..
in class file stepdef read the filename.... look for the this file in class path..
laod this file using clasloader and use objectmapper to map this json.
I am interacting with ES using Java rest client from a spring boot app and my response is JSON object.
See sample below.
What would would be the most effective way to remove certain unwanted fields ?
For example : Remove "took","successful".""_source.accountname"
Since this is a nested json object and there is high possibility of getting a larger response,I am trying to find out the most effective way.I am worried that looping through each nested object migh chew up time
{
"took" : 63,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1000,
"max_score" : null,
"hits" : [ {
"_index" : "bank",
"_type" : "account",
"_id" : "0",
"sort": [0],
"_score" : null,
"_source" : {"account_number":0,"balance":16623,"firstname":"Bradshaw","lastname":"Mckenzie","age":29,"gender":"F","address":"244 Columbus Place","employer":"Euron","email":"bradshawmckenzie#euron.com","city":"Hobucken","state":"CO"}
}, {
"_index" : "bank",
"_type" : "account",
"_id" : "1",
"sort": [1],
"_score" : null,
"_source" : {"account_number":1,"balance":39225,"firstname":"Amber","lastname":"Duke","age":32,"gender":"M","address":"880 Holmes Lane","employer":"Pyrami","email":"amberduke#pyrami.com","city":"Brogan","state":"IL"}
}, ...
]
}
}
I have gone through various threads, but couldn't find the particular answer in python.
I have a json file
{
"StoreID" : "123",
"Status" : 3,
"data" : {
"Response" : {
"section" : "25",
"elapsed" : 277.141,
"products" : {
"prd_1": {
"price" : 11.99,
"qty" : 10,
"upc" : "0787493"
},
"prd_2": {
"price" : 9.99,
"qty" : 2,
"upc" : "0763776"
},
"prd_3": {
"price" : 29.99,
"qty" : 8,
"upc" : "9948755"
}
},
"type" : "Tagged"
}
}
}
I need to convert this json file into the format below, by changing json object 'products' into an array form.
{
"StoreID" : "123",
"Status" : 3,
"data" : {
"Response" : {
"section" : "25",
"elapsed" : 277.141,
"products" : [
{
"price" : 11.99,
"qty" : 10,
"upc" : "0787493"
},
{
"price" : 9.99,
"qty" : 2,
"upc" : "0763776"
},
{
"price" : 29.99,
"qty" : 8,
"upc" : "9948755"
}
],
"type" : "Tagged"
}
}
}
Is there any good way to do it in python. Mostly I saw people are using java, but not in python. Can you please let me know a way to do it in python.
Just get the values() of products dictionary and that will give you an array of values. Code below works from me assuming your json is in file1.txt Also note
import json
with open('file1.txt') as jdata:
data = json.load(jdata)
d = data
d["data"]["Response"]["products"] = d["data"]["Response"]["products"].values()
print(json.dumps(d))
output:
{"Status": 3, "StoreID": "123", "data": {"type": "Tagged", "Response": {"section": "25", "products": [{"price": 9.99, "upc": "0763776", "qty": 2}, {"price": 29.99, "upc": "9948755", "qty": 8}, {"price": 11.99, "upc": "0787493", "qty": 10}], "elapsed": "277.141"}}}
Would something like this work for you?
import json
import copy
a = json.load(open("your_data.json", "r"))
b = copy.deepcopy(a)
t = a.get('data').get('Response').get('products')
b['data']['Response']['products'] = t.values() # Originally was: [t[i] for i in t]
You can give back JSON with json.dumps(b)