I'm coding a console app to Insert data from JSOIN files into Elasticsearch 7.5.1. The _id field should be an int and act like auto-increment. To make that happen, before inserting I get the last ID inserted and increment 1.
My problem is that I'm having trouble getting the last ID, because the ordering is happening on a string. This is what I mean: if you have 10 items and the last ID is 10, when querying and sorting it will return 9.
This is my query when using Postman:
GET my_index/_search
{
"size": 1,
"query": {
"match_all": {}
},
"sort": [{
"_id": {
"order": "desc"
}
}
]
}
And my Java function using their client:
private static String getLastElasticSearchId(String index)
{
RestHighLevelClient client = getElasticSearchClient();
SearchRequest searchRequest = new SearchRequest(index);
SearchSourceBuilder b = new SearchSourceBuilder();
b.query(QueryBuilders.matchAllQuery());
b.sort(new FieldSortBuilder("_id").order(SortOrder.DESC));
b.from(0);
b.size(1);
searchRequest.source(b);
try {
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
if(hits.getTotalHits().value > 0){
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
return hit.getId();
}
}
else {
return "0";
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
client.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return null;
}
Both return 9 as the last ID, even having another item in the index with 10 as ID.
I cannot apply string padding in the IDs as answered here.
How can I achieve what I need?
Try this:
GET my_index/_search
{
"size": 1,
"query": {
"match_all": {}
},
"sort" : {
"_script" : {
"type" : "number",
"script" : {
"lang": "painless",
"source": "Integer.parseInt(doc['_id'].value)"
},
"order" : "desc"
}
}
}
Hope this helps
Related
I'm using Elasticsearch 7.5.1 and its Java client. I'm inserting in my index using ints as _id and I have to increment it manually. For that, I need to retrieve the last ID inserted. I know how to do it using PostMan:
GET my_index/_search
{
"size": 1,
"query": {
"match_all": {}
},
"sort": [{
"_id": {
"order": "desc"
}
}
]
}
But I don't know how to do it using their Java client. I'm trying with the code below but it's returning one of those auto-generated IDs. My index has only 3 items with Ids 1,2 and 3.
private static String getLastElastticSearchId(String index)
{
RestHighLevelClient client = getElasticSearchClient();
SearchRequest searchRequest = new SearchRequest();
SearchSourceBuilder b = new SearchSourceBuilder();
b.query(QueryBuilders.matchAllQuery());
b.sort(new FieldSortBuilder("_id").order(SortOrder.DESC));
b.from(0);
b.size(1);
searchRequest.source(b);
try {
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
if(hits.getTotalHits().value > 0){
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
return hit.getId();
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
client.close();
} catch (IOException e) {
e.printStackTrace();
}
}
How to can I change my code to get the last ID?
I just found what was wrong. I had to tell in which index to look for when instantiating the request:
SearchRequest searchRequest = new SearchRequest(index);
I'm new to development part using API and JSON file, so be patient with me. I'm trying to implement a Rest API in java where I need to send a Post request to an URL to process but when I tried it in my localhost, the result is that what I receive is just empty Json file.
I was using the dependency from org.json.simple.JSONObject, but I need to change the dependency to org.json.JSONObject. I know that they are two different library and that's why I am a bit stuck. I looked on the forum and on the internet but I didn't find a solution for my own problem. If it is possible I want to also ask if there is a way to convert a String to a JSON.
Here is the main class.
public class DataService {
public static JSONObject processData(JSONObject jsonObject) {
System.out.println(jsonObject);
System.out.println(jsonObject.toString());
ObjectMapper mapper = new ObjectMapper();
mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
Data data = new Data();
try {
data = mapper.readValue(jsonObject.toString(), Data.class);
} catch (IOException e) {
e.printStackTrace();
}
List<DataJson> timeserie = data.getData();
List<Double> values = new ArrayList<Double>();
DataJson inter;
for (int i = 0; i<timeserie.size(); i++){
inter = timeserie.get(i);
values.add(inter.getValue());
}
int EmbeddingDimension;
EmbeddingDimension = data.getEmbeddingDimension();
data.setResult(DynamicProperties.PermEn(values, EmbeddingDimension));
String url = "http://localhost:8080/crosscpp/toolbox/test";
OkHttpClient client = new OkHttpClient();
ObjectMapper objectMapper = new ObjectMapper();
RequestBody body = null;
try {
body = RequestBody.create(
MediaType.parse("application/json; charset=utf-8"), objectMapper.writeValueAsString(data));
} catch (JsonProcessingException e) {
e.printStackTrace();
}
Request request = new Request.Builder()
.url(url)
.post(body)
.build();
Call call = client.newCall(request);
try {
Response response = call.execute();
String result = response.body().string();
JSONObject json = null;
/*JSONParser parser = new JSONParser();
JSONObject json = null;
try {
json = (JSONObject) parser.parse(result);
} catch (ParseException e) {
e.printStackTrace();
}*/ //Look for another solution.
return json;
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
The Json file I will send.
{
"inputAeonSubscriptionUrl": "xxxx",
"outputAeonPublicationUrl": "xxxx",
"EmbeddingDimension": 3,
"offerId": "xxxxxx",
"measurement-channel-id": "1",
"id": "xxxxxx",
"submissionDate": {
"min": "2019-04-09",
"max": "2019-05-07"
},
"travelDates": {
"min": "2019-05-13",
"max": "2019-05-17"
},
"travelledDuration": {
"min": 1,
"max": 2
},
"geoBoundingBox": {
"latitude-max": 51.507561,
"latitude-min": 51.497715,
"longitude-min": 7.482349,
"longitude-max": 7.500885
},
"data": [
{
"value": 1,
"timestamp": "2019-04-09"
},
{
"value": 3,
"timestamp": "2019-04-09"
},
{
"value": 2,
"timestamp": "2019-04-09"
},
{
"value": 1,
"timestamp": "2019-04-09"
},
{
"value": 2,
"timestamp": "2019-04-10"
},
{
"value": 3,
"timestamp": "2019-04-10"
},
{
"value": 2,
"timestamp": "2019-04-10"
}
]
}
I am expecting a result where it send back the received JSON file with an added attribute where it shows the process done on the values.
I am new to elastic search , i am reading the doc so far its good but i am not able to write a method to get by range, Below is a method that gets by ID which works perfectly but how can i do this to get a list of data that match the price range.
public Map<String, Object> getTourById(String id){
GetRequest getRequest = new GetRequest(INDEXTOUR, TYPETOUR, id);
GetResponse getResponse = null;
try {
getResponse = restHighLevelClient.get(getRequest);
} catch (java.io.IOException e){
e.printStackTrace();
e.getLocalizedMessage();
}
Map<String, Object> sourceAsMap = getResponse.getSourceAsMap();
return sourceAsMap;
}
Above method works fine now below is the method to get by range and return datas that matches the price
public Map<String, Object> getTourByPriceRange(int minPrice, int maxPrice) {
GetRequest getRequest = new GetRequest(INDEXTOUR, TYPETOUR, "requires an ID");
QueryBuilder qb = QueryBuilders
.rangeQuery("price")
.from(minPrice)
.to(maxPrice)
.includeLower(true)
.includeUpper(true);
GetResponse getResponse = null;
try {
getResponse = restHighLevelClient.get(getRequest);
} catch (java.io.IOException e) {
e.printStackTrace();
e.getLocalizedMessage();
}
Map<String, Object> sourceAsMap = getResponse.getSourceAsMap();
return sourceAsMap;
}
Only the query builder is accurate in the above method, the result i want to get is a bunch of data that price fall in the range given.
{
"departure": {
"city": "\u0438\u043d\u0441\u043a",
"date": "2018-08-10"
},
"extras": [],
"hotel": {
"beach": {
"distance": 0,
"type": "\u041f\u0435\u0447\u0430\u043d\u044b\u0439"
},
"country": "\u0413\u0440\u0446\u0438\u044f",
"distanceToAirport": 0,
"facilities": [
"Standard Without Balcony"
],
"food": "\u0422\u043e\u043b\u044c\u043a\u043e \u0437\u0430\u0432\u0442\u0440\u0430\u043a\u0438",
"photoUrl": "https://s1.tez-tour.com/hotel/7021893.jpg",
"regionName": "\u0425\u0430\u043b\u043a\u0438\u0434\u0438\u043a\u0438 - \u041a\u0430\u0441\u0441\u0430\u043d\u0434\u0440\u0430",
"stars": 4,
"title": "KALLIKRATEIA"
},
"nights": 8,
"people": 1,
"price": 2595 // i want to use price as parameters for the search method
}
Good job so far!! In order to issue a search request you need to use SearchRequest and not GetRequest which is meant to retrieve a single document by ID.
QueryBuilder qb = QueryBuilders
.rangeQuery("price")
.from(minPrice)
.to(maxPrice)
.includeLower(true)
.includeUpper(true);
SearchRequest searchRequest = new SearchRequest(INDEXTOUR);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(qb);
searchRequest.types(TYPETOUR);
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = null;
try {
searchResponse = restHighLevelClient.search(searchRequest);
} catch (java.io.IOException e) {
e.printStackTrace();
e.getLocalizedMessage();
}
I have a special need where in I need to query elastic search for a field say F1.
F1 is not there in all documents . My search should be for missing F1 documents and F1 with some value.
F1 is missing or F1 = 'A1'
I am not sure if this is possible in elastic search. Here is my query and I know that is not right query. Would appreciate somebody could correct it or the Java program.
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"bool": {
"should": {
"terms": {
"F1": [
"Some Value"
]
}
}
}
},
{
"missing": {
"field": "F1"
}
}
]
}
}
}
}
}
This is my java code building the query.
public QueryBuilder getQueryBuilder() {
QueryBuilder qb;
//what do we want to return if there is no request? Nothing or all.
if (filters == null || filters.isEmpty()) {
qb = QueryBuilders.matchAllQuery();
} //build our filtered query using given filters in our request
else {
Set<String> keys = filters.keySet();
BoolFilterBuilder boolFilterBuilder = FilterBuilders.boolFilter();
for (String key : keys) {
Set<Object> values = filters.get(key);
if (values == null || values.toString().isEmpty()) {
continue; //Ignore nothing to do.
}
if (key.equalsIgnoreCase(Constants.MISSING_FILTER)) {
Iterator i = values.iterator();
while (i.hasNext()) {
boolFilterBuilder.must(FilterBuilders.missingFilter((String) i.next()));
}
} else {
boolFilterBuilder.must(buildShouldQuery(key, values));
}
}
qb = QueryBuilders.filteredQuery(createSearchQueryBuilder(), boolFilterBuilder);
}
return qb;
}
Try whit this.
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"should": [
{ "terms": { "F1": ["Some Value"] },
{ "missing": { "field": "F1" } }
]
}
}
}
}
}
I am stuck here and would like to extract using java the second link of the facebook query below
{
"data": [
{
"attachment": {
"media": [
{
"photo": {
"images": [
{
"src": "https://fbcdn-photos-h-a.akamaihd.net/hphotos-ak-prn2/1508634_699393523428883_996610253_s.png"
},
{
"src": "https://fbcdn-sphotos-h-a.akamaihd.net/hphotos-ak-prn2/s720x720/1508634_699393523428883_996610253_n.png"
}
]
}
}
]
}
}
]
}
my code below, is obviously not working
try
{
List<JsonObject> queryResults = facebookClient.executeFqlQuery(query, JsonObject.class);
if(!queryResults.isEmpty())
{
JsonObject facebookPosturl_J = queryResults.get(0);
facebook_post = facebookPosturl_J.getString("src");
}
}
catch (Exception e){logger.warn("Unexpected error", e);}
Try calling:
facebookPosturl_J.getJsonArray("data").getJsonObject(0).getJsonObject("attachment").getJsonArray("media").getJsonObject(0).getJsonObject("photo").getJsonArray("images").getJsonObject(1).getString("src")