We Keep Coding

Elasticsearch: sort by a numberic field

I'm coding a console app to Insert data from JSOIN files into Elasticsearch 7.5.1. The _id field should be an int and act like auto-increment. To make that happen, before inserting I get the last ID inserted and increment 1.
My problem is that I'm having trouble getting the last ID, because the ordering is happening on a string. This is what I mean: if you have 10 items and the last ID is 10, when querying and sorting it will return 9.
This is my query when using Postman:
GET my_index/_search
{
"size": 1,
"query": {
"match_all": {}
},
"sort": [{
"_id": {
"order": "desc"
}
}
]
}
And my Java function using their client:
private static String getLastElasticSearchId(String index)
{
RestHighLevelClient client = getElasticSearchClient();
SearchRequest searchRequest = new SearchRequest(index);
SearchSourceBuilder b = new SearchSourceBuilder();
b.query(QueryBuilders.matchAllQuery());
b.sort(new FieldSortBuilder("_id").order(SortOrder.DESC));
b.from(0);
b.size(1);
searchRequest.source(b);
try {
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
if(hits.getTotalHits().value > 0){
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
return hit.getId();
}
}
else {
return "0";
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
client.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return null;
}
Both return 9 as the last ID, even having another item in the index with 10 as ID.
I cannot apply string padding in the IDs as answered here.
How can I achieve what I need?

Try this:
GET my_index/_search
{
"size": 1,
"query": {
"match_all": {}
},
"sort" : {
"_script" : {
"type" : "number",
"script" : {
"lang": "painless",
"source": "Integer.parseInt(doc['_id'].value)"
},
"order" : "desc"
}
}
}
Hope this helps

Getting this error whilt trying to parse JSON data W/System.err: org.json.JSONException: No value for customers

I am trying to retreive customer object from my rest api. I generated the api using spring data jpa. I have used volley to retrive the information from the api. I can't tell what i did wrong. As i am new to android i don't have much idea. can some one help me to parse the customer object from my Json api.
my api looks like this:
{
"_embedded": {
"customers": [
{
"firstName": "Alexander",
"lastName": "arnold",
"email": "trentarnold#liverpool.com",
"password": "cornertakenquickly",
"_links": {
"self": {
"href": "http://localhost:8080/api/customers/1"
},
"customer": {
"href": "http://localhost:8080/api/customers/1"
}
}
},
{
"firstName": "test",
"lastName": "tester",
"email": "dulalsujan911#gmail.com",
"password": "12345678",
"_links": {
"self": {
"href": "http://localhost:8080/api/customers/2"
},
"customer": {
"href": "http://localhost:8080/api/customers/2"
}
}
}
]
},
"_links": {
"self": {
"href": "http://localhost:8080/api/customers{?page,size,sort}",
"templated": true
},
"profile": {
"href": "http://localhost:8080/api/profile/customers"
}
},
"page": {
"size": 20,
"totalElements": 2,
"totalPages": 1,
"number": 0
}
}
my code looks like this i am using volley :
// connects to the api and stores the retrived JSON values in the respective list
public void connectToApi(){
// initialize lists
emailList = new ArrayList<>();
passwordList = new ArrayList<>();
final String BASE_URL ="http://192.168.1.67:8080/api/customers";
// final String BASE_URL ="http://10.0.2.2:8080/api/customers";
// creating a request ques for HTTP request
RequestQueue requestQueue = Volley.newRequestQueue(this);
// Setting HTTP GET request to retrieve the data from the SERVER
JsonObjectRequest objectRequest = new JsonObjectRequest(Request.Method.GET,BASE_URL
,null
, new Response.Listener<JSONObject>() {
#Override
public void onResponse(JSONObject response) {
try {
JSONArray jsonArray = response.getJSONArray("customers");
for(int i=0; i<jsonArray.length();i++){
JSONObject customer = jsonArray.getJSONObject(i);
emailList.add(customer.getString("email"));
passwordList.add(customer.getString("password"));
}
} catch (Exception e) {
e.printStackTrace();
}
}
} , new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
Log.e("REST error",error.toString() );
}
}
);
requestQueue.add(objectRequest);
}

Do this :
#Override
public void onResponse(JSONObject response) {
try {
JSONObject json = new JSONObject(response);
JSONObject json_embedded = json.getJSONObject("_embedded");// need to access JSONObject("_embedded")
JSONArray jsonArray = json_embedded.getJSONArray("customers"); // then get JSONARRAY
for(int i=0; i<jsonArray.length();i++){
JSONObject customer = jsonArray.getJSONObject(i);
emailList.add(customer.getString("email"));
passwordList.add(customer.getString("password"));
}
} catch (Exception e) {
e.printStackTrace();
}
}
NOTE : your json array (customers) is in _embedded that's why
it is showing exception.

You need to access first to _embedded object.
try {
JSONObject embedded = response.getJSONObject("_embedded");
JSONArray jsonArray = embedded.getJSONArray("customers");
for(int i=0; i<jsonArray.length();i++){
JSONObject customer = jsonArray.getJSONObject(i);
emailList.add(customer.getString("email"));
passwordList.add(customer.getString("password"));
}
} catch (Exception e) {
e.printStackTrace();
}

Convert Json payload to java objects

I need to implement below json payload to java object including JSONArray and JSONObject
I tried using below java code in order to implement the same
DWYT_productOrderResponse CreateProductOrderResponse = new DWYT_productOrderResponse();
ResultHeader rslthdr = new ResultHeader();
JSONObject productOrder = new JSONObject();
JSONArray orderIt = new JSONArray();
JSONObject produt = new JSONObject();
// List<ProductCharacteristic> ProductChara = new ArrayList<ProductCharacteristic>();
Map ProductChara = new LinkedHashMap();
//Map orderItem = new LinkedHashMap();
// OrderIteam[] orderItem;
if (1 == 1) {
String output = null;
try {
productOrder.put("externalId", CreateOrderReq.getProductOrder().getExternalId());
productOrder.put("description", CreateOrderReq.getProductOrder().getDescription());
ProductChara.put("name", "CustomerName");
ProductChara.put("value", CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getCustomerName());
ProductChara.put("name", "CustomerContactNumber");
ProductChara.put("value", CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getCustomerContactNumber());
ProductChara.put("name", "CRMAddress");
ProductChara.put("value", CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getCRMAddress());
ProductChara.put("name", "CustomerEmail");
ProductChara.put("value", CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getCustomerEmail());
ProductChara.put("name", "CustomerGovetID");
ProductChara.put("value", CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getCustomerGovtID());
ProductChara.put("name", "ODBNo");
ProductChara.put("value", CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getODBNO());
produt.put("productCharacteristic", ProductChara);
produt.put("product", produt);
orderIt.put(produt);
productOrder.put("orderItem", orderIt);
productOrder.put("productOrder", productOrder);
} catch (JSONException e) {
e.printStackTrace();
}
}
here is the result of json payload of the above code
{
"externalId":"CRM000000912",
"description":"Activation Request",
"orderItem":[
{
"productCharacteristic":{
"name":"CustomerName",
"value":"xxxx",
"name":"CustomerContactNumber",
"value":"5600000232",
"name":"CRMAddress",
"value":"xxxxxxx",
"name":"CustomerEmail",
"value":"xxx#xx.xxx",
"name":"CustomerGovetID",
"value":"1223232323232322",
"name":"ODBNo",
"value":"RYH-736834-JKS"
}
}
]
}
here is the json payload that I want to parse
{
"externalId": " 12345678",
"orderItem": [{
"product": {
"productCharacteristic": [{
"name": "CustomerName",
"value": "someone"
}, {
"name":
"CustomerContactNumber",
"value": "13524687502"
}, {
"name": "CRMAddress",
"value": "xxxxxx"
}, {
"name": "CustomerEmail ",
"value": "XXX"
}, {
"name": " CustomerGovet.ID",
"value": "XXX"
}, {
"name": " ODBNo.",
"value": "XXX"
}
]
}
}
]
}
Is there any easy way or java code that can help me to get the expected output.

SOLVED: now I got the expected json payload after adding some enhancement to my code
try {
productOrder.put("externalId", CreateOrderReq.getProductOrder().getExternalId());
productOrder.put("description", CreateOrderReq.getProductOrder().getDescription());
ProductChara.put("name","CustomerName");
ProductChara.put("value",CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getCustomerName());
ProductCharaArray.put(ProductChara);
ProductChara.put("name","CustomerContactNumber");
ProductChara.put("value",CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getCustomerContactNumber());
ProductCharaArray.put(ProductChara);
ProductChara.put("name","CRMAddress");
ProductChara.put("value",CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getCRMAddress());
ProductCharaArray.put(ProductChara);
ProductChara.put("name","CustomerEmail");
ProductChara.put("value",CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getCustomerEmail());
ProductCharaArray.put(ProductChara);
ProductChara.put("name","CustomerGovetID");
ProductChara.put("value",CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getCustomerGovtID());
ProductCharaArray.put(ProductChara);
ProductChara.put("name","ODBNo");
ProductChara.put("value",CreateOrderReq.getProductOrder().getOrderItem().getProduct().getProductCharacteristic().getODBNO());
ProductCharaArray.put(ProductChara);
produt.put("productCharacteristic",ProductCharaArray);
orderItm.put("product",produt);
orderItemArray.put(orderItm);
productOrder.put("orderItem", orderItemArray);
System.out.println(productOrder.toString());
} catch (JSONException e) {
e.printStackTrace();
}

Recieving empty json file when using Postman in API

I'm new to development part using API and JSON file, so be patient with me. I'm trying to implement a Rest API in java where I need to send a Post request to an URL to process but when I tried it in my localhost, the result is that what I receive is just empty Json file.
I was using the dependency from org.json.simple.JSONObject, but I need to change the dependency to org.json.JSONObject. I know that they are two different library and that's why I am a bit stuck. I looked on the forum and on the internet but I didn't find a solution for my own problem. If it is possible I want to also ask if there is a way to convert a String to a JSON.
Here is the main class.
public class DataService {
public static JSONObject processData(JSONObject jsonObject) {
System.out.println(jsonObject);
System.out.println(jsonObject.toString());
ObjectMapper mapper = new ObjectMapper();
mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
Data data = new Data();
try {
data = mapper.readValue(jsonObject.toString(), Data.class);
} catch (IOException e) {
e.printStackTrace();
}
List<DataJson> timeserie = data.getData();
List<Double> values = new ArrayList<Double>();
DataJson inter;
for (int i = 0; i<timeserie.size(); i++){
inter = timeserie.get(i);
values.add(inter.getValue());
}
int EmbeddingDimension;
EmbeddingDimension = data.getEmbeddingDimension();
data.setResult(DynamicProperties.PermEn(values, EmbeddingDimension));
String url = "http://localhost:8080/crosscpp/toolbox/test";
OkHttpClient client = new OkHttpClient();
ObjectMapper objectMapper = new ObjectMapper();
RequestBody body = null;
try {
body = RequestBody.create(
MediaType.parse("application/json; charset=utf-8"), objectMapper.writeValueAsString(data));
} catch (JsonProcessingException e) {
e.printStackTrace();
}
Request request = new Request.Builder()
.url(url)
.post(body)
.build();
Call call = client.newCall(request);
try {
Response response = call.execute();
String result = response.body().string();
JSONObject json = null;
/*JSONParser parser = new JSONParser();
JSONObject json = null;
try {
json = (JSONObject) parser.parse(result);
} catch (ParseException e) {
e.printStackTrace();
}*/ //Look for another solution.
return json;
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
The Json file I will send.
{
"inputAeonSubscriptionUrl": "xxxx",
"outputAeonPublicationUrl": "xxxx",
"EmbeddingDimension": 3,
"offerId": "xxxxxx",
"measurement-channel-id": "1",
"id": "xxxxxx",
"submissionDate": {
"min": "2019-04-09",
"max": "2019-05-07"
},
"travelDates": {
"min": "2019-05-13",
"max": "2019-05-17"
},
"travelledDuration": {
"min": 1,
"max": 2
},
"geoBoundingBox": {
"latitude-max": 51.507561,
"latitude-min": 51.497715,
"longitude-min": 7.482349,
"longitude-max": 7.500885
},
"data": [
{
"value": 1,
"timestamp": "2019-04-09"
},
{
"value": 3,
"timestamp": "2019-04-09"
},
{
"value": 2,
"timestamp": "2019-04-09"
},
{
"value": 1,
"timestamp": "2019-04-09"
},
{
"value": 2,
"timestamp": "2019-04-10"
},
{
"value": 3,
"timestamp": "2019-04-10"
},
{
"value": 2,
"timestamp": "2019-04-10"
}
]
}
I am expecting a result where it send back the received JSON file with an added attribute where it shows the process done on the values.

querying on field missing from some document in elasticsearch

I have a special need where in I need to query elastic search for a field say F1.
F1 is not there in all documents . My search should be for missing F1 documents and F1 with some value.
F1 is missing or F1 = 'A1'
I am not sure if this is possible in elastic search. Here is my query and I know that is not right query. Would appreciate somebody could correct it or the Java program.
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"bool": {
"should": {
"terms": {
"F1": [
"Some Value"
]
}
}
}
},
{
"missing": {
"field": "F1"
}
}
]
}
}
}
}
}
This is my java code building the query.
public QueryBuilder getQueryBuilder() {
QueryBuilder qb;
//what do we want to return if there is no request? Nothing or all.
if (filters == null || filters.isEmpty()) {
qb = QueryBuilders.matchAllQuery();
} //build our filtered query using given filters in our request
else {
Set<String> keys = filters.keySet();
BoolFilterBuilder boolFilterBuilder = FilterBuilders.boolFilter();
for (String key : keys) {
Set<Object> values = filters.get(key);
if (values == null || values.toString().isEmpty()) {
continue; //Ignore nothing to do.
}
if (key.equalsIgnoreCase(Constants.MISSING_FILTER)) {
Iterator i = values.iterator();
while (i.hasNext()) {
boolFilterBuilder.must(FilterBuilders.missingFilter((String) i.next()));
}
} else {
boolFilterBuilder.must(buildShouldQuery(key, values));
}
}
qb = QueryBuilders.filteredQuery(createSearchQueryBuilder(), boolFilterBuilder);
}
return qb;
}

Try whit this.
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"should": [
{ "terms": { "F1": ["Some Value"] },
{ "missing": { "field": "F1" } }
]
}
}
}
}
}