We Keep Coding

How to update many records with `dateAdd` in Java

With a collection of
[
{
"user": "abc",
"seconds": 1111,
"time": ISODate("2020-05-05T00:00:00Z")
},
{
"user": "abc",
"seconds": 2222,
"time": ISODate("2020-05-05T00:00:00Z")
}
]
I need to have another field, which adds the seconds to the time of individual record.
This seems to be possible with dateAdd which is added in version 5 (the database is version 5).
However, I am not able to achieve that with MongoDB Java driver 4.6.0.
var alerts = db.getCollection("alerts");
alerts.updateMany(eq("user", user),
set(new Field<>("time2",
new Document("$dateAdd",
new Document("startDate", "$time")
.append("unit", "second")
.append("amount", "$seconds"))
)));

Since mongoDB version 4.0, before $dateAdd of version 5.0, you can use $toDate:
db.collection.update({},
[
{
$set: {
time: {
$toDate: {
$add: [
{$toLong: "$time"},
{$multiply: ["$seconds", 1000]}
]
}
}
}
}
],
{multi: true})
See how it works on the playground example

Jayway JsonPath filtering with predicates

I've recently taken up Jayway JsonPath and I've had trouble with how the inpath filtering works.
So my JSON looks like this:
At the top I have shareables. These shareables have an array called user, which contains an ID and a name, and they also contain an item called dataset, which can contain any json.
These shareables can exist within the dataset as well.
My working JSON looks like this:
{
"shareable": {
"user": [
{
"ID": 1,
"Name": "Bob"
},
{
"ID": 2,
"Name": "Charles"
}
],
"dataSet": [
{
"insulinMeasurement":
{
"timestamp": "Tuesday Morning",
"measurement": 174,
"unit": "pmol/L"
}
},
{
"insulinMeasurement":
{
"timestamp": "Tuesday Noon",
"measurement": 80,
"unit": "pmol/L"
}
},
{ "shareable": {
"user": [
{
"ID": 3,
"Name": "Jim"
}
],
"dataSet": [
{
"insulinMeasurement":
{
"timestamp": "Tuesday Evening",
"measurement": 130,
"unit": "pmol/L"
}
}
]
}
},
{ "unshareable": {
"user": [
{
"ID": 2,
"Name": "Bob"
}
],
"dataSet": [
{
"insulinMeasurement":
{
"timestamp": "Tuesday Night",
"measurement": 130,
"unit": "pmol/L"
}
}
]
}
}
]
}
}
So what I want is, all shareables that have a user with a certain ID. So I figured the path I would use would look like this:
$..shareable[ ?(#.user[*].ID == 1 )]
which here has a hardcoded ID. This returns nothing while
$..shareable[ ?(#.user[0].ID == 1 )]
returns any shareable where the first ID is 1.
I also tried something along the lines of
$..shareable[ ?(#.user[?(#.ID == 1)]
which I figure should return any shareable that has a user with an ID of 1.
Am I going about this the wrong way? Do I need to somehow iterate through the user objects that exist?

Well, I figured it out, so if anyone stumbles across this, the query should look as follows:
$..shareable[?( " + user + " in #.user[*].ID )]
where user is just the int of the userId. Basically the right hand side creates a list of all IDs that shareable contains, and checks if the requested ID exists therein.

Completion Suggester in elasticsearch in mutifield

I'm using elasticsearch for the first time. I'm trying to use completion suggester in multi-field key, although I don't see any error but I don't get the response.
Mapping creation:
PUT /products5/
{
"mappings":{
"products" : {
"properties" : {
"name" : {
"type":"text",
"fields":{
"text":{
"type":"keyword"
},
"suggest":{
"type" : "completion"
}
}
}
}
}
}
}
Indexing:
PUT /products5/product/1
{
"name": "Apple iphone 5"
}
PUT /products5/product/2
{
"name": "iphone 4 16GB"
}
PUT /products5/product/3
{
"name": "iphone 3 SS 16GB black"
}
PUT /products5/product/4
{
"name": "Apple iphone 4 S 16 GB white"
}
PUT /products5/product/5
{
"name": "Apple iphone case"
}
Query:
POST /products5/product/_search
{
"suggest":{
"my-suggestion":{
"prefix":"i",
"completion":{
"field":"name.suggest"
}
}
}
}
Output:
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 0,
"max_score": 0,
"hits": []
},
"suggest": {
"my-suggestion": [
{
"text": "i",
"offset": 0,
"length": 1,
"options": []
}
]
}
}
Please guide me what is the mistake, I tried every possible options.

From the first perspective this looks accurate. Probably the reason why you don't have correct response is that you added documents in the index before you created mapping in the index. And documents are not indexed according to the mapping you specified

I have found an issue in your mapping name. There is an inconsistency between name of the mapping and value which you specifies in the url when you're creating new documents. You create a mapping in the index with the name products. And when you add new documents you're specifying product as a name of the mapping of your index and it doesn't end with s. You have a typo.

ElasticSearch: Update multi fields in Script Java Plugin

I saw this snippet where update_by_query can update source directly
POST twitter/_update_by_query
{
"script": {
"inline": "ctx._source.likes++",
"lang": "painless"
},
"query": {
"term": {
"user": "kimchy"
}
}
}
instead of using painless, I wrote a native script plugin in Java because of my complex business logic.
{
"subtotal": 1000,
"markup": 2,
"total": 2000,
"items": [
{
"subtotal": 100,
"markup": 2,
"total": 200
},
{
"subtotal": 500,
"markup": 2,
"total": 1000
}
]
}
User can set markup value in the application. If user change markup to 3, I want to update markup and total field including the ones in nested object. (NOTE: I can't use painless because in my case, the logic is more complicated than just multiplies those fields. That's why I use Java)
// my plugin code
public Object run() {
// change field value of "markup"
// change field value of "total"
return true;
}
My Code is almost similar with https://github.com/imotov/elasticsearch-native-script-example/blob/master/src/main/java/org/elasticsearch/examples/nativescript/script/TFIDFScoreScript.java
I was trying with source().put("markup", 3) but I kept getting NullPointerException
ElasticSearch Version: 5.0.0
Thank you

elasticsearch - Return the tokens of a field

How can I have the tokens of a particular field returned in the result
For example, A GET request
curl -XGET 'http://localhost:9200/twitter/tweet/1'
returns
{
"_index" : "twitter",
"_type" : "tweet",
"_id" : "1",
"_source" : {
"user" : "kimchy",
"postDate" : "2009-11-15T14:12:12",
"message" : "trying out Elastic Search"
}
}
I would like to have the tokens of '_source.message' field included in the result

There is also another way to do it using the following script_fields script:
curl -H 'Content-Type: application/json' -XPOST 'http://localhost:9200/test-idx/_search?pretty=true' -d '{
"query" : {
"match_all" : { }
},
"script_fields": {
"terms" : {
"script": "doc[field].values",
"params": {
"field": "message"
}
}
}
}'
It's important to note that while this script returns the actual terms that were indexed, it also caches all field values and on large indices can use a lot of memory. So, on large indices, it might be more useful to retrieve field values from stored fields or source and reparse them again on the fly using the following MVEL script:
import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;
import java.io.StringReader;
// Cache analyzer for further use
cachedAnalyzer=(isdef cachedAnalyzer)?cachedAnalyzer:doc.mapperService().documentMapper(doc._type.value).mappers().indexAnalyzer();
terms=[];
// Get value from Fields Lookup
//val=_fields[field].values;
// Get value from Source Lookup
val=_source[field];
if(val != null) {
tokenStream=cachedAnalyzer.tokenStream(field, new StringReader(val));
CharTermAttribute termAttribute = tokenStream.addAttribute(CharTermAttribute);
while(tokenStream.incrementToken()) {
terms.add(termAttribute.toString())
};
tokenStream.close();
}
terms
This MVEL script can be stored as config/scripts/analyze.mvel and used with the following query:
curl 'http://localhost:9200/test-idx/_search?pretty=true' -d '{
"query" : {
"match_all" : { }
},
"script_fields": {
"terms" : {
"script": "analyze",
"params": {
"field": "message"
}
}
}
}'

If you mean the tokens that have been indexed you can make a terms facet on the message field. Increase the size value in order to get more entries back, or set to 0 to get all terms.
Lucene provides the ability to store the term vectors, but there's no way to have access to it with elasticsearch by now (as far as I know).
Why do you need that? If you only want to check what you're indexing you can have a look at the analyze api.

Nowadays, it's possible with the Term vectors API:
curl http://localhost:9200/twitter/_termvectors/1?fields=message
Result:
{
"_index": "twitter",
"_id": "1",
"_version": 1,
"found": true,
"took": 0,
"term_vectors": {
"message": {
"field_statistics": {
"sum_doc_freq": 4,
"doc_count": 1,
"sum_ttf": 4
},
"terms": {
"elastic": {
"term_freq": 1,
"tokens": [
{
"position": 2,
"start_offset": 11,
"end_offset": 18
}
]
},
"out": {
"term_freq": 1,
"tokens": [
{
"position": 1,
"start_offset": 7,
"end_offset": 10
}
]
},
"search": {
"term_freq": 1,
"tokens": [
{
"position": 3,
"start_offset": 19,
"end_offset": 25
}
]
},
"trying": {
"term_freq": 1,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 6
}
]
}
}
}
}
}
Note: Mapping types (here: tweets) have been removed in Elasticsearch 8.x (see migration guide).