We are using the spring-data-elasticsearch project to interface with our elasticsearch clusters, and have been using it now for around a year. Recently, we moved to elasticsearch 5.x (from 2.x) where we now have the "keyword" datatype.
I would like to index these keywords as lowercase values, which I know can be done with field normalizers. I can't find anywhere in the documentation or online where I can add a normalizer to a field through the annotation based mapping.
E.g
#Field(type = FieldType.keyword, <some_other_param = some_normalizer>)
Is this something that can be done? I know that we can use JSON based mapping definitions as well, so I will fall back to that option if needed, but would like to be able to do it this way if possible.
Any help would be very appreciated!
Since the pull request of #xhaggi has been merged (spring-data-elasticsearch 3.1.3+ or Spring Boot 2.1.1), we have a normalizer field in the #Field annotation.
To use it, we need:
declare a #Field or an #InnerField with params type = FieldType.Keyword, normalizer = "%NORMALIZER_NAME%"
add #Setting(settingPath = "%PATH_TO_NORMALIZER_JSON_FILE%") at the class level.
put the normalizer mapping into a json file at %PATH_TO_NORMALIZER_JSON_FILE%
Example of usage
FYI, for anyone looking at this, the answer is there is not a way to do this at this time.
You can do this, however, by creating your mappings file as JSON in the Elasticsearch format. See:
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html
You can then create that JSON file and link it to your Domain model with.
#Mapping(mappingPath = "some/path/mapping.json")
Note that this is not, in my experience, compatible with the provided annotation based mapping for fields.
There is a pending issue https://jira.spring.io/browse/DATAES-492 waiting for review.
Related
I'm using simple Java classes which are the schema for my mongo db table.
There are several frameworks for serialization/ deserialization to/ from JSON and CRUD operations for mongo (I've looked into Jackson serializer and Morphia).
But none of them seems to provide a solution for handling changes:
Let's say I have this class as my schema:
Class Person
{
String name;
int age;
String occupation;
}
In my code, I will probably use a setter in some place for age:
Person newDbEntry = new Person();
newDbEntry.setAge(45);
newDbEntry.setOccupation("Carpenter");
Now let's say that at some point of the development process, it was decided that age field name needs to be changed to "theAge", and it was also decided to remove "occupation" field from this collection completely- to a new table.
The problem that I'm faced with is that all my queries look like this:
JsonObject query = new JsonObject().put("age",new JsonObject().put("$gte", 22);
In other words, all field names are written in queries as Strings (and also in all other mongo APIs- update, findAndModify, etc).
I'm looking for a way to "bind" all mentions of the field "age" in my code with the POJO class- so that when something in the POJO schema changes (like renaming this field), I'll have (ideally) compiler errors in all queries that mention this field.
As it currently stands, changes to schema cause no compiler errors and - more critically - usually no runtime errors. The old string query just quietly returns no results, or something similar. This makes changes to the schema very hard to implement.
How should this be done correctly?
Here's the solution that I ended up using:
Project lombok now supports FieldNames generation:
https://projectlombok.org/features/experimental/FieldNameConstants
So instead of using the name hardcoded as string:
serviceRepository.setField(id, “service.serviceName”, “newName”);
I use:
serviceRepository.setField(id, ConnectivityServiceDetails.Fields.service + "." + ConnectivityService.Fields.serviceName, “newName”);
This way, when we search in Intellij for usages of this field (or try to refactor it), it will find those places also automatically.
I'm writing a demo app using Spring & MongoDB as a database.
My main domain class looks like:
#Document
public class Person {
#Id
private String id;
//Some other fields
private DBObject additionalData;
}
The key is that additionalData is a subdocument with no schema specified, it is kind of user-defined JSON. But when I am parsing this json (using (DBObject) JSON.parse(value) expression), it is stored as a string in MongoDB, and I need it to be a nested document structure.
Searched for couple of hours, found no solution. Any ideas?
I'm not really sure of the expected result of casting the result of
JSON.parse(value)
to DBObject, which is an interface, not a class.
Try casting the result to an implementation of DBObject BasicDBObject (or BasicDBList), or a Map<String, Object> as mentioned in the comments (it is also an interface, but it does work).
If you're working with Spring Data Rest, you will probably not need to deserialize "manually", Spring will do it for you. Check this answer for a basic example of what to do.
Having data with no schema specified may not be the best idea around (mongodb saves you from doing it at the database level, but you should do it at the application level), but I use similar tricks in production, and you can somehow make it work.
Given an #Entity declared with the following fields:
#Id
private String idgeo;
private String isoCtryCd;
private String randomField;
with the default spring configuration I get resource paths ending with .../{idgeo}.
Is there an option in the spring configuration to use other (unique) fields as the resource path ending? In my example it'd be .../{isoCtryCd}
Thank you!
Actually this feature will be introduced in Spring Data Rest 2.5. Currently there is a 2.5.0.M1 milestone release containing this feature.
This part of the documentation shows how to use a different entity attribute for item resource uris.
http://docs.spring.io/spring-data/rest/docs/2.5.0.M1/reference/html/#_customizing_item_resource_uris
Good morning,
I´m trying to combine regular expression with Spring data mongodb repository using Query annotation.
What I want is search one substring inside one string attribute of my mongo document.
I have been looking in google and here, but I did not find anything elegant, and I was wondering if Spring data has something official about this using the repositories.
Regards.
It seems like an old question, so maybe you've already had a solution but here how I handled the same issue :
#Query(value = "{'title': {$regex : ?0, $options: 'i'}}")
Foo findByTitleRegex(String regexString);
using the /?0/ notation won't work since Spring Data places a String value with quotes
I would like to change the entity property from String to long. I have seen Nick answering similar problem in Change IntegerProperty to FloatProperty of existing AppEngine DataStore but I am writing in Java and need some code example since I don't know anything about the mapreduce.
e.g. we want to change userId from String to Long of this class.
I also would like to get advice on my thinking of storing date in long instead of String so that the time information can be consumed readily from android, GWT and more(over Rest Json or RPC). Right now, GWT does not have Jodatime and it has limited support of Java.util.Date and parsing.
If you really want to convert from String to Long, I can't see any other choice except to write a conversion snippet using raw GAE, eg:
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.PreparedQuery;
import com.google.appengine.api.datastore.Query;
Query q = new Query (Task.class.getName());
PreparedQuery pq = DatastoreServiceFactory.getDatastoreService ().prepare (q);
for (Entity entity : pq.asIterable ())
{
String orig = entity.getProperty ("userId").toString ();
entity.removeProperty ("userId");
entity.setProperty ("userId", Long.parseLong (orig));
}
What is your persistence interface? JDO (mine), JPA, Objectify, Twig, raw GAE/J API? I don't think that many people can give you a code example without knowing this.
Also, please give the code extract of your existing (an underlying date-time, I presume) persistent entity including the data member you talk about.
Your class is using JPA not JDO. The latest version (v2.x) of the GAE JPA plugin allows persistence of (java.util.)Date as Long or String. This wouldn't cater for your migration of data (see the reply by Jonathan for that) but would allow you to persist future Date fields as Long. IIRC you can specify the "jdbcType" (DataNucleus extension annotation) as INTEGER would trigger that.