We Keep Coding

Updating elasticsearch entities with bulk

I have this database data as below (ES 7.xx) version
{
"id":"1234",
"expirationDate":"17343234234",
"paths":"http:localhost:9090",
"work":"software dev",
"family":{
"baba":"jams",
"mother":"ela"
}
},
{
"id":"00021",
"expirationDate":"0123234",
"paths":"http:localhost:8080",
"work":"software engi",
"family":{
"baba":"stev",
"mother":"hela"
}
}
how can i update the entity which its expirationDate smaller than current Time? to be the current time for example:
the id 00021 is expired because its expiration date is smaller than today then it should updated to current time.
something like void updateExpiredEntity(List<ids> ids,Long currentTime) suing void bulkUpdate(List<UpdateQuery> queries, BulkOptions bulkOptions, IndexCoordinates index);
Please provide me some code implementation
is it correct like this?
public void update(UUID id,Long currentDate) {
UpdateQuery updateQuery = UpdateQuery.builder(id.toString()).withRouting("expirationDate=currentDate")
.build();
elasticsearchTemplate.bulkUpdate(List.of(updateQuery), IndexCoordinates.of("index"));
}
}

If you are using Elasticsearch 7.xx, I will assume that you have use Spring Data Elasticsearch version 4.0.x that comes with Spring boot 2.3.x. Since it's the version that support Elasticsearch 7.xx.
There're many things that have change in this Spring Data Elasticsearch version. Update document by query is one of them. Unlike before that we autowired ElasticsearchTemplate, we now have to use ElasticsearchRestTemplate and RestHighLevelClient instead.
In your case if you might want to use RestHighLevelClient to update document by query. Assume that you stored expirationDate as number mapping type in seconds unit then the code that you have asked for should look like this.
public class ElasticsearchService {
#Autowired
private ElasticsearchRestTemplate elasticsearchRestTemplate;
#Autowired
private RestHighLevelClient highLevelClient;
public void updateExpireDateDemo() throws IOException {
String indexName = "test";
Date currentDate = new Date();
Long seconds = (Long) (currentDate.getTime() / 1000);
UpdateByQueryRequest request = new UpdateByQueryRequest(indexName);
request.setQuery(new RangeQueryBuilder("expirationDate").lte(seconds));
Script updateScript = new Script(
ScriptType.INLINE, "painless",
"ctx._source.expirationDate=" + seconds + ";",
Collections.emptyMap());
request.setScript(updateScript);
highLevelClient.updateByQuery(request, RequestOptions.DEFAULT);
}
}
I'm not quite get why you really need to use the bulkUpdate but if that's the case then. You have to query the record that need to be update from Elasticsearch to get and id of each document first. Then you can update with list of UpdateQuery. So your code will look like this.
#Service
public class ElasticsearchService {
#Autowired
private ElasticsearchRestTemplate elasticsearchRestTemplate;
public void updateExpireDateByBulkDemo() throws IOException {
String indexName = "test";
Date currentDate = new Date();
Long seconds = (Long) (currentDate.getTime() / 1000);
List<UpdateQuery> updateList = new ArrayList();
RangeQueryBuilder expireQuery = new RangeQueryBuilder("expirationDate").lte(seconds);
NativeSearchQuery query = new NativeSearchQueryBuilder().withQuery(expireQuery).build();
SearchHits<Data> searchResult = elasticsearchRestTemplate.search(query, Data.class, IndexCoordinates.of(indexName));
for (SearchHit<Data> hit : searchResult.getSearchHits()) {
String elasticsearchDocumentId = hit.getId();
updateList.add(UpdateQuery.builder(elasticsearchDocumentId).withScript("ctx._source.expirationDate=" + seconds + ";").build());
}
if (updateList.size() > 0) {
elasticsearchRestTemplate.bulkUpdate(updateList, IndexCoordinates.of(indexName));
}
}
}
However, this only update first page of the search result. If you require to update every record matched your query then you have to use searchScroll method in oder to get every document id instead.

Is there any way to write custom or native queries in Java JPA (DocumentDbRepository) while firing a query to azure-cosmosdb?

Connected to azure-cosmosdb and able to fire default queries like findAll() and findById(String Id). But I can't write a native query using #Query annotation as the code is not considering it. Always considering the name of the function in respository class/interface. I need a way to fire a custom or native query to azure-cosmos db. ?!
Tried with #Query annotation. But not working.
List<MonitoringSessions> findBySessionID(#Param("sessionID") String sessionID);
#Query(nativeQuery = true, value = "SELECT * FROM MonitoringSessions M WHERE M.sessionID like :sessionID")
List<MonitoringSessions> findSessions(#Param("sessionID") String sessionID);
findBySessionID() is working as expected. findSessions() is not working. Below root error came while running the code.
Caused by: org.springframework.data.mapping.PropertyReferenceException: No property findSessions found for type MonitoringSessions

Thanks for the response. I got what I exactly wanted from the below link. Credit goes to Author of the link page.
https://cosmosdb.github.io/labs/java/technical_deep_dive/03-querying_the_database_using_sql.html
public class Program {
private final ExecutorService executorService;
private final Scheduler scheduler;
private AsyncDocumentClient client;
private final String databaseName = "UniversityDatabase";
private final String collectionId = "StudentCollection";
private int numberOfDocuments;
public Program() {
// public constructor
executorService = Executors.newFixedThreadPool(100);
scheduler = Schedulers.from(executorService);
client = new AsyncDocumentClient.Builder().withServiceEndpoint("uri")
.withMasterKeyOrResourceToken("key")
.withConnectionPolicy(ConnectionPolicy.GetDefault()).withConsistencyLevel(ConsistencyLevel.Eventual)
.build();
}
public static void main(String[] args) throws InterruptedException, JSONException {
FeedOptions options = new FeedOptions();
// as this is a multi collection enable cross partition query
options.setEnableCrossPartitionQuery(true);
// note that setMaxItemCount sets the number of items to return in a single page
// result
options.setMaxItemCount(5);
String sql = "SELECT TOP 5 s.studentAlias FROM coll s WHERE s.enrollmentYear = 2018 ORDER BY s.studentAlias";
Program p = new Program();
Observable<FeedResponse<Document>> documentQueryObservable = p.client
.queryDocuments("dbs/" + p.databaseName + "/colls/" + p.collectionId, sql, options);
// observable to an iterator
Iterator<FeedResponse<Document>> it = documentQueryObservable.toBlocking().getIterator();
while (it.hasNext()) {
FeedResponse<Document> page = it.next();
List<Document> results = page.getResults();
// here we iterate over all the items in the page result
for (Object doc : results) {
System.out.println(doc);
}
}
}
}

MissingPropertyException: No such property: request for class: Script1

How to run a groovy script in java from a servlet? With the below code, I have a
MissingPropertyException: No such property: request for class: Script1
This is my script
<script type="server/groovy">
import com.daimler.Car
def id = request.getParameter("id")
car = Car.lookup(id)
</script>
This is my Java code
public void runScript(HttpServletRequest request) {
Script script = groovyShell.parse("def id = request.getParameter(\"id\")\n" +
" car = Car.lookup(id)");
Map bindings = script.getBinding().getVariables();
bindings.put("id",1);
Object ret = script.run(); //a+b+3
//and if you changed variables in script you can get their values
Object aAfter = bindings.get("car");
}

You need to pass the HttpServletRequest request object in the bindings as well:
Map bindings = script.getBinding().getVariables();
bindings.put("id",1);
bindings.put("request", request);
Object ret = script.run(); //a+b+3 (...)

Update model - Play 2.4.3

I have the following code in my update function.
I use the form that I pre-fill with data but when I submit the form i CANNOT update the existing object even if I set client.id manually.
Is it correct to use the commented code in order to first get the object from DB and then update every field one by one?
Form<Client> clientForm = form(Client.class).bindFromRequest();
Map<String, String> data = clientForm.data();
if (clientForm.hasErrors()) {
return badRequest(addEditClient.render("Edit Client", "Update Client", clientID, false, clientForm));
} else {
// do stuff
Client client = clientForm.get();
client.contact = User.findByID("1");
for(ClientTask ct : client.taskSet) {
Task t = Task.findByID(ct.t.id);
ct.t = t;
}
client.id = clientID;
//Client c = Client.findByID(clientID);
//c.name = "test12";
//etc...
//c.save();
client.save();
return GO_HOME;
}

Hibernate update without select

I've a question on hibernate operation: update.
Here a bit of code:
Campaign campaign = campaignDAO.get(id);
campaign.setStatus(true);
campaignDAO.update(campaign);
If I just have all the data of the campaign object, is there any way to perform an update without perform the first select (campaignDAO.get(id)) ?
Thanks,
Alessio

HQL will definitely help you.
In order to maintain the separation of concerns, you can add a more specialized method in you DAO object:
public void updateStatusForId(long id, boolean status){
//provided you obtain a reference to your session object
session.createQuery("UPDATE Campaign SET status = " + status + " WHERE id = :id").setParameter("id", id).executeUpdate();
//flush your session
}
Then you could simply call this method from your business method. You can check the generated SQL statements inside the logs of your app by setting the show_sql hibernate property to true.

You can use session.load(). It will not hit the database. Here you can find its details and example code.

I had worte a extension to solve this issue in Nhibernate
how to use!
first of all you need enable dynamic-update="true"
using (ISession session = sessionFactory.OpenSession())
{
Customer c1 = new Customer();
c1.CustomerID = c.CustomerID;
session.Mark(c1);
// c1.Name = DateTime.Now.ToString();
c1.Phone = DateTime.Now.ToString();
//需要开启动态更新
session.UpdateDirty(c1);
session.Flush();
}
UpdateExtension.cs
public static class UpdateExtension
{
static readonly Object NOTNULL = new Object();
public static void UpdateDirty<TEntity>(this ISession session, TEntity entity)
{
SessionImpl implementor = session as SessionImpl;
EntityEntry entry = implementor.PersistenceContext.GetEntry(entity);
if (entry == null)
{
throw new InvalidOperationException("找不到对应的实例，请先使用Mask方法标记");
}
IEntityPersister persister = entry.Persister;
// 如果某列不可以为空，新的Entity里也不想更新他。
// 那么LoadState 里的值应该和Entity 中的值相同
Object[] CurrentState = entry.Persister.GetPropertyValues(entity, EntityMode.Poco);
Object[] LoadedState = entry.LoadedState;
int[] dirtys = persister.FindDirty(CurrentState
, LoadedState
, entity
, (SessionImpl)session);
if (dirtys == null || dirtys.Length == 0)
{
return;
}
persister.Update(entry.Id
, CurrentState
, dirtys
, true
, LoadedState
, entry.Version
, entity
, entry.RowId
, (SessionImpl)session);
implementor.PersistenceContext.RemoveEntry(entity);
implementor.PersistenceContext.RemoveEntity(entry.EntityKey);
session.Lock(entity, LockMode.None);
// 防止(implementor.PersistenceContext.EntityEntries.Count == 0)
}
public static void Mark<TEntity>(this ISession session, TEntity entity)
{
session.Lock(entity, LockMode.None);
}
}
here is update sql
command 0:UPDATE Customers SET Phone = #p0 WHERE CustomerID = #p1;#p0 = '2014/12/26 0:12:56' [Type: String (4000)], #p1 = 1 [Type: Int32 (0)]
Only update Phone column .
event Name property can not be null. we can work very well.