We Keep Coding

How do you test listCollections in Mongo

I have a spring-boot project, with spring-data-mongo dependency. Everything is working great. I have a service which basically does a listCollections on one of the DB and clears the content of all the collections in that DB.
Code:
public void clearContentsOfAllCollections() {
MongoDatabase db = this.mongoTemplate.getMongoDbFactory().getDb("any-db-name");
LOGGER.info("=================================================================");
LOGGER.info("Clearing collections in DB - {}", db.getName());
MongoIterable<String> collectionNames = db.listCollectionNames();
for (final String collectionName : collectionNames) {
LOGGER.info("Clearing collection - {}", collectionName);
db.getCollection(collectionName).deleteMany(new Document());
}
LOGGER.info("Successfully cleared DB - {}", db.getName());
LOGGER.info("=================================================================");
}
When I try to write unit test, this is what I have
#Test
public void dropDBTest() {
SimpleMongoClientDbFactory simpleMongoClientDbFactory = Mockito.mock(SimpleMongoClientDbFactory.class);
MongoDatabase mongoDatabase = Mockito.mock(MongoDatabase.class);
Mockito.when(mongoTemplate.getMongoDbFactory()).thenReturn(simpleMongoClientDbFactory);
Mockito.when(simpleMongoClientDbFactory.getDb("db-name"))
.thenReturn(mongoDatabase);
// How do I convert this array list into a Mongo iterable
List<String> collectionList = Arrays.asList("collection-1", "collection-2");
}
The problem is - I do not know how I can return the content of collectionList, when I have to mock database.listCollectionNames() (from package com.mongodb.client).
Morever, the iterable.iterator() is of type MongoCursor. How do I test this? Am I missing something?

public class MongoDBTest {
#Test
public void testListCollectionNames() {
// Set up the mock MongoDatabase
MongoDatabase mockDatabase = Mockito.mock(MongoDatabase.class);
// Set up the mock MongoIterable<String>
MongoIterable<String> mockIterable = Mockito.mock(MongoIterable.class);
Mockito.when(mockDatabase.listCollectionNames()).thenReturn(mockIterable);
final var collectionNames = new ArrayList<>();
collectionNames.add("testCollection")
doReturn(collectionNames).when(mockIterable).into(new ArrayList<>());
// Get the list of collection names
List<String> collectionNames = mockDatabase.listCollectionNames().into(new ArrayList<>());
// Assert that the test collection is in the list
assertEquals(true, collectionNames.contains("testCollection"));
}
}

Updating elasticsearch entities with bulk

I have this database data as below (ES 7.xx) version
{
"id":"1234",
"expirationDate":"17343234234",
"paths":"http:localhost:9090",
"work":"software dev",
"family":{
"baba":"jams",
"mother":"ela"
}
},
{
"id":"00021",
"expirationDate":"0123234",
"paths":"http:localhost:8080",
"work":"software engi",
"family":{
"baba":"stev",
"mother":"hela"
}
}
how can i update the entity which its expirationDate smaller than current Time? to be the current time for example:
the id 00021 is expired because its expiration date is smaller than today then it should updated to current time.
something like void updateExpiredEntity(List<ids> ids,Long currentTime) suing void bulkUpdate(List<UpdateQuery> queries, BulkOptions bulkOptions, IndexCoordinates index);
Please provide me some code implementation
is it correct like this?
public void update(UUID id,Long currentDate) {
UpdateQuery updateQuery = UpdateQuery.builder(id.toString()).withRouting("expirationDate=currentDate")
.build();
elasticsearchTemplate.bulkUpdate(List.of(updateQuery), IndexCoordinates.of("index"));
}
}

If you are using Elasticsearch 7.xx, I will assume that you have use Spring Data Elasticsearch version 4.0.x that comes with Spring boot 2.3.x. Since it's the version that support Elasticsearch 7.xx.
There're many things that have change in this Spring Data Elasticsearch version. Update document by query is one of them. Unlike before that we autowired ElasticsearchTemplate, we now have to use ElasticsearchRestTemplate and RestHighLevelClient instead.
In your case if you might want to use RestHighLevelClient to update document by query. Assume that you stored expirationDate as number mapping type in seconds unit then the code that you have asked for should look like this.
public class ElasticsearchService {
#Autowired
private ElasticsearchRestTemplate elasticsearchRestTemplate;
#Autowired
private RestHighLevelClient highLevelClient;
public void updateExpireDateDemo() throws IOException {
String indexName = "test";
Date currentDate = new Date();
Long seconds = (Long) (currentDate.getTime() / 1000);
UpdateByQueryRequest request = new UpdateByQueryRequest(indexName);
request.setQuery(new RangeQueryBuilder("expirationDate").lte(seconds));
Script updateScript = new Script(
ScriptType.INLINE, "painless",
"ctx._source.expirationDate=" + seconds + ";",
Collections.emptyMap());
request.setScript(updateScript);
highLevelClient.updateByQuery(request, RequestOptions.DEFAULT);
}
}
I'm not quite get why you really need to use the bulkUpdate but if that's the case then. You have to query the record that need to be update from Elasticsearch to get and id of each document first. Then you can update with list of UpdateQuery. So your code will look like this.
#Service
public class ElasticsearchService {
#Autowired
private ElasticsearchRestTemplate elasticsearchRestTemplate;
public void updateExpireDateByBulkDemo() throws IOException {
String indexName = "test";
Date currentDate = new Date();
Long seconds = (Long) (currentDate.getTime() / 1000);
List<UpdateQuery> updateList = new ArrayList();
RangeQueryBuilder expireQuery = new RangeQueryBuilder("expirationDate").lte(seconds);
NativeSearchQuery query = new NativeSearchQueryBuilder().withQuery(expireQuery).build();
SearchHits<Data> searchResult = elasticsearchRestTemplate.search(query, Data.class, IndexCoordinates.of(indexName));
for (SearchHit<Data> hit : searchResult.getSearchHits()) {
String elasticsearchDocumentId = hit.getId();
updateList.add(UpdateQuery.builder(elasticsearchDocumentId).withScript("ctx._source.expirationDate=" + seconds + ";").build());
}
if (updateList.size() > 0) {
elasticsearchRestTemplate.bulkUpdate(updateList, IndexCoordinates.of(indexName));
}
}
}
However, this only update first page of the search result. If you require to update every record matched your query then you have to use searchScroll method in oder to get every document id instead.

Is there any way to write custom or native queries in Java JPA (DocumentDbRepository) while firing a query to azure-cosmosdb?

Connected to azure-cosmosdb and able to fire default queries like findAll() and findById(String Id). But I can't write a native query using #Query annotation as the code is not considering it. Always considering the name of the function in respository class/interface. I need a way to fire a custom or native query to azure-cosmos db. ?!
Tried with #Query annotation. But not working.
List<MonitoringSessions> findBySessionID(#Param("sessionID") String sessionID);
#Query(nativeQuery = true, value = "SELECT * FROM MonitoringSessions M WHERE M.sessionID like :sessionID")
List<MonitoringSessions> findSessions(#Param("sessionID") String sessionID);
findBySessionID() is working as expected. findSessions() is not working. Below root error came while running the code.
Caused by: org.springframework.data.mapping.PropertyReferenceException: No property findSessions found for type MonitoringSessions

Thanks for the response. I got what I exactly wanted from the below link. Credit goes to Author of the link page.
https://cosmosdb.github.io/labs/java/technical_deep_dive/03-querying_the_database_using_sql.html
public class Program {
private final ExecutorService executorService;
private final Scheduler scheduler;
private AsyncDocumentClient client;
private final String databaseName = "UniversityDatabase";
private final String collectionId = "StudentCollection";
private int numberOfDocuments;
public Program() {
// public constructor
executorService = Executors.newFixedThreadPool(100);
scheduler = Schedulers.from(executorService);
client = new AsyncDocumentClient.Builder().withServiceEndpoint("uri")
.withMasterKeyOrResourceToken("key")
.withConnectionPolicy(ConnectionPolicy.GetDefault()).withConsistencyLevel(ConsistencyLevel.Eventual)
.build();
}
public static void main(String[] args) throws InterruptedException, JSONException {
FeedOptions options = new FeedOptions();
// as this is a multi collection enable cross partition query
options.setEnableCrossPartitionQuery(true);
// note that setMaxItemCount sets the number of items to return in a single page
// result
options.setMaxItemCount(5);
String sql = "SELECT TOP 5 s.studentAlias FROM coll s WHERE s.enrollmentYear = 2018 ORDER BY s.studentAlias";
Program p = new Program();
Observable<FeedResponse<Document>> documentQueryObservable = p.client
.queryDocuments("dbs/" + p.databaseName + "/colls/" + p.collectionId, sql, options);
// observable to an iterator
Iterator<FeedResponse<Document>> it = documentQueryObservable.toBlocking().getIterator();
while (it.hasNext()) {
FeedResponse<Document> page = it.next();
List<Document> results = page.getResults();
// here we iterate over all the items in the page result
for (Object doc : results) {
System.out.println(doc);
}
}
}
}

How do you execute a MongoDB query stored as string in Java?

I'm kind of new to the MongoDB Java driver and I was wondering how you could execute a query stored as a string. Is this the best way to execute them, or what would be a better approach?
I've stumbled across the piece of the below on another stackoverflow thread, but haven't been able to get anything useful out of it. The output does not contain the result of the query at all.
The code I'm running right now:
#Test
public void testExecuteStoredQueries() {
String code = "db.getCollection('users').find({})";
final BasicDBObject command = new BasicDBObject();
String formattedCode = String.format("function() { return %s ; }", code);
System.out.println("Formatted code:");
System.out.println(formattedCode);
command.put("eval", formattedCode);
Document result = DbEngine.getInstance().getDatabase().runCommand(command);
System.out.println(result.toJson());
}
Summarized output:
{
"retval": {
"_mongo": "....",
"_db": "...",
"_collection": "...",
"_ns": "cezy.users",
"_query": {},
"_fields": null,
"_limit": 0,
"_skip": 0,
"_batchSize": 0,
"_options": 0,
"_cursor": null,
"_numReturned": 0,
"_special": false
},
"ok": 1
}

I use morphia when i have to deal with objects. As when you retrieve the data from MongoDb, for the long values you get extended Json instead of Json Response. Parsing Extended Json could be a trouble and might break the code. As Gson doesn't support the conversion from Extended Json to Json.
private void createDatastore(boolean createIndexes) {
Morphia morphia = new Morphia();
morphia.map(classname.class);
datastore = morphia.createDatastore(mongoClient, databaseName);
if (createIndexes) {
datastore.ensureIndexes();
}
}
#Override
public Datastore getDatastore() {
return this.datastore;
}
#Test
public void testExecuteStoredQueries() {
String code = "db.getCollection('users').find({})";
String formattedCode = String.format("function() { return %s ; }", code);
final BasicDBObject basicObject = new BasicDBObject(new BasicDBObject("$in", formattedCode));
Query<ClassName> query = getDatastore().createQuery(<Classname>.class).filter("_eval", basicObject);
List<Classname> List = query.asList();
//if you want to access each object and perform some task
List.forEach((cursor) -> {
//perform your task
});
}

Removing the function creation and adding ".toArray()" pretty much solved the problem.
#Test
public void testExecuteStoredQueries() {
String code = "db.users.find({}).toArray();";
final BasicDBObject command = new BasicDBObject();
command.put("eval", code);
Document result = DbEngine.getInstance().getDatabase().runCommand(command);
System.out.println(result.toJson());
assertNotNull(result.get("retval"));
}
The array is in the "retval" field of the response.

How to save an Object containing List of Objects in MongoDB collection

I have a Java Class named Status and it consists of 3 fields namely : String Email, String Status, ArrayList(Comment) commentList. Comment is another Java class consisting of 2 String field. I have worked with MySQL and i know how i could save this Status object in database table. Now i am learning MongoDB and i need this Status object to store in my db collection. How do i do it? As of now i have tried the following and failed to store the Status object. Can anyone please help me with it?
I have tried the following :
public static MongoClient getMongoConnection() {
MongoClient mongo = null;
try {
mongo = new MongoClient("localhost", 27017);
} catch (Exception e) {
e.printStackTrace();
}
return mongo;
}
public static void main(String[] args) {
MongoClient mongo = getMongoConnection();
DB db = mongo.getDB("myTestdatabase");
DBCollection myReviews = db.getCollection("myStatus");
BasicDBObject obj = new BasicDBObject();
List<Object> commentList = new BasicDBList();
commentList.add(new Comment("Looks like a nice status", "email#gmail.com"));
commentList.add(new Comment("This is a nice status", "email#gmail.com"));
Status status = new Status();
status.setStatus("This is my new Status");
status.setEmail("email#gmail.com");
obj.put("status", status.getStatus());
obj.put("email", status.getEmail());
obj.put("comments", commentList);
myReviews.insert(obj);
}
I get this error.
org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class com.Comment.

mongo.getDB is depreciated. Use the below to create your DB. Make sure you import bson as well.
MongoClient mongo = new MongoClient("localhost", 27017);
MongoDatabase db = mongo.getDatabase("myDB");
MongoCollection items = db.getCollection("myCollection");
Document document = new Document(); // Create the document to be inserted to the DB
document.put("First Name", "John");
document.put("Last Name", "Smith");
items.insertOne(document); // Insert document to DB