We Keep Coding

Lookup in MongoDB using spring data

I have a query that would be fairly simple using SQL databases. However, in MongoDB, I'm not sure on how to do it. I am building a notification system. I have a collection that stores notifications, and another that creates a document when the user has seen the notification, as such:
notifications collection:
_id: 1
content: "some content"
targetGroup: "somegroup"
seen-notification collection
_id: 1
notificationId: 1
userName: "johndoe"
I'm building an endpoint that should return all notifications in a specific group, that the user has not already seen. In a SQL pseudo-code, I'm trying to do:
SELECT
*
FROM
notifications AS n
INNER JOIN
seen-notification AS sn
ON n._id = sn.notificationId
WHERE
sn.notificationId IS NULL
AND n.targetGroup = "somegroup"
So, basically, a right join with a where clause. I'm using spring data with a MongoRepository. Right now, i'm making two different selects and iterating through them to remove the already seen. I've looked into mongo's $lookup and several SO questions regarding it, but I just couldn't get it work with spring data and MongoRepository. I'm fairly new to mongo with spring data, so maybe there's a way to accomplish this with $lookup and I just didn't figure out how.

try
https://mongoplayground.net/p/-dZHmP9yj-c
expects collection names of notifications and seenNotifications
#Aggregation(pipeline = {
"{$match: { targetGroup: ?0 } }",
"{$lookup: { from: \"seenNotifications\", localField: \"_id\", foreignField: \"notificationId\", as: \"seenNotificationsDocs\"}}",
"{$match: { seenNotificationsDocs: { $size: 0 } } }",
"{$project: { seenNotificationsDocs: 0 } }",
})
List<Notifications> findUnseenNotificationByTargetGroup(String targetGroup);
#Document("notifications")
public class Notifications {
#Id
private Integer id;
private String content;
private String targetGroup;
//getter & setter
}

Firebase data retrieval java

{
"Account1" :
{
Push_key(): { Carplate: "ABC1234" }
Push_key(): { Carplate: "ABC" }
Push_key(): { Carplate: "A" }
}
}
This is how the database looks like.
I would like to retrieve the third data which contains "A" alone ONLY.
I am using startAt() and endAt() for data retrieval:
Query query = ref.child("Account1").orderByChild("Carplate").startAt("A").endAt("A"+"\uf8ff");
But it returns all 3 records. (I think its due to they are all started at "A".)
Need help! Please!

You should look at the equalTo() method for this (from the doc):
The equalTo() method allows you to filter based on exact matches. As is the case with the other range queries, it will fire for each matching child node.
To adapt it to your query you might try:
Query query = ref.child("Account1").orderByChild("Carplate").equalTo("A");

How to build dynamic queries with Spring Data Redis Repositories?

I'm testing Redis with spring-data-redis using repositories like this:
public interface CreditCardRepository extends CrudRepository<CreditCard, String>{
List<CreditCard> findByIssuer(String issuer);
List<CreditCard> findByCreditNetwork(String creditNetwork);
List<CreditCard> findByCreditNetworkAndIssuer(String creditNetwork, String issuer);
}
Above methods will query over Redis structures like:
creditcard:creditNetwork:mastercard
creditcard:creditNetwork:visa
creditcard:issuer:company1
creditcard:issuer:company2
Right now my CreditCard object contains two attributes (issuer, network and the id), so it's easy to search objects like this:
private List<CreditCard> searchCardFromCache(CreditCardGetReq req) {
if (req.getIssuer() != null && req.getNetwork() != null) {
return ccRepository.findByIssuerAndCreditNetwork(req.getIssuer(), req.getNetwork().name());
}
if (req.getIssuer() != null) {
return ccRepository.findByIssuer(req.getIssuer());
}
if (req.getNetwork() != null) {
return ccRepository.findByCreditNetwork(req.getNetwork().name());
}
return null;
}
However, I don't like this code since I will have to create a combination of all the properties and will be very messy. In the future, I plan to have 15 properties so the 'if' chain is not possible.
I would like to ask you how can I create dynamic queries using spring-data-redis, so Redis can return the intersection based on the object properties in a better way than checking each property?
Have tried using MethodHandle by hardcoding (I previously deleted from the repository findByIssuerAndCreditNetwork) a method name that would be dynamic generated like this:
MethodType methodType = MethodType.methodType(cardList.getClass(), String.class, String.class);
// Dynamic create 'findByIssuerAndCreditNetwork'
MethodHandle methodHandle = MethodHandles.lookup().findVirtual(CreditCardRepository.class, "findByIssuerAndCreditNetwork", methodType);
But seems MethodHandle does not work since I got below error:
java.lang.NoSuchMethodException: no such method: com.creditcard.dao.CreditCardRepository.findByIssuerAndCreditNetwork(String,String)ArrayList/invokeInterface

Right now, there's no support to create dynamic queries. It sounds a bit as if Query by Example could be the thing you're looking for. Spring Data MongoDB and Spring Data JPA already implement Query by Example.
A query is created by the data store module to match an example domain object:
Person person = new Person();
person.setFirstname("Dave");
Example<Person> example = Example.of(person);
MongoRepository repo = …
List<Person> result = repo.findAll(example); // returns all objects that with Dave in firstname
Query by Example is not supported by Spring Data Redis right now but it should be possible to provide basic support.
I created a ticket DATAREDIS-605 to track the progress of this feature.

Spring Data Mongo: How to save batch ignoring all duplicate key errors?

I have the following domain object:
#Document
class Foo {
#Id
private final String bar;
private final String baz;
// getters, setters, constructor omitted
}
Which is inserted as follows:
Collection<Foo> foos = ...;
mongoTemplate.insert(foos, Foo.class);
How to save all results in one call ignoring all duplicate key exceptions ?

In my case it was not suitable to allow modification/overwriting of the existing documents as in #marknorkin's answer. Instead, I only wanted to insert new documents. I came up with this using MongoOperations, which is injectable in Spring. The code below is in Kotlin.
try {
// we do not want to overwrite existing documents, especially not behind the event horizon
// we hence use unordered inserts and supresss the duplicate key exceptions
// as described in: https://docs.mongodb.com/v3.2/reference/method/db.collection.insertMany/#unordered-inserts
mongoOps.bulkOps(BulkOperations.BulkMode.UNORDERED, EventContainer::class.java)
.insert(filtered)
.execute()
} catch (ex: BulkOperationException) {
if (!isDuplicateKeyException(ex)) {
throw ex
}
}
With this little helper
private fun isDuplicateKeyException(ex: BulkOperationException): Boolean {
val duplicateKeyErrorCode = 11000
return ex.errors.all { it.code == duplicateKeyErrorCode }
}

I searched through spring data mongo documentation and other resources, but didn't find expected answer.
Seems like Mongo inserts batch docs until unique key constraint is met, and it's up to DB to decide.
So for example if you need to insert 100 docs and document on position 50 already exists in DB then the first 49 will be inserted and the second 50 will not.
What I came up is the next solution:
Set<String> ids = foos.stream().map(Foo::getBar).collect(toSet()); // collect all ids from docs that will be inserted
WriteResult writeResult = mongoTemplate.remove(new Query(Criteria.where("_id").in(ids)), Foo.class); // perform remove with collected ids
mongoTemplate.insert(foos, Foo.class); // now can safely insert batch
So DB will be called twice.
Also as bar is indexed field the remove operation will be fast.

How to return only specific fields for a query in Spring Data MongoDB?

How can we select specific fields in Spring Data Mongo. I tried the following but I got cast exception from Foo to String.
Using #Query
#Query(value="{path : ?0}", fields="{path : 0}")
String findPathByPath(String path);
Non #Query
String findPathByPath(String path);
Here is the document model
#Document(collection = "foo")
public class Foo {
String name, path;
…
}

MongoDB only returns JSON documents for standard queries. What you'd like to see can be achieved by still returning a List<Foo>. The fields property in #Query will cause only the fields set to 1 being returned.
#Query(value="{ path : ?0}", fields="{ path : 0 }")
List<Foo> findByPath(String path);
We usually recommend introducing a dedicted DTO for that so that you prevent the partially filled Foo instance from being handed to save(…) in turn.
Another option is using the aggreation framework but that's more involved.

You can use
Query query = new Query();
query.fields().include("path");

You can use
public interface PersonRepository extends MongoRepository<Person, String>
#Query(value="{ 'firstname' : ?0 }",fields="{ 'firstname' : 1, 'lastname' : 1}")
List<Person> findByThePersonsFirstname(String firstname);
}
More information in spring data documentation

You can use below query to get specific fields.
#Query(fields="{path : 1}")
Foo findPathByPath(String path);
Records present in DB
{
"name" : "name2",
"path" : "path2"
},
{
"name" : "name3",
"path" : "path3"
}
Below query will return Foo object if path=Path3
{
"name": null,
"path": "path3"
}
we need to specify required fields with fieldName:1 and if don't require then specify it with 0.

I found this question while trying to get the value of a field from a specific object in my collection. From what my research shows, Mongo doesn't provide a way to natively return just a specific field's value from an object. (Disappointing since it seems pretty basic to be able to return just a specific value from a field like I would do in SQL or JSONPath).
To get around this, I wrote the following method using Spring MongoDB with Java 11:
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.data.mongodb.core.MongoTemplate; //not used, just showing which type the template is
import java.util.Arrays;
import static java.util.Objects.requireNonNull;
/**
* Use this method to get a specific field from an object saved in Mongo. The objectId will be
* the id of the object to fetch, and the fieldValueToReturn will be the field to return.
*
* #return the value of the provided field-path converted to the class type provided
*/
public <T> T getFieldValueById(String objectId, String fieldValueToReturn, String collectionName, Class<T> classTypeToReturn) {
var query = new Query().addCriteria(Criteria.where("_id").is(objectId));
query.fields().include(fieldValueToReturn);
var result = mongoTemplate.findOne(query, org.bson.Document.class, collectionName);
requireNonNull(result, "Did not find any documents with id '" + objectId + "' in collection: " + collectionName);
return result.getEmbedded(Arrays.asList(fieldValueToReturn.split("\\.")), classTypeToReturn);
}
The getEmbedded call allows us to get the value of the nested field within the returned Bson document.
To use the method, just call it like this:
getFieldValueById("A1234", "field.nestedfield.nestedfield", "collectionName", String.class);
Hopefully this helps out someone else looking on how to do this.
As a side note, I'm not sure how to extend this to return a list of objects - if I get to that dilemma and solve it, I will try to update this answer. I'm also not sure if this is slower than running a Mongo aggregate query - I haven't tried running any performance comparisons between the two methods.
EDIT 2022-09-30: To return a list of a custom Pojo, it looks like you'll have to use an aggregate query via spring-data-mongodb. Also it seems basic queries are faster than aggregate queries, so use basic queries where possible.

You can directly pass your json query with #Query annotation, for example:
#Query("{ 'firstname' : 'john' }")
Here is the link to all json based queries in Spring Data MongoDb - https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mongodb.repositories.queries.json-based

You can do the following.
In your repository, you have the method:
String findPathByPath(String path);
If the document looks like this (below), and you want to only return path
#Document(collection = "foo")
public class Foo {
String name;
String path;
String type;
…
}
Then create a Projection interface, e.g.
#Projection(name = "flattenedFoo", types = Foo.class)
public interface FlattenedInvoice {
String getPath(); // This returns the path value in Foo class
}
You can use the getter methods to get the fields from Foo that you are interested in.
Then in your get request, you would have to specify the projectionName.
e.g. with (#Resource path)
#RestResource(path = "findByPath", rel = "findByPath")
String findPathByPath(String path);
You could then say (In a get request):
..../findByPath?path=target_path&projection=flattenedFoo
this would then return a json with only the fields specifies in FlattenedFoo interface.