We Keep Coding

Spring Webflux Reactive Mongo Bulk Operations (Java)

https://github.com/spring-projects/spring-data-mongodb/issues/2821
https://jira.spring.io/browse/DATAMONGO-1922?redirect=false
I have been looking for ReactiveBulk operations to update documents as a batch in Spring WebFlux.
Like in the Mongo Template
var bulkOps = mongoTemplate.bulkOps()
for(dto : List<DTO> DTOs) {
Query query = new Query();
query.addCriteria(Criteria.where(ID).is(dto.getId()));
Update update = new Update()
.set(STATUS, dto.getStatus())
bulkOps.updateOne(query, update)
}
bulkOps.execute();
is there a workaround to implement that operation in reactive way since reactivemongotemplate look like does not support that operation currently?
similiar topic in so: Bulk Update with ReactiveMongoTemplate

Quickly remind that Bulk is different than UpdateMulti.
Bulk is meant to write multiple queries, therefore, update various objects. On the other side, UpdateMulti is intended to update all lines where expression matches
As for reactive bulk, you should be able to use ReactiveMongoTemplate and implement something like that:
reactiveMongoTemplate.getCollection("collection_name")
.flatMap(mongoCollection -> {
List<UpdateOneModel<Document>> operations = DTOs.stream()
.map(dto -> {
Document doc = new Document("status", dto.getStatus());
reactiveMongoTemplate.getConverter().write(dto, doc);
Document filter = new Document("id", dto.getId());
return new UpdateOneModel<Document>(filter, new Document("$set", doc));
}
}).toList();
return Mono.from(mongoCollection.bulkWrite(operations));
});
You can also add custom options to bulkWrite() if you desire.
If more filter is needed, you can append them to the document
Document filter = new Document("id", dto.getId())
.append("country", dto.getCountry);

MongoDB "Invalid BSON Field Name"

I know that there's probably a better way to do this however I'm completely stumped. I'm writing a Discord bot in which a user is able to add points to other users, however I can't figure out how to replace a user's "points". My code is as follows:
BasicDBObject cursor = new BasicDBObject();
cursor.put(user.getAsMember().getId(), getMongoPoints(user.getAsMember()));
if(cursor.containsKey(user.getAsMember().getId())) {
Document old = new Document(user.getAsMember().getId(), getMongoPoints(user.getAsMember()));
Document doc = new Document(user.getAsMember().getId(), getMongoPoints(user.getAsMember()) + Integer.parseInt(amount.getAsString()));
collection.findOneAndUpdate(old, doc);
}
My getMongoPoints function:
public static int getMongoPoints(Member m) {
ConnectionString connectionString = new ConnectionString("database");
MongoClientSettings settings = MongoClientSettings.builder()
.applyConnectionString(connectionString)
.build();
MongoClient mongoClient = MongoClients.create(settings);
MongoDatabase database = mongoClient.getDatabase("SRU");
MongoCollection<Document> collection = database.getCollection("points");
DistinctIterable<Integer> docs = collection.distinct(m.getId(), Integer.class);
MongoCursor<Integer> result = docs.iterator();
return result.next();
}
I've tried findOneAndReplace, however that simply makes a new entry without deleting the old one. The error I receive is: Invalid BSON field name 262014495440896000
Everything else works, include writing to the database itself which is why I'm stumped. Any help would be greatly appreciated and I apologize if this is written poorly.

BSON field names must be string. From the spec:
Zero or more modified UTF-8 encoded characters followed by '\x00'. The (byte*) MUST NOT contain '\x00', hence it is not full UTF-8.
To use 262014495440896000 as a field name, convert it to string first.

Keycloack is a MessageDigestPasswordEncoder sha512 exists?

I am in the process of migrating users from an OAuth 2 system made with Symfony to Keycloak.
Create the users in Keycloak with the encrypted password is ok, but I can't find an algorithm equivalent to mine.
example of user creation:
Post
{
"firstName": "test_encryption",
"lastName":"test_encryption",
"email":"jeremy.rafflin.test#ageo.fr",
"credentials": [{
"type":"password",
"secretData":"{\"value\":\"zeR2Uapc+a/2qD5QR56gh3mVb+KOeZ2XU+rkWMK6B5A=\",\"salt\":\"OThjajM1WnVZWlI3UzZOLk12WjJsQS9VWWZXQXp0WGZGLm5tL2hGSVFzbw==\"}​​​​​​​",
"credentialData": "{\"algorithm\":\"sha512\",\"hashIterations\":5000}"
}]
}
For the current encryption in PHP I am using https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Security/Core/Encoder/MessageDigestPasswordEncoder.php.
Which amounts to:
$password = 'toto';
$salt = '1234';
$salted = $password.'{'.$salt.'}';
$digest = hash('sha512', $salted, true);
for ($i=1; $i<5000; $i++) {
$digest = hash('sha512', $digest.$salted, true);
}
$encodedPassword = base64_encode($digest);
In the spring documentation, https://andifalk.github.io/reactive-spring-security-5-workshop/workshop-tutorial.html I see :
package org.springframework.security.crypto.factory;
public class PasswordEncoderFactories {
...
public static PasswordEncoder createDelegatingPasswordEncoder() {
String encodingId = "bcrypt";
Map<String, PasswordEncoder> encoders = new HashMap<>();
encoders.put(encodingId, new BCryptPasswordEncoder());
encoders.put("ldap", new LdapShaPasswordEncoder());
encoders.put("MD4", new Md4PasswordEncoder());
encoders.put("MD5", new MessageDigestPasswordEncoder("MD5"));
encoders.put("noop", NoOpPasswordEncoder.getInstance());
encoders.put("pbkdf2", new Pbkdf2PasswordEncoder());
encoders.put("scrypt", new SCryptPasswordEncoder());
encoders.put("SHA-1", new MessageDigestPasswordEncoder("SHA-1"));
encoders.put("SHA-256", new MessageDigestPasswordEncoder("SHA-256"));
encoders.put("sha256", new StandardPasswordEncoder());
return new DelegatingPasswordEncoder(encodingId, encoders);
}
...
}
So I thought my algorithm matched MessageDigestPasswordEncoder ("SHA-512") and tried to create a user, but it doesn't work:
"credentialData": "{"algorithm":"SHA-512","hashIterations":5000}"
Does my algorithm exist in keycloak or do I have to create a custom credential algorithm ?

Does my algorithm exist in keycloak or do I have to create a custom
credential algorithm ?
From the Keycloak Documentation:
Here’s an explanation of each policy type:
HashAlgorithm
Passwords are not stored as clear text. Instead they are
hashed using standard hashing algorithms before they are stored or
validated. The only built-in and default algorithm available is
PBKDF2.
Nevertheless, Keycloak allows you to customized and deploy your own algorithm by taking advantage of Service Provider Interfaces.

How to perform Amazon Cloud Search with .net code?

I am learning Amazon Cloud Search but I couldn't find any code in either C# or Java (though I am creating in C# but if I can get code in Java then I can try converting in C#).
This is just 1 code I found in C#: https://github.com/Sitefinity-SDK/amazon-cloud-search-sample/tree/master/SitefinityWebApp.
This is 1 method i found in this code:
public IResultSet Search(ISearchQuery query)
{
AmazonCloudSearchDomainConfig config = new AmazonCloudSearchDomainConfig();
config.ServiceURL = "http://search-index2-cdduimbipgk3rpnfgny6posyzy.eu-west-1.cloudsearch.amazonaws.com/";
AmazonCloudSearchDomainClient domainClient = new AmazonCloudSearchDomainClient("AKIAJ6MPIX37TLIXW7HQ", "DnrFrw9ZEr7g4Svh0rh6z+s3PxMaypl607eEUehQ", config);
SearchRequest searchRequest = new SearchRequest();
List<string> suggestions = new List<string>();
StringBuilder highlights = new StringBuilder();
highlights.Append("{\'");
if (query == null)
throw new ArgumentNullException("query");
foreach (var field in query.HighlightedFields)
{
if (highlights.Length > 2)
{
highlights.Append(", \'");
}
highlights.Append(field.ToUpperInvariant());
highlights.Append("\':{} ");
SuggestRequest suggestRequest = new SuggestRequest();
Suggester suggester = new Suggester();
suggester.SuggesterName = field.ToUpperInvariant() + "_suggester";
suggestRequest.Suggester = suggester.SuggesterName;
suggestRequest.Size = query.Take;
suggestRequest.Query = query.Text;
SuggestResponse suggestion = domainClient.Suggest(suggestRequest);
foreach (var suggest in suggestion.Suggest.Suggestions)
{
suggestions.Add(suggest.Suggestion);
}
}
highlights.Append("}");
if (query.Filter != null)
{
searchRequest.FilterQuery = this.BuildQueryFilter(query.Filter);
}
if (query.OrderBy != null)
{
searchRequest.Sort = string.Join(",", query.OrderBy);
}
if (query.Take > 0)
{
searchRequest.Size = query.Take;
}
if (query.Skip > 0)
{
searchRequest.Start = query.Skip;
}
searchRequest.Highlight = highlights.ToString();
searchRequest.Query = query.Text;
searchRequest.QueryParser = QueryParser.Simple;
var result = domainClient.Search(searchRequest).SearchResult;
//var result = domainClient.Search(searchRequest).SearchResult;
return new AmazonResultSet(result, suggestions);
}
I have already created domain in Amazon Cloud Search using AWS console and uploaded document using Amazon predefine configuration option that is movie Imdb json file provided by Amazon for demo.
But in this method I am not getting how to use this method, like if I want to search Director name then how do I pass in this method as because this method parameter is of type ISearchQuery?

I'd suggest using the official AWS CloudSearch .NET SDK. The library you were looking at seems fine (although I haven't look at it any detail) but the official version is more likely to expose new CloudSearch features as soon as they're released, will be supported if you need to talk to AWS support, etc, etc.
Specifically, take a look at the SearchRequest class -- all its params are strings so I think that obviates your question about ISearchQuery.
I wasn't able to find an example of a query in .NET but this shows someone uploading docs using the AWS .NET SDK. It's essentially the same procedure as querying: creating and configuring a Request object and passing it to the client.
EDIT:
Since you're still having a hard time, here's an example. Bear in mind that I am unfamiliar with C# and have not attempted to run or even compile this but I think it should at least be close to working. It's based off looking at the docs at http://docs.aws.amazon.com/sdkfornet/v3/apidocs/
// Configure the Client that you'll use to make search requests
string queryUrl = #"http://search-<domainname>-xxxxxxxxxxxxxxxxxxxxxxxxxx.us-east-1.cloudsearch.amazonaws.com";
AmazonCloudSearchDomainClient searchClient = new AmazonCloudSearchDomainClient(queryUrl);
// Configure a search request with your query
SearchRequest searchRequest = new SearchRequest();
searchRequest.Query = "potato";
// TODO Set your other params like parser, suggester, etc
// Submit your request via the client and get back a response containing search results
SearchResponse searchResponse = searchClient.Search(searchRequest);

How to return raw JSON directly from a mongodb query in Java?

I have the following code:
#RequestMapping(value = "/envinfo", method = RequestMethod.GET)
#ResponseBody
public Map getEnvInfo()
{
BasicQuery basicQuery = new BasicQuery("{_id:'51a29f6413dc992c24e0283e'}", "{'envinfo':1, '_id': false }");
Map envinfo= mongoTemplate.findOne(basicQuery, Map.class, "jvmInfo");
return envinfo;
}
As you can notice, the code:
Retrieves JSON from MongoDB
Converts it to a Map object
The Map object is then converted to JSON by Spring MongoData before it is returned to the browser.
Is it possible to directly return the raw json from MongoDb without going through the intermediate conversion steps?

There's two way's you can do this right now:
1. Using the CollectionCallback on MongoTemplate
You can use a CollectionCallback to deal with the returned DBObject directly and simply toString() it:
template.execute("jvmInfo", new CollectionCallback<String>() {
String doInCollection(DBCollection collection) {
DBCursor cursor = collection.find(query)
return cursor.next().toString()
}
}
Yo'll still get the exception translation into Spring's DataAccessExceptions. Note, that this is slightly brittle as we expect only a single result to be returned for the query but that's probably something you have to take care of when trying to produce a String anyway.
2. Register a Converter from DBObject to String
You can implement a Spring Converter to do the toString() for you.
class DBObjectToStringConverter implements Converter<DBObject, String> {
public String convert(DBObject source) {
return source == null ? null : source.toString();
}
}
You can then either use the XML configuration or override customConversions() to return a new CustomConversions(Arrays.asList(new DBObjectToStringConverter())) to get it registered with your MongoConverter. You can then simply do the following:
String result = mongoTemplate.findOne(basicQuery, String.class, "jvmInfo");
I will add the just showed converter to Spring Data MongoDB and register it by default for the upcoming 1.3 GA release and port the fix back to 1.2.x as part of the fix for DATAMONGO-743.

As Oliver points out, you can use Spring Data for that, but an alternative which you may or may not prefer would be to use MongoDB's more low-level Java Driver. Take a look at the MongoDB Java Driver 3.x or MongoDB Java Driver 2.x documentation for instructions on using that driver.
Basically, what you need to do is this:
MongoDB Java Driver 3.x
MongoClient mongoClient = new MongoClient();
MongoDatabase db = mongoClient.getDatabase("test");
MongoCollection coll = db.getCollection("testCollection");
BasicDBObject query = new BasicDBObject("_id", "51a29f6413dc992c24e0283e");
try (MongoCursor<Document> cursor = collection.find(query).iterator()) {
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
}
MongoDB Java Driver 2.x
MongoClient mongoClient = new MongoClient();
DB db = mongoClient.getDB("test");
DBCollection coll = db.getCollection("testCollection");
BasicDBObject query = new BasicDBObject("_id", "51a29f6413dc992c24e0283e");
try (DBCursor cursor = coll.find(query)) {
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
}
That will print out all the documents in the collection that have a field _id with a value 51a29f6413dc992c24e0283e.

For me the following worked perfectly:
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.query.Query;
import org.bson.Document;
List<Document> documents = mongoTemplate.find(query, Document.class, "collection_name");