We Keep Coding

Is there any way to write custom or native queries in Java JPA (DocumentDbRepository) while firing a query to azure-cosmosdb?

Connected to azure-cosmosdb and able to fire default queries like findAll() and findById(String Id). But I can't write a native query using #Query annotation as the code is not considering it. Always considering the name of the function in respository class/interface. I need a way to fire a custom or native query to azure-cosmos db. ?!
Tried with #Query annotation. But not working.
List<MonitoringSessions> findBySessionID(#Param("sessionID") String sessionID);
#Query(nativeQuery = true, value = "SELECT * FROM MonitoringSessions M WHERE M.sessionID like :sessionID")
List<MonitoringSessions> findSessions(#Param("sessionID") String sessionID);
findBySessionID() is working as expected. findSessions() is not working. Below root error came while running the code.
Caused by: org.springframework.data.mapping.PropertyReferenceException: No property findSessions found for type MonitoringSessions

Thanks for the response. I got what I exactly wanted from the below link. Credit goes to Author of the link page.
https://cosmosdb.github.io/labs/java/technical_deep_dive/03-querying_the_database_using_sql.html
public class Program {
private final ExecutorService executorService;
private final Scheduler scheduler;
private AsyncDocumentClient client;
private final String databaseName = "UniversityDatabase";
private final String collectionId = "StudentCollection";
private int numberOfDocuments;
public Program() {
// public constructor
executorService = Executors.newFixedThreadPool(100);
scheduler = Schedulers.from(executorService);
client = new AsyncDocumentClient.Builder().withServiceEndpoint("uri")
.withMasterKeyOrResourceToken("key")
.withConnectionPolicy(ConnectionPolicy.GetDefault()).withConsistencyLevel(ConsistencyLevel.Eventual)
.build();
}
public static void main(String[] args) throws InterruptedException, JSONException {
FeedOptions options = new FeedOptions();
// as this is a multi collection enable cross partition query
options.setEnableCrossPartitionQuery(true);
// note that setMaxItemCount sets the number of items to return in a single page
// result
options.setMaxItemCount(5);
String sql = "SELECT TOP 5 s.studentAlias FROM coll s WHERE s.enrollmentYear = 2018 ORDER BY s.studentAlias";
Program p = new Program();
Observable<FeedResponse<Document>> documentQueryObservable = p.client
.queryDocuments("dbs/" + p.databaseName + "/colls/" + p.collectionId, sql, options);
// observable to an iterator
Iterator<FeedResponse<Document>> it = documentQueryObservable.toBlocking().getIterator();
while (it.hasNext()) {
FeedResponse<Document> page = it.next();
List<Document> results = page.getResults();
// here we iterate over all the items in the page result
for (Object doc : results) {
System.out.println(doc);
}
}
}
}

How do you execute a MongoDB query stored as string in Java?

I'm kind of new to the MongoDB Java driver and I was wondering how you could execute a query stored as a string. Is this the best way to execute them, or what would be a better approach?
I've stumbled across the piece of the below on another stackoverflow thread, but haven't been able to get anything useful out of it. The output does not contain the result of the query at all.
The code I'm running right now:
#Test
public void testExecuteStoredQueries() {
String code = "db.getCollection('users').find({})";
final BasicDBObject command = new BasicDBObject();
String formattedCode = String.format("function() { return %s ; }", code);
System.out.println("Formatted code:");
System.out.println(formattedCode);
command.put("eval", formattedCode);
Document result = DbEngine.getInstance().getDatabase().runCommand(command);
System.out.println(result.toJson());
}
Summarized output:
{
"retval": {
"_mongo": "....",
"_db": "...",
"_collection": "...",
"_ns": "cezy.users",
"_query": {},
"_fields": null,
"_limit": 0,
"_skip": 0,
"_batchSize": 0,
"_options": 0,
"_cursor": null,
"_numReturned": 0,
"_special": false
},
"ok": 1
}

I use morphia when i have to deal with objects. As when you retrieve the data from MongoDb, for the long values you get extended Json instead of Json Response. Parsing Extended Json could be a trouble and might break the code. As Gson doesn't support the conversion from Extended Json to Json.
private void createDatastore(boolean createIndexes) {
Morphia morphia = new Morphia();
morphia.map(classname.class);
datastore = morphia.createDatastore(mongoClient, databaseName);
if (createIndexes) {
datastore.ensureIndexes();
}
}
#Override
public Datastore getDatastore() {
return this.datastore;
}
#Test
public void testExecuteStoredQueries() {
String code = "db.getCollection('users').find({})";
String formattedCode = String.format("function() { return %s ; }", code);
final BasicDBObject basicObject = new BasicDBObject(new BasicDBObject("$in", formattedCode));
Query<ClassName> query = getDatastore().createQuery(<Classname>.class).filter("_eval", basicObject);
List<Classname> List = query.asList();
//if you want to access each object and perform some task
List.forEach((cursor) -> {
//perform your task
});
}

Removing the function creation and adding ".toArray()" pretty much solved the problem.
#Test
public void testExecuteStoredQueries() {
String code = "db.users.find({}).toArray();";
final BasicDBObject command = new BasicDBObject();
command.put("eval", code);
Document result = DbEngine.getInstance().getDatabase().runCommand(command);
System.out.println(result.toJson());
assertNotNull(result.get("retval"));
}
The array is in the "retval" field of the response.

Manage Simultaneous calls to a REST Service without Duplicate an ID in JAVA

I´m build a REST service through Spring and Swagger with CMIS Protocol in Maven. My services works well until the moment when I´m doing simultaneous calls through Jmeter to stress the system. Context: the service get an ID from an ID creator to generate a node in Alfresco. The id cannot be repeated in the system. this id creator take the last created node and sum +1. I tried with Thread.sleep and TimeUnit. The Jmeter Answer in 10 calls in one sec that the Node was created 10 times with the same ID.
Extract of current code: Controller:
#ApiOperation(value = "Crea un Tipo de documento")
#RequestMapping(value = "/create2", method = RequestMethod.POST)
ResponseMessage createTypeDocument2(#RequestParam String description) throws InterruptedException {
return typeDocumentService.createTypeDocument(description);
}
Service:
#Transactional
public ResponseMessage createTypeDocument(String description) throws InterruptedException {
Session session = obtieneSesion();
int id = this.idCreator2();
int aux = 0;
if(searchDocuments(id)==null) {
aux=0;
}else {
aux = Integer.parseInt(searchDocuments(id).getProperty("bc:id").getValueAsString());
}
ResponseMessage rm = new ResponseMessage();
TimeUnit.MILLISECONDS.sleep(new Random().nextInt(500));
Thread.sleep(1000);
if(id != aux ) {
SimpleDateFormat sdf = new SimpleDateFormat("SSS");
Date date = new Date();
String nombre = sdf.format(date.getTime());
DocumentTypeDTO docFilter = new DocumentTypeDTO();
Folder root = (Folder) session.getObjectByPath("/DataList-BCH");
HashMap<String, Object> metadata = new HashMap<String, Object>();
metadata.put(PropertyIds.OBJECT_TYPE_ID, "***************");
metadata.put(PropertyIds.NAME, nombre);
metadata.put("bc:id", id);
metadata.put("bc:available", activo);
metadata.put("bc:description", description);
Document newDoc = root.createDocument(metadata, null, null);
docFilter.setId( Integer.parseInt(newDoc.getProperty("bc:id").getValueAsString()));
docFilter.setDescription(newDoc.getProperty("bc:description").getValueAsString());
docFilter.setUuid(newDoc.getId().replaceAll(";1.0", "").replace("workspace://SpacesStore/", ""));
rm.setMensaje("Exitoso");
rm.setCodigo(200);
rm.setObjeto(docFilter);
return rm;
}else {
rm.setCodigo(-1);
rm.setMensaje("La ID ya existe");
return rm;
}
}
SearchDocument method if the object is not found return null. I try multiple options without results, for that reason I need your help to resolve it. Thanks in Advice
RESOLVED IN STACKOVERFLOW IN SPANISH:
RESOLVED

JPA conditional insertion

I have a Java Spring based web application and I want to insert a record to a table only if the table does not contain any rows that are "similar" (according to some specific, irrelevant criteria) to the new row.
Because this is a multi-threaded environment, I cannot use a SELECT+INSERT two-step combination as it would expose me to a race condition.
The same question was first asked and answered here and here several years ago. Unfortunately, the questions have got only a little attention and the provided answer is not sufficient to my needs.
Here's the code I currently have and it's not working:
#Component("userActionsManager")
#Transactional
public class UserActionsManager implements UserActionsManagerInterface {
#PersistenceContext(unitName = "itsadDB")
private EntityManager manager;
#Resource(name = "databaseManager")
private DB db;
...
#SuppressWarnings("unchecked")
#Override
#PreAuthorize("hasRole('ROLE_USER') && #username == authentication.name")
public String giveAnswer(String username, String courseCode, String missionName, String taskCode, String answer) {
...
List<Submission> submissions = getAllCorrectSubmissions(newSubmission);
List<Result> results = getAllCorrectResults(result);
if (submissions.size() > 0
|| results.size() > 0) throw new SessionAuthenticationException("foo");
manager.persist(newSubmission);
manager.persist(result);
submissions = getAllCorrectSubmissions(newSubmission);
results = getAllCorrectResults(result);
for (Submission s : submissions) manager.lock(s, LockModeType.OPTIMISTIC_FORCE_INCREMENT);
for (Result r : results ) manager.lock(r, LockModeType.OPTIMISTIC_FORCE_INCREMENT);
manager.flush();
...
}
#SuppressWarnings("unchecked")
private List<Submission> getAllCorrectSubmissions(Submission newSubmission) {
Query q = manager.createQuery("SELECT s FROM Submission AS s WHERE s.missionTask = ?1 AND s.course = ?2 AND s.user = ?3 AND s.correct = true");
q.setParameter(1, newSubmission.getMissionTask());
q.setParameter(2, newSubmission.getCourse());
q.setParameter(3, newSubmission.getUser());
return (List<Submission>) q.getResultList();
}
#SuppressWarnings("unchecked")
private List<Result> getAllCorrectResults(Result result) {
Query q = manager.createQuery("SELECT r FROM Result AS r WHERE r.missionTask = ?1 AND r.course = ?2 AND r.user = ?3");
q.setParameter(1, result.getMissionTask());
q.setParameter(2, result.getCourse());
q.setParameter(3, result.getUser());
return (List<Result>) q.getResultList();
}
...
}
According to the answer provided here I am supposed to somehow use OPTIMISTIC_FORCE_INCREMENT but it's not working. I suspect that the provided answer is erroneous so I need a better one.
edit:
Added more context related code. Right now this code still has a race condition. When I make 10 simultaneous HTTP POST requests approximately 5 rows will get erroneously inserted. Other 5 requests are rejected with HTTP error code 409 (conflict). The correct code would guarantee that only 1 row would get inserted to the database no matter how many concurrent requests I make. Making the method synchronous is not a solution since the race condition still manifests for some unknown reason (I tested it).

Unfortunately after several days of research I was unable to find a short and simple solution to my problem. Since my time budget is not unlimited I had to come up with a workaround. Call it a kludge if you may.
Since the whole HTTP request is a transaction, it will be rolled back at the sight of any conflicts. I am using this for my advantage by locking a special entity within the context of the whole HTTP request. Should multiple HTTP requests be received at the same time, all but one will result in some PersistenceException.
In the beginning of the transaction I am checking whether no other correct answers have been submitted yet. During that check the lock is already effective so no race condition could happen. The lock is effective until the answer is submitted. This basically simulates a critical section as a SELECT+INSERT two step query on the application level (in pure MySQL I would have used the INSERT IF NOT EXISTS construct).
This approach has some drawbacks. Whenever two students submit an answer at the same time, one of them will be thrown an exception. This is sort of bad for performance and bandwidth because the student who received HTTP STATUS 409 has to resubmit their answer.
To compensate the latter, I am automatically retrying to submit the answer on the server side a couple of times between randomly chosen time intervals. See the according HTTP request controller code is below:
#Controller
#RequestMapping("/users")
public class UserActionsController {
#Autowired
private SessionRegistry sessionRegistry;
#Autowired
#Qualifier("authenticationManager")
private AuthenticationManager authenticationManager;
#Resource(name = "userActionsManager")
private UserActionsManagerInterface userManager;
#Resource(name = "databaseManager")
private DB db;
.
.
.
#RequestMapping(value = "/{username}/{courseCode}/missions/{missionName}/tasks/{taskCode}/submitAnswer", method = RequestMethod.POST)
public #ResponseBody
Map<String, Object> giveAnswer(#PathVariable String username,
#PathVariable String courseCode, #PathVariable String missionName,
#PathVariable String taskCode, #RequestParam("answer") String answer, HttpServletRequest request) {
init(request);
db.log("Submitting an answer to task `"+taskCode+"` of mission `"+missionName+
"` in course `"+courseCode+"` as student `"+username+"`.");
String str = null;
boolean conflict = true;
for (int i=0; i<10; i++) {
Random rand = new Random();
int ms = rand.nextInt(1000);
try {
str = userManager.giveAnswer(username, courseCode, missionName, taskCode, answer);
conflict = false;
break;
}
catch (EntityExistsException e) {throw new EntityExistsException();}
catch (PersistenceException e) {}
catch (UnexpectedRollbackException e) {}
try {
Thread.sleep(ms);
} catch(InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
if (conflict) str = userManager.giveAnswer(username, courseCode, missionName, taskCode, answer);
if (str == null) db.log("Answer accepted: `"+answer+"`.");
else db.log("Answer rejected: `"+answer+"`.");
Map<String, Object> hm = new HashMap<String, Object>();
hm.put("success", str == null);
hm.put("message", str);
return hm;
}
}
If for some reason the controller is unable to commit the transaction 10 times in a row then it will try one more time but will not attempt to catch the possible exceptions. When an exception is thrown on the 11th try then it will be processed by the global exception controller and the client will receive HTTP STATUS 409. The global exception controller is defined below.
#ControllerAdvice
public class GlobalExceptionController {
#Resource(name = "staticDatabaseManager")
private StaticDB db;
#ExceptionHandler(SessionAuthenticationException.class)
#ResponseStatus(value=HttpStatus.FORBIDDEN, reason="session has expired") //403
public ModelAndView expiredException(HttpServletRequest request, Exception e) {
ModelAndView mav = new ModelAndView("exception");
mav.addObject("name", e.getClass().getSimpleName());
mav.addObject("message", e.getMessage());
return mav;
}
#ExceptionHandler({UnexpectedRollbackException.class,
EntityExistsException.class,
OptimisticLockException.class,
PersistenceException.class})
#ResponseStatus(value=HttpStatus.CONFLICT, reason="conflicting requests") //409
public ModelAndView conflictException(HttpServletRequest request, Exception e) {
ModelAndView mav = new ModelAndView("exception");
mav.addObject("name", e.getClass().getSimpleName());
mav.addObject("message", e.getMessage());
synchronized (db) {
db.setUserInfo(request);
db.log("Conflicting "+request.getMethod()+" request to "+request.getRequestURI()+" ("+e.getClass().getSimpleName()+").", Log.LVL_SECURITY);
}
return mav;
}
//ResponseEntity<String> customHandler(Exception ex) {
// return new ResponseEntity<String>("Conflicting requests, try again.", HttpStatus.CONFLICT);
//}
}
Finally, the giveAnswer method itself utilizes a special entity with a primary key lock_addCorrectAnswer. I lock that special entity with the OPTIMISTIC_FORCE_INCREMENT flag which makes sure that no two transactions can have overlapping execution times for the giveAnswer method. The respective code can be seen below:
#Component("userActionsManager")
#Transactional
public class UserActionsManager implements UserActionsManagerInterface {
#PersistenceContext(unitName = "itsadDB")
private EntityManager manager;
#Resource(name = "databaseManager")
private DB db;
.
.
.
#SuppressWarnings("unchecked")
#Override
#PreAuthorize("hasRole('ROLE_USER') && #username == authentication.name")
public String giveAnswer(String username, String courseCode, String missionName, String taskCode, String answer) {
.
.
.
if (!userCanGiveAnswer(user, course, missionTask)) {
error = "It is forbidden to submit an answer to this task.";
db.log(error, Log.LVL_MAJOR);
return error;
}
.
.
.
if (correctAnswer) {
.
.
.
addCorrectAnswer(newSubmission, result);
return null;
}
newSubmission = new Submission(user, course, missionTask, answer, false);
manager.persist(newSubmission);
return error;
}
private void addCorrectAnswer(Submission submission, Result result) {
String var = "lock_addCorrectAnswer";
Global global = manager.find(Global.class, var);
if (global == null) {
global = new Global(var, 0);
manager.persist(global);
manager.flush();
}
manager.lock(global, LockModeType.OPTIMISTIC_FORCE_INCREMENT);
manager.persist(submission);
manager.persist(result);
manager.flush();
long submissions = getCorrectSubmissionCount(submission);
long results = getResultCount(result);
if (submissions > 1 || results > 1) throw new EntityExistsException();
}
private long getCorrectSubmissionCount(Submission newSubmission) {
Query q = manager.createQuery("SELECT count(s) FROM Submission AS s WHERE s.missionTask = ?1 AND s.course = ?2 AND s.user = ?3 AND s.correct = true");
q.setParameter(1, newSubmission.getMissionTask());
q.setParameter(2, newSubmission.getCourse());
q.setParameter(3, newSubmission.getUser());
return (Long) q.getSingleResult();
}
private long getResultCount(Result result) {
Query q = manager.createQuery("SELECT count(r) FROM Result AS r WHERE r.missionTask = ?1 AND r.course = ?2 AND r.user = ?3");
q.setParameter(1, result.getMissionTask());
q.setParameter(2, result.getCourse());
q.setParameter(3, result.getUser());
return (Long) q.getSingleResult();
}
}
It is important to note that the entity Global has to have a version annotation in its class for the OPTIMISTIC_FORCE_INCREMENT to work (see code below).
#Entity
#Table(name = "GLOBALS")
public class Global implements Serializable {
.
.
.
#Id
#Column(name = "NAME", length = 32)
private String key;
#Column(name = "INTVAL")
private int intVal;
#Column(name = "STRVAL", length = 4096)
private String strVal;
#Version
private Long version;
.
.
.
}
Such an approach can be optimized even further. Instead of using the same lock name lock_addCorrectAnswer for all giveAnswer calls, I could generate the lock name deterministically from the name of the submitting user. For example, if the student's username is Hyena then the primary key for the lock entity would be lock_Hyena_addCorrectAnswer. That way multiple students could submit answers at the same time without receiving any conflicts. However, if a malicious user spams the HTTP POST method for submitAnswer 10x in parallel they will be prevented by the this locking mechanism.

spring data mongodb query document

I am facing this issue(getting null response) when i am trying to Query in Java using
I need to based on placed time stamp range and releases desc and status.
// My document as follows:
<ordersAuditRequest>
<ordersAudit>
<createTS>2013-04-19 12:19:17.165</createTS>
<orderSnapshot>
<orderId>43060151</orderId>
<placedTS>2013-04-19 12:19:17.165</placedTS>
<releases>
<ffmCenterDesc>TW</ffmCenterDesc>
<relStatus>d </relStatus>
</releases>
</ordersAudit>
</ordersAuditRequest>
I am using following query but it returns null.
Query query = new Query();
query.addCriteria(Criteria.where("orderSnapshot.releases.ffmCenterDesc").is(ffmCenterDesc)
.and("orderSnapshot.releases.relStatus").is(relStatus)
.andOperator(
Criteria.where("orderSnapshot.placedTS").gt(orderPlacedStart),
Criteria.where("orderSnapshot.placedTS").lt(orderPlacedEnd)
)
);

I can't reproduce your problem, which suggests that the issue is with the values in the database and the values you're passing in to the query (i.e. they're not matching). This is not unusual when you're trying to match dates, as you need to make sure they're stored as ISODates in the database and queried using java.util.date in the query.
I have a test that shows your query working, but I've made a number of assumptions about your data.
My test looks like this, hopefully this will help point you in the correct direction, or if you give me more feedback I can re-create your problem more accurately.
#Test
public void shouldBeAbleToQuerySpringDataWithDates() throws Exception {
// Setup - insert test data into the DB
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd' 'hh:mm:ss.SSS");
MongoTemplate mongoTemplate = new MongoTemplate(new Mongo(), "TheDatabase");
// cleanup old test data
mongoTemplate.getCollection("ordersAudit").drop();
Release release = new Release("TW", "d");
OrderSnapshot orderSnapshot = new OrderSnapshot(43060151, dateFormat.parse("2013-04-19 12:19:17.165"), release);
OrdersAudit ordersAudit = new OrdersAudit(dateFormat.parse("2013-04-19 12:19:17.165"), orderSnapshot);
mongoTemplate.save(ordersAudit);
// Create and run the query
Date from = dateFormat.parse("2013-04-01 01:00:05.000");
Date to = dateFormat.parse("2014-04-01 01:00:05.000");
Query query = new Query();
query.addCriteria(Criteria.where("orderSnapshot.releases.ffmCenterDesc").is("TW")
.and("orderSnapshot.releases.relStatus").is("d")
.andOperator(
Criteria.where("orderSnapshot.placedTS").gt(from),
Criteria.where("orderSnapshot.placedTS").lt(to)
)
);
// Check the results
List<OrdersAudit> results = mongoTemplate.find(query, OrdersAudit.class);
Assert.assertEquals(1, results.size());
}
public class OrdersAudit {
private Date createdTS;
private OrderSnapshot orderSnapshot;
public OrdersAudit(final Date createdTS, final OrderSnapshot orderSnapshot) {
this.createdTS = createdTS;
this.orderSnapshot = orderSnapshot;
}
}
public class OrderSnapshot {
private long orderId;
private Date placedTS;
private Release releases;
public OrderSnapshot(final long orderId, final Date placedTS, final Release releases) {
this.orderId = orderId;
this.placedTS = placedTS;
this.releases = releases;
}
}
public class Release {
String ffmCenterDesc;
String relStatus;
public Release(final String ffmCenterDesc, final String relStatus) {
this.ffmCenterDesc = ffmCenterDesc;
this.relStatus = relStatus;
}
}
Notes:
This is a TestNG class, not JUnit.
I've used SimpleDateFormat to create Java Date classes, this is just for ease of use.
The XML value you pasted for relStatus included spaces, which I have stripped.
You showed us the document structure in XML, not JSON, so I've had to assume what your data looks like. I've translated it almost directly into JSON, so it looks like this in the database:
{
"_id" : ObjectId("51d689843004ec60b17f50de"),
"_class" : "OrdersAudit",
"createdTS" : ISODate("2013-04-18T23:19:17.165Z"),
"orderSnapshot" : {
"orderId" : NumberLong(43060151),
"placedTS" : ISODate("2013-04-18T23:19:17.165Z"),
"releases" : {
"ffmCenterDesc" : "TW",
"relStatus" : "d"
}
}
}
You can find what yours really looks like by doing a db.<collectionName>.findOne() call in the mongoDB shell.