We Keep Coding

Cannot create Mongo Materialized view using Java Spring Boot

I have a collection name Order with many properties. I wanna create a materialized view with 3 fields from Order to destStationList. I'm using the following java method to do that.
private void createMaterializedView() {
String collectionName = "orders";
String viewName = "destStationList";
// Attempt to create the view
if(mongoTemplate.collectionExists(viewName)){
mongoTemplate.dropCollection(viewName);
}
CommandResult result = mongoTemplate.executeCommand("{" +
"aggregate: '" + viewName + "', " +
"pipeline: [ { $project: { travelDate: 1, trainNumber: 1, to: 1 } }, { $merge: { into: \"destStationList\", whenMatched: \"replace\", whenNotMatched: \"insert\" } } ]," +
"cursor: { }"+
"}");
if(result.ok()) {
LOGGER.info("Successfully created view '{}' on collection '{}'", viewName, collectionName);
}
else {
System.out.println("Failed to create view '" + viewName + "' on collection '" + collectionName + "' - " + result.getErrorMessage());
}
}
I have tested the following mongo shell command to check that in my local mongodb. It works well.
db.runCommand( {
aggregate: "order",
pipeline: [ { $project: { travelDate: 1, trainNumber: 1, to: 1 } }, { $merge: { into: "destStationList", whenMatched: "replace", whenNotMatched: "insert" } } ],
cursor: { }
} );
For your information, I'm using MongoDB version 4.4.
The problem is that executing the java method shows that the view is created successfully. But when I run the command mongoTemplate.collectionExists("destStationList") it returns false and also cannot retrieve data by querying from the view.
Can anyone please help me with it? How can I create a mongo materialized view using java?

Should it be collectionName instead of viewName in the aggregate?
CommandResult result = mongoTemplate.executeCommand("{" +
"aggregate: '" + collectionName + "', " +
"pipeline: [ { $project: { travelDate: 1, trainNumber: 1, to: 1 } }, { $merge: { into: \"destStationList\", whenMatched: \"replace\", whenNotMatched: \"insert\" } } ]," +
"cursor: { }"+
"}");
You may also want to use the viewName variable rather than hardcode it into the String.

Parallel stream doesn't look like working in parallel, completely

1. Set's parallelStream doesn't use enough thread.
Java8 parallelStream doesn't working exactly parallel.
In my computer, java8 set's parallelStream is not using enough thread when task's count is smaller than processor's count.
public class ParallelStreamSplitTest {
#Test
public void setStreamParallelTest() {
System.out.printf("Total processor count : %d \n", Runtime.getRuntime().availableProcessors());
long start = System.currentTimeMillis();
IntStream.range(1, 8).boxed().collect(Collectors.toCollection(HashSet::new)).parallelStream().forEach((index) -> {
System.out.println("Starting " + Thread.currentThread().getName() + ", index=" + index + ", " + new Date());
try {
Thread.sleep(1000);
} catch (Exception e) {
}
});
long end = System.currentTimeMillis();
System.out.println(Thread.currentThread().getName() + "'s elapsed time : " + (end - start));
}
#Test
public void intStreamParallelTest() {
System.out.printf("Total processor count : %d \n", Runtime.getRuntime().availableProcessors());
long start = System.currentTimeMillis();
IntStream.range(1, 8).parallel().forEach(index -> {
System.out.println("Starting " + Thread.currentThread().getName() + ", index=" + index + ", " + new Date());
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
});
long end = System.currentTimeMillis();
System.out.println(Thread.currentThread().getName() + "'s elapsed time : " + (end - start));
}
}
In my code, setStreamParallelTest takes 4 seconds whereas intStreamParallelTest takes 1 second.
I expect that setStreamParallelTest also done in 1 seconds.
Is it bug?
2. Is it okay to use parallel stream to call another api in web application? If it is wrong, why?
My web application need to call another api server in parallel. So I use parallel stream to call api.
Sets.newHashSet(api1, api2, api3, api4).parallelStream().forEach(api -> callApiSync(api))
I think all requests bound for my server share a fork-join pool. so, It looks dangerous when one of api's response is slow.
Is it correct?

The contract for parallelStream says:
Returns a possibly parallel Stream with this collection as its source. It is allowable for this method to return a sequential stream.
If you want to invoke several tasks in parallel, use an ExecutorService.

Can I get multiple response from micro Service which will Query MongoDB

I have Service which will give us response back by querying MongoDB on the basis of some parameter provided
#RequestMapping(value = "/custRef/{custRef}", method = RequestMethod.GET)
#ResponseBody
public ResponseEntity<String> getServiceId(#PathVariable("custRef") String custRef) throws InterruptedException {
System.out.println("Thread.currentThread().getName() :"+Thread.currentThread().getName());
String serviceId=//calling Mongo Service and getting the result
if(custRef == null) {
return new ResponseEntity<String>("No service id available for the given FO Id:" + custRef,HttpStatus.NOT_FOUND);
}
return new ResponseEntity<String>(serviceId,HttpStatus.OK);
}
I have another client which will call the above service by providing the proper parameter. I want to call the above service by using 10 threads. can I get the response back in same frequency from the above service or do I need to do any configuration on the server where above one is running
ExecutorService es = Executors.newFixedThreadPool(50);
for (RouterInfo router : listOfcpeRouterInfo){
Future<String> serviceIDS = es.submit(new CalculationTaskA(router.getCustomerRef(), rieClient));
}
#Override
public String call() throws Exception {
String currentThreadName = Thread.currentThread().getName();
log.info("##### [" + currentThreadName + "] <" + taskId + "> STARTIING #####");
// System.out.println("[" + currentThreadName + "] <" + taskId + ">
// Sleeping for " + sleepTime + " millis");
// TimeUnit.MILLISECONDS.sleep(sleepTime);
//
String serviceId = null;
try {
///
serviceId = rieClient.getObject(customerRef);
log.info("serviceId for given foid: " + customerRef + " is " + serviceId);
} catch (ParseException pe) {
log.error("error while parsing Data", pe);
}
log.info("****** [" + currentThreadName + "] <" + taskId + "> DONE ******");
return serviceId;
}
calling above service
enter code here
Inside getObject I am doing below
ResponseEntity<String> response=restTemplate.exchange(this.serviceIdUrl+"/{foId}",HttpMethod.GET,entity,String.class,foId);

By default, Spring Boot applications are multithreaded so your code sample
ExecutorService es = Executors.newFixedThreadPool(50);
for (RouterInfo router : listOfcpeRouterInfo){ Future serviceIDS = es.submit(new CalculationTaskA(router.getCustomerRef(), rieClient)); }
would only be needed in your "Mongo" Service if asynchronous calls are to be made by the "Mongo" service.

Parallel Execution: How to update Excel WorkBook using Synchronized method, only after synchronized method execution is completed?

I have written a non-static synchronized method in a class to avoid multiple threads updating the Excel Workbook at the same time. For example Thread with ID #9 and #10 are trying to update Excel, expected is that Thread#9 should complete the operation of synchronized method and then Thread#10 should be able to invoke the method which is not happening for us. Below is the code snippet.
public synchronized boolean put(String parameterName, String parameterValue) {
try {
System.out.println("Start Time:= " + Thread.currentThread().getId() + " :: " + LocalDateTime.now().toString());
updateData(data.get("XLFileName"), data.get("XLSheetName"), parameterName, parameterValue, data.get("TestCaseID"), data.get("Iteration"));
System.out.println("End Time:= " + Thread.currentThread().getId() + " :: " + LocalDateTime.now().toString());
return true;
} catch (FilloException flex) {
flex.printStackTrace();
// extentTest.log(LogStatus.FATAL, "A", flex.getMessage());
return false;
}
}
public void updateData(String XLFileName, String XLSheet, String parameterName, String parameterValue, String testCaseID, String iteration) throws FilloException {
Fillo fillo = new Fillo();
Connection connection = fillo.getConnection(XLFileName);
String strQuery = "Update " + XLSheet + " Set " + parameterName + "='" + parameterValue + "' where TestCaseID = '" + testCaseID + "' and Iteration = '" + iteration + "'";
connection.executeUpdate(strQuery);
connection.close();
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Current Output:
Start Time:= 10 :: 2017-03-20T15:03:31.041
1 row(s) affected
Start Time:= 9 :: 2017-03-20T15:03:31.619
1 row(s) affected
End Time:= 10 :: 2017-03-20T15:03:41.201
End Time:= 9 :: 2017-03-20T15:03:41.675
Expected Output:
Start Time:= 10 :: 2017-03-20T15:03:31.041
1 row(s) affected
End Time:= 10 :: 2017-03-20T15:03:41.201
Start Time:= 9 :: 2017-03-20T15:03:31.619
1 row(s) affected
End Time:= 9 :: 2017-03-20T15:03:41.675
Tried with synchronized block as well.

Can you try with Synchronized block with lock on class as shown below.
public synchronized boolean put(String parameterName, String parameterValue) {
synchronized (DataBook.class) {
try {
System.out.println("Start Time:= " + Thread.currentThread().getId() + " :: " + LocalDateTime.now().toString());
updateData(data.get("XLFileName"), data.get("XLSheetName"), parameterName, parameterValue, data.get("TestCaseID"), data.get("Iteration"));
System.out.println("End Time:= " + Thread.currentThread().getId() + " :: " + LocalDateTime.now().toString());
return true;
} catch (FilloException flex) {
flex.printStackTrace();
return false;
}
}
}

Since this is not static, the lock used here is an intrinsic lock, i.e. a lock per object. What it means is that you have a lock per thread, so each thread has the right to execute its own put method, thus the output you observe.
What I would suggest to obtain the desired output is to make this method static, so that the lock is associated to the class, and only one Thread will be able to access it at a time, giving you the expected output (or almost, thread 9 could start sooner than Thread 10):
public static synchronized boolean put(String parameterName, String parameterValue)

Java - Updating static variables

I have two classes in java that need to run at the same time - A Crawler class ( that basically implements a web crawler, and keeps printing out urls as it encounters them ), and an Indexer class, which as of now, is supposed to simply print the urls crawled.
For this, my Indexer class has a Queue :
public static Queue<String> urls = new LinkedList();
And in the toVisit() function of my Crawler class, I have the following :
Indexer.urls.add( url ) // where url is a String
The Crawler is working totally fine, since it prints out all the urls that it has encountered, but for some reason, these urls do not get added to the Queue in my Indexer class. Any idea why this may be the case ?
The toVisit() method from Crawler.java is as follows :
public void visit(Page page) {
int docid = page.getWebURL().getDocid();
String url = page.getWebURL().getURL();
String domain = page.getWebURL().getDomain();
String path = page.getWebURL().getPath();
String subDomain = page.getWebURL().getSubDomain();
String parentUrl = page.getWebURL().getParentUrl();
System.out.println("Docid: " + docid);
System.out.println("URL: " + url);
System.out.println("Domain: '" + domain + "'");
System.out.println("Sub-domain: '" + subDomain + "'");
System.out.println("Path: '" + path + "'");
System.out.println("Parent page: " + parentUrl);
Indexer.urls.add( url );
System.out.println("=============");
}
Code from my Indexer class :
public static Queue<String> urls = new LinkedList();
public static void main( String[] args )
{
while( urls.isEmpty() )
{
//System.out.println("Empty send queue");
Thread.sleep(sleepTime);
}
System.out.println( urls.poll() );
}

Okay, so I solved my problem by doing as suggested by BigMike. I implemented the Runnable interface in my two classes, and then ran those 2 classes as threads within the main function of a new third class.
Thanks everyone for all your help ! :)