We Keep Coding

MongoClientURI connection string throwing error "com.mongodb.MongoSocketReadException: Prematurely reached end of stream"

public void MongoDBClient(String user, String pwd, String dbName, String collectionName) {
MongoClientURI uri = new MongoClientURI("mongodb+srv://" + user + ":" + pwd + "#cluster0.mff6p.mongodb.net/"
+ dbName + "?retryWrites=true&w=majority&connectTimeoutMS=30000&socketTimeoutMS=30000");
try (MongoClient mongoClient = new MongoClient(uri)) {
MongoDatabase database = mongoClient.getDatabase(dbName);
MongoCollection<Document> collection = database.getCollection(collectionName);
Document query = new Document("_id", new ObjectId("5f05e46281048f54ac98c455"));
Document result = collection.find(query).iterator().next();
System.out.println(result);
System.out.println("Test3: " + result.getString("Cluster"));
}
}
Getting exception in the above code -
INFO: Exception in monitor thread while connecting to server cluster0-shard-00-01.mff6p.mongodb.net:27017
com.mongodb.MongoSocketReadException: Prematurely reached end of stream
at com.mongodb.internal.connection.SocketStream.read(SocketStream.java:112)
at com.mongodb.internal.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:580)
at com.mongodb.internal.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:445)
at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:299)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:259)
at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:83)
at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:33)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:105)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:62)
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:129)
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:117)
at java.base/java.lang.Thread.run(Thread.java:834)
Can anyone help me, i think there is an issue with the connection string url, MongoClientURI...

Its seems that your connection was terminated by the mongo server. It could have multiple reasons and best way to debug is to look at the server logs.
Usually the following is the issue:
From documentation:
For long running applications, it is often prudent to enable
"keepAlive" with a number of milliseconds. Without it, after some
period of time you may start to see "connection closed" errors for
what seems like no reason.
Try to enable to keepAlive property.
server: {
socketOptions: {
keepAlive: 100,
connectTimeoutMS: 30000
}
}

Try like this:
const MongoClient = require('mongodb').MongoClient;
const dburl = "mongodb+srv://" + user + ":" + pwd + "#cluster0.mff6p.mongodb.net/"
+ dbName + "?retryWrites=true&w=majority&connectTimeoutMS=30000&socketTimeoutMS=30000";
MongoClient.connect(dburl,{useNewUrlParser:true,useUnifiedTopology:true},(err,client) => {
if(err){
console.log("Error",err);
}
else{
console.log("Connected");
}
});

Implement Multi-threading on Java Program

I'm writing a little Java program which uses PsExec.exe from cmd launched using ProcessBuilder to copy and install an application on networked PC (the number of PC that will need to be installed can vary from 5 to 50).
The program works fine if I launched ProcessBuilder for each PC sequentially.
However to speed things up I would like to implement some form of MultiThreading which could allow me to install 5 PC's at the time concurrently (one "batch" of 5 X Processbuilder processes untill all PC's have been installed).
I was thinking of using a Fixed Thread Pool in combination with a Callable interface (each execution of PsExec returns a value which indicates if the execution was succesfull and which I have to evaluate).
The code used for the ProcessBuilder is:
// Start iterating over all PC in the list:
for(String pc : pcList)
{
counter++;
logger.info("Starting the installation of remote pc: " + pc);
updateMessage("Starting the installation of remote pc: " + pc);
int exitVal = 99;
logger.debug("Exit Value set to 99");
try
{
ProcessBuilder pB = new ProcessBuilder();
pB.command("cmd", "/c",
"\""+psExecPath+"\"" + " \\\\" + pc + userName + userPassword + " -c" + " -f" + " -h" + " -n 60 " +
"\""+forumViewerPath+"\"" + " -q "+ forumAddress + remotePath + "-overwrite");
logger.debug(pB.command().toString());
pB.redirectError();
Process p = pB.start();
InputStream stErr = p.getErrorStream();
InputStreamReader esr = new InputStreamReader(stErr);
BufferedReader bre = new BufferedReader(esr);
String line = null;
line = bre.readLine();
while (line != null)
{
if(!line.equals(""))
logger.info(line);
line = bre.readLine();
}
exitVal = p.waitFor();
} catch (IOException ex)
{
logger.info("Exception occurred during installation of PC: \n"+pc+"\n "+ ex);
notInstalledPc.add(pc);
}
if(exitVal != 0)
{
notInstalledPc.add(pc);
ret = exitVal;
updateMessage("");
updateMessage("The remote pc: " + pc + " was not installed");
logger.info("The remote pc: " + pc + " was not installed. The error message returned was: \n"+getError(exitVal) + "\nProcess exit code was: " + exitVal);
}
else
{
updateMessage("");
updateMessage("The remote pc: " + pc + " was succesfully installed");
logger.info("The remote pc: " + pc + " was succesfully installed");
}
Now I've read some info on how to implement Callable and I would like to enclose my ProcessBuilder in a Callable interface and then submit all the Tasks for running in the for loop.
Am I on the right track?

You can surely do that. I suppose you want to use Callable instead of runnable to get the result of your exitVal ?
It doesn't seem like you have any shared data between your threads, so I think you should be fine. Since you even know how many Callables you are going to make you could create a collection of your Callables and then do
List<Future<SomeType>> results = pool.invokeAll(collection)
This would allow for easier handling of your result. Probably the most important thing you need to figure out when deciding on whether or not to use a threadpool is what to do if your program terminates while threads are still running; Do you HAVE to finish what you're doing in the threads, do you need to have seamless handling of errors etc.
Check out java threadpools doc: https://docs.oracle.com/javase/tutorial/essential/concurrency/pools.html
or search the web, there are tons of posts/blogs about when or not to use threadpools.
But seems like you're on the right track!

Thank you for your reply! It definitely put me on the right track. I ended up implementing it like this:
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(5); //NEW
List<Future<List<String>>> resultList = new ArrayList<>();
updateMessage("Starting the installation of all remote pc entered...");
// Start iterating over all PC in the list:
for(String pc : pcList)
{
counter++;
logger.debug("Starting the installation of remote pc: " + pc);
psExe p = new psExe(pc);
Future<List<String>> result = executor.submit(p);//NEW
resultList.add(result);
}
for(Future<List<String>> future : resultList)
{.......
in the last for loop I read the result of my operations and write them on screen or act according to the result returned.
I still have a couple of questions as it is not really clear to me:
1 - If I have 20 PC and submit all the callable threads to the pool in my first For loop, do I get it correctly that only 5 threads will be started (threadpool size = 5) but all will already be created and put in Wait status, and only as soon as the first running Thread is complete and returns a result value the next one will automatically start until all PC are finished?
2 - What is the difference (advantage) of using invokeall() as you suggested compared to the method I used (submit() inside the for loop)?
Thank you once more for your help...I really Love this Java stuff!! ;-)

Gradle P4Java java.net.SocketTimeoutException: Read timed out

I am using P4Java library in my build.gradle file to sync a large zip file (>200MB) residing at a remote Perforce repository but I am encountering a "java.net.SocketTimeoutException: Read timed out" error either during the sync process or (mostly) during deleting the temporary client created for the sync operation. I am referring http://razgulyaev.blogspot.in/2011/08/p4-java-api-how-to-work-with-temporary.html for working with temporary clients using P4Java API.
I tried increasing the socket read timeout from default 30 sec as suggested in http://answers.perforce.com/articles/KB/8044 and also by introducing sleep but both approaches didn't solved the problem. Probing the server to verify the connection using getServerInfo() right before performing sync or delete operations results in a successful connection check. Can someone please point me as to where I should look for answers?
Thank you.
Providing the code snippet:
void perforceSync(String srcPath, String destPath, String server) {
// Generating the file(s) to sync-up
String[] pathUnderDepot = [
srcPath + "*"
]
// Increasing timeout from default 30 sec to 60 sec
Properties defaultProps = new Properties()
defaultProps.put(PropertyDefs.PROG_NAME_KEY, "CustomBuildApp")
defaultProps.put(PropertyDefs.PROG_VERSION_KEY, "tv_1.0")
defaultProps.put(RpcPropertyDefs.RPC_SOCKET_SO_TIMEOUT_NICK, "60000")
// Instantiating the server
IOptionsServer p4Server = ServerFactory.getOptionsServer("p4java://" + server, defaultProps)
p4Server.connect()
// Authorizing
p4Server.setUserName("perforceUserName")
p4Server.login("perforcePassword")
// Just check if connected successfully
IServerInfo serverInfo = p4Server.getServerInfo()
println 'Server info: ' + serverInfo.getServerLicense()
// Creating new client
IClient tempClient = new Client()
// Setting up the name and the root folder
tempClient.setName("tempClient" + UUID.randomUUID().toString().replace("-", ""))
tempClient.setRoot(destPath)
tempClient.setServer(p4Server)
// Setting the client as the current one for the server
p4Server.setCurrentClient(tempClient)
// Creating Client View entry
ClientViewMapping tempMappingEntry = new ClientViewMapping()
// Setting up the mapping properties
tempMappingEntry.setLeft(srcPath + "...")
tempMappingEntry.setRight("//" + tempClient.getName() + "/...")
tempMappingEntry.setType(EntryType.INCLUDE)
// Creating Client view
ClientView tempClientView = new ClientView()
// Attaching client view entry to client view
tempClientView.addEntry(tempMappingEntry)
tempClient.setClientView(tempClientView)
// Registering the new client on the server
println p4Server.createClient(tempClient)
// Surrounding the underlying block with try as we want some action
// (namely client removing) to be performed in any way
try {
// Forming the FileSpec collection to be synced-up
List<IFileSpec> fileSpecsSet = FileSpecBuilder.makeFileSpecList(pathUnderDepot)
// Syncing up the client
println "Syncing..."
tempClient.sync(FileSpecBuilder.getValidFileSpecs(fileSpecsSet), true, false, false, false)
}
catch (Exception e) {
println "Sync failed. Trying again..."
sleep(60 * 1000)
tempClient.sync(FileSpecBuilder.getValidFileSpecs(fileSpecsSet), true, false, false, false)
}
finally {
println "Done syncing."
try {
p4Server.connect()
IServerInfo serverInfo2 = p4Server.getServerInfo()
println '\nServer info: ' + serverInfo2.getServerLicense()
// Removing the temporary client from the server
println p4Server.deleteClient(tempClient.getName(), false)
}
catch(Exception e) {
println 'Ignoring exception caught while deleting tempClient!'
/*sleep(60 * 1000)
p4Server.connect()
IServerInfo serverInfo3 = p4Server.getServerInfo()
println '\nServer info: ' + serverInfo3.getServerLicense()
sleep(60 * 1000)
println p4Server.deleteClient(tempClient.getName(), false)*/
}
}
}
One unusual thing which I observed while deleting tempClient was it was actually deleting the client but still throwing "java.net.SocketTimeoutException: Read timed out" which is why I ended up commenting the second delete attempt in the second catch block.

Which version of P4Java are you using? Have you tried this out with the newest P4Java? There are notable fixes dealing with RPC sockets since the 2013.2 version forward as can be seen in the release notes:
http://www.perforce.com/perforce/doc.current/user/p4javanotes.txt
Here are some variations that you can try where you have your code to increase timeout and instantiating the server:
a] Have you tried to passing props in its own argument,? For example:
Properties prop = new Properties();
prop.setProperty(RpcPropertyDefs.RPC_SOCKET_SO_TIMEOUT_NICK, "300000");
UsageOptions uop = new UsageOptions(prop);
server = ServerFactory.getOptionsServer(ServerFactory.DEFAULT_PROTOCOL_NAME + "://" + serverPort, prop, uop);
Or something like the following:
IOptionsServer p4Server = ServerFactory.getOptionsServer("p4java://" + server, defaultProps)
You can also set the timeout to "0" to give it no timeout.
b]
props.put(RpcPropertyDefs.RPC_SOCKET_SO_TIMEOUT_NICK, "60000");
props.put(RpcPropertyDefs.RPC_SOCKET_POOL_SIZE_NICK, "5");
c]
Properties props = System.getProperties();
props.put(RpcPropertyDefs.RPC_SOCKET_SO_TIMEOUT_NICK, "60000");
IOptionsServer server =
ServerFactory.getOptionsServer("p4java://perforce:1666", props, null);
d] In case you have Eclipse users using our P4Eclipse plugin, the property can be set in the plugin preferences (Team->Perforce->Advanced) under the Custom P4Java Properties.
"sockSoTimeout" : "3000000"
REFERENCES
Class RpcPropertyDefs
http://perforce.com/perforce/doc.current/manuals/p4java-javadoc/com/perforce/p4java/impl/mapbased/rpc/RpcPropertyDefs.html
P4Eclipse or P4Java: SocketTimeoutException: Read timed out
http://answers.perforce.com/articles/KB/8044

Thread VS RabbitMQ Worker resource consumption

I am using JAVA ExecutorService threads to send amazon emails, this helps me to make concurrent connection with AmazonSES via API and sends mails at lightning speed. So amazon accepts some number of connection in a sec, so for me its 50 requests in a second. So I execute 50 threads in a second and send around 1million emails daily.
This is working pretty good, but now the number of mails is going to be increased. And I don't want to invest more into RAM and processors.
One of my friend suggested me to use RabbitMQ Workers instead of threads, so instead of 50 threads, I ll be having 50 workers which will do that job.
So before changing some code to test the resource management, I just want to know will there be any huge difference in consumption? So currently when I execute my threads, JAVA consumes 20-30% of memory. So if I used workers will it be low or high?
Or is their any alternative option to this?
Here is my thread email sending function:
#Override
public void run() {
Destination destination = new Destination().withToAddresses(new String[] { this.TO });
Content subject = new Content().withData(SUBJECT);
Content textBody = new Content().withData(BODY);
Body body = new Body().withHtml(textBody);
Message message = new Message().withSubject(subject).withBody(body);
SendEmailRequest request = new SendEmailRequest().withSource(FROM).withDestination(destination).withMessage(message);
Connection connection = new Connection();
java.util.Date dt = new java.util.Date();
java.text.SimpleDateFormat sdf = new java.text.SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
String insert = "";
try {
System.out.println("Attempting to send an email to " + this.TO);
ctr++;
client.sendEmail(request);
insert = "INSERT INTO email_histories (campaign_id,contact_id,group_id,is_opened,mail_sent_at,mail_opened_at,ip_address,created_at,updated_at,is_sent) VALUES (" + this.campaign_id + ", " + this.contact_id + ", " + this.group_id + ", false, '" + sdf.format(dt) + "', null, null, '" + sdf.format(dt) + "', '" + sdf.format(dt) + "', true);";
connection.insert(insert);
System.out.println("Email sent! to " + this.TO);
} catch (Exception ex) {
System.out.println("The email was not sent.");
System.out.println("Error message: " + ex.getMessage());
}
}

I have no experience with RabbitMQ, so I'll have to leave that for others to answer.
Or is their any alternative option to this?
Instead of using one thread per mail, move that code inside a runable. Add a shared Semaphore with the number of permits = the number of mails you want to send per second. Take one permit per mail, refill permits every second from another thread (i.e. a separate SchedledExecutorService or a Timer). Then adjust the Executor thread pool size to whatever your server can handle.

From a RabbitMQ perspective there is a small amount of memory and network resource consumed, although pretty constant for each connection. If you use a pool of worker threads to read off of the RabbitMQ queue or queues it is possible that it will save you some resources because you are not garbage collecting the individual threads. As far as alternatives are concerned I would use a Thread Pool in any case. Although perhaps too heavyweight for your use, Spring Framework has a very good thread pool that I have used in the past.

AmazonCloudWatchClient not sending HTTP requests

first time i'm using aws api in java to get the cloud watch statistics for my ec2-instance. i googled about this and i found some code snippet. here it is
AmazonCloudWatchClient cloudWatch = new AmazonCloudWatchClient(
new BasicAWSCredentials(AccessKey, SecretKey));
cloudWatch.setEndpoint("ec2-<my-static-ip>.compute-1.amazonaws.com");
long offsetInMilliseconds = 1000 * 60 * 60 * 24;
Dimension instanceDimension = new Dimension();
instanceDimension.setName("Instanceid");
instanceDimension.setValue(InstanceId);
GetMetricStatisticsRequest request = new GetMetricStatisticsRequest()
.withStartTime(
new Date(new Date().getTime()
- offsetInMilliseconds))
.withNamespace("AWS/EC2")
.withPeriod(60 * 60)
.withDimensions(
new Dimension().withName("InstanceId").withValue(
InstanceId))
.withMetricName("CPUUtilization")
.withStatistics("Average", "Maximum")
.withEndTime(new Date());
GetMetricStatisticsResult getMetricStatisticsResult = cloudWatch
.getMetricStatistics(request);
double avgCPUUtilization = 0;
List dataPoint = getMetricStatisticsResult.getDatapoints();
for (Object aDataPoint : dataPoint) {
Datapoint dp = (Datapoint) aDataPoint;
avgCPUUtilization = dp.getAverage();
System.out.println(InstanceId
+ " instance's average CPU utilization : "
+ dp.getAverage());
}
} catch (AmazonServiceException ase) {
System.out
.println("Caught an AmazonServiceException, which means the request was made "
+ "to Amazon EC2, but was rejected with an error response for some reason.");
System.out.println("Error Message: " + ase.getMessage());
System.out.println("HTTP Status Code: " + ase.getStatusCode());
System.out.println("AWS Error Code: " + ase.getErrorCode());
System.out.println("Error Type: " + ase.getErrorType());
System.out.println("Request ID: " + ase.getRequestId());
}
so, using this code i tried to get statistics, but first time it throws error saying
com.amazonaws.AmazonClientException: Unable to execute HTTP request:Connection to https://ec2-<my-static-ip>.compute-1.amazonaws.com refused
then i thought it was sending https requests. so i enabled ssl on my instance and tried, then i'm getting below exception.
com.amazonaws.AmazonClientException: Unable to execute HTTP request: peer not authenticated
i was using OpenJDK in my instance, so i thought that may causing the problem. then i removed openjdk and installed Oracle JDK 1.7. but still same problem.
My questions are,
1) how can i send only HTTP (instead of HTTPS) requests to get statistics?
2)how to get rid of this problem, so that i can get my results?
But please don't ask me to read any docs, because i messed up by searching in net, blogs,forums, docs... etc. then i end up here. so, please just provide me solution or tell me where i'm going wrong.
Can anybody please help me out this issue.
thank you in Advance.

Got Solution.
1) removed setting end point for AmazonCloudWatchClient.
2) problem with the AWS credentials (Access key ID, Secret key).So, i created another set of credentials and gave CloudWatchFullAccess policy for the user.
Now it is working like Charm... :-)
Thanks.