We Keep Coding

Starting and monitoring spark application from a Java application

I am working on an application to execute spark batch application from a java application.
There is one main class which starts the thread to start spark application. It uses zookeeper to find the leader among machines which would start the spark application. Main method looks like this:
public static void main(String[] args) throws IOException {
final int id = Integer.valueOf(args[0]);
final String zkURL = args[1];
final ExecutorService service = Executors.newSingleThreadExecutor();
final Future<?> status = service.submit(new ProcessNode(id, zkURL));
try {
status.get();
} catch (InterruptedException | ExecutionException e) {
LOG.fatal(e.getMessage(), e);
service.shutdown();
}
Once the leader is selected , following code would run on it to start spark application.
protected Boolean executeCommand() {
try {
final Runtime rt = Runtime.getRuntime();
final Process proc = rt.exec("sh start-sparkapp.sh");
final int exitVal = proc.waitFor();
BufferedReader buf = new BufferedReader(new InputStreamReader(proc.getInputStream()));
String line = "";
while ((line=buf.readLine())!=null) {
System.out.println(line);
}
System.out.println(" commandToExecute exited with code: " + exitVal);
proc.destroy();
} catch (final Exception e) {
System.out.println("Exception occurred while Launching process : " + e.getMessage());
return Boolean.FALSE;
}
return Boolean.TRUE;
}
But this starts a long running spark job. So I believe, next part of code would be executed only when spark job is finished. My requirement is as soon as spark application is started , the control goes to the next part of code, where I am monitoring the status of same spark application. i.e I start the spark application and monitor the status of spark application from same java application.
Assume I have a method montior which monitors the status of application
public String monitor(ApplicationId id)
Any suggestion how to achieve this?

Since you will be monitoring your Spark application using the method public String monitor(ApplicationId id) , I am assuming you do not want your current thread to wait on the process using proc.waitFor(). Additionally, you do not want to print the normal output of the process to your console. Both these operations make your thread wait on the spawned process. Moreover, your monitor method should take the process-id of the spawned process, rather than the spark applicationId, as input.
So the modified code could look like:
protected Boolean executeCommand() {
try {
final Runtime rt = Runtime.getRuntime();
final Process proc = rt.exec("sh start-sparkapp.sh");
/*
*Call to method monitor(ProcessId id)
*/
} catch (final Exception e) {
System.out.println("Exception occurred while Launching process : " + e.getMessage());
return Boolean.FALSE;
}
return Boolean.TRUE;
}

cron schedule freeze

I'm using Cron to schedule an the upload of a file to the server in specific time given by the adminstrator. i created an interface on java, where the user can choose the time of execution of the upload program, and submit the chosen values, once submitted the following method is executed:
public class Reminder {
String minute;
//static int i=0;
String heur;
String substr=",";
String patterns;
List<String> list = null;
List<String> lines = new ArrayList<>();
Timer timer;
FTPUploadFileDemo up=new FTPUploadFileDemo();
public void start() throws IOException {
/************ Get the chosen values from the administrator saved in a CSV file *********************************************************/
BufferedReader reader;
try {
reader = new BufferedReader(new FileReader("C:/Users/BACKENDPC1/Desktop/timer.csv"));
String line = null;
while ((line = reader.readLine()) != null) {
lines.add(line);
}} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();}
/**********************create cron patterns *********************************************/
patterns="";
for(int i=0; i<lines.size();i++) {
heur=lines.get(i).substring(0, lines.get(i).indexOf(substr));
minute=lines.get(i).substring(lines.get(i).indexOf(substr) + substr.length());
System.out.println("Time selected is: "+heur+","+minute);
patterns=patterns+minute+" "+heur+" * * *|";
}
System.out.println(patterns);
// Creates the scheduler.
Scheduler scheduler = new Scheduler();
// Schedules the task, once every minute.
scheduler.schedule(patterns,new RemindTask());
scheduler.start();
try {
Thread.sleep(1L * 60L * 1000L);
} catch (InterruptedException e) {
System.out.println(e);
}
// Stops the scheduler.
scheduler.stop();
}
class RemindTask extends TimerTask {
public void run() {
up.Uplaod();
}
}
}
the scheduling works and it runs but, every time the user interface i created freeze, i don't get any error and the program keeps running but the i can't use the interface anymore. can any one help me please.

public void start() throws IOException {
..............
try {
Thread.sleep(1L * 60L * 1000L);
} catch (InterruptedException e) {
System.out.println(e);
}
...............
}
Why you pause main thread for 60 secounds? Scheduler run his own tasks in separate thread, so you shouldn't interrupt execution of main thread.
ALSO, try to put breakpoints and debug your program step by step and localize problem
And don't write math operations like this:
1L * 60L * 1000L
will be enough to write:
1L * 60 * 1000
In addition, every time format your code:
In Eclipse: Ctrl + Shift + F
In IntelliJ IDEA: Ctrl + Alt + L

OutOfMemoryError: unable to create new native thread using ExecutorService

I launched my instance overnight to see how it handled things and when I came by this morning, I was facing a
Exception in thread "pool-535-thread-7" java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:691)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:943)
at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:992)[info] application - Connecting to server A
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
The aim of my code is quite simple : Every 5 minutes, I connect to a list of remote server, send a request (via socket) and that's it.
Here's my code :
My "cron" task :
/** will create a new instance of ExecutorService every 5 minutes, loading all the websites in the database to check their status **/
/** Maybe that's where the problem is ? I need to empty (GC ?) this ExecutorService ? **/
Akka.system().scheduler().schedule(
Duration.create(0, TimeUnit.MILLISECONDS), // Initial delay 0 milliseconds
Duration.create(5, TimeUnit.MINUTES), // Frequency 5 minutes
new Runnable() {
public void run() {
// We get the list of websites to check
Query<Website> query = Ebean.createQuery(Website.class, "WHERE disabled = false AND removed IS NULL");
query.order("created ASC");
List<Website> websites = query.findList(); // Can be 1, 10, 100, 1000. In my test case, I had only 9 websites.
ExecutorService executor = Executors.newFixedThreadPool(NTHREDS);
for (Website website : websites) {
CheckWebsite task = new CheckWebsite(website);
executor.execute(task);
}
// This will make the executor accept no new threads
// and finish all existing threads in the queue
executor.shutdown();
}
},
Akka.system().dispatcher()
);
My CheckWebsite class :
public class CheckWebsite implements Runnable {
private Website website;
public CheckWebsite(Website website) {
this.website = website;
}
#Override
public void run() {
WebsiteLog log = website.checkState(); // This is where the request is made, I copy paste the code just after
if (log == null) {
Logger.error("OHOH, WebsiteLog should not be null for website.checkState() in CheckWebsite class :s");
return;
}
try {
log.save();
catch (Exception e) {
Logger.info ("An error occured :/");
Logger.info(e.getMessage());
e.printStackTrace();
}
}
}
My checkState() method in Website.class :
public WebsiteLog checkState() {
// Since I use Socket and the connection can hang indefinitely, I use an other ExecutorService in order to limit the time spent
// The duration is defined via Connector.timeout, Which will be the next code.
ExecutorService executor = Executors.newFixedThreadPool(1);
Connector connector = new Connector(this);
try {
final long startTime = System.nanoTime();
Future<String> future = executor.submit(connector);
String response = future.get(Connector.timeout, TimeUnit.MILLISECONDS);
long duration = System.nanoTime() - startTime;
return PlatformLog.getLastOccurence(this, response, ((int) duration/ 1000000));
}
catch (Exception e) {
return PlatformLog.getLastOccurence(this, null, null);
}
}
Here's the Connector.class. I removed useless part here (like Catches) :
public class Connector implements Callable<String> {
public final static int timeout = 2500; // WE use a timeout of 2.5s, which should be enough
private Website website;
public Connector(Website website) {
this.website = website;
}
#Override
public String call() throws Exception {
Logger.info ("Connecting to " + website.getAddress() + ":" + website.getPort());
Socket socket = new Socket();
try {
socket.connect(new InetSocketAddress(website.getIp(), website.getPort()), (timeout - 50));
BufferedReader input = new BufferedReader(new InputStreamReader(socket.getInputStream()));
String response = input.readLine();
socket.close();
return response;
}
catch (Exception e) {
e.printStackTrace();
throw e;
}
finally {
// I take the precaution to close the socket here in order to avoid a memory leak
// But if the previous ExecutorService force the close of this thread before
// I can't guarantee it will be closed :/
if (socket != null && !socket.isClosed()) {
socket.close();
}
}
}
}
I'm new to Java multithreading so I probably made big mistake. I suspect some area that could be potentially the reason, but my lack of knowledge requires me to ask for your help :)
As a summary, here's the potentials areas :
Creating a new ExecutorService every 5 minutes. Maybe I can reuse the old one ? Or do I need to close the current one when finished (if so, how ?).
The fact that I create an ExecutorService that will create an ExecutorService (in the checkstate() method)
The fact that the Connector class can be (violently) stopped by the ExecutorService running it, if it takes too long, resulting in a socket not closed (and then a memory leak) ?
Also, as you can see, the exception occured for the thread "pool-535-thread-7" which mean it didn't happen soon.
I store the last_occured check in the database, and the creation of the log entry (in WebsiteLog), the delta is around 5 hours (so, for every 5 minutes, the thread crashed after around 60 calls).
Update : Here's the revisited checkState method to include the shutdown call :
public PlatformLog checkState() {
ExecutorService executor = Executors.newFixedThreadPool(1);
Connector connector = new Connector(this);
String response = null;
Long duration = null;
try {
final long startTime = System.nanoTime();
Future<String> future = executor.submit(connector);
response = future.get(Connector.timeout, TimeUnit.MILLISECONDS);
duration = System.nanoTime() - startTime;
}
catch (Exception e) {}
executor.shutdown();
if (duration != null) {
return WebsiteLog.getLastOccurence(this, response, (duration.intValue()/ 1000000));
}
else {
return WebsiteLog.getLastOccurence(this, response, null);
}
}

I'm not sure this is the only problem, but you are creating an ExecutorService in your checkState() method but you don't shut it down.
According to the JavaDocs for Executors.newFixedThreadPool():
The threads in the pool will exist until it is explicitly shutdown.
The threads staying alive will cause the ExecutorService not to be garbage collected (which would call shutdown() on your behalf. Hence you are leaking a thread each time this is called.

Background thread blocks Tomcat 7 application startup [duplicate]

This question already has answers here:
How to run a background task in a servlet based web application?
(5 answers)
Closed 7 years ago.
EDIT:
The current code is the working solution, the one which does not block the application, it
incorporates the suggestion made in the approved answer.
I want a background thread to download an MS Access database continuously, while my tomcat 7 web application is running, the thread does download the database, however it seems to block my application's startup as I'm unable to access any page from the service, this is the code that I'm using:
public class DatabaseUpdater implements ServletContextListener {
private Thread thread = null;
private final Runnable updater = new Runnable() {
private boolean hasExpired(File mdbFile) throws IOException {
if (!mdbFile.exists())
return true;
Long ttl = Long.parseLong(Configuration.getValueForOS("db.http-expiration"));
Date now = new Date();
Date fileDate = new Date(mdbFile.lastModified());
return (now.getTime() - fileDate.getTime()) > ttl;
}
#Override
public void run() {
while (true) {
if (Thread.currentThread().isInterrupted())
throw new RuntimeException("Application Shutdown");
try {
String databases[] = new String[]{"database1", "database2"};
for (String database : databases) {
String fileName = database + "." + StringUtil.randomString(8) + ".mdb";
String fileLocation = Configuration.getValueForOS("db.path");
File mdbFile = new File(fileLocation, fileName);
File currentDatabaseFile = new File(fileLocation, database + ".mdb");
if (hasExpired(currentDatabaseFile)) {
URL url = new URL(Configuration.getValueForOS("db.url." + database));
InputStream in = url.openConnection().getInputStream();
OutputStream out = new FileOutputStream(mdbFile);
FileUtil.streamBridge(in, out);
FileUtil.close(in, out);
while (currentDatabaseFile.exists() && !currentDatabaseFile.delete()) ;
while (!mdbFile.renameTo(currentDatabaseFile)) ;
}
}
// Put the thread to sleep so the other threads do not starve
Thread.sleep(Long.parseLong(
Configuration.getValueForOS("db.http-expiration"));
} catch (IOException ioe) {
ioe.printStackTrace();
} catch (InterruptedException ie) {
ie.printStackTrace();
}
}
}
};
#Override
public void contextInitialized(ServletContextEvent servletContextEvent) {
this.thread = new Thread(updater);
thread.start();
}
#Override
public void contextDestroyed(ServletContextEvent servletContextEvent) {
if (this.thread.isAlive())
this.thread.interrupt();
}
}
What could be causing?
I based my implementation on this question: Background Thread for a Tomcat servlet app

Given that your code loops forever, you're probably starving all the other threads in the VM. Try sleeping the thread once in a while.

To check if a file is written completely

How do I know if a software is done writing a file if I am executing that software from java?For example, I am executing geniatagger.exe with an input file RawText that will produce an output file TAGGEDTEXT.txt. When geniatagger.exe is finished writing the TAGGEDTEXT.txt file, I can do some other staffs with this file. The problem is- how can I know that geniatagger is finished writing the text file?
try{
Runtime rt = Runtime.getRuntime();
Process p = rt.exec("geniatagger.exe -i "+ RawText+ " -o TAGGEDTEXT.txt");
}

You can't, or at least not reliably.
In this particular case your best bet is to watch the Process complete.
You get the process' return code as a bonus, this could tell you if an error occurred.
If you are actually talking about this GENIA tagger, below is a practical example which demonstrates various topics (see explanation about numbered comments beneath the code). The code was tested with v1.0 for Linux and demonstrates how to safely run a process which expects both input and output stream piping to work correctly.
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.util.concurrent.Callable;
import org.apache.commons.io.IOUtils;
public class GeniaTagger {
/**
* #param args
*/
public static void main(String[] args) {
tagFile(new File("inputText.txt"), new File("outputText.txt"));
}
public static void tagFile(File input, File output) {
FileInputStream ifs = null;
FileOutputStream ofs = null;
try {
ifs = new FileInputStream(input);
ofs = new FileOutputStream(output);
final FileInputStream ifsRef = ifs;
final FileOutputStream ofsRef = ofs;
// {1}
ProcessBuilder pb = new ProcessBuilder("geniatagger.exe");
final Process pr = pb.start();
// {2}
runInThread(new Callable<Void>() {
public Void call() throws Exception {
IOUtils.copy(ifsRef, pr.getOutputStream());
IOUtils.closeQuietly(pr.getOutputStream()); // {3}
return null;
}
});
runInThread(new Callable<Void>() {
public Void call() throws Exception {
IOUtils.copy(pr.getInputStream(), ofsRef); // {4}
return null;
}
});
runInThread(new Callable<Void>() {
public Void call() throws Exception {
IOUtils.copy(pr.getErrorStream(), System.err);
return null;
}
});
// {5}
pr.waitFor();
// output file is written at this point.
} catch (Exception e) {
e.printStackTrace();
} finally {
// {6}
IOUtils.closeQuietly(ifs);
IOUtils.closeQuietly(ofs);
}
}
public static void runInThread(final Callable<?> c) {
new Thread() {
public void run() {
try {
c.call();
} catch (Exception e) {
e.printStackTrace();
} finally {
}
}
}.start();
}
}
Use a ProcessBuilder to start your process, it has a better interface than plain-old Runtime.getRuntime().exec(...).
Set up stream piping in different threads, otherwhise the waitFor() call in ({5}) might never complete.
Note that I piped a FileInputStream to the process. According to the afore-mentioned GENIA page, this command expects actual input instead of a -i parameter. The OutputStream which connects to the process must be closed, otherwhise the program will keep running!
Copy the result of the process to a FileOutputStream, the result file your are waiting for.
Let the main thread wait until the process completes.
Clean up all streams.

If the program exits after generating the output file then you can call Process.waitFor() to let it run to completion then you can process the file. Note that you will likely have to drain both the standard output and error streams (at least on Windows) for the process to finish.
[Edit]
Here is an example, untested and likely fraught with problems:
// ...
Process p = rt.exec("geniatagger.exe -i "+ RawText+ " -o TAGGEDTEXT.txt");
drain(p.getInputStream());
drain(p.getErrorStream());
int exitCode = p.waitFor();
// Now you should be able to process the output file.
}
private static void drain(InputStream in) throws IOException {
while (in.read() != -1);
}