We Keep Coding

Force stop Java Files.copy() running on external thread

The answer here seemed to be a valid solution before Java 8:
How to cancel Files.copy() in Java?
But now it doesn't work, because ExtendedCopyOption.INTERRUPTIBLE is private.
Basically, I need to download a file from some given URL and save it to my local file-system using Files.copy().
Currently, I am using a JavaFX Service because I need to show the progress in a ProgressBar.
However, I don't know how to block the thread running Files.copy() if the operation takes too long.
Using Thread.stop() is at least not wanted. Even Thread.interrupt() fails.
I also want the operation to terminate gracefully if the internet connection becomes unavailable.
To test the case when no internet connection is available, I'm removing my ethernet cable and putting it back after 3 seconds.
Unfortunately, Files.copy() returns only when I put back the ethernet cable, while I would like it to fail immediately.
As I can see, internally Files.copy() is running a loop, which prevents the thread from exiting.
Tester(Downloading OBS Studio exe):
/**
* #author GOXR3PLUS
*
*/
public class TestDownloader extends Application {
/**
* #param args
*/
public static void main(String[] args) {
launch(args);
}
#Override
public void start(Stage primaryStage) throws Exception {
// Block From exiting
Platform.setImplicitExit(false);
// Try to download the File from URL
new DownloadService().startDownload(
"https://github.com/jp9000/obs-studio/releases/download/17.0.2/OBS-Studio-17.0.2-Small-Installer.exe",
System.getProperty("user.home") + File.separator + "Desktop" + File.separator + "OBS-Studio-17.0.2-Small-Installer.exe");
}
}
DownloadService:
Using #sillyfly comment with FileChannel and removing File.copy seems to work only with calling Thread.interrupt() but it is not exiting when the internet is not available..
import java.io.File;
import java.net.URL;
import java.net.URLConnection;
import java.nio.channels.Channels;
import java.nio.channels.FileChannel;
import java.nio.file.StandardOpenOption;
import java.util.logging.Level;
import java.util.logging.Logger;
import javafx.concurrent.Service;
import javafx.concurrent.Task;
/**
* JavaFX Service which is Capable of Downloading Files from the Internet to the
* LocalHost
*
* #author GOXR3PLUS
*
*/
public class DownloadService extends Service<Boolean> {
// -----
private long totalBytes;
private boolean succeeded = false;
private volatile boolean stopThread;
// CopyThread
private Thread copyThread = null;
// ----
private String urlString;
private String destination;
/**
* The logger of the class
*/
private static final Logger LOGGER = Logger.getLogger(DownloadService.class.getName());
/**
* Constructor
*/
public DownloadService() {
setOnFailed(f -> System.out.println("Failed with value: " + super.getValue()+" , Copy Thread is Alive? "+copyThread.isAlive()));
setOnSucceeded(s -> System.out.println("Succeeded with value: " + super.getValue()+" , Copy Thread is Alive? "+copyThread.isAlive()));
setOnCancelled(c -> System.out.println("Succeeded with value: " + super.getValue()+" , Copy Thread is Alive? "+copyThread.isAlive()));
}
/**
* Start the Download Service
*
* #param urlString
* The source File URL
* #param destination
* The destination File
*/
public void startDownload(String urlString, String destination) {
if (!super.isRunning()) {
this.urlString = urlString;
this.destination = destination;
totalBytes = 0;
restart();
}
}
#Override
protected Task<Boolean> createTask() {
return new Task<Boolean>() {
#Override
protected Boolean call() throws Exception {
// Succeeded boolean
succeeded = true;
// URL and LocalFile
URL urlFile = new URL(java.net.URLDecoder.decode(urlString, "UTF-8"));
File destinationFile = new File(destination);
try {
// Open the connection and get totalBytes
URLConnection connection = urlFile.openConnection();
totalBytes = Long.parseLong(connection.getHeaderField("Content-Length"));
// --------------------- Copy the File to External Thread-----------
copyThread = new Thread(() -> {
// Start File Copy
try (FileChannel zip = FileChannel.open(destinationFile.toPath(), StandardOpenOption.CREATE,
StandardOpenOption.TRUNCATE_EXISTING, StandardOpenOption.WRITE)) {
zip.transferFrom(Channels.newChannel(connection.getInputStream()), 0, Long.MAX_VALUE);
// Files.copy(dl.openStream(), fl.toPath(),StandardCopyOption.REPLACE_EXISTING)
} catch (Exception ex) {
stopThread = true;
LOGGER.log(Level.WARNING, "DownloadService failed", ex);
}
System.out.println("Copy Thread exited...");
});
// Set to Daemon
copyThread.setDaemon(true);
// Start the Thread
copyThread.start();
// -------------------- End of Copy the File to External Thread-------
// ---------------------------Check the %100 Progress--------------------
long outPutFileLength;
long previousLength = 0;
int failCounter = 0;
// While Loop
while ((outPutFileLength = destinationFile.length()) < totalBytes && !stopThread) {
// Check the previous length
if (previousLength != outPutFileLength) {
previousLength = outPutFileLength;
failCounter = 0;
} else
++failCounter;
// 2 Seconds passed without response
if (failCounter == 40 || stopThread)
break;
// Update Progress
super.updateProgress((outPutFileLength * 100) / totalBytes, 100);
System.out.println("Current Bytes:" + outPutFileLength + " ,|, TotalBytes:" + totalBytes
+ " ,|, Current Progress: " + (outPutFileLength * 100) / totalBytes + " %");
// Sleep
try {
Thread.sleep(50);
} catch (InterruptedException ex) {
LOGGER.log(Level.WARNING, "", ex);
}
}
// 2 Seconds passed without response
if (failCounter == 40)
succeeded = false;
// --------------------------End of Check the %100 Progress--------------------
} catch (Exception ex) {
succeeded = false;
// Stop the External Thread which is updating the %100
// progress
stopThread = true;
LOGGER.log(Level.WARNING, "DownloadService failed", ex);
}
//----------------------Finally------------------------------
System.out.println("Trying to interrupt[shoot with an assault rifle] the copy Thread");
// ---FORCE STOP COPY FILES
if (copyThread != null && copyThread.isAlive()) {
copyThread.interrupt();
System.out.println("Done an interrupt to the copy Thread");
// Run a Looping checking if the copyThread has stopped...
while (copyThread.isAlive()) {
System.out.println("Copy Thread is still Alive,refusing to die.");
Thread.sleep(50);
}
}
System.out.println("Download Service exited:[Value=" + succeeded + "] Copy Thread is Alive? "
+ (copyThread == null ? "" : copyThread.isAlive()));
//---------------------- End of Finally------------------------------
return succeeded;
}
};
}
}
Interesting questions:
1-> What does java.lang.Thread.interrupt() do?

I strongly encourage you to use a FileChannel.
It has the transferFrom() method which returns immediately when the thread running it is interrupted.
(The Javadoc here says that it should raise a ClosedByInterruptException, but it doesn't.)
try (FileChannel channel = FileChannel.open(Paths.get(...), StandardOpenOption.CREATE,
StandardOpenOption.WRITE)) {
channel.transferFrom(Channels.newChannel(new URL(...).openStream()), 0, Long.MAX_VALUE);
}
It also has the potential to perform much better than its java.io alternative.
(However, it turns out that the implementation of Files.copy() may elect to delegate to this method instead of actually performing the copy by itself.)
Here's an example of a reusable JavaFX Service that lets you fetch a resource from the internet and save it to your local file-system, with automatic graceful termination if the operation takes too long.
The service task (spawned by createTask()) is the user of the file-channel API.
A separate ScheduledExecutorService is used to handle the time constraint.
Always stick to the good practices for extending Service.
If you choose to use such an high-level method, you won't be able to track down the progress of the task.
If the connection becomes unavailable, transferFrom() should eventually return without throwing an exception.
To start the service (may be done from any thread):
DownloadService downloadService = new DownloadService();
downloadService.setRemoteResourceLocation(new URL("http://speedtest.ftp.otenet.gr/files/test1Gb.db"));
downloadService.setPathToLocalResource(Paths.get("C:", "test1Gb.db"));
downloadService.start();
and then to cancel it (otherwise it will be automatically cancelled after the time expires):
downloadService.cancel();
Note that the same service can be reused, just be sure to reset it before starting again:
downloadService.reset();
Here is the DownloadService class:
public class DownloadService extends Service<Void> {
private static final long TIME_BUDGET = 2; // In seconds
private final ScheduledExecutorService watchdogService =
Executors.newSingleThreadScheduledExecutor(new ThreadFactory() {
private final ThreadFactory delegate = Executors.defaultThreadFactory();
#Override
public Thread newThread(Runnable r) {
Thread thread = delegate.newThread(r);
thread.setDaemon(true);
return thread;
}
});
private Future<?> watchdogThread;
private final ObjectProperty<URL> remoteResourceLocation = new SimpleObjectProperty<>();
private final ObjectProperty<Path> pathToLocalResource = new SimpleObjectProperty<>();
public final URL getRemoteResourceLocation() {
return remoteResourceLocation.get();
}
public final void setRemoteResourceLocation(URL remoteResourceLocation) {
this.remoteResourceLocation.set(remoteResourceLocation);
}
public ObjectProperty<URL> remoteResourceLocationProperty() {
return remoteResourceLocation;
}
public final Path getPathToLocalResource() {
return pathToLocalResource.get();
}
public final void setPathToLocalResource(Path pathToLocalResource) {
this.pathToLocalResource.set(pathToLocalResource);
}
public ObjectProperty<Path> pathToLocalResourceProperty() {
return pathToLocalResource;
}
#Override
protected Task<Void> createTask() {
final Path pathToLocalResource = getPathToLocalResource();
final URL remoteResourceLocation = getRemoteResourceLocation();
if (pathToLocalResource == null) {
throw new IllegalStateException("pathToLocalResource property value is null");
}
if (remoteResourceLocation == null) {
throw new IllegalStateException("remoteResourceLocation property value is null");
}
return new Task<Void>() {
#Override
protected Void call() throws IOException {
try (FileChannel channel = FileChannel.open(pathToLocalResource, StandardOpenOption.CREATE,
StandardOpenOption.WRITE)) {
channel.transferFrom(Channels.newChannel(remoteResourceLocation.openStream()), 0, Long.MAX_VALUE);
}
return null;
}
};
}
#Override
protected void running() {
watchdogThread = watchdogService.schedule(() -> {
Platform.runLater(() -> cancel());
}, TIME_BUDGET, TimeUnit.SECONDS);
}
#Override
protected void succeeded() {
watchdogThread.cancel(false);
}
#Override
protected void cancelled() {
watchdogThread.cancel(false);
}
#Override
protected void failed() {
watchdogThread.cancel(false);
}
}

There is one important aspect not covered by the other answers/comments; and that is a wrong assumption of yours:
What I want is it to fail immediately when no internet connection is there.
It is not that easy. The TCP stack/state machine is actually a pretty complicated thing; and depending on your context (OS type; TCP stack implementation, kernel parameters, ...), there can be situations where a network partition takes place and a sender doesn't notice for 15 or more minutes. Listen here for more details on that.
In other words: "just pulling the plug" is no way equal to "immediately breaking" your existing TCP connection. And just for the record: you don't need to plug cables manually to simulate network outages. In a reasonable test setup, tools like iptables aka firewalls can do that for you.

You seem to need an Asynchronous/Cancellable HTTP GET which can be tough.
The problem is that if read stalls waiting for more data (cable is pulled) it won't quit until either the socket dies or new data comes in.
There are a few path you could follow, tinkering with socket factories to set a good timeout, using http client with timeouts and others.
I would have a look at Apache Http Components which has non blocking HTTP based on java NIO Sockets.

Why is my boolean not being changed?

So I'm trying to create a client/server program. I want to know when my client disconnects of his own accord, so I've setup a heartbeat system. Every 6 seconds my client sends a ping to my server, if the client doesn't send a ping for a total of 30 seconds the client is considered disconnected and removed from the current connections list (for which I plan to implement a GUI). Or at least, that's the plan.
ConnectionManager.java
public class ConnectionManager implements Runnable{
static Socket connection;
private ArrayList<Thread> allConnections;
private ArrayList<Connection> allConnectionList;
private ServerSocket server;
private int id = 0;
public ConnectionManager() {
allConnections = new ArrayList<Thread>();
allConnectionList = new ArrayList<Connection>();
}
#Override
public void run() {
try {
server = new ServerSocket(5555);
System.out.println("Server is running!");
while(true) {
connection = server.accept();
Connection a = new Connection(connection, id);
Runnable runnable = a;
allConnectionList.add(a);
allConnections.add(new Thread(runnable));
allConnections.get(allConnections.size() - 1).start();
id++;
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public void removeConnection(int id) {
allConnections.remove(id);
allConnectionList.remove(id);
}
Connection.java
public class Connection implements Runnable {
private Socket a;
public boolean amIActive;
private int id;
public Connection(Socket a, int id) {
amIActive = true;
this.a = a;
this.id = id;
}
public void onConnect() {
try {
String TimeStamp = new java.util.Date().toString();
String formattedAddress = a.getInetAddress().toString().replace("/", "");
System.out.println("Received connection from: " + formattedAddress + " at " + TimeStamp);
Runnable runnable = new ConnectionListener(this);
Thread connectionThread = new Thread(runnable);
connectionThread.start();
String returnCode = "Server repsonded to " + a.getInetAddress().toString().replace("/", "") + " at "+ TimeStamp + (char) 13;
BufferedOutputStream os = new BufferedOutputStream(a.getOutputStream());
OutputStreamWriter osw = new OutputStreamWriter(os, "US-ASCII");
osw.write(returnCode);
osw.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
#Override
public void run() {
onConnect();
System.out.println("We got this far!");
while(amIActive) {
whileTrue();
}
System.out.println("This code never gets run because we get stuck in the while loop above");
Main.b.removeConnection(id);
System.out.println("Connection was closed from " + a.getInetAddress());
}
public void setOffline(boolean state) {
this.amIActive = state;
}
public void whileTrue() {
}
public Socket getSocket() {
return a;
}
ConnectionListener.java
public class ConnectionListener implements Runnable{
public Connection myConnection;
public boolean receivedHeartbeat;
public int missedHeartbeats = 0;
public ConnectionListener(Connection a) {
this.myConnection = a;
}
#Override
public void run() {
Runnable runnable = new Heartbeat(this);
Thread thread = new Thread(runnable);
thread.start();
while(myConnection.amIActive) {
try {
BufferedInputStream is;
is = new BufferedInputStream(myConnection.getSocket().getInputStream());
InputStreamReader isr = new InputStreamReader(is);
StringBuffer process = new StringBuffer();
int character;
while((character = isr.read()) != 13) { //GETTING STUCK HERE BECAUSE STUPID.
if(character == -1) {
myConnection.setOffline(true);
} else {
process.append((char)character);
}
}
handleInput(process);
} catch (Exception e) {
e.printStackTrace();
}
}
}
public void handleInput(StringBuffer process) {
String messageSent = process.toString();
if(messageSent.equals("Ping!")) {
receivedHeartbeat = true;
}
}
Heartbeat.java
public class Heartbeat implements Runnable{
private ConnectionListener b;
public Heartbeat(ConnectionListener a) {
b = a;
}
#Override
public void run() {
while(true) {
try {
Thread.sleep(1000);
if(b.missedHeartbeats > 5) {
b.myConnection.amIActive = false;
System.out.println("Setting amIActiveToFalse!");
}
if(b.receivedHeartbeat) {
b.receivedHeartbeat = false;
} else {
b.missedHeartbeats++;
}
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
My console is spammed with System.out.println("Setting amIActiveToFalse!"); from Heartbeat.java. But the while loop in Connection.java keeps running. I believe this might be something to do with my threading, but I can't figure it out.

When you have a non-volatile variable, there is no guarentee of visability of a change in one thread to another. In particular, if the JVM detects that a thread doesn't alter a boolean it can inline it, meaning you will never see the value change.
The simple solution is to make the boolean volatile and it will not be inlined and one thread will see when another changes it.
For more details http://vanillajava.blogspot.com/2012/01/demonstrating-when-volatile-is-required.html

The trivial answer to this is: make the variable volatile.
Without this, it is allowed for the thread changing the value to basically keep its updates in cache, committing them to main memory some time later.
This allows threaded code to run much faster, since it can keep its variables in cache rather than having to fetch from main memory. However, the consequence of this is that other threads don't see the update.
Making the variable volatile prevents this from happening: a thread always reads the value from main memory, and writes are immediately committed.
I say that this is the trivial answer because it doesn't necessarily fix all of your problems. There may also be an atomicity issue: in between one thread reading the variable and writing it again, another thread might sneak in and change its value, which may or may not put the first thread into an undefined state from the perspective of its invariants.
Specifically:
if(b.receivedHeartbeat) { b.receivedHeartbeat = false;
It is possible that some other thread can change b.receivedHeartbeat to false after this thread evaluates it to true, so this iteration is erroneously counted as a "non-missed" heartbeat.
This can be fixed by making the variable a (non-volatile) AtomicBoolean, on which there is an atomic compare-and-set method, which avoids such race conditions.
Java Concurrency In Practice is a great reference on these issues, I wholeheartedly recommend it. Look for the topics "visibility" and "atomicity".
Also read the advanced chapter on the Java Memory Model. That made me doubt myself at first, but made me a much stronger programmer after I digested it.

There are a couple issues I saw while debugging the code you posted, but I was able to successfully get the heartbeat functionality working.
In the Connection Listener class I don't think the if statement with .equals("Ping!") will match, because of the newline character at the end of each line.
In the Connection Listener class I would probably put the socket's Input Stream at the top of the loop not inside the loop. (I don't think this will break it but it's probably nicer this way)
ConnectionListener Updates:
public void run() {
Runnable runnable = new Heartbeat(this);
Thread thread = new Thread(runnable);
thread.start();
BufferedReader br = null;
try {
//is = new BufferedInputStream(myConnection.getSocket().getInputStream());
br = new BufferedReader(new InputStreamReader(myConnection.getSocket().getInputStream()));
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
while(myConnection.amIActive) {
try {
String processLine = br.readLine();
System.out.println("handleInput:" + processLine);
handleInput(processLine);
} catch (Exception e) {
System.out.println("Exception!");
e.printStackTrace();
}
}
}
public void handleInput(String messageSent) {
if(messageSent.startsWith("Ping!")) { //Need to use startsWith, or add newline character
receivedHeartbeat = true;
System.out.println("receivedHeartbeat!");
}
}
Also, in your Heartbeat class make sure you reset the missedHeartbeats counter to 0 on true:
if(b.receivedHeartbeat) {
b.receivedHeartbeat = false;
b.missedHeartbeats = 0;
} else {
b.missedHeartbeats++;
}

Tapestry Hibernate session closed after exceeding ExecutorService fixed thread pool

I'm a tapestry-hibernate user and I'm experiencing an issue where my session remains closed once I exceed my Executors.newFixedThreadPool(1);
I have the following code which will work perfectly for the first thread while the remaining threads experience a closed session. If I increase the thread pool to 10, all the threads will run without issue. As soon as I exceed the fixedThreadPool, I get the session closed exception. I do not know how to open it since it's managed by tapestry-hibernate. If I use newCachedThreadPool, everything works perfectly. Does anybody know what might be happening here?
public void setupRender() {
ExecutorService executorService = Executors.newFixedThreadPool(1);
final ConcurrentHashMap<String, Computer> map = new ConcurrentHashMap<>();
final String key = "myKey";
final Date date = new Date();
List<Future> futures = new ArrayList<>();
for (int i = 0; i < 10; i++) {
final int thread = i;
Future future = executorService.submit(new Callable() {
#Override
public String call() {
try {
Computer computer = new Computer("Test Computer thread");
computer = getComputer(map, key, key, computer);
Monitor monitor = new Monitor();
monitor.setComputer(computer);
session.save(monitor);
session.flush();
System.out.println("thread " + thread);
try {
sessionManager.commit();
} catch (HibernateException ex) {
sessionManager.abort();
} finally {
session.close();
}
} catch (Exception ex) {
System.out.println("ex " + ex);
}
System.out.println( new Date().getTime() - date.getTime());
return "completed";
}
});
futures.add(future);
}
for(Future future : futures) {
try {
System.out.println(future.get());
} catch (InterruptedException | ExecutionException ex) {
Logger.getLogger(MultiThreadDemo.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
public synchronized Computer getComputer(ConcurrentHashMap<String, Computer> map, String key, String thread, Computer computer) {
if (map.putIfAbsent(key, computer) == null) {
session.save(computer);
} else {
computer = map.get(key);
}
return computer;
}

I've told you this before.... you MUST either use ParallelExecutor OR call PerThreadManager.cleanup(). You need to understand that tapestry-hibernate has PerThread scoped services that MUST be cleaned up if you are using them outside of a normal request/response (or ParallelExecutor).
I also don't think you should be calling session.close(). You should mimmic CommitAfterWorker.
It would probably look something like:
#Inject PerThreadManager perThreadManager;
#Inject HibernateSessionManager sessionManager; // this is a proxy to a per-thread value
#Inject Session session; // this is a proxy to a per-thread value
public void someMethod() {
ExecutorService executorService = ...;
executorService.submit(new Callable() {
public String call() {
try {
Monitor monitor = ...
session.save(monitor);
session.flush(); // optional
sessionManager.commit();
} catch (Exception ex) {
sessionManager.abort();
} finally {
// this allows Session and HibernateSessionManager to
// clean up after themselves
perThreadManager.cleanup();
}
return ...
}
});
}
If you choose to use the ParallelExecutor (and Invokable) instead of Executors.newFixedThreadPool(1) you can remove the references to PerThreadManager since it automatically cleans up the thread.

OutOfMemoryError: unable to create new native thread using ExecutorService

I launched my instance overnight to see how it handled things and when I came by this morning, I was facing a
Exception in thread "pool-535-thread-7" java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:691)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:943)
at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:992)[info] application - Connecting to server A
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
The aim of my code is quite simple : Every 5 minutes, I connect to a list of remote server, send a request (via socket) and that's it.
Here's my code :
My "cron" task :
/** will create a new instance of ExecutorService every 5 minutes, loading all the websites in the database to check their status **/
/** Maybe that's where the problem is ? I need to empty (GC ?) this ExecutorService ? **/
Akka.system().scheduler().schedule(
Duration.create(0, TimeUnit.MILLISECONDS), // Initial delay 0 milliseconds
Duration.create(5, TimeUnit.MINUTES), // Frequency 5 minutes
new Runnable() {
public void run() {
// We get the list of websites to check
Query<Website> query = Ebean.createQuery(Website.class, "WHERE disabled = false AND removed IS NULL");
query.order("created ASC");
List<Website> websites = query.findList(); // Can be 1, 10, 100, 1000. In my test case, I had only 9 websites.
ExecutorService executor = Executors.newFixedThreadPool(NTHREDS);
for (Website website : websites) {
CheckWebsite task = new CheckWebsite(website);
executor.execute(task);
}
// This will make the executor accept no new threads
// and finish all existing threads in the queue
executor.shutdown();
}
},
Akka.system().dispatcher()
);
My CheckWebsite class :
public class CheckWebsite implements Runnable {
private Website website;
public CheckWebsite(Website website) {
this.website = website;
}
#Override
public void run() {
WebsiteLog log = website.checkState(); // This is where the request is made, I copy paste the code just after
if (log == null) {
Logger.error("OHOH, WebsiteLog should not be null for website.checkState() in CheckWebsite class :s");
return;
}
try {
log.save();
catch (Exception e) {
Logger.info ("An error occured :/");
Logger.info(e.getMessage());
e.printStackTrace();
}
}
}
My checkState() method in Website.class :
public WebsiteLog checkState() {
// Since I use Socket and the connection can hang indefinitely, I use an other ExecutorService in order to limit the time spent
// The duration is defined via Connector.timeout, Which will be the next code.
ExecutorService executor = Executors.newFixedThreadPool(1);
Connector connector = new Connector(this);
try {
final long startTime = System.nanoTime();
Future<String> future = executor.submit(connector);
String response = future.get(Connector.timeout, TimeUnit.MILLISECONDS);
long duration = System.nanoTime() - startTime;
return PlatformLog.getLastOccurence(this, response, ((int) duration/ 1000000));
}
catch (Exception e) {
return PlatformLog.getLastOccurence(this, null, null);
}
}
Here's the Connector.class. I removed useless part here (like Catches) :
public class Connector implements Callable<String> {
public final static int timeout = 2500; // WE use a timeout of 2.5s, which should be enough
private Website website;
public Connector(Website website) {
this.website = website;
}
#Override
public String call() throws Exception {
Logger.info ("Connecting to " + website.getAddress() + ":" + website.getPort());
Socket socket = new Socket();
try {
socket.connect(new InetSocketAddress(website.getIp(), website.getPort()), (timeout - 50));
BufferedReader input = new BufferedReader(new InputStreamReader(socket.getInputStream()));
String response = input.readLine();
socket.close();
return response;
}
catch (Exception e) {
e.printStackTrace();
throw e;
}
finally {
// I take the precaution to close the socket here in order to avoid a memory leak
// But if the previous ExecutorService force the close of this thread before
// I can't guarantee it will be closed :/
if (socket != null && !socket.isClosed()) {
socket.close();
}
}
}
}
I'm new to Java multithreading so I probably made big mistake. I suspect some area that could be potentially the reason, but my lack of knowledge requires me to ask for your help :)
As a summary, here's the potentials areas :
Creating a new ExecutorService every 5 minutes. Maybe I can reuse the old one ? Or do I need to close the current one when finished (if so, how ?).
The fact that I create an ExecutorService that will create an ExecutorService (in the checkstate() method)
The fact that the Connector class can be (violently) stopped by the ExecutorService running it, if it takes too long, resulting in a socket not closed (and then a memory leak) ?
Also, as you can see, the exception occured for the thread "pool-535-thread-7" which mean it didn't happen soon.
I store the last_occured check in the database, and the creation of the log entry (in WebsiteLog), the delta is around 5 hours (so, for every 5 minutes, the thread crashed after around 60 calls).
Update : Here's the revisited checkState method to include the shutdown call :
public PlatformLog checkState() {
ExecutorService executor = Executors.newFixedThreadPool(1);
Connector connector = new Connector(this);
String response = null;
Long duration = null;
try {
final long startTime = System.nanoTime();
Future<String> future = executor.submit(connector);
response = future.get(Connector.timeout, TimeUnit.MILLISECONDS);
duration = System.nanoTime() - startTime;
}
catch (Exception e) {}
executor.shutdown();
if (duration != null) {
return WebsiteLog.getLastOccurence(this, response, (duration.intValue()/ 1000000));
}
else {
return WebsiteLog.getLastOccurence(this, response, null);
}
}

I'm not sure this is the only problem, but you are creating an ExecutorService in your checkState() method but you don't shut it down.
According to the JavaDocs for Executors.newFixedThreadPool():
The threads in the pool will exist until it is explicitly shutdown.
The threads staying alive will cause the ExecutorService not to be garbage collected (which would call shutdown() on your behalf. Hence you are leaking a thread each time this is called.

Java - FutureTask not working?

I am creating a Java application. On start up, my application will download all required files. My application will parse XML file and download files from the URL from XML files. I want my application download files "step by step" so i use FutureTask My problem is, FutureTask is not working for my application.
Here is some part of my code.
Startup.class
public void startDownloading()
{
Thread t = new Thread(new Runnable()
{
public void run()
{
downloader.startDownload();
}
});
t.run();
}
}
Downloader.class
private LibrariesDownloader ld;
private RDownloader rd;
public Downloader()
{
this.ld = new LibrariesDownloader(launcher);
this.rd = new RDownloader(launcher);
}
public void startDownload()
{
ExecutorService executor = Executors.newFixedThreadPool(2);
FutureTask<Void> libDownloader = new FutureTask<Void>(ld);
FutureTask<Void> resDownloader = new FutureTask<Void>(rd);
executor.execute(libDownloader);
if(libDownloader.isDone())
{
executor.execute(resDownloader);
}
}
LibrariesDownloader.class(& RDownloader.class(Code almost the same, only URL is different))
public class LibrariesDownloader implements Callable<Void>
{
private Proxy proxy = Proxy.NO_PROXY;
#Override
public Void call() throws Exception
{
try
{
URL resourceUrl = new URL("http://www.exmaple.com/libraries.xml");
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder();
Document doc = db.parse(resourceUrl.openConnection(proxy).getInputStream());
NodeList nodeLst = doc.getElementsByTagName("Content");
for (int i = 0; i < nodeLst.getLength(); i++)
{
Node node = nodeLst.item(i);
if (node.getNodeType() == 1)
{
Element element = (Element)node;
String key = element.getElementsByTagName("Key").item(0).getChildNodes().item(0).getNodeValue();
final File path = new File("C://test/", key);
final String url = "http://www.exmaple.com/dl/" + key;
final String fileName = key;
SwingWorker<Void, Void> worker = new SwingWorker<Void, Void>()
{
#Override
protected Void doInBackground() throws Exception
{
try
{
URL fileURL = new URL(url);
org.apache.commons.io.FileUtils.copyURLToFile(fileURL, path);
}
catch(Exception e)
{
URL redownloadURL = new URL("http://www.example.com/dl/" + fileName);
File p = new File("C://test/", fileName);
org.apache.commons.io.FileUtils.copyURLToFile(redownloadURL, p);
}
return null;
}
#Override
public void done()
{
System.out.println(fileName + " had downloaded successfully");
}
};
worker.execute();
}
}
}
catch(Exception e)
{
launcher.println("An error was found when trying to download libraries file " + e);
}
return null;
}
}
There is ton of <Key></Key> in my XML file. My application can execute LibrariesDownloader and download all libraries files. After all libraries files had downloaded, my application just stop there. It will not execute RDownloader.
Is that any code wrong in my application? Thanks for helping me.

You start a new thread with
t.run();
It should be t.start(). The thread scheduler calls the run().
You probably want a busy/wait loop or a timeout for LibrariesDownloader
if(libDownloader.isDone())
{
executor.execute(resDownloader);
}
should be
Future<?> future = executor.submit(libDownloader);
while (!future.isDone()) {
//bad spin wait
}
executor.execute(resDownloader);
Better still, make a single ThreadPoolExecutor with Executors.newSingleThreadExecutor() or the more robust Executors.newFixedThreadPool(1) and submit both of them. The 2nd task will be queued.
The code snippet would look like
ExecutorService executor = Executors.newFixedThreadPool(1);
executor.execute(libDownloader);
executor.execute(resDownloader);