I want to sandbox an application where end users can submit Java code to a server where it is compiled and executed (kind of a web-based IDE as part of an educational game). Most aspects can easily be handled by using either the standard security manager or verification of used APIs against a whitelist using ASM or similar.
An open problem is how to deal with infinite loops. As threads have their own stack, StackOverFlowErrors seem to be thread-local. I have done a little spike and came up with this:
public class TryToSurviveAStackOverflow {
public static void main(String[] args) throws Exception {
final Runnable infiniteLoop = new Runnable() {
#Override public void run() {
run();
}
};
Runnable sandboxed = new Runnable() {
#Override public void run() {
try {
Thread.sleep(10000); // some time to connect VisualVM to monitor this
infiniteLoop.run();
}
catch (StackOverflowError x) {
System.err.println("Thread crashed with stack overflow");
} catch (InterruptedException e) {
System.err.println("Thread interruped");
}
}
};
Thread thread = new Thread(sandboxed,"infinite loop");
thread.start();
thread.join();
System.out.println(thread.getState());
System.out.println("I am still alive");
}
}
This seems to work, but how safe is this? In particular, what happens to the stack space used by the unsafe thread? I can see that the state of the thread is set to TERMINATED.
Any help / pointers are highly appreciated !
Cheers, Jens
Related
I'm using a java 1.8 java.util.concurrent.LinkedBlockingQueue, and when I call:
LinkedBlockingQueue.poll(5000, TimeUnit.MILLISECONDS)
it is very occasionally throwing an InterruptedException:
java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088)
at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
which I think is happening because the following is returning true during the (indirect) call to checkInterruptWhileWaiting() at AbstractQueuedSynchronizer:2079
Unsafe.compareAndSwapInt(...)
As a side note, Unsafe.compareAndSwapInt returns a boolean, but what does that boolean mean? I can't find any documentation on that Class/function.
I suspect that something is going on in another thread to cause this issue, but I'm not sure where to look right now.
Any help on understanding why the InterruptedException is being thrown would be very helpful. I would really like to be able to reproduce it in a small test program, but it's in a big messy program right now so I'm trying to understand what things could cause this so that I can create a test program to reproduce it.
Is there any other thread in your app that calls Thread.interrupt()? This is what's happening in awaitInNanos():
if (Thread.interrupted())
throw new InterruptedException();
If you control the threads, then you can override the interrupt method just for testing:
Thread thread = new Thread() {
#Override
public void run() {
// do something longrunning
}
#Override
public void interrupt() {
// try-finally ensures to run both super.interrupt() and the deubg code
try {
super.interrupt();
} finally {
// you can use any logging services that you already have
System.out.println("--> Interrupted from thread: " + Thread.currentThread().getName());
Thread.dumpStack();
}
}
};
If you create the threads manually, you can override interrupt(). If you use executors, then you can provide a ThreadFactory, that creates the threads with the right interrupt() method.
Here is a main() method that plays with this debug technique. Please note that you need to enter a line in STDIN or manually kill the process. Otherwise it's going to run forever (jvm restart).
public static void main(String[] args) {
Thread thread = new Thread() {
#Override
public void run() {
System.out.println("--> asdf");
try (BufferedReader br = new BufferedReader(new InputStreamReader(System.in))) {
br.readLine();
} catch (Exception ex) {
throw new RuntimeException(ex);
}
}
#Override
public void interrupt() {
// try-finally ensures to run both super.interrupt() and the deubg code
try {
super.interrupt();
} finally {
// you can use any logging services that you already have
System.out.println("--> Interrupted from thread: " + Thread.currentThread().getName());
Thread.dumpStack();
}
}
};
thread.start();
System.out.println("--> before interrupt");
thread.interrupt();
System.out.println("--> after interrupt");
}
I've been using Java for a few years, but my thread knowledge is rubbish. I've Googled pretty heavily and found some good information about general use of ProgressMonitorDialog but nothing like my exact circumstances.
I'm currently using a ProgressMonitorDialog as a wrapper around an instance of IRunnableWithProgress, which in turn is a wrapper around a Thread. This works fine but now I'm trying to make the cancel button trigger an interrupt on the running thread, which I can handle to gracefully terminate the operation.
One important thing to note is that I have two plugins; "Data" and "UI". The data plugin contains all of the real work, and must be independent from the UI or any Eclipse plugins. The UI plugin should be as thin as possible.
Here's a distilled version of the code I've got so far.
Data:
public class Data {
public static Thread createThread() {
return new Thread() {
#Override
public void run() {
Thing t = new Thing();
t.operationA();
t.operationB();
t.operationC();
}
}
}
}
UI:
public class UI {
public void doOperation() {
try {
new ProgressMonitorDialog(getShell()).run(true, true, new MyOperation());
}
catch (Exception e) {
e.printStatckTrace();
}
}
public class MyOperation implements IRunnableWithProgress {
#Override
public void run(IProgressMonitor monitor) throws InterruptedException, InvocationTargetException {
monitor.beginTask("Task", 2);
try {
Thread myThread = Data.createThread();
myThread.start();
monitor.worked(1);
while (myThread.isAlive() && !monitor.isCanceled()) {}
if (monitor.isCanceled()) {
myThread.interrupt();
}
monitor.worked(1);
}
finally {
monitor.done();
}
}
}
}
So when the cancel button is clicked, myThread.interrupt() is called. Now the thread needs to respond to the interrupt. Data.createThread() now looks something like this:
public static Thread createThread() {
return new Thread() {
#Override
public void run() {
Thing t = new Thing();
t.operationA();
if (Thread.currentThread.isInterrupted()) {
// Tidy up
return;
}
t.operationB();
if (Thread.currentThread.isInterrupted()) {
// Tidy up
return;
}
t.operationC();
if (Thread.currentThread.isInterrupted()) {
// Tidy up
return;
}
}
}
}
It might be rather verbose polling the interrupted state like this, but I can't see this causing any problems.
But, what if Thing.operationA() wasn't atomic, and could be interrupted within that function:
public class Thing {
public void operationA() {
atomic1();
// How would I detect and handle a change to the interrupted state here?
atomic2();
}
public void operationB() {
// One atomic operation
}
public void operationC() {
// One atomic operation
}
}
How would I detect and handle a change to the interrupted state between atomic1() and atomic2()? Is it as simple as polling Thread.currentThread.isInterrupted() again? Or will I need to pass around some volatile object to track the interrupted state? Should I be throwing InterruptedException somewhere?
My second question is about tracking and reporting progress. I understand how IProgressMonitor.worked() should be used. As already seen, my Data thread contains 3 operations. Is it possible to pass that information up to the UI so I can track the progress in the ProgressMonitorDialog?
Ideally, something like this:
public static Thread createThread() {
return new Thread() {
#Override
public void run(IProgressMonitor monitor) {
Thing t = new Thing();
t.operationA();
monitor.worked(1);
if (Thread.currentThread.isInterrupted()) {
// Tidy up
return;
}
t.operationB();
monitor.worked(1);
if (Thread.currentThread.isInterrupted()) {
// Tidy up
return;
}
t.operationC();
monitor.worked(1);
if (Thread.currentThread.isInterrupted()) {
// Tidy up
return;
}
}
}
}
However as stated, Data cannot depend on Eclipse and therefore passing the IProgressMonitor doesn't work in this case.
Could I have a variable tracking progress in my thread, and then call something like myThread.getProgress() asynchronously from the UI thread to update the progress bar with new work? I'm not sure how feasible this is (it popped into my head as I was writing this question) so I'll try that next.
Lots of information and question marks in here, sorry if my style is a bit scattered. I could elaborate more if needs be but this is already a wall of text. Any information, advice or ideas appreciated.
Between atomic1() and atomic2() you do need to check for Thread.currentThread.isInterrupted() to cleanup in case of canceling. No need to throw an exception if you handle what is needed.
As for progress tracking, you can create your own listener object in the Data plugin and allow passing it to the thread. the UI will instantiate it and pass it to the thread. this way the Data can pass progress events to the UI without dependencies.
I have the following thread which simply prints a dot every 200ms:
public class Progress {
private static boolean threadCanRun = true;
private static Thread progressThread = new Thread(new Runnable()
{
public void run() {
while (threadCanRun) {
System.out.print('.');
System.out.flush();
try {
progressThread.sleep(200);
} catch (InterruptedException ex) {}
}
}
});
public static void stop()
{
threadCanRun = false;
progressThread.interrupt();
}
public static void start()
{
if (!progressThread.isAlive())
{
progressThread.start();
} else
{
threadCanRun = true;
}
}
}
I start the thread with this code (for now):
System.out.println("Working.");
Progress.start();
try {
Thread.sleep(10000); //To be replaced with code that does work.
} catch (InterruptedException ex) {}
Progress.stop();
What's really strange is this:
If I use System.out.println('.'); , the code works exactly as expected. (Apart from the fact that I don't want a new line each time).
With System.out.print('.');, the code waits for ten seconds, and then shows the output.
System.out.println:
Print dot, wait 200ms, print dot, wait 200ms etc...
System.out.print:
Wait 5000ms, Print all dots
What is happening, and what can I do to go around this behaviour?
EDIT:
I have also tried this:
private static synchronized void printDot()
{
System.err.print('.');
}
and printDot() instead of System.out.print('.');
It still doesn't work.
EDIT2:
Interesting. This code works as expected:
System.out.print('.');
System.out.flush(); //Makes no difference with or without
System.out.println();
This doesn't:
System.err.print('.');
System.err.flush();
System.out.print('.');
System.out.flush();
Solution: The issue was netbeans related. It worked fine when I run it as a jar file from java -jar.
This is one of the most frustrating errors I have seen in my life. When I try to run this code with breakpoints in debug mode, everything works correctly.
The stdout is line buffered.
Use stderr, or flush the PrintStream after each print.
(This is weird code -- there are much cleaner ways to write and manage threads. But, that's not the issue.)
Your IDE must be buffering by line. Try running it directly on the command line. (And hope that the shell isn't buffering either, but shouldn't.)
The println method automatically flushes the output buffer, the print method not. If you want to see the output immediately, a call to System.out.flush might help.
I think this is because the println() method is synchronized
(This is not an answer; the asker, David, requested that I follow up on a secondary point about rewriting the threading. I am only able to post code this way.)
public class Progress {
private ProgressRunnable progressRunnable = new ProgressRunnable();
public void start() {
new Thread(progressRunnable).start();
}
public void stop() {
progressRunnable.stop();
}
private class ProgressRunnable implements Runnable {
private final AtomicBoolean running = new AtomicBoolean(true);
#Override
public void run() {
while (running.get()) {
System.out.print('.');
System.out.flush();
try {
Thread.sleep(200);
} catch (InterruptedException e) {
}
}
}
private void stop() {
running.set(false);
}
}
public static void main(String[] args) {
Progress progress = new Progress();
progress.start();
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
}
progress.stop();
}
}
I tested your code, with System.out.print() and System.out.flush(). The code works for me, except for the code:
while (!threadCanRun)
{
Thread.yield();
}
in Progress class. Doing that, the thread is pausing allowing other thread to execute, as you can see in the thread api page. Removing this part, the code works.
But I don't understand why do you need the yield method. If you call Progress.stop(), this will cause to invoke the yield method. After the thread will stop with interrupt, (after waiting a huge amount of time on my pc).
If you want to allow other threads executing and the current thread pausing, consider the join() method.
If you want to stop the current thread, maybe you can consider to remove the
while(!threadCanRun) loop, or place Thread.currentThread().join() before Thread.interrupt() in the stop() method to wait for the completion of other threads, or simply call the p.stop() method .
Take a look to these posts.
(individual question from IDE-Style program running )
Your best option is probably to fork off a new JVM through ProcessBuilder.
It is however possible to kill the internal program (and all the threads it spawned) using ThreadGroups. (I don't recommend it though. It uses the stop method which according to docs is "Deprecated. This method is inherently unsafe. See Thread.stop() for details."):
public class Main {
public static void main(String args[]) throws InterruptedException {
ThreadGroup internalTG = new ThreadGroup("internal");
Thread otherProcess = new Thread(internalTG, "Internal Program") {
public void run() {
OtherProgram.main(new String[0]);
}
};
System.out.println("Starting internal program...");
otherProcess.start();
Thread.sleep(1000);
System.out.println("Killing internal program...");
internalTG.stop();
}
}
class OtherProgram {
public static void main(String[] arg) {
for (int i = 0; i < 5; i++)
new Thread() {
public void run() {
System.out.println("Starting...");
try {
sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Stopping...");
}
}.start();
}
}
Output:
Starting internal program...
Starting...
Starting...
Starting...
Starting...
Starting...
Killing internal program...
Run the code in a separate JVM. Use debugger interfaces to control that JVM.
Instrument the byte code of the classes you intend to run. Insert cancellation checks at appropriate places, as well as trapping access to global JVM resources.
The second choice is probably an endless source of annoying bugs.
I have created a threaded service the following way:
public class TCPClientService extends Service{
...
#Override
public void onCreate() {
...
Measurements = new LinkedList<String>();
enableDataSending();
}
#Override
public IBinder onBind(Intent intent) {
//TODO: Replace with service binding implementation
return null;
}
#Override
public void onLowMemory() {
Measurements.clear();
super.onLowMemory();
}
#Override
public void onDestroy() {
Measurements.clear();
super.onDestroy();
try {
SendDataThread.stop();
} catch(Exception e){
...
}
}
private Runnable backgrounSendData = new Runnable() {
public void run() {
doSendData();
}
};
private void enableDataSending() {
SendDataThread = new Thread(null, backgrounSendData, "send_data");
SendDataThread.start();
}
private void addMeasurementToQueue() {
if(Measurements.size() <= 100) {
String measurement = packData();
Measurements.add(measurement);
}
}
private void doSendData() {
while(true) {
try {
if(Measurements.isEmpty()) {
Thread.sleep(1000);
continue;
}
//Log.d("TCP", "C: Connecting...");
Socket socket = new Socket();
socket.setTcpNoDelay(true);
socket.connect(new InetSocketAddress(serverAddress, portNumber), 3000);
//socket.connect(new InetSocketAddress(serverAddress, portNumber));
if(!socket.isConnected()) {
throw new Exception("Server Unavailable!");
}
try {
//Log.d("TCP", "C: Sending: '" + message + "'");
PrintWriter out = new PrintWriter( new BufferedWriter( new OutputStreamWriter(socket.getOutputStream())),true);
String message = Measurements.remove();
out.println(message);
Thread.sleep(200);
Log.d("TCP", "C: Sent.");
Log.d("TCP", "C: Done.");
connectionAvailable = true;
} catch(Exception e) {
Log.e("TCP", "S: Error", e);
connectionAvailable = false;
} finally {
socket.close();
announceNetworkAvailability(connectionAvailable);
}
} catch (Exception e) {
Log.e("TCP", "C: Error", e);
connectionAvailable = false;
announceNetworkAvailability(connectionAvailable);
}
}
}
...
}
After I close the application the phone works really slow and I guess it is due to thread termination failure.
Does anyone know what is the best way to terminate all threads before terminating the application?
Addendum: The Android framework provides many helpers for one-off work, background work, etc, which may be preferable over trying to roll your own thread in many instances. As mentioned in a below post, AsyncTask is a good starting point to look into. I encourage readers to look into the framework provisions first before even beginning to think about doing their own threading.
There are several problems in the code sample you posted I will address in order:
1) Thread.stop() has been deprecated for quite some time now, as it can leave dependent variables in inconsistent states in some circumstances. See this Sun answer page for more details (Edit: that link is now dead, see this page for why not to use Thread.stop()). A preferred method of stopping and starting a thread is as follows (assuming your thread will run somewhat indefinitely):
private volatile Thread runner;
public synchronized void startThread(){
if(runner == null){
runner = new Thread(this);
runner.start();
}
}
public synchronized void stopThread(){
if(runner != null){
Thread moribund = runner;
runner = null;
moribund.interrupt();
}
}
public void run(){
while(Thread.currentThread() == runner){
//do stuff which can be interrupted if necessary
}
}
This is just one example of how to stop a thread, but the takeaway is that you are responsible for exiting a thread just as you would any other method. Maintain a method of cross thread communcation (in this case a volatile variable, could also be through a mutex, etc) and within your thread logic, use that method of communication to check if you should early exit, cleanup, etc.
2) Your measurements list is accessed by multiple threads (the event thread and your user thread) at the same time without any synchronization. It looks like you don't have to roll your own synchronization, you can use a BlockingQueue.
3) You are creating a new Socket every iteration of your sending Thread. This is a rather heavyweight operation, and only really make sense if you expect measurements to be extremely infrequent (say one an hour or less). Either you want a persistent socket that is not recreated every loop of the thread, or you want a one shot runnable you can 'fire and forget' which creates a socket, sends all relevant data, and finishes. (A quick note about using a persistent Socket, socket methods which block, such as reading, cannot be interrupted by Thread.interrupt(), and so when you want to stop the thread, you must close the socket as well as calling interrupt)
4) There is little point in throwing your own exceptions from within a Thread unless you expect to catch it somewhere else. A better solution is to log the error and if it is irrecoverable, stop the thread. A thread can stop itself with code like (in the same context as above):
public void run(){
while(Thread.currentThread() == runner){
//do stuff which can be interrupted if necessary
if(/*fatal error*/){
stopThread();
return; //optional in this case since the loop will exit anyways
}
}
}
Finally, if you want to be sure a thread exits with the rest of your application, no matter what, a good technique is to call Thread.setDaemon(true) after creation and before you start the thread. This flags the thread as a daemon thread, meaning the VM will ensure that it is automatically destroyed if there are no non-daemon threads running (such as if your app quits).
Obeying best practices with regards to Threads should ensure that your app doesn't hang or slow down the phone, though they can be quite complex :)
Actually, you don't need the "runner" variable as described above, something like:
while (!interrupted()) {
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
break;
}
}
But generally, sitting in a Thread.sleep() loop is a really bad idea.
Look at the AsyncTask API in the new 1.5 API. It will probably solve your problem more elegantly than using a service. Your phone is getting slow because the service never shuts down - there's nothing that will cause the service to kill itself.