Assume that I have some code that because of a bug sometimes does
while(true){/*...*/}
It is in a third part library that I have no source code for.
In Java I can isolate it in a CompletableFuture. But since the 3th party code is not cooperating Timeout will not work on The future. Stopping a custom Thread pool also does not work. I tested that with a literal 'while true...'
Java's Thread.stop() is deprecated, but works more or less.
How should I stop such never returning non cooperating code after a certain time in Java? Should I use a process?
Same question for C# and NodeJS.
In c# context you have Thread.Abort() which will try to intercept on each managed code execution step and stop threads execution. Although, this option is not recommended since it can cause thread to be left in corrupted state while it is being killed, unmanaged resources hanging and other unwanted behavior.
A better idea would be to bound execution of this 3rd party code in a different process and end process with your provided time out. That incurs a greater performance penalty but you do not risk of any resources or locks hanging.
Related
TL;DR:
Is there a way to find out, that JVM-shutdown is only prevented from the threads started by my code? Is it for example possible to automatically trigger AutoCloseable.close() on Shutdown?
Context
I am building a library, that should be used by several customers. This means, besides providing a documentation, I can't enforce certain things.
Architecture
(I try to describe it as abstract as possible and avoid unnecessary details)
I have a "Manager" object (which is kind of a Factory), that is used to create a "Service" object, that in turn needs some data to work accordingly. Since that data is loaded from some "slow" backend service (which also might change from time to time), I use a separate (Daemon)-Thread that checks for updates and injects new data into that service as soon as available. (This also means that unless the first update, that service is simply in "noop mode". But that's ok.)
Now the "Updater" (which runs in my daemon thread) uses a library that again starts a thread when opening a connection and it's necessary to call "close" to ensure that this secondary thread is stopped - otherwise it is not possible, to shutdown the JVM properly.
As a safety-net I call the close() method inside the finalize() method of my "Manager" (which keeps a reference to all Updater instances). This is not 100% safe, since it's not predictable when GC runs (even more during shutdown!), but it's my only option.
Update: Here is some abstract example code that illustrates the architecture and the according problem
Problem
This architecture causes two possible pitfalls:
If the implementation does not keep a reference to the instance of the manager, it will be garbage collected at some point and trough the finalize method the necessary background updates will be stopped.
If the implementation keeps an instances of the manager, it must call the close method during the shutdown of the according system, otherwise the JVM can't terminate properly.
So my actual problem is the "potential unreliability" of the developers, which are using that library.
Does anyone have an idea how to build a solution, that could handle both pitfalls?
It would be nice to have some Auto-AutoCloseable ;) that is called during Shutdown (e.g by the DestroyJavaVM Thread or similar).
Solutions I tried unsuccessfully
Inside the Updater I am closing the "problematic" connection inside a "try-finally" block, but that daemon thread is not interrupted / stopped automatically as well.
I registered a Runtime.getRuntime().addShutdownHook(...) that would close all connections, but this shutdown hook is never called since a Shutdown is only initiated when all user-threads are stopped.
Update: Solved at my implementation, but not the problem
I solved my problem as I found that the third party library (RabbitMQ Client) offers a setThreadFactory method that I can use to ensure the spawned Threads are Daemon-Threads.
Good luck for me with my 3rd party library, but the described problem is still possible.
You want the AutoCloseable resources to be closed so the shutdown is orderly, I guess.
AutoCloseable objects should be used (by your library clients) in a manner that ensures they are closed when they are no longer needed. In almost all cases, they should be using a try-with-resources block, so they are closed even if an exception is thrown.
You should take advantage of that by requiring your library clients perform a controlled shutdown of each thread when they receive a request to shutdown the program. A thread performs a controlled shutdown by returning from each Runnable.run method, or throwing an exception from each Runnable.run method. I believe this is the only reliable means of closing resources, because it ensures nested resource allocations are reallocated in the correct order. More generally, as a library writer you can not know what other operations your library clients might want to do on shutdown, so you should give them complete control over the shutdown.
You can help them do that by having your library code properly handle InterruptedException and the Thread.interrupted flag.
Ok, I guess there is no good solution for that problem, but since I found that my 3rd party library gives the possibility to create Daemon-Threads, this is exactly what was necessary to fix it.
Maybe this is also a problem that should be solved on the "human level" by providing a good documentation and ensure a proper usage of AutoCloseable. A good developer should know how to deal with that.
From a technical point of view I found these possible solutions, which are just "safety nets" no one should rely on.
call "close()" inside the finalize method
I implemented a service, that can be used to register AutoClosable resources. It runs inside a Daemon-Thread and checks every few seconds, if it finds the JavaDestroyVM Thread and in such a case, closes all registered AutoClosables and stops itself.
Disclosure: If a system runs "outside of the main method" (the JavaDestroyVM Thread will run all the time), this is solution won't work.
Update: The RegisterAutoClosable-Service was a very ugly/hacky solution - I deleted it and plan to change the design, to avoid such situations, but finally it's the responsibility of the implementing developer to close opened resources properly.
I have several threads in my Android application that should be executed concurrently. I am not running them on several cores. REAL parallel execution at exactly the same time is not necessary. If Android switches between the different tasks or temporarily pauses certain threads, it's Ok.
The problem: Some threads are highly time consuming and computationally expensive (complex algorithms), but are not real time critical. Other threads and the Android UI thread are real time critical and should not be blocked or heavily delayed by the time consuming processes. Nevertheless, the time consuming processes should also be executed if there are no other more important tasks to perform. Ideally, the highly important threads should safely pause the less important threads. In Java people used to implement suspend() commands, but they are now depricated.
What is the recommended way to solve this problem in Android?? In Java I would have used Sleep commands or wait and notify methods. What is recommended way in Android?
Thanks for the anwer, guys!
Edit:
I was thinking about threads. But I wrote "processes" instead of "threads", just in case regular threads are not the best way to solve the problem in Android. I am open to anything. I have only ONE App. Sorry if terms get mixed up a little. But I wanted to keep the options and ideas open.
Thanks for the comments and answers so far. But I am more interested in a GENERAL, recommended, established strategy and good practice to solve this kind of problem in Android. I could also do some hacks, but I was really hoping for a clean and established solution. I am more looking for an answer like: "The common, established approach is to use strategy A + strategy B..."
In my opinion you should use a different component (Service) and set it to run on a different process - here you can see how
By doing so you can put all the computing in a different process that not related to your man UI component lifecycle.
If the dependency of algorithms' logics and the main UI is loose I wouldn't try to implement them via Threads concurrency.
We are running a Java server app that is using ScheduledThreadPoolExecutor to manage some work. There are multiple instances running, for different types of work, but each instance only uses one thread. It's not important why this is as there's really no way around it. What we noticed on the production server is that one of these instances stopped working at some point, completely and silently. Restarting the server brought it back again, but the problem isn't solved.
I know that using scheduleAtFixedRate will stop if the task throws an exception at some point, but this isn't the case here. We had a recurrent task that simply stopped executing, and new tasks that used the schedule() method and still didn't execute. I presume that the thread it was using died and didn't start again.
My question is, are there any circumstances under which this could happen? Is there anything I should look out for?
It looks like the simplest explanation is the answer: all threads hang.
In my case the cause of this seems to be HTTP requests that never timeout. This can happen in certain situations and I am yet to find a good solution for the problem. I think the best option is to implement a timeout on the scheduled task itself to make sure we avoid any issues.
I'm seeing strange behavior and I don't know how to gain any further insight into and am hoping someone can help.
Background: I have a query that takes a long time to return results so instead of making the user wait for the data directly upon request I execute this query via a Timer object at regular intervals and store the results in a static variable. Therefore, when the user requests the data I always just pull from the static variable, therefore making the response virtually instant. So far so good.
Issue: The behavior I'm seeing, however, is that if I make a request for the data just as the background (Timer) request has begun to query the data, my user's request waits for the data to come back before responding -- forcing the user to wait. It's as if tomcat is behaving synchronously with the threads (I know it's not -- it just looks that way).
This is in a Production environment and, for the most part, everything works great but for users there are times when the site just hangs for them and they feel it's unreliable (well, in a sense it is).
What I've done: Being that the requests for the data were in a static method I thought "A ha! The threads are syncronized which is causing the delay!" so i pulled all of my static methods out, removed the syncronization and forced each call to instantiate it's own object to retrieve the data (to keep it thread safe). There isn't any syncronization on a semaphore to the static variable either.
I've also installed javamelody to try and gain some insight into what's going on but nothing new thus far. I have noticed a lot (majority) of threads are in "WAITING" state but they also have 0ms for User and CPU time so don't think that is pointing to anything(?).
Running Tomcat 5.5 (no apache layer), struts 2, Java 1.5
If anyone has any idea why a simple request to a static variable hangs for longer background processes I would really appreciate it! Or if you know how I can gain insight that would be great too.
Thanks!
One possible explanation is that the threads are actually blocking at the database level due to database locking (or something) caused by the long-running query.
The way to figure out what is going on is to find out exactly where the blocked threads are blocking. A thread dump can be produced by sending a SIGQUIT (or equivalent) to the JVM, and included stack traces for all Java thread stacks. Alternatively, you can get the same information (and more) by attaching a debugger, etcetera. Either way, the class name and line number of the top frame of each stack should allow you to look at the source code and figure out (at least) what kind of locking or blocking is going on.
For those who would like to know I eventually found VisualVM (http://visualvm.java.net/download.html). It's perfect. I run Tomcat from eclipse like I normally do and it appears within the VisualVM client. Right-mouse click the tomcat icon, choose Thread Dump and, boom, I've got it all.
Thanks, all, for the help and pointers towards the right direction!
I'm working on a multi-user Java webapp, where it is possible for clients to use the webapp API to do potentially naughty things, by passing code which will execute on our server in a sandbox.
For example, it is possible for a client to write a tight while(true) loop that impacts the performance of other clients.
Can you guys think of ways to limit the damage caused by these sorts of behaviors to other clients' performance?
We are using Glassfish for our application server.
The halting problem show that there is no way that a computer can reliably identify code that will not terminate.
The only way to do this reliably is to execute your code in a separate JVM which you then ask the operating system to shut down when it times out. A JVM not timing out can process more tasks so you can just reuse it.
One more idea would be byte-code instrumentation. Before you load the code sent by your client, manipulate it so it adds a short sleep in every loop and for every method call (or method entry).
This avoids clients clogging a whole CPU until they are done. Of course, they still block a Thread object (which takes some memory), and the slowing down is for every client, not only the malicious ones. Maybe make the first some tries free, then scale the waiting time up with each try (and set it down again if the thread has to wait for other reasons).
Modern day app servers use Thread Pooling for better performance. The problem is that one bad apple can spoil the bunch. What you need is an app server with one thread or maybe process per request. Of course there are going to be trade offs. but the OS will handle making sure that processing time gets allocated evenly.
NOTE: After researching a little more what you need is an engine that will create another process per request. If not a user can either cripple you servlet engine by having servlets with infinite loops and then posting multiple requests. Or he could simply do a System.exit in his code and bring everybody down.
You could use a parent thread to launch each request in a separate thread as suggested already, but then monitor the CPU time used by the threads using the ThreadMXBean class. You could then have the parent thread kill any threads that are misbehaving. This is if, of course, you can establish some kind of reasonable criteria for how much CPU time a thread should or should not be using. Maybe the rule could be that a certain initial amount of time plus a certain additional amount per second of wall clock time is OK?
I would make these client request threads have lower priority than the thread responsible for monitoring them.