I have a potential race condition in my Java code that uses a BlockingQueue, and I'd like to know how to modify the code to avoid it:
private static BlockingQueue<FileToFTP> ftpQueue = new LinkedBlockingQueue<FileToFTP>();
private static boolean shuttingDown = false;
private static FileToFTP pendingFile = null;
private static int uploadsPending = 0;
private static Thread ftpThread = new Thread(new Runnable() {
public void run() {
try {
for(;;) {
FileToFTP f2f = ftpQueue.take(); // this blocks until there is something to take
// FIXME: If the main thread takes over right
// at this instant, then uploadsRemaining()
// is not correct!!
uploadsPending = 1;
int qs = ftpQueue.size();
logIt(qs, f2f);
pendingFile = f2f;
if(!doUploadFile(f2f.getPath(), f2f.getFilename(), f2f.getRenameTo(), f2f.isBinary())) {
if(shuttingDown) {
log.info("Upload " + f2f + " failed!");
} else {
ftpQueue.offer(f2f); // put it back on to retry later
}
uploadsPending = 0;
} else {
pendingFile = null;
uploadsPending = 0;
}
if(qs == 0) logIt("");
}
} catch (InterruptedException consumed) {
// Allow thread to exit
}
}
});
public static int uploadsRemaining() {
return ftpQueue.size() + uploadsPending;
}
Please see the "FIXME" comment in the code. Thanks!!
Maybe I'm misinterpreting what you want, but it sounds like you may be better off using an ExecutorService to actually run things. You can create those using Exectors.newSingleThreadExecutor() or Executors.newFixedThreadPool(2) (2 being an example of the number of threads for it to use).
You can then either .execute or .submit Runnables on the ExecutorService. submit will return a Future<T> object that can be used to track the status of a particular job submitted to an ExecutorService.
Having said that, you may need to create a new class to do it as the Runnable / Callable class would need to have a FileToFTP variable in it. (Side note: I try to avoid inner classes, so if you're wondering why I didn't suggest doing it that way...)
The problem then becomes "how do I tell how many files are pending?" Unfortunately, the only easy way is to make it a static property of another class... with either static get/set methods or as a public/protected property. An AtomicInteger is ideal for this, so you may want to consider using one as a protected static one in your calling class. It has dedicated increment/decrement commands as well as ones that both adjust the value and return it.
Related
In one of the interview, a coding question was asked to me and I had to find the problem in that code and suggest proper solution.
Please find below the entire code:
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;
public class Atomic {
static AtomicInteger count = new AtomicInteger(0);
static int counter = 0;
public static class Runnable extends Thread {
public void run() {
while (count.getAndSet(1) != 0) {
try {
Thread.sleep(3000);
} catch (Exception e) {
}
}
counter = counter + 1;
count.set(0);
}
}
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(10);
for (int i = 0; i < 10; i++) {
Runnable runnable = new Runnable();
executor.execute(runnable);
}
executor.shutdown();
}
}
This code is running properly. But question is , there is some problem in this code if number of threads get increased or if I run For loop for almost 10000 times.
I tried to find the problem, but couldn't find one.
There are several things wrong with this code. You've not stated with "there is some problem" means, but here are the things that jump out.
Firstly, the counter variable is not updated safely. Multiple threads don't have guaranteed visibility of the last-written value; nor do you have the guarantee that no other thread has updated its value in between the read and the write.
The simple solution to this: change counter to an AtomicInteger, and use getAndIncrement or incrementAndGet to increment it.
Secondly, public static class Runnable extends Thread { is extremely dubious.
Don't hide the names of commonly-known Java classes (this is hiding java.lang.Runnable)
Don't extend Thread directly, especially when all you need is a java.lang.Runnable to add execute with an ExecutorService.
A more suitable class declaration would be:
public static class MyRunnable implements Runnable {
(or whatever you want to call it)
Or you can just declare an anonymous class:
executor.execute(new Runnable() { /* body */ });
Or you can just declare a lambda:
executor.execute(() -> { /* body */ });
Thirdly, count doesn't really seem to be serving an obvious purpose here. The logic of the runnable seems to be:
If "flag" is false:
Set "flag" to true
Increment a variable
Set "flag" to false
Otherwise:
Wait 3 seconds
Try again
count is playing the role of "flag" here. It's effectively just an AtomicBoolean.
But you don't need a separate count variable at all, if you make the counter an AtomicInteger:
while (true) {
int current = counter.get();
if (counter.compareAndSet(current, current + 1)) {
// Nothing else is trying to update "current" at the same time:
// we updated it. Stop.
break;
}
// Something else is trying to update at the same time.
// Sleep for 3 seconds.
Thread.sleep(3000);
}
I am working for multithreaded application, so i have a thread which has one Global ArrayList variable, I am fetching list of values from database and assigning it to that variable. I have restricted how many list should fetch, to 100. In backend i am using hibernate.
I have kept this thread running in background periodically, But what i wanted to do that, whenever this thread runs, it will fetch next 100 list of items. And also i don't want to use static and don't want to maintain it in a database, i am confuse how to maintain a counter that will tell my fetching method that next 100 item to fetch. can anybody help, how can i achieve this? Following is my code sample,
public class DemoThread implements Runnable {
private ArrayList <<>> samplePojoList = null;
#Override
public void run() {
MyDaoClass dao = null;
try {
// this dao method fetching list of 100 results from database,
// for each iteration i want to fetch next 100 result from
// database
samplePojoList = dao.getSampleList();
} catch (Exception e) {}
}
}
You can very well make use of ThreadLocal variable in this case.
Please see example below:
private ThreadLocal<Integer> counter = new ThreadLocal<Integer>();
Here is a simplified example how you can pass a counter to your threads and maintain a synchronized count among your consumer threads. In order to maintain the counter count consistent among threads, it should be accessed and increased in the synchronized block.
My assumption is that you want a counter to be shared among all your consumer threads, not one counter per consumer thread.
class Counter{
int count = 0;
}
class MyThread implements Runnable {
Counter counter;
MyThread(Counter counter){
this.counter = counter;
}
public void run() {
int count;
synchronized(counter){
count = counter.count;
counter.count++;
}
// fetch 100 times the value of this counter
System.out.println(count); // just a stub code
}
}
public class CounterDemo {
public static void main(String[] args) {
Counter counter = new Counter();
MyThread t = new MyThread(counter);
new Thread(t).start();
}
}
You can use AtomicReference to achieve the same.
Sample code for String:
String initialReference = "value 1";
AtomicReference<String> someRef =
new AtomicReference<String>(initialReference);
String newReference = "value 2";
boolean exchanged = someRef.compareAndSet(initialReference, newReference);
System.out.println("exchanged: " + exchanged);
You can replace String with ArrayList in above example.
Have a look at this SE post for more details:
When to use AtomicReference in Java?
I have two primary threads. One spawns new threads and the other listens for results, like so:
//Spawner
while(!done) {
spawnNewProcess(nextId, parameters);
myListener.listenFor(nextId);
nextId ++;
}
The spawnNewProcess method takes a widely variable amount of time. When it finishes, it will put a result object into a map that can be accessed by Id.
The listener thread runs like so:
//Listener
while(!done) {
for (int id : toListenFor) {
if (resultMap.contains(id)) {
result = resultMap.get(id);
toListenFor.remove(id);
process(result);
}
}
}
I can't change the spawnNewProcess method, nor how it stores results. What I want to do is set a maximum limit on how many can be going concurrently. My first inclination would be to just have a variable track that number. If the max would be exceeded, then the spawner waits. When a result comes back, the listener will notify it. Like this:
//Spawner2
AtomicInteger numSpawns = new AtomicInteger(0);
int maxSpawns = 10;
while(!done) {
if (numSpawns.intValue() >= maxSpawns) {
this.wait(0);
}
numSpawns.getAndIncrement;
spawnNewProcess(nextId, parameters);
myListener.listenFor(nextId);
nextId ++;
}
And the Listener be:
//Listener2
while(!done) {
for (int id : toListenFor) {
if (resultMap.contains(id)) {
result = resultMap.get(id);
toListenFor.remove(id);
numSpawns.getAndDecrement();
Spawner.notify();
process(result);
}
}
}
Will this work? Are there potential deadlocks that I'm missing? It wouldn't be a huge deal if somehow 11 or 9 spawns were running at the same time instead of 10. Or is there a much better way that I'm oblivious to?
Use a Semaphore.
import java.util.concurrent.Semaphore;
private Semaphore sem = new Semaphore(NUM_MAX_CONCURRENT);
// Spawner
while(!done) {
sem.acquire(); // added by corsiKa
spawnNewProcess(nextId, parameters);
myListener.listenFor(nextId);
nextId ++;
}
// listener
while(!done) {
for (int id : toListenFor) {
if (resultMap.contains(id)) {
result = resultMap.get(id);
toListenFor.remove(id);
sem.release(); // added by corsiKa
process(result);
}
}
}
To control the number of spawners running, use a Executors.newFixedThreadPool(size), which will always run no more than a fixed amount of tasks at once. Then wrap the spawning tasks in a Runnable and pass them to the ExecutorService.
while(!done) {
task = new Runnable() { public void run() {
spawnNewProcess(nextId, parameters);
} });
exec.submit(task);;
nextId ++;
}
To get the results back, use a SynchronousQueue or ConcurrentLinkedQueue, which will allow you to pass objects between threads without using lower-level concurrency objects.
I am writing a multithreaded parser.
Parser class is as follows.
public class Parser extends HTMLEditorKit.ParserCallback implements Runnable {
private static List<Station> itemList = Collections.synchronizedList(new ArrayList<Item>());
private boolean h2Tag = false;
private int count;
private static int threadCount = 0;
public static List<Item> parse() {
for (int i = 1; i <= 1000; i++) { //1000 of the same type of pages that need to parse
while (threadCount == 20) { //limit the number of simultaneous threads
try {
Thread.sleep(50);
} catch (InterruptedException ex) {
ex.printStackTrace();
}
}
Thread thread = new Thread(new Parser());
thread.setName(Integer.toString(i));
threadCount++; //increase the number of working threads
thread.start();
}
return itemList;
}
public void run() {
//Here is a piece of code responsible for creating links based on
//the thread name and passed as a parameter remained i,
//connection, start parsing, etc.
//In general, nothing special. Therefore, I won't paste it here.
threadCount--; //reduce the number of running threads when current stops
}
private static void addItem(Item item) {
itenList.add(item);
}
//This method retrieves the necessary information after the H2 tag is detected
#Override
public void handleText(char[] data, int pos) {
if (h2Tag) {
String itemName = new String(data).trim();
//Item - the item on which we receive information from a Web page
Item item = new Item();
item.setName(itemName);
item.setId(count);
addItem(item);
//Display information about an item in the console
System.out.println(count + " = " + itemName);
}
}
#Override
public void handleStartTag(HTML.Tag t, MutableAttributeSet a, int pos) {
if (HTML.Tag.H2 == t) {
h2Tag = true;
}
}
#Override
public void handleEndTag(HTML.Tag t, int pos) {
if (HTML.Tag.H2 == t) {
h2Tag = false;
}
}
}
From another class parser runs as follows:
List<Item> list = Parser.parse();
All is good, but there is a problem. At the end of parsing in the final list "List itemList" contains 980 elements onto, instead of 1000. But in the console there is all of 1000 elements (items). That is, some threads for some reason did not call in the handleText method the addItem method.
I already tried to change the type of itemList to ArrayList, CopyOnWriteArrayList, Vector. Makes the method addItem synchronized, changed its call on the synchronized block. All this only changes the number of elements a little, but the final thousand can not be obtained.
I also tried to parse a smaller number of pages (ten). As the result the list is empty, but in the console all 10.
If I remove multi-threading, then everything works fine, but, of course, slowly. That's not good.
If decrease the number of concurrent threads, the number of items in the list is close to the desired 1000, if increase - a little distanced from 1000. That is, I think, there is a struggle for the ability to record to the list. But then why are synchronization not working?
What's the problem?
After your parse() call returns, all of your 1000 Threads have been started, but it is not guaranteed that they are finished. In fact, they aren't that's the problem you see. I would heavily recommend not write this by yourself but use the tools provided for this kind of job by the SDK.
The documentation Thread Pools and the ThreadPoolExecutor are e.g. a good starting point. Again, don't implement this yourself if you are not absolutely sure you have too, because writing such multi-threading code is pure pain.
Your code should look something like this:
ExecutorService executor = Executors.newFixedThreadPool(20);
List<Future<?>> futures = new ArrayList<Future<?>>(1000);
for (int i = 0; i < 1000; i++) {
futures.add(executor.submit(new Runnable() {...}));
}
for (Future<?> f : futures) {
f.get();
}
There is no problem with the code, it is working as you have coded. the problem is with the last iteration. rest all iterations will work properly, but during the last iteration which is from 980 to 1000, the threads are created, but the main process, does not waits for the other thread to complete, and then return the list. therefore you will be getting some odd number between 980 to 1000, if you are working with 20 threads at a time.
Now you can try adding Thread.wait(50), before returning the list, in that case your main thread will wait, some time, and may be by the time, other threads might finish the processing.
or you can use some syncronization API from java. Instead of Thread.wait(), use CountDownLatch, this will help you to wait for the threads to complete the processing, and then you can create new threads.
I have a situation where I have a callback that I want to execute once. For the sake of argument let's say it looks like this:
final X once = new X(1);
Runnable r = new Runnable() {
#Override public void run() {
if (once.use())
doSomething();
}
}
where X is some concurrent object with the following behavior:
constructor: X(int N) -- allocates N use permits
boolean use(): If there is at least 1 use permit, consume one of them and return true. Otherwise return false. This operation is atomic with respect to multiple threads.
I know I can use java.util.concurrent.Semaphore for this, but I don't need the blocking/waiting aspect of it, and I want this to be a one-time use thing.
AtomicInteger doesn't look sufficient unless I do something like
class NTimeUse {
final private AtomicInteger count;
public NTimeUse(int N) { this.count = new AtomicInteger(N); }
public boolean use() {
while (true)
{
int n = this.count.get();
if (n == 0)
return false;
if (this.count.compareAndSet(n, n-1))
return true;
}
}
and I feel queasy about the while loop.
CountDownLatch won't work, because the countDown() method has no return value and can't be executed atomically w/r/t getCount().
Should I just use Semaphore or is there a more appropriate class?
In the case of single permit you can use AtomicBoolean:
final AtomicBoolean once = new AtomicBoolean(true);
Runnable r = new Runnable() {
#Override public void run() {
if (once.getAndSet(false))
doSomething();
}
}
If you need many permits, use your solution with compareAndSet(). Don't worry about the loop, getAndIncrement() works the same way under the cover.
yes. AtomicInteger is non-blocking. You can use getAndDecrement().
You can use something like
if(counter.getAndDecrement() > 0) {
// something
} else {
counter.set(0);
}
This will work provided you don't call it two billion times between the decrement and the set. i.e. you would need to have two billion threads stop between these two statements.
Again you can use AtomicLong for extra paranoia.
// This implements an unfair locking scheme:
while ( mayContinue() ) {
// acquire the permit and check if it was legally obtained
if ( counter.decrementAndGet() > 0 )
return true;
// return the illegally acquired permit
counter.incrementAndGet();
}
return false;
Setting the counter back to zero if you discover the permit was illegally obtained creates a race condition when another thread releases a permit. This only works for situations where there are 2 or 3 threads at most. Some other backoff or latching mechanism needs to be added if you have more.