Java concurrency exercise. Asynchronous download - java

I'm doing an exercise about Java concurrency using wait, notify to study for an exam.
The exam will be written, so the code does have to be perfect since we can't try to compile and check errors.
This is the text of the exercise:
General idea:
when the downloader is instanced the queue and the hashmap are created and passed to all the threads. (shared data)
the download method add the url to the queue and call notifyAll to wake up the Downloader Threads.
the getData method waits until there are data in the hashmap for the provided url. When data are available it returns to the caller.
the downloader thread runs an infinity loop. It waits until an url is present in the queue. When it receives an url it downloads it and puts the bytes in the hashmap calling notifyAll to wake up a possible user waiting in getData method.
This is the code that I produced:
public class Downloader{
private Queue downloadQueue;
private HashMap urlData;
private final static THREADS_NUMBER;
public Downloader(){
this.downloadQueue = new Queue();
this.urlData = new HashMap();
for(int i = 0; i < THREADS_NUMBER; i++){
new DownTh(this.downloadQueue, this.urlData).start();
}
}
void syncronized download(String URL){
downloadQueue.add(url);
notifyAll();
return;
}
byte[] syncronized getData(String URL){
while(urlData.get(URL) == null ){
wait()
}
return urlData.get(URL);
}
}
public class DownTh extend Thread{
private Queue downloadQueue;
private HashMap urlData;
public DownTh(Queue downloadQueue, HashMap urlData){
this.downloadQueue = downloadQueue
this.urlData = urlData;
}
public void run(){
while(true){
syncronized{
while(queue.isEmpty()){
wait()
}
String url = queue.remove();
urlData.add(url, Util.download(url))
notifyAll()
}
}
}
}
Can you help me and tell me if the logic is right?

Let's assume for a second that all those great classes in Java that handle synchronization do not exist, because this is a synthetic task, and all you got to handle is sychronized, wait and notify.
The first question to answer in simple words is: "Who is going to wait on what?"
The download thread is going to wait for an URL to download.
The caller is going to wait for the result of that download thread.
What does this mean in detail? We need at least one synchronization element between the caller and the download thread (your urlData), also there should be one data object handling the download data itself for convenience, and to check whether or not the download has yet been completed.
So the detailed steps that will happen are:
Caller requests new download.create: DownloadResultwrite: urlData(url -> DownloadResult)wake up 1 thread on urlData.
Thread X must find data to download and process it or/then fall asleep again.read: urlData (find first unprocessed DownloadResult, otherwise wait on urlData)write: DownloadResult (acquire it)write: DownloadResult (download result)notify: anyone waiting on DownloadResultrepeat
Caller must be able to asynchronously check/wait for download result.read: urlDataread: DownloadResult (wait on DownloadResult if required)
As there are reads and writes from different threads on those objects, synchronization is required when accessing the objects urlData or DownloadResult.
Also there will be a wait/notify association:
caller -> urlData -> DownTh
DownTh -> DownloadResult -> caller
After careful analysis the following code would fulfill the requirements:
public class DownloadResult {
protected final URL url; // this is for convenience
protected boolean inProgress;
protected byte[] result;
public DownloadResult(final URL url) {
this.url = url;
this.inProgress = false;
}
/* Try to lock this against tother threads if not already acquired. */
public synchronized boolean acquire() {
if (this.inProgress == false) {
this.inProgress = true;
return true;
} else {
return false;
}
}
public void download() {
final byte[] downloadedBytes = Util.download(this.url); // note how this is done outside the synchronized block to avoid unnecessarily long blockings
synchronized (this) {
this.result = downloadedBytes;
this.notifyAll(); // wake-up ALL callers
}
}
public synchronized byte[] getResult() throws InterruptedException {
while (this.result == null) {
this.wait();
}
return this.result;
}
}
protected class DownTh extends Thread {
protected final Map<URL, DownloadResult> urlData;
public DownTh(final Map<URL, DownloadResult> urlData) {
this.urlData = urlData;
this.setDaemon(true); // this allows the JVM to shut down despite DownTh threads still running
}
protected DownloadResult getTask() {
for (final DownloadResult downloadResult : urlData.values()) {
if (downloadResult.acquire()) {
return downloadResult;
}
}
return null;
}
#Override
public void run() {
DownloadResult downloadResult;
try {
while (true) {
synchronized (urlData) {
while ((downloadResult = this.getTask()) == null) {
urlData.wait();
}
}
downloadResult.download();
}
} catch (InterruptedException ex) {
// can be ignored
} catch (Error e) {
// log here
}
}
}
public class Downloader {
protected final Map<URL, DownloadResult> urlData = new HashMap<>();
// insert constructor that creates the threads here
public DownloadResult download(final URL url) {
final DownloadResult result = new DownloadResult(url);
synchronized (urlData) {
urlData.putIfAbsent(url, result);
urlData.notify(); // only one thread needs to wake up
}
return result;
}
public byte[] getData(final URL url) throws InterruptedException {
DownloadResult result;
synchronized (urlData) {
result = urlData.get(url);
}
if (result != null) {
return result.getResult();
} else {
throw new IllegalStateException("URL " + url + " not requested.");
}
}
}
In real Java things would be done differently, by using Concurrent classes and/or Atomic... classes, so this is just for educational purposes. For further reading see "Callable Future".

Related

Can Handler return a Future object?

Can I get a future object from a handler?
Handler handler = new Handler(getMainLooper());
Future<String> future = handler.post(new Callable<String>() {
public String call() throw Exception {
// run in the main thread
return askForPassword();
}
}); // can I do something like this?
String password = future.get(); // wait until finish
// do network things...
I have a network thread and I need to ask the user for password, since I need to show an input dialog I have to do this on Main thread, but handler can not return values.
I can do the same thing by
Handler handler = new Handler(getMainLooper());
String password = null;
handler.post(() -> {
// run in the main thread
password = askForPassword();
});
while (password == null) { /*wait until finish*/ }
// do network things...
But this looks stupid and inconvenient
Handler is fundamentally asynchronous and thus whatever you put there is not guaranteed to run immediately (moreover, you can postDelayed or postAtTime). Then it is clear, that you can not return any value from the Handler directly to the code that posted it. So you have to work this around.
Another obstacle is that in Java your closure can capture only final variables and you have to work this around as well.
New API (CompletableFuture)
Unfortunately original Future Java API is not suitable for composition. If you target new devices only and thus can use newer CompletableFuture, you may do something like this:
CompletableFuture<String> askForPasswordNewApi() {
// your UI code that creates Future
}
void doNetworkThingNewApi() {
// some network stuff
final CompletableFuture<String> passwordFutureWrapper = new CompletableFuture<String>();
Handler handler = new Handler(getMainLooper());
handler.post(new Runnable() {
#Override
public void run() {
// run in the main thread
CompletableFuture<String> future = askForPasswordNewApi();
// bind the real future to the outer one
future.handle((r, ex) -> {
if (ex != null)
passwordFutureWrapper.completeExceptionally(ex);
else
passwordFutureWrapper.complete(r);
return 0;
});
}
});
// wait until finish
// don't forget to handle InterruptedException here
String password = passwordFutureWrapper.get();
// do more network things...
}
The idea is rather simple: create outer final variable passwordFutureWrapper that can be captured by the Handler and bind this wrapper to the real future
Side note: if your askForPassword already returns Future but you can't use the new API, you probably have re-implemented something similar to CompletableFuture anyway, so you just need to modify this code a bit to allow binding of one future to another.
Old API
If you can't use CompletableFuture in your code yet, but still somehow has a method with a Future-based signature:
Future<String> askForPasswordOldApi()
you may do it more explicitly:
void doNetworkThingOldApi() {
// some network stuff
final CountDownLatch syncLock = new CountDownLatch(1);
final Future<String>[] futureWrapper = new Future<String>[1];
Handler handler = new Handler(getMainLooper());
handler.post(new Runnable() {
#Override
public void run() {
// run in the main thread
final CompletableFuture<String> future = askForPasswordOldApi();
futureWrapper[0] = future;
syncLock.countDown();
}
});
String password;
try {
// 1 minute should be quite enough for synchronization between threads
if (!syncLock.await(1, TimeUnit.MINUTES)) {
// log error, show some user feedback and then stop further processing
return;
}
password = futureWrapper[0].get(); // wait until finish
} catch (InterruptedException ex) {
// log error, show some user feedback and then stop further processing
return;
} catch (ExecutionException ex) {
// log error, show some user feedback and then stop further processing
return;
}
// do more network things...
}
The idea here is following:
Use single-element array as a simple container to work around final-closure limitations
Use CountDownLatch to ensure synchronization between the network and the UI threads i.e. that futureWrapper[0] is not null by the time we start waiting on the result with get.
Update (design for library API)
If you are designing API and want to have a single entry for login with different additional scenarios handled by a callback, I'd do using custom implementation of something similar to CompletableFuture:
public interface ResultHandler<T> {
void resolve(T result);
void cancel();
}
class ResultHandlerImpl<T> implements ResultHandler<T> {
enum State {
Empty,
Resolved,
Cancelled
}
private final Object _lock = new Object();
private State _state = State.Empty;
private T _result;
#Override
public void resolve(T result) {
synchronized (_lock) {
if (_state != State.Empty) // don't override current state
return;
_result = result;
_state = State.Resolved;
_lock.notifyAll();
}
}
#Override
public void cancel() {
synchronized (_lock) {
if (_state != State.Empty) // don't override current state
return;
_state = State.Cancelled;
_lock.notifyAll();
}
}
public boolean isCancelled() {
synchronized (_lock) {
return _state == State.Cancelled;
}
}
public boolean isDone() {
synchronized (_lock) {
return _state == State.Resolved;
}
}
public T get() throws InterruptedException, CancellationException {
while (_state == State.Empty) {
synchronized (_lock) {
_lock.wait();
}
}
if (_state == State.Resolved)
return _result;
else
throw new CancellationException();
}
}
I would probably make interface public but implementation ResultHandlerImpl package-private so it would be harder for the users to mess up with implementation details. Then in callback methods I'd pass my callback ResultHandler as a parameter (actually it obviously would be a ResultHandlerImpl):
public interface LoginCallback {
void askForPassword(ResultHandler<String> resultHandler);
}
And my login method would look something like this (assuming you have private methods tryRestoreSession that doesn't need password and loginWithPassword that requires):
public boolean login(final LoginCallback loginCallback) {
if (tryRestoreSession()) {
return true;
} else {
final ResultHandlerImpl<String> passwordHandler = new ResultHandlerImpl<>();
Handler handler = new Handler(getMainLooper());
handler.post(new Runnable() {
#Override
public void run() {
// run in the main thread
loginCallback.askForPassword(passwordHandler);
}
});
String password;
try {
password = passwordHandler.get();
} catch (CancellationException e) {
return false;
} catch (InterruptedException e) {
return false;
}
return loginWithPassword(password);
}
}
What I think is important here:
I think that passing a callback to LoginCallback makes it easier to write an asynchronous UI-based implementation using standard Java.
ResultHandlerImpl has cancel method. So if the user, for example, forgot the password there is a way to cancel whole login process and not get stuck with a background thread waiting for a password forever
ResultHandlerImpl uses explicit synchronization and wait/notifyAll to establish happens-before relationship between actions on different threads to avoid.
ResultHandlerImpl uses wait/notifyAll so the background thread doesn't consume CPU (and battery) while waiting for the UI.
UPDATED using "wait and notify" instead of looping
UPDATED 2 synchronized methods
Finally I end up with object wrapper(thansk to #SergGr ) and handler
class ObjectWrapper<T> {
T object;
boolean ready;
synchronized void set(T object) {
this.object = object;
this.ready = true;
notifyAll();
}
T get() {
while (!ready) {
synchronized(this) {
try {
wait();
} catch (InterruptedException e) {
return null;
}
}
}
return object;
}
}
In my network thread
Handler handler = new Handler(getMainLooper());
ObjectWarpper<String> wrapper = new ObjectWarpper<>();
handler.post(() -> wrapper.set(askForPassword()));
String password = wrapper.get();

Unit testing asynchronous computation that has to be interrupted manually

I have got a class that records eyetracking data asynchronously. There are methods to start and stop the recording process. The data is collected in a collection and the collection can only be accessed if the recording thread has finished its work. It basically encapsulates all the threading and synchronizing so the user of my library doesn't have to do it.
The heavily shortened code (generics and error handling omitted):
public class Recorder {
private Collection accumulatorCollection;
private Thread recordingThread;
private class RecordingRunnable implements Runnable {
...
public void run() {
while(!Thread.currentThread().isInterrupted()) {
// fetch data and collect it in the accumulator
synchronized(acc) { acc.add(Eyetracker.getData()) }
}
}
}
public void start() {
accumulatorCollection = new Collection();
recordingThread = new Thread(new RecordingRunnable(accumulatorCollection));
recordingThread.start();
}
public void stop() {
recordingThread.interrupt();
}
public void getData() {
try {
recordingThread.join(2000);
if(recordingThread.isAlive()) { throw Exception(); }
}
catch(InterruptedException e) { ... }
synchronized(accumulatorCollection) { return accumulatorCollection; }
}
}
The usage is quite simple:
recorder.start();
...
recorder.stop();
Collection data = recorder.getData();
My problem with the whole thing is how to test it. Currently i am doing it like this:
recorder.start();
Thread.sleep(50);
recorder.stop();
Collection data = recorder.getData();
assert(stuff);
This works, but it is non-deterministic and slows down the test suite quite a bit (i marked these tests as integration tests, so they have to be run separately to circumvent this problem).
Is there a better way?
There is a better way using a CountDownLatch.
The non-deterministic part of the test stems from two variables in time you do not account for:
creating and starting a thread takes time and the thread may not have started executing the runnable when Thread.start() returns (the runnable will get executed, but it may be a bit later).
the stop/interrupt will break the while-loop in the Runnable but not immediately, it may be a bit later.
This is where a CountDownLatch comes in: it gives you precise information about where another thread is in execution. E.g. let the first thread wait on the latch, while the second "counts down" the latch as last statement within a runnable and now the first thread knows that the runnable finished. The CountDownLatch also acts as a synchronizer: whatever the second thread was writing to memory, can now be read by the first thread.
Instead of using an interrupt, you can also use a volatile boolean. Any thread reading the volatile variable is guaranteed to see the last value set by any other thread.
A CountDownLatch can also be given a timeout which is useful for tests that can hang: if you have to wait to long you can abort the whole test (e.g. shutdown executors, interrupt threads) and throw an AssertionError. In the code below I re-used the timeout to wait for a certain amount of data to collect instead of 'sleeping'.
As an optimization, use an Executor (ThreadPool) instead of creating and starting threads. The latter is relative expensive, using an Executor can really make a difference.
Below the updated code, I made it runnable as an application (main method). (edit 28/02/17: check maxCollect > 0 in while-loop)
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public class Recorder {
private final ExecutorService executor;
private Thread recordingThread;
private volatile boolean stopRecording;
private CountDownLatch finishedRecording;
private Collection<Object> eyeData;
private int maxCollect;
private final AtomicBoolean started = new AtomicBoolean();
private final AtomicBoolean stopped = new AtomicBoolean();
public Recorder() {
this(null);
}
public Recorder(ExecutorService executor) {
this.executor = executor;
}
public Recorder maxCollect(int max) { maxCollect = max; return this; }
private class RecordingRunnable implements Runnable {
#Override public void run() {
try {
int collected = 0;
while (!stopRecording) {
eyeData.add(EyeTracker.getData());
if (maxCollect > 0 && ++collected >= maxCollect) {
stopRecording = true;
}
}
} finally {
finishedRecording.countDown();
}
}
}
public Recorder start() {
if (!started.compareAndSet(false, true)) {
throw new IllegalStateException("already started");
}
stopRecording = false;
finishedRecording = new CountDownLatch(1);
eyeData = new ArrayList<Object>();
// the RecordingRunnable created below will see the values assigned above ('happens before relationship')
if (executor == null) {
recordingThread = new Thread(new RecordingRunnable());
recordingThread.start();
} else {
executor.execute(new RecordingRunnable());
}
return this;
}
public Collection<Object> getData(long timeout, TimeUnit tunit) {
if (started.get() == false) {
throw new IllegalStateException("start first");
}
if (!stopped.compareAndSet(false, true)) {
throw new IllegalStateException("data already fetched");
}
if (maxCollect <= 0) {
stopRecording = true;
}
boolean recordingStopped = false;
try {
// this establishes a 'happens before relationship'
// all updates to eyeData are now visible in this thread.
recordingStopped = finishedRecording.await(timeout, tunit);
} catch(InterruptedException e) {
throw new RuntimeException("interrupted", e);
} finally {
stopRecording = true;
}
// if recording did not stop, do not return the eyeData (could stil be modified by recording-runnable).
if (!recordingStopped) {
throw new RuntimeException("recording");
}
// only when everything is OK this recorder instance can be re-used
started.set(false);
stopped.set(false);
return eyeData;
}
public static class EyeTracker {
public static Object getData() {
try { Thread.sleep(1); } catch (Exception ignored) {}
return new Object();
}
}
public static void main(String[] args) {
System.out.println("Starting.");
ExecutorService exe = Executors.newSingleThreadExecutor();
try {
Recorder r = new Recorder(exe).maxCollect(50).start();
int dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(100).start();
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
r.maxCollect(0).start();
Thread.sleep(100);
dsize = r.getData(2000, TimeUnit.MILLISECONDS).size();
System.out.println("Collected " + dsize);
} catch (Exception e) {
e.printStackTrace();
} finally {
exe.shutdownNow();
System.out.println("Done.");
}
}
}
Happy coding :)

How to check the size() or isEmpty() for ConcurrentLinkedQueue

I am trying to prototype a simple structure for a Web crawler in Java. Until now the prototype is just trying to do the below:
Initialize a Queue with list of starting URLs
Take out a URL from Queue and submit to a new Thread
Do some work and then add that URL to a Set of already visited URLs
For the Queue of starting URLs, I am using a ConcurrentLinkedQueue for synchronizing.
To spawn new Threads I am using ExecutorService.
But while creating a new Thread, the application needs to check if the ConcurrentLinkedQueue is empty or not. I tried using:
.size()
.isEmpty()
But both seem not to be returning the true state of ConcurrentLinkedQueue.
The problem is in below block:
while (!crawler.getUrl_horizon().isEmpty()) {
workers.submitNewWorkerThread(crawler);
}
And because of this, ExecutorService creates all the Threads in its limit, even if the input is only 2 URLs.
Is there a problem with the way multi-threading is being implemented here? If not, what is the better way to check the state of ConcurrentLinkedQueue?
Starting class for the application:
public class CrawlerApp {
private static Crawler crawler;
public static void main(String[] args) {
crawler = = new Crawler();
initializeApp();
startCrawling();
}
private static void startCrawling() {
crawler.setUrl_visited(new HashSet<URL>());
WorkerManager workers = WorkerManager.getInstance();
while (!crawler.getUrl_horizon().isEmpty()) {
workers.submitNewWorkerThread(crawler);
}
try {
workers.getExecutor().shutdown();
workers.getExecutor().awaitTermination(10, TimeUnit.MINUTES);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private static void initializeApp() {
Properties config = new Properties();
try {
config.load(CrawlerApp.class.getClassLoader().getResourceAsStream("url-horizon.properties"));
String[] horizon = config.getProperty("urls").split(",");
ConcurrentLinkedQueue<URL> url_horizon = new ConcurrentLinkedQueue<>();
for (String link : horizon) {
URL url = new URL();
url.setURL(link);
url_horizon.add(url);
}
crawler.setUrl_horizon(url_horizon);
} catch (IOException e) {
e.printStackTrace();
}
}
}
Crawler.java which maintains the Queue of URLs and Set of already visited URLs.
public class Crawler implements Runnable {
private ConcurrentLinkedQueue<URL> url_horizon;
public void setUrl_horizon(ConcurrentLinkedQueue<URL> url_horizon) {
this.url_horizon = url_horizon;
}
public ConcurrentLinkedQueue<URL> getUrl_horizon() {
return url_horizon;
}
private Set<URL> url_visited;
public void setUrl_visited(Set<URL> url_visited) {
this.url_visited = url_visited;
}
public Set<URL> getUrl_visited() {
return Collections.synchronizedSet(url_visited);
}
#Override
public void run() {
URL url = nextURLFromHorizon();
scrap(url);
addURLToVisited(url);
}
private URL nextURLFromHorizon() {
if (!getUrl_horizon().isEmpty()) {
URL url = url_horizon.poll();
if (getUrl_visited().contains(url)) {
return nextURLFromHorizon();
}
System.out.println("Horizon URL:" + url.getURL());
return url;
}
return null;
}
private void scrap(URL url) {
new Scrapper().scrap(url);
}
private void addURLToVisited(URL url) {
System.out.println("Adding to visited set:" + url.getURL());
getUrl_visited().add(url);
}
}
URL.java is just a class with private String url and overriden hashCode() and equals().
Also, Scrapper.scrap() just has dummy implementation until now:
public void scrap(URL url){
System.out.println("Done scrapping:"+url.getURL());
}
WorkerManager to create Threads:
public class WorkerManager {
private static final Integer WORKER_LIMIT = 10;
private final ExecutorService executor = Executors.newFixedThreadPool(WORKER_LIMIT);
public ExecutorService getExecutor() {
return executor;
}
private static volatile WorkerManager instance = null;
private WorkerManager() {
}
public static WorkerManager getInstance() {
if (instance == null) {
synchronized (WorkerManager.class) {
if (instance == null) {
instance = new WorkerManager();
}
}
}
return instance;
}
public Future submitNewWorkerThread(Runnable run) {
return executor.submit(run);
}
}
Problem
The reason why you end up creating more Threads than there are URLs in the queue is because it is possible (and in fact likely) that none of the Threads of the Executor start until you go through the while loop a lot of times.
Whenever working with threads you should always keep in mind that the threads are scheduled independently and run at their own pace except when you explicitly synchronize them. In this case, the threads can start at any time after the submit() call, even though it seems you'd like each one to start and go past nextURLFromHorizon before a next iteration in your while loop.
Solution
Consider dequeuing the URL from the queue before submitting the Runnable to the Executor. I also suggest defining a CrawlerTask that is submitted to the Executor once, rather than a Crawler that is submitted repeatedly. In such design you wouldn't even need a thread-safe container for the URLs to-be-scraped.
class CrawlerTask extends Runnable {
URL url;
CrawlerTask(URL url) { this.url = url; }
#Override
public void run() {
scrape(url);
// add url to visited?
}
}
class Crawler {
ExecutorService executor;
Queue urlHorizon;
//...
private static void startCrawling() {
while (!urlHorizon.isEmpty()) {
executor.submit(new CrawlerTask(urlHorizon.poll());
}
// ...
}
}

Thread safe access to arraylist: two producer one consumer. Lock object is enough?

I have two thread that can produce value and add it in a arraylist,
and other thread can access to it to read a value.
My problem is that the producer can access to the list in the same time that the consumer use data.
This is my code :
public class CommandTree
{
Lock lock = new ReentrantLock();
ArrayList<Command> cmdToSend = null;
JSONObject sendCmdMap;
public CommandTree(JSONObject sendCmdMap)
{
this.cmdToSend = new ArrayList<Command>();
this.sendCmdMap = sendCmdMap;
}
private synchronized void addMacroCmd(String macro, int fmt, int tgt, int sid,int count,JSONArray sli,String paramName,JSONObject params,int function)
{
boolean check = false;
int i = 0;
lock.lock();
try
{
for(i=0; i<cmdToSend.size(); i++)
{
if(cmdToSend.get(i).getMacroName().equalsIgnoreCase(macro))
{
check = true;
break;
}
}
if(check == false)
{
cmdToSend.add(new Command(macro,fmt,tgt,sid,count,function,sli));
}
if(paramName != null)
{
if(check)
cmdToSend.get(i).setParameter(paramName,params);
else
cmdToSend.get(cmdToSend.size()-1).setParameter(paramName,params);
}
}
finally
{
lock.unlock();
}
}
private void addParameter(String macro,int fmt, int tgt, int sid,int count,JSONArray sli,String paramName,JSONObject params,int function)
{
lock.lock();
try
{
this.addMacroCmd(macro, fmt, tgt, sid, count,sli, paramName,params,function);
}
finally
{
lock.unlock();
}
}
public int getSize()
{
return cmdToSend.size();
}
public void reset()
{
lock.lock();
try
{
cmdToSend.clear();
}
finally
{
lock.unlock();
}
}
/*
public Command getNextCommandInLoop()
{
return cmdToSend.;
}
*/
public Command getNextCommand(int i)
{
Command result;
lock.lock();
try
{
result = cmdToSend.get(i);
}
finally
{
lock.unlock();
}
return result;
}
public synchronized boolean populateCommandTree(String i,String target) throws JSONException
{
JSONObject tgtCmd = (JSONObject) sendCmdMap.get(target);
JSONObject cmdObject;
Iterator<String> iter = tgtCmd.keys();
while (iter.hasNext())
{
String key = iter.next();
if(key.equalsIgnoreCase(i))
{
//it is a general commands
JSONObject macro = (JSONObject)tgtCmd.opt(key);
cmdObject = (JSONObject) macro.opt("cmd");
addMacroCmd(key,cmdObject.optInt("fmt"),cmdObject.optInt("tgt"),cmdObject.optInt("sid"),cmdObject.optInt("count"),cmdObject.optJSONArray("sli"),null,null,macro.optInt("function"));
return true;
}
else
{
//It is a parameter, we have to search its general command
cmdObject = (JSONObject)tgtCmd.opt(key);
if(cmdObject == null)
{
continue;
}
JSONObject parameter = cmdObject.optJSONObject("Parameter");
if( parameter == null)
{
//There isn't the requested command, we iterate on the next one
continue;
}
else
{
if(((JSONObject) parameter).optJSONObject(i) != null)
{
JSONObject cmdStructure = (JSONObject)cmdObject.opt("cmd");
//We have found the command, save it in commandSendCache
addMacroCmd(key,cmdStructure.optInt("fmt"),cmdStructure.optInt("tgt"),cmdStructure.optInt("sid"),cmdStructure.optInt("count"),cmdStructure.optJSONArray("sli"),i,parameter.optJSONObject(i),cmdObject.optInt("function"));
return true;//(JSONObject)tgtCmd.opt(key);
}
else
{
continue;
}
}
}
}
return false;
}}
I read some post on that case, but I don't understand very well. I thought to post my code in this way I can understand in better way.
Other problem is that one producer is a UI thread, and I worried if there is problem to stop the UI thread for some times.
I also thought to use ConcurrentLinkedQueue because some time I need to loop on the list, and I always extract the value from the first position, but with concurrentLInkedQueue I don't know how can implementate the loop and in what way I can implementate the addMacroCmd method..
In my case I think to use lock object and ArrayList.
Do you have some suggestion ? I want to learn in better way the concurrency, but it not very easy for me :(
EDIT : the following is the part of code that add and remove the command :
public synchronized void readSensorData(String[] sensor, String target)
{
cmdTree.reset();
for(int i=0;i<sensor.length;i++)
{
try
{
cmdTree.populateCommandTree(sensor[i],target);
}
catch (JSONException e)
{
}
}
writeExecutor.execute(this.writeCommandTree);
}
/**
*
* #param i
* #param target
* #return
* #throws JSONException when the command requested doesn't exists
*/
private ByteArrayOutputStream f = new ByteArrayOutputStream();
ExecutorService writeExecutor = Executors.newSingleThreadExecutor();
Semaphore mutex = new Semaphore(0);
volatile boolean diagnostic = false;
volatile int index = 0;
Runnable writeCommandTree = new Runnable()
{
#Override
public void run()
{
while(index < cmdTree.getSize())
{
writeCmd();
try
{
mutex.acquire();
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}
sendAnswerBroadcast("answer", answer);
answer = new JSONObject();
index = 0;
}
};
and the mutex is release when arrive a new response .
Addictional information :
The readSensorData() is called when button on the ux (UI Thread) is
pressed and in same case from other Thread B. WriteCommandTree is only
execute in the executor (Other Thread C).
I change the name of getnextcommand into getcommand
- getcommand(int i) is called in the callback of the response (sometime is in other thread (i'm forget to that function ...) and in writecmd inside writecommandtree
- getsize in the writecommandTree in the thread C
Don't get headaches just for synchronizing a list, simply use the Java standard library :
List<Command> commands = Collections.synchronizedList(new ArrayList<>());
By the way, a naive implementation of this would simply to wrap an unsafe list and add synchronized to all the methods.
You can use blockingQueue to achieve the same. Refer simple tutorial about blockingQueue :http://tutorials.jenkov.com/java-util-concurrent/blockingqueue.html
There are several problems with this code:
It is unlikely that you need both a ReentrantLock and synchronization.
The getSize method is not synchronized at all. If, e.g., reset is called from a thread other than the one from which getSize is called, the program is incorrect.
sendCmdMap is leaked in CommandTree's constructor. If the thread that creates the CommandTree is different from the thread that calls populateCommandTree, the program is incorrect.
Note, btw, that using a synchronized view of cmdToSend would not fix any of these problems.
What you need to do, here, is this:
Producers need to seize a lock, hand a command to the CommandTree and then delete all references to it.
Consumers need to seize the same lock and get a reference to a command, deleting it from the CommandTree.
For problems like this, there is no better reference than "Java Concurrency in Practice"

Java Simple Concurrent Server

I must design a simple server which receives messages from multiple nodes and stores them in a message repository.
The code for the server is :
public class CommunicationServer implements Runnable {
private List<String> messages;
private MessageRepository messageRepository;
private boolean serverBusy;
public CommunicationServer() {
messages = new ArrayList<String>();
messageRepository = new MessageRepository();
serverBusy = false;
}
#Override
public void run() {
try {
while (!Thread.interrupted()) {
synchronized (this) {
if (messages.size() > 10) {
serverBusy = true;
addMessageToRepository();
notifyAll();
}
else {
serverBusy = false;
wait();
}
}
}
}
catch (InterruptedException e) {
System.out.println(e.getMessage());
}
}
public synchronized void receiveMessage(String message) {
if (messages.size() < 10) {
messages.add(message);
}
}
private void addMessageToRepository() {
if (messages.size() != 0) {
messageRepository.addMessage(messages.remove(0));
}
}
public void showMessageRepository() {
messageRepository.showStoredMessages();
}
public synchronized boolean isServerBusy() {
return serverBusy;
}
}
The code for the node is:
public class Node implements Runnable {
private static final String CHARACTERS = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVXYZ0123456789";
private static final int MESSAGE_LENGHT = 5;
private Random random = new Random();
private CommunicationServer communicationServer;
public Node(CommunicationServer communicationServer) {
this.communicationServer = communicationServer;
}
#Override
public void run() {
try {
while (!Thread.interrupted()) {
while (communicationServer.isServerBusy()) {
wait();
}
communicationServer.receiveMessage(generateRandomString());
}
}
catch (InterruptedException e) {
System.out.println(e.getMessage());
}
}
private String generateRandomString() {
StringBuffer randomMessage = new StringBuffer();
for (int i = 0; i < MESSAGE_LENGHT; i++) {
randomMessage.append(CHARACTERS.charAt(random.nextInt(51)));
}
return randomMessage.toString();
}
}
In main I just create a thread for the server and 5 threads for the nodes and let them run for some time. The server sleeps until it receives 10 messages, after that it must wake up to process the messages. The problem is I can't figure it out where to call notifyAll() in order to wake the thread responsible for the server.
Making CommunicationServer implements Runnable doesn't really make sense and exposes your basic misunderstanding: you identify the actors in your process (server, client) with threads. A thread is not the actor; a thread is where actor's code gets executed.
So, when in your CommunicationServer you say wait(), you don't make the server wait for messages; you make that particular thread wait on the server object as its monitor. Likewise, then you say notifyAll(), you are not "notifying all servers"; you are notifying all threads waiting on that particular monitor. It should be some code in the client which notifies the threads that are currently waiting on the server's monitor, and some code in the server which notifies those waiting on the client monitor.
As a general rule of thumb, when you find yourself using both wait() and notify() within the same synchronized block, you can be pretty sure there's something wrong with your logic.

Categories