We Keep Coding

Android AsyncTask progress update hang

So I had a crack at coding my own AsyncTask class like system that runs on a ThreadPoolExecutor natively. Everything was working fine until I decided to implement the progress side of things. The progress works much like AsyncTask, the onProgressUpdate function is called on the UI thread. The problem I'm experiencing is that whenever there is a System.out or Log.x line in the onProgressUpdate it hangs indefinitely with no error or warning oddly. The code is as below:
public abstract class Task<A, B> {
private static final Executor EXECUTOR = getExecutor();
private static final int DEFAULT_PRIORITY = Thread.MIN_PRIORITY;
private static final int DEFAULT_PROGRESS_INCREMENT = 1;
private static final Executor getExecutor() {
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newCachedThreadPool();
executor.setCorePoolSize(1);
executor.allowCoreThreadTimeOut(false);
// TODO set rejection handler
//executor.setRejectedExecutionHandler(new Handler());
// TODO set thread factory
executor.prestartCoreThread();
return executor;
}
public static class ExecutionListener<B> {
public void onPreExecute() {
Log.i("TASK", "Pre - Thread: " + Thread.currentThread().getId());
}
public void onPostExecute(B output) {
Log.i("TASK", "Post - Thread: " + Thread.currentThread().getId() + " - Output: " + output);
}
public void onProgressUpdate(int progress) {
Log.d("TASK", "Hello");
}
}
private Handler handler;
private ExecutionListener<B> executionListener;
private volatile int progress = 0;
private AtomicBoolean progressPublished = new AtomicBoolean(true);
private B output;
public Task() {
this.handler = new Handler();
this.executionListener = new ExecutionListener();
}
public void setExecutionListener(ExecutionListener executionListener) {
if(executionListener == null) {
this.executionListener = new ExecutionListener();
}
else {
this.executionListener = executionListener;
}
}
protected void updateProgress(int progressMade) {
Log.d("TASK", "Test");
progress += progressMade;
if(progressPublished.compareAndSet(true, false)) {
if(!handler.post(new Runnable() {
#Override
public void run() {
Log.d("TASK", new Integer(progress).toString() + " - a");
executionListener.onProgressUpdate(progress);
// Hangs below
progressPublished.lazySet(true);
Log.d("TASK", new Integer(progress).toString() + " - b");
}
})) {
Log.d("TASK", "Failed to post");
}
}
}
protected void updateProgress() {
updateProgress(DEFAULT_PROGRESS_INCREMENT);
}
protected abstract B doTask(A input);
public void execute(final A input, final int priority) {
EXECUTOR.execute(new Runnable() {
#Override
public void run() {
Thread.currentThread().setPriority(priority);
handler.post(new Runnable() {
#Override
public void run() {
executionListener.onPreExecute();
}
});
output = doTask(input);
if(!handler.post(new Runnable() {
#Override
public void run() {
Log.d("TASK", "Done");
executionListener.onPostExecute(output);
}
})) {
Log.d("TASK", "Failed to post post");
}
}
});
}
public void execute(final A input) {
execute(input, DEFAULT_PRIORITY);
}
}
The ExecutionListener is just a class to override the methods to be run on the UI much like AsyncTask's methods for doing the same. The code uses Runnable objects to execute the doTask method and send updates / the result to the appropriate method in the ExecutionListener.
The Thread.currentThread() parts are just to ensure things are running on the thread I intended them to. The problem only shows itself when running a Task that frequently calls updateProgress() - I have tried putting a thread sleep in the onProgressUpdate() method and that seems so solve things, though that obviously isn't a good solution.
It also only seems to have a problem with Log.x / System.out - I don't know whether the call frequency of either of them could cause this kind of issue. I'm at a loss with this progress feature and logging so any advice would be greatly appreciated - I've also found this quite hard to explain so please ask if you need me to clarify anything!

Turns out the Thread.currentThread().getId() is at fault. Removing that part fixes everything. Follow up question here: Is Java Thread getId() thread-safe?

How to check the size() or isEmpty() for ConcurrentLinkedQueue

I am trying to prototype a simple structure for a Web crawler in Java. Until now the prototype is just trying to do the below:
Initialize a Queue with list of starting URLs
Take out a URL from Queue and submit to a new Thread
Do some work and then add that URL to a Set of already visited URLs
For the Queue of starting URLs, I am using a ConcurrentLinkedQueue for synchronizing.
To spawn new Threads I am using ExecutorService.
But while creating a new Thread, the application needs to check if the ConcurrentLinkedQueue is empty or not. I tried using:
.size()
.isEmpty()
But both seem not to be returning the true state of ConcurrentLinkedQueue.
The problem is in below block:
while (!crawler.getUrl_horizon().isEmpty()) {
workers.submitNewWorkerThread(crawler);
}
And because of this, ExecutorService creates all the Threads in its limit, even if the input is only 2 URLs.
Is there a problem with the way multi-threading is being implemented here? If not, what is the better way to check the state of ConcurrentLinkedQueue?
Starting class for the application:
public class CrawlerApp {
private static Crawler crawler;
public static void main(String[] args) {
crawler = = new Crawler();
initializeApp();
startCrawling();
}
private static void startCrawling() {
crawler.setUrl_visited(new HashSet<URL>());
WorkerManager workers = WorkerManager.getInstance();
while (!crawler.getUrl_horizon().isEmpty()) {
workers.submitNewWorkerThread(crawler);
}
try {
workers.getExecutor().shutdown();
workers.getExecutor().awaitTermination(10, TimeUnit.MINUTES);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private static void initializeApp() {
Properties config = new Properties();
try {
config.load(CrawlerApp.class.getClassLoader().getResourceAsStream("url-horizon.properties"));
String[] horizon = config.getProperty("urls").split(",");
ConcurrentLinkedQueue<URL> url_horizon = new ConcurrentLinkedQueue<>();
for (String link : horizon) {
URL url = new URL();
url.setURL(link);
url_horizon.add(url);
}
crawler.setUrl_horizon(url_horizon);
} catch (IOException e) {
e.printStackTrace();
}
}
}
Crawler.java which maintains the Queue of URLs and Set of already visited URLs.
public class Crawler implements Runnable {
private ConcurrentLinkedQueue<URL> url_horizon;
public void setUrl_horizon(ConcurrentLinkedQueue<URL> url_horizon) {
this.url_horizon = url_horizon;
}
public ConcurrentLinkedQueue<URL> getUrl_horizon() {
return url_horizon;
}
private Set<URL> url_visited;
public void setUrl_visited(Set<URL> url_visited) {
this.url_visited = url_visited;
}
public Set<URL> getUrl_visited() {
return Collections.synchronizedSet(url_visited);
}
#Override
public void run() {
URL url = nextURLFromHorizon();
scrap(url);
addURLToVisited(url);
}
private URL nextURLFromHorizon() {
if (!getUrl_horizon().isEmpty()) {
URL url = url_horizon.poll();
if (getUrl_visited().contains(url)) {
return nextURLFromHorizon();
}
System.out.println("Horizon URL:" + url.getURL());
return url;
}
return null;
}
private void scrap(URL url) {
new Scrapper().scrap(url);
}
private void addURLToVisited(URL url) {
System.out.println("Adding to visited set:" + url.getURL());
getUrl_visited().add(url);
}
}
URL.java is just a class with private String url and overriden hashCode() and equals().
Also, Scrapper.scrap() just has dummy implementation until now:
public void scrap(URL url){
System.out.println("Done scrapping:"+url.getURL());
}
WorkerManager to create Threads:
public class WorkerManager {
private static final Integer WORKER_LIMIT = 10;
private final ExecutorService executor = Executors.newFixedThreadPool(WORKER_LIMIT);
public ExecutorService getExecutor() {
return executor;
}
private static volatile WorkerManager instance = null;
private WorkerManager() {
}
public static WorkerManager getInstance() {
if (instance == null) {
synchronized (WorkerManager.class) {
if (instance == null) {
instance = new WorkerManager();
}
}
}
return instance;
}
public Future submitNewWorkerThread(Runnable run) {
return executor.submit(run);
}
}

Problem
The reason why you end up creating more Threads than there are URLs in the queue is because it is possible (and in fact likely) that none of the Threads of the Executor start until you go through the while loop a lot of times.
Whenever working with threads you should always keep in mind that the threads are scheduled independently and run at their own pace except when you explicitly synchronize them. In this case, the threads can start at any time after the submit() call, even though it seems you'd like each one to start and go past nextURLFromHorizon before a next iteration in your while loop.
Solution
Consider dequeuing the URL from the queue before submitting the Runnable to the Executor. I also suggest defining a CrawlerTask that is submitted to the Executor once, rather than a Crawler that is submitted repeatedly. In such design you wouldn't even need a thread-safe container for the URLs to-be-scraped.
class CrawlerTask extends Runnable {
URL url;
CrawlerTask(URL url) { this.url = url; }
#Override
public void run() {
scrape(url);
// add url to visited?
}
}
class Crawler {
ExecutorService executor;
Queue urlHorizon;
//...
private static void startCrawling() {
while (!urlHorizon.isEmpty()) {
executor.submit(new CrawlerTask(urlHorizon.poll());
}
// ...
}
}

Binding an API callback to an RxJava Observable

I'm trying make a reactive application that listens to a network socket on a separate thread for prices and got a bit stumped with how exactly to construct the Observable. Much of the interfaces I have are constrained by the API I am using and therefore cannot change. I distilled what I am trying to do as a test below, but I can't see how to fill in the body of the getPriceReactive() method such that the prices are printed on the console by the subscriber (see the comment in the code).
public class PriceObservableTest {
// This interface is defined externally and used by the API
private interface ITickHandler {
void priceReceived(double price);
}
// Stores the price (currently just one double for illustration)
private class Tick {
double price = Double.NaN;
}
// Implementation of handler called by API when it receives a price
private class TickHandler implements ITickHandler {
private final Tick tick;
TickHandler() { this.tick = new Tick(); }
#Override public void priceReceived(double x) { tick.price = x; }
}
// This class emulates the API delivering prices from the socket
private class PriceSource {
private final Thread thread;
PriceSource(final ITickHandler handler) {
thread = new Thread(new Runnable() {
final Random r = new Random();
#Override public void run() {
while (!Thread.currentThread().isInterrupted()) {
try {
Thread.sleep(100);
handler.priceReceived(r.nextDouble() * 100);
} catch (InterruptedException e) {
break;
}
}
System.out.println("Price thread closed");
}
});
}
void subscribe() { thread.start(); }
void unsubscribe() { thread.interrupt(); }
}
#Test
public void simpleTest() throws Exception {
final ITickHandler handler = new TickHandler();
// Simulate some prices received periodically from a socket
PriceSource prices = new PriceSource(handler);
Observable<Tick> reactive = getPriceReactive(handler);
reactive.subscribe(new Subscriber<Tick>() {
#Override public void onCompleted() { }
#Override public void onError(Throwable e) { }
#Override public void onNext(Tick tick) {
System.out.println("Received price: " + tick.price);
}});
// Observe prices for 1 second. The subscriber should print them to console
prices.subscribe();
Thread.sleep(1000);
prices.unsubscribe();
}
// Returns an observable that reacts to price changes
private Observable<Tick> getPriceReactive(ITickHandler handler) {
return Observable.create(new Observable.OnSubscribe<Tick>() {
#Override public void call(Subscriber<? super Tick> subscriber) {
// How to call subscriber.onNext() whenever
// priceReceived() is called with a new price?
}
});
}
}
Somehow subscriber.onNext() needs to be called whenever the API calls priceReceived(), but I can't quite see how to achieve this. Of course I could store a reference to the subscriber in the TickHandler but this kind of defeats the purpose of having an Observable, doesn't it?

Transition to Observable in ITickHandler implementation. You are not controlling the subscriber(s) but the publisher
private class TickHandler implements ITickHandler {
private final Tick tick;
private final PublishSubject<Tick> priceSubject;
TickHandler() {
this.tick = new Tick();
this.priceSubject = PublishSubject.create();
}
#Override public void priceReceived(double x)
{
tick.price = x;
priceSubject.onNext(tick);
}
public Observable<Tick> priceReceivedObservable()
{
return priceSubject.asObservable();
}
}
And you can use it in your tests like:
final ITickHandler handler = new TickHandler();
PriceSource prices = new PriceSource(handler);
handler.priceReceivedObservable()
.subscribe(new Subscriber<Tick>() {
#Override public void onCompleted() { }
#Override public void onError(Throwable e) { }
#Override public void onNext(Tick tick) {
System.out.println("Received price: " + tick.price);
}});
I warn you, it's not tested since I don't do a lot of Java :)

Call a method only once after 1 second from a method which get called 2-3 times

The below method onReceivedTitlegets called 2-3 times with in a second when webview url changes. I want to call a method in it, when onReceivedTitle is being called last time. I am doing this because I just want to monitor url changes with in webview. shouldOverrideUrlLoading is not getting called when url changes through ajax.
class MyWebChromeClient extends WebChromeClient {
#Override
public void onReceivedTitle(WebView view, String title) {
Log.v("onReceivedTitle", "=>" + title);
// callAMehod();
super.onReceivedTitle(view, title);
}
}

If you want to throttle how often a method call causes another method call you can do so for example via a Handler. The simplest version enqueues a delayed message on the first call and any subsequent call while there is an enqueued message will not enqueue a new one. That results in 1 call every X time to go though - but it take at least that amount of time until the first action happens.
Example implementation (you can put that class unmodified somewhere in your code)
public abstract class ThrottleExecutor {
private final long mMinDelay;
public ThrottleExecutor(long minDelay) {
mMinDelay = minDelay;
}
/** Implement to do something */
public abstract void doThrottled();
public final void scheduleExecution() {
if (mHandler.hasMessages(0)) {
// message already enqueued, do nothing
} else {
// otherwise enqueue a message for later
mHandler.sendEmptyMessageDelayed(0, mMinDelay);
}
}
public final void cancelExecution() {
mHandler.removeMessages(0);
}
private final Handler mHandler = new Handler(Looper.getMainLooper()) {
#Override
public void handleMessage(Message msg) {
doThrottled();
}
};
}
And then use it for example like so
class Usage {
private ThrottleExecutor mThrottle = new ThrottleExecutor(2000) {
#Override
public void doThrottled() {
// happens at most every 2000ms
methodToBeThrottled();
}
};
void methodThatHappensTooOften() {
mThrottle.scheduleExecution();
}
void methodToBeThrottled() {
Log.d("TAG", "triggered at 2000ms before");
}
}

You might want to use Handler and do something like this:
class MyWebChromeClient extends WebChromeClient {
private boolean mOnReceivedTitleInvoked;
#Override
public synchronized void onReceivedTitle(final WebView view, final String title) {
if (!mOnReceivedTitleInvoked) {
mOnReceivedTitleInvoked = true;
Log.v("onReceivedTitle", "=>" + title);
handler.postDelayed(new Runnable() {
#Override
public void run() {
super.onReceivedTitle(view, title);
mOnReceivedTitleInvoked = false;
}
}, 1000);
}
}
}
Although you might want to reconsider the onReceivedTitle behaviour.

How to stack up incoming data

Each time a back-end message comes I add it to JList and JList is being refreshed using fireIntervalAdded. The problem is that in one second 20 messages may arrive and each one of them will call fireIntervalAdded. What I would like to do is to stack all messages in List and send one big stack of data to JList. My current solution does not seem to work, it always sends one message instead of one big stack:
private class StackingListener implements MessageListener {
private List<Message> messages = new LinkedList<Message> ();
private int waiting = 0;
#Override
public void messageReceived(MessageEvent event) {
stackData(event.getData());
}
private void stackData(Message data) {
messages.add(data);
if (waiting <= 0) {
waiting = 3;
new Thread(new Runnable() {
#Override
public void run() {
while(--waiting > 0) {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
List<Message> list = new ArrayList<Message>(messages);
messages.clear();
logger.info("Adding list with size of " + list.size());
controller.getListModel().addFullElements(list);
}
}).run();
} else {
waiting = 3;
}
}
}
I think I'm doing something very wrong. The idea of this code is to stack up messages while Thread is sleeping. But seems like Thread.sleep stops everything, not only the current thread.
Thanks

You are using Thread.run() which just calls the run() method in the current thread. What you intended to use was Thread.start() creates a thread and call run() in that new thread.
However I wouldn't structure the code like this at all. I believe a simpler apporach is to use a queue.
class StackingListener implements MessageListener, Runnable {
private final BlockingQueue<Message> messages = new LinkedBlockingDeque<Message>();
private final ScheduledExecutorService service = Executors.newSingleThreadScheduledExecutor(); {
service.scheduleAtFixedRate(this, 500, 500, TimeUnit.MILLISECONDS);
}
#Override
public void messageReceived(MessageEvent event) {
messages.add(event.getData());
}
#Override
public void run() {
final List<Message> list = new ArrayList<Message>();
messages.drainTo(list);
logger.info("Adding list with size of " + list.size());
// add to the GUI component in a thread safe manner.
SwingUtilities.invokeLater(new Runnable() {
#Override
public void run() {
controller.getListModel().addFullElements(list);
}
});
}
public void stop() {
service.shutdown();
}
}