I want to implement the observer pattern in Android. The observable and observers have to work in their own threads (HandlerThread). The observable object produces some results and notifies their observers every milliseconds. The observers register by passing themselves and Handler reference as arguments.
Example code:
public class Observable {
public interface Observer {
void notify(List<Object> results);
}
private HandlerThread handlerThread;
private List<Object> results = new ArrayList<>();
private final Map<Observer, ObserverWrapper> observers = new HashMap<>();
private void working() {
//Critial section...
//modifying results list...
synchronized (observers) {
for (ObserverWrapper wrapper : observers.values()) {
wrapper.notify(results);
}
}
}
public void register(Observer observer, Handler handler) {
synchronized (observers) {
//Create observer wrapper and add to map here...
}
}
public void unregister(Observer observer) {
synchronized (observers) {
//Remove observer from the map...
}
}
private static final class ObserverWrapper {
private final Observer observer;
private final Handler handler;
public ObserverWrapper(Observer observer, Handler handler) {
this.observer = observer;
this.handler = handler;
}
public void notify(final List<Object> results) {
//The observable thread
handler.post(new Runnable() {
void run() {
//The observer thread
//Critical section...
observer.notify(results);
}
});
}
}
}
The question is: how to synchronize results list that is passed to all observers? I cannot use a copy for each observer because it would cause high memory usage.
There are three options I can think of:
Use the results object as a monitor. That requires minimum changes but this a) not very clean as object itself doesn't say anything about being a monitor. b) Given that the updates are coming every few milliseconds, there is a concern that observer threads will be locking the producer thread for extended periods of time. Both producer and the observer will have to:
synchronized(results) {
// ...
}
Pass an explicit lock object to notify() and use it explicitely:
private final Object lock = new Object();
synchronized(lock) {
// Update results
}
...
synchronized(lock) {
// Read results
}
You mentioned that you don't want to copy results to reduce GC footprint, but this can be the most concurrent option. I'd do it if I were writing it. There is actually a better option, more optimistic option in terms of memory, which is CopyOnWriteArrayList, it will only copy if there is a simultaneous read and write:
https://developer.android.com/reference/java/util/concurrent/CopyOnWriteArrayList.html
Based on the discussions, here is the most concurrent way to do it:
private void working() {
//Critial section...
//modifying results list...
// Create a copy for observers to process
List resultsToProcess = new ArrayList(results);
synchronized (observers) {
for (ObserverWrapper wrapper : observers.values()) {
wrapper.notify(resultsToProcess);
}
}
}
Related
I want to have multiple threads, each of them linked to a "group", which will do operations. Each of these threads, will in a loop look into a queue of Operations to do, and it is not empty, they will take next Operation and process it.
It is as simple as this:
public class GroupThreadManager {
private static ConcurrentHashMap<Long, GroupThread> threads = new ConcurrentHashMap<>();
private synchronized static void newOperation(Operation op) throws IOException {
final Long idGroup = op.getIdGroup();
if (!threads.containsKey(idGroup )) {
threads.put(idGroup , new GroupThread());
}
threads.get(idGroup ).start(op);
}
private synchronized static void interrupThread(Long id) {
if(threads.remove(id) == null) {
log.info("THIS SHOULDNT HAVE HAPPENED!!!!!");
}
}
}
public class GroupThread implements Runnable {
private Thread worker;
private ConcurrentLinkedQueue<Operation> operations = new ConcurrentLinkedQueue<>();
private Long idGroup;
public void start(Operation op) throws IOException {
addOperation(op);
if (worker == null) {
idGroup = op.getIdGroup();
worker = new Thread(this);
worker.start();
}
}
public synchronized void addOperation(Operation op) {
operations .add(user);
}
private synchronized int size() {
return operations.size();
}
public void run() {
while (size() > 0) {
operations.poll()
.compute() // do something here
}
GroupThreadManager.interrupThread(idUser);
}
}
If the run method was implemented with while (true) I would have no problem. The issue comes when I want that thread to process all the operations it has, and whenever it becomes out of operations, I want to make that Thread end. I have been trying to make proper synchronization to create/get the Thread from the GroupThreadManager, but I always come into either deadlocks, either Thread ending with new Operations to proceed due to missing synchronization.
The idea is that from another part of the program I can just call GroupThreadManager.newOperation(new Operation()) and this manager automatically gives me the correct thread for that groupId (contained in Operation), creating it, giving me the existing one, or stopping & deleting it when it detects there are no new operations for it
As we know, ThreadPoolExecutor uses some BlockingQueue as a queue of incoming tasks. What I want is to have ThreadPoolExecutor that has a second queue for the task results which are ready. I want to use this queue as a source for input/output services which send or store these results.
Why I want to create a separate queue? Because I want to decouple action of sending results from action of obtaining results. Also, I suppose any Exceptions and Delays that accompany input/output operations should not affect my ThreadPoolExecutor which is calculating the result.
I have created some naive implementation of this. I would like to get some criticism on this. May be, it can be implemented with out-of-the-box Java classes better? I use Java 7.
public class ThreadPoolWithResultQueue {
interface Callback<T> {
void complete(T t);
}
public abstract static class CallbackTask<T> implements Runnable {
private final Callback callback;
CallbackTask(Callback callback) {
this.callback = callback;
}
public abstract T execute();
final public void run() {
T t = execute();
callback.complete(t);
}
}
public static class CallBackTaskString extends CallbackTask<String> {
public CallBackTaskString(Callback callback) {
super(callback);
}
#Override
public String execute() {
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
}
return hashCode() + "-" + System.currentTimeMillis();
}
}
public static void main(String[] args) throws InterruptedException {
BlockingQueue<Runnable> workQueue = new LinkedBlockingQueue<Runnable>();
final BlockingQueue<String> resultQueue = new LinkedBlockingQueue<String>();
Callback<String> addToQueueCallback = new Callback<String>() {
#Override
public void complete(String s) {
System.out.println("Adding Result To Queue " + s);
resultQueue.add(s); //adding to outgoing queue. some other executor (or same one?) will process it
}
};
ThreadPoolExecutor executor = new ThreadPoolExecutor(3, 5, 1000l, TimeUnit.DAYS, workQueue);
for (int i = 0; i <= 5; i++) {
executor.submit(new CallBackTaskString(addToQueueCallback));
};
System.out.println("All submitted.");
executor.shutdown();
executor.awaitTermination(10l, TimeUnit.SECONDS);
System.out.println("Result queue size " + resultQueue.size());
}
}
For the sake of makinf a library component, you would have to wrap things up...
You could extend The thread pool executor which has a number of methods to intercept the submitted tasks, so you would queue thing out to a queue passed in the constructor.
That's basically ExecutorCompletionService, but you would allow the user to plug a queue instead of appearing as one.
Otherwise, this is typical proxying of the task. Fair job.
I am currently writing a java application to do a brute force attack on an MD5 hash. I made a JFrame that takes the uses hash input.
for example i hashed the word "password" to MD5. (seen in picture 1) when the JButton is clicked
it will send the hash to a Comparative class that will iterate trough all the possibilities in sequential order. IE go from AAAAAAAA. then to AAAAAAAB and so on. and hashes them. If the hash of the possibility matched the hash of the word "password". A dialog box will appear informing the user that a match has been found.
My problem is that i don't know how to return a value from a thread in a class that implements Runnable to the GUI class without using setters and getters.
public void actionPerformed(ActionEvent e)
{
if(e.getSource().equals(bruteForceButton))
{
enterLabel.setText("Brute force in process");
String enteredHash = input.getText();
int lengthOfPass = (int) length.getSelectedItem();
//Send the information to the comparative class
HashComparative comp = new HashComparative();
comp.setHash(enteredHash);
comp.setLengthOfPass(lengthOfPass);
Thread t1 = new Thread(comp);
t1.start();
hash.setText(comp.getHash());
}
this starts the thread sending the appropriate data to the comparative class.
And here is the other class.
public class HashComparative implements Runnable
{
private String h;
private int l;
private Thread thread;
public void start()
{
thread = new Thread(this, "");
thread.start();
}
#Override
public void run()
{
try
{
//CODE WILL GO HERE TO COMPUTE HASHES
setHash(h);
}
catch(Exception e)
{
e.printStackTrace();
}
}
public void setHash(String h)
{
this.h = h;
}
//Returns the hash currently being compared
public String getHash()
{
return h;
}
}
While the brute force is in operation. I want the display the hashes that it tries by sending the hash string back to the GUI class and set the text to a JLabel. For Example
I have looked online but can't find an answer to this. Any help will be appreciated
What you need to implement is rather Callable instead of Runnable as this interface allows you to return a value which is what you want here. As you can only provide Runnable to a Thread, you need to use a pool thread instead, that you can create from Executors. It will allow you to submit your task as a Callable or a Runnable, and then get a Future to be able to wait until the task is completed successfully or not.
In your case you can create a pool of only one thread using Executors.newSingleThreadExecutor()
ExecutorService executor = Executors.newSingleThreadExecutor();
...
Then your class HashComparative will be:
public class HashComparative implements Callable<String> {
private final String enteredHash;
private final int lengthOfPass;
public HashComparative(final String enteredHash, final int lengthOfPass) {
this.enteredHash = enteredHash;
this.lengthOfPass = lengthOfPass;
}
#Override
public String call() throws Exception {
//CODE WILL GO HERE TO COMPUTE HASHES
return h;
}
...
}
And finally you will submit your task as next:
HashComparative comp = new HashComparative(enteredHash, lengthOfPass);
Future<String> future = executor.submit(comp);
hash.setText(future.get());
What can be used to display the unmatching hashes?
The design pattern that matches the best with this kind of need is Observer. Indeed here you want to make your UI observes the progress of your computation. Out of the box, you have Observable and Observer that you can use to implement this pattern.
So here for the sake of simplicity, you could make your class HashComparative extends Observable to be able to notify the Observers (your UI here) any time a new hash is being processed.
public class HashComparative extends Observable implements Callable<String> {
private String h;
...
public void setHash(String h) {
this.h = h;
setChanged();
notifyObservers();
}
public String getHash() {
return h;
}
}
Then in your UI, you can change the text accordingly by implementing Observer
final HashComparative comp = new HashComparative(enteredHash, lengthOfPass);
comp.addObserver(new Observer() {
#Override
public void update(final Observable o, final Object arg) {
HashComparative hc = (HashComparative)o;
hash.setText(hc.getHash());
}
});
If it goes too fast, you could also use a ScheduledExecutorService to only change the hash displayed after a given amount of time. You could for example create it using Executors.newSingleThreadScheduledExecutor() to have only one thread then refresh the hash as next:
final HashComparative comp = new HashComparative(enteredHash, lengthOfPass);
scheduler.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
hash.setText(comp.getHash());
}
}, 1L, 1L, TimeUnit.SECONDS);
This will refresh the hash displayed every second. With this approach you have no need to extend Observable anymore, you will only need to make h volatile as it will be modified and accessed by several threads concurrently
public class HashComparative implements Callable<String> {
private volatile String h;
...
}
I made a producer-consumer program. It's just a program in core java without any GUI(Swing or SWT). It has one producer who put objects into the queue.
Also there is a few consumers who must add some staff(for example String) into Every object in that shared queue. So, every consumer must handle every object in a shared queue.
In this case - every BookShelf must have items from All consumers in "books" ArrayList. consumers.
Question: What condition should I use in consumers to finish their threads correctly?
Here are the code fragments of the program. Maybe I implemented it in wrong way.
Here is an object for the queue:
public class BookShelf {
private int id;
private String name;
private int height;
private int weigh;
List<String> books = Collections.synchronizedList(new ArrayList<String>());
public BookShelf(int id, String name) {
this.id = id;
this.name = name;
}
public void addBook(String book) {
books.add(book);
}
public boolean eq(String book) {
synchronized (books) {
for (String b: books) {
if (b.equalsIgnoreCase(book)) {
return true;
}
}
}
return false;
}
other setters and getters..
}
Here is the producer class:
public class Producer implements Runnable {
private BlockingQueue myQueue;
public Producer(BlockingQueue myQueue) {
this.myQueue = myQueue;
}
public void run() {
for(int i=0; i<7; i++){
try {
System.out.println("Produced: " + i);
BookShelf myBookShelf = new BookShelf(i, "book #" + i);
myQueue.put(myBookShelf);
} catch (InterruptedException ex) {
//Proper handle
}
}
}
}
Here is one of consumers class:
public class Consumer implements Runnable {
private BlockingQueue myQueue;
public Consumer(BlockingQueue myQueue) {
this.myQueue = myQueue; }
public void run() {
while(true){
try {
BookShelf tempBookShelf = (BookShelf) myQueue.take();
//eq() is my method to check if ArraList has a book.
if (tempBookShelf.eq("Abc book")) {
System.out.println("It already has book");
myQueue.put(tempBookShelf);
Thread.sleep(2000);
} else {
tempBookShelf.addBook("Abc book");
myQueue.put(tempBookShelf);
Thread.sleep(2000);
}
} catch (InterruptedException ex) {
//Proper handle
}
}
}
}
Here is main class:
public class ProducerConsumerTest {
public static void main(String[] args) {
BlockingQueue sharedQueue = new LinkedBlockingQueue();
Thread prodThread = new Thread(new Producer(sharedQueue));
Thread consThread = new Thread(new Consumer(sharedQueue));
Thread consThread2 = new Thread(new Consumer2(sharedQueue));
prodThread.start();
consThread.start();
consThread2.start();
}
}
Register each consumer with the producer. Each consumer has its own queue and the producer puts the object into all the queues. Each consumer then process on the same instance of the object.
public interface Consumer{
public void process(BookShelf bs);
}
public class Producer implements Runnable{
private final List<Consumer> consumers = new CopyOnWriteArrayList<Consumer>(); // thread safe but not efficient with lots of changes
public void register(Consumer c){
consumers.add(c); // thread safe
}
public void run(){
for(;;){
BookShelf bs = generateBookShelfByWhateverMeans();
for (Consumer c : consumers){
c.process(bs);
}
}
}
}
public class BookShelfConsumer implements Runnable, Consumer{
private final BlockingQueue<BookShelf> queue = new LinkedTransferQueue<BookShelf>(); // unbounded & thread safe
public void process(BookShelf bs){
queue.offer(bs); // non-blocking
}
public void run(){
for(;;){
BookShelf bs = queue.take(); // blocks until got object or interrupted
// catch InterruptedException
// do whatever this consumer is supposed to do with the object
}
}
}
I would try using SwingWorker instead. It has a done() method that is executed when it's finished. See this page for some code examples.
If it's not Swing you are using, there is a similar function in Swt called Jobs. Check this page for examples. It also has a done() method being executed when the job is done.
Also there is a few(N number) consumers who must add some staff(for example String) into Every object in that shared queue
I assume you mean every consumer must add their thing to every object which ever enters the queue. In that case, this is not a producer-consumer problem, this is more like an observer-observable problem. Basically, when a new BookShelf is created, that is the Observable. All of the Observers should be notified about the BookShelf and given the opportunity to add their own Book.
I recommend using a ConcurrentLinkedQueue in Bookshelf instead of a synchronized list - it's lock free (doesn't need to be synchronized) and will probably be more efficient.
To end your consumers, change their while(true) loops to while(!cancel) loops. Give each consumer a cancel boolean as an instance variable that initializes to false, and give them a cancel() method that sets cancel to true. Call cancel() on your consumers when you're done with them. If you will always be canceling all of your consumers at once (instead of selectively canceling some but not others), then you can use a static cancel instead of an instance cancel.
I have a thread inside a class like this-
import java.util.Observable;
public class Download extends Observable {
private int state = 0;
private final Thread myThread = new Thread(() -> {
/*
some work to do here
*/
setChanged();
notifyObservers(state);
});
public void download(int state) {
if (!myThread.isAlive()) {
this.state = state;
myThread.start();
}
}
public Thread getThread() {
return myThread;
}
public static void MyMethod() throws InterruptedException {
Download down = new Download();
down.addObserver((Observable ob, Object dat) -> {
System.out.println(ob);
if ((int) dat == 1) {
down.download(2);
} else {
System.out.println("success");
}
});
down.download(1);
down.getThread().join();
}
public static void main() throws InterruptedException {
MyMethod();
}
}
The problem is I never get it to print the "success" message.
I assume, it is because all observers are being notified from inside of MyThread. So when down.download(2) is called from the observer inside MyMethod(), the previous thread is still running and the call is ignored.
How can I notify all observers from the main thread, not from the myThread?
You are calling down.download(2) from within the execution of MyThread, therefore the thread is still alive which means that your download method does nothing because of if(!myThread.isAlive()).
I would recommend you to use the Executor framework and Listenable Futures from Guava instead of creating threads manually. Example code from the Guava wiki:
ListeningExecutorService service =
MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(10));
ListenableFuture<Explosion> explosion = service.submit(new Callable<Explosion>() {
public Explosion call() {
return pushBigRedButton();
}
});
Futures.addCallback(explosion, new FutureCallback<Explosion>() {
// we want this handler to run immediately after we push the big red button!
public void onSuccess(Explosion explosion) {
walkAwayFrom(explosion);
}
public void onFailure(Throwable thrown) {
battleArchNemesis(); // escaped the explosion!
}
});
Note that Futures.addCallback(..) also has an overload which allows you to determine which executor should execute the callback, this seems to be what you want.