I am supposed to create 2 Threads. One reads from data from file and creates objects of class Merchandise. The file itself consists of over 10,000 lines:
IdOfMerchandise Weight
First thread creates Merchandise objects line and every 200 objects it writes about it. The problem I have is, that I need a second thread, working at the same time as the first one, getting these objects and summing up overall weight, writing report every 100 added.
How can i use the thread to get object data at the same time as they are created in the other thread? Is using HashMap good idea to store newly created class objects with 2 variables?
When you pass data from one thread to another thread, you need a thread-safe data structure. As you correctly pointed out, HashMap is not thread-safe. For thread-safe collections in Java, look at the java.util.concurrent package. One of the simplest ways how to implementing a producer-consumer patterns is with LinkedBlockingQueue.
Here is a complete example with two threads, one producing objects, the other one consuming and printing something every 100 objects:
AtomicBoolean finished = new AtomicBoolean(false);
LinkedBlockingQueue<String> queue = new LinkedBlockingQueue<>();
Thread thread1 = new Thread(() -> {
for (int i = 0; i < 10000; i++) {
String createdObject = Integer.toString(i);
queue.offer(createdObject);
}
finished.set(true);
});
Thread thread2 = new Thread(() -> {
int count = 0;
while (!finished.get() || !queue.isEmpty()) {
try {
String object = queue.poll(100, TimeUnit.MILLISECONDS);
if (count++ % 100 == 0) {
System.out.println(object);
}
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
});
thread1.run(); thread2.run();
thread1.join(); thread2.join();
You may notice one thing - apart from the produced items, the threads also need to exchange other information - when the producer is finished. Again, you cannot safely exchange this information without synchronization. You can use AtomicBoolean as in the example, or a volatile field.
Seems like a producer-consumer problem. This official link will help you understanding and implementing the concept.
Guarded Blocks
The basic idea is, consumer can not consume unless producer has produced something.
Related
I am Looping through a Map , and i have a number of threads.
the queue in the map contains Actions. my goal is to give every thread an Action to do.. but no 2 threads(or more) can run 2 task(or more) from one queue
it means every thread is gonna search for a queue , and Lock the Queue some how and check if the queue has actions if yes it runs one of them if no search for another Queue to run fro them actions.
NOTE: number of Queues can be greater than Number of Threads
e
I tried to synchronize on the 'Map.Entry'
public void run() {
while (true) {
Action<?> act;
for (Map.Entry entry :ActionMap.entrySet()) {
Synchronized(entry)
{
act = ((Queue<Action>)entry.getValue()).poll();
if (act == null)
break;
}
}
}
}
the problem is that if another thread is searching for an action to do is gonna be stuck in the synchronized Line And Wait for the first thread to finish the task or to finish waiting and that is not what i want.
i want all of the threads to search for queues if some a thread reaches a queue that another thread is working on just skip it and continue searching
so I digged around and found semaphore so I reached this
Semaphore Gate = new Semaphore(1);
public void run() {
while (true) {
Action<?> act;
for (Map.Entry entry :ActionMap.entrySet()) {
if( Gate.tryAcquire());
{
act = ((Queue<Action>)entry.getValue()).poll();
if (act == null){
Gate.Release();
break;
}
else {
act.handle();
Gate.Release();
}
}
}
}
}
put the problem with this that Gate.aquire() is gonna Lock all entries
it means for 2 diffirent entries and 2 different threads only one thread can access the gate and execute the Action
so finally dose any one have a design pattern that can help me ?
thank you ...
You could use java.util.concurrent types of map for this. They are thread safe, so you dont need Syncronize.
Synchronize means : the resource(which is synchronized) can't be modified by multiple threads simultaneously. e.g MAP returned by Collections.synchronizedMap(Map) will be a synchronized map and can be modified by one thread at a time, but Concurrent Collections allows multiple threads to access different parts of a collection at a given time, based on the requirement. e.g we have an overloaded constructor for ConcurentHashMap which takes input concurrencyLevel as number of threads which can access the collection simultaneously.
I'm trying to figure out a way to implement the following in Java.
Thread1 will add jobs to queue1.
Another different thread (Thread2) will add jobs to queue2.
In the run() method of Thread1 I wait until there's a job in queue 1, and let's say I will print it, if and only if there are no awaiting jobs in queue2.
How may I notify Thread1 that Thread2 has added a job in queue2?
Here is Thread1 Class
public class Thread1 implements Runnable {
private List queue1 = new LinkedList();
public void processData(byte [] data, int count) {
byte[] dataCopy = new byte[count];
System.arraycopy(data, 0, dataCopy, 0, count);
synchronized(queue1) {
queue1.add(data);
queue1.notify();
}
}
public void run() {
byte [] data;
while(true) {
// Wait for data to become available
synchronized(queue1) {
while(queue1.isEmpty()) {
try {
queue1.wait();
} catch (InterruptedException e) {}
}
data = (byte[]) queue1.remove(0);
}
// print data only if queue2 has no awaiting jobs in it
}
}
You have not quite well explained your question and I am not sure what you are trying to ask -its very confusing to read what you have written. Also, I don't see any code for Thread-2 and Queue-2.
So I am going to put general advice,
1.Use existing implementation of Blocking Queue instead of doing private List queue1 = new LinkedList(); and then doing synchronized(queue1).
Here is documentation of BlockingQueue interface. You can use class , LinkedBlockingQueue as implementation.
2.Sample code - If you browse above link of BlockingQueue documentation, you see code at the bottom highlighting as how to write consumers and producers. There you don't see instance of queue getting created inside Thread class but set via constructor - that way you can share a single queue with as many threads as you like - by passing reference to queue in Runnable constructor.
3.BlockingQueue implementations are thread-safe - so you don't have to synchronizeon queue instances. You can freely pass queue instances to as many threads as you like believing that its methods will be called in synchronized way.
So I suggest that you try to rewrite whatever program you are trying to write using above construct and code samples and come back for any more questions.
Hope it helps !!
I have the following method:
void store(SomeObject o) {
}
The idea of this method is to store o to a permanent storage but the function should not block. I.e. I can not/must not do the actual storage in the same thread that called store.
I can not also start a thread and store the object from the other thread because store might be called a "huge" amount of times and I don't want to start spawning threads.
So I options which I don't see how they can work well:
1) Use a thread pool (Executor family)
2) In store store the object in an array list and return. When the array list reaches e.g. 1000 (random number) then start another thread to "flush" the array list to storage. But I would still possibly have the problem of too many threads (thread pool?)
So in both cases the only requirement I have is that I store persistantly the objects in exactly the same order that was passed to store. And using multiple threads mixes things up.
How can this be solved?
How can I ensure:
1) Non blocking store
2) Accurate insertion order
3) I don't care about any storage guarantees. If e.g. something crashes I don't care about losing data e.g. cached in the array list before storing them.
I would use a SingleThreadExecutor and a BlockingQueue.
SingleThreadExecutor as the name sais has one single Thread. Use it to poll from the Queue and persist objects, blocking if empty.
You can add not blocking to the queue in your store method.
EDIT
Actually, you do not even need that extra Queue - JavaDoc of newSingleThreadExecutor sais:
Creates an Executor that uses a single worker thread operating off an unbounded queue. (Note however that if this single thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.) Tasks are guaranteed to execute sequentially, and no more than one task will be active at any given time. Unlike the otherwise equivalent newFixedThreadPool(1) the returned executor is guaranteed not to be reconfigurable to use additional threads.
So I think it's exactly what you need.
private final ExecutorService persistor = Executors.newSingleThreadExecutor();
public void store( final SomeObject o ){
persistor.submit( new Runnable(){
#Override public void run(){
// your persist-code here.
}
} );
}
The advantage of using a Runnable that has a quasi-endless-loop and using an extra queue would be the possibility to code some "Burst"-functionality. For example you could make it wait to persist only when 10 elements are in queue or the oldest element has been added at least 1 minute ago ...
I suggest using a Chronicle-Queue which is a library I designed.
It allows you to write in the current thread without blocking. It was originally designed for low latency trading systems. For small messages it takes around 300 ns to write a message.
You don't need to use a back ground thread, or a on heap queue and it doesn't wait for the data to be written to disk by default. It also ensures consistent order for all readers. If the program dies at any point after you call finish() the message is not lost. (Unless the OS crashes/loses power) It also supports replication to avoid data loss.
Have one separate thread that gets items from the end of a queue (blocking on an empty queue), and writes them to disk. Your main thread's store() function just adds items to the beginning of the queue.
Here's a rough idea (though I assume there will be cleaner or faster ways for doing this in production code, depending on how fast you need things to be):
import java.util.*;
import java.io.*;
import java.util.concurrent.*;
class ObjectWriter implements Runnable {
private final Object END = new Object();
BlockingQueue<Object> queue = new LinkedBlockingQueue();
public void store(Object o) throws InterruptedException {
queue.put(o);
}
public ObjectWriter() {
new Thread(this).start();
}
public void close() throws InterruptedException {
queue.put(END);
}
public void run() {
while (true) {
try {
Object o = queue.take();
if (o == END) {
// close output file.
return;
}
System.out.println(o.toString()); // serialize as appropriate
} catch (InterruptedException e) {
}
}
}
}
public class Test {
public static void main(String[] args) throws Exception {
ObjectWriter w = new ObjectWriter();
w.store("hello");
w.store("world");
w.close();
}
}
The comments in your question make it sound like you are unfamilier with multi-threading, but it's really not that difficult.
You simply need another thread responsible for writing to the storage which picks items off a queue. - your store function just adds the objects to the in-memory queue and continues on it's way.
Some psuedo-ish code:
final List<SomeObject> queue = new List<SomeObject>();
void store(SomeObject o) {
// add it to the queue - note that modifying o after this will also alter the
// instance in the queue
synchronized(queue) {
queue.add(queue);
queue.notify(); // tell the storage thread there's something in the queue
}
}
void storageThread() {
SomeObject item;
while (notfinished) {
synchronized(queue) {
if (queue.length > 0) {
item = queue.get(0); // get from start to ensure same order
queue.removeAt(0);
} else {
// wait for something
queue.wait();
continue;
}
}
writeToStorage(item);
}
}
My application needs to keep an access log of requests to a certain resource and multiple threads will be recording log entries. The only pertinent piece of information is the timestamp of the request and the stats being retrieved will be how many requests occurred in the last X seconds. The method that returns the stats for a given number of seconds also needs to support multiple threads.
I was thinking of approaching the concurrency handling using the Locks framework, with which I am not the most familiar, hence this question. Here is my code:
import java.util.LinkedList;
import java.util.concurrent.locks.ReentrantLock;
public class ConcurrentRecordStats
{
private LinkedList<Long> recLog;
private final ReentrantLock lock = new ReentrantLock();
public LinkedConcurrentStats()
{
this.recLog = new LinkedList<Long>();
}
//this method will be utilized by multiple clients concurrently
public void addRecord(int wrkrID)
{
long crntTS = System.currentTimeMillis();
this.lock.lock();
this.recLog.addFirst(crntTS);
this.lock.unlock();
}
//this method will be utilized by multiple clients concurrently
public int getTrailingStats(int lastSecs)
{
long endTS = System.currentTimeMillis();
long bgnTS = endTS - (lastSecs * 1000);
int rslt = 0;
//acquire the lock only until we have read
//the first (latest) element in the list
this.lock.lock();
for(long crntRec : this.recLog)
{
//release the lock upon fetching the first element in the list
if(this.lock.isLocked())
{
this.lock.unlock();
}
if(crntRec > bgnTS)
{
rslt++;
}
else
{
break;
}
}
return rslt;
}
}
My questions are:
Will this use of ReentrantLock insure thread safety?
Is it needed to use a lock in getTrailingStats?
Can I do all this using synchronized blocks? The reason I went with locks is because I wanted to have the same lock in both R and W sections so that both writes and reading of the first element in the list (most recently added entry) is done a single thread at a time and I couldn't do that with just synchronized.
Should I use the ReentrantReadWriteLock instead?
The locks can present a major performance bottleneck. An alternative is to use a ConcurrentLinkedDeque: use offerFirst to add a new element, and use the (weakly consistent) iterator (that won't throw a ConcurrentModificationException) in place of your for-each loop. The advantage is that this will perform much better than your implementation or than the synchronizedList implementation, but the disadvantage is that the iterator is weakly consistent - thread1 might add elements to the list while thread2 is iterating through it, which means that thread2 won't count those new elements. However, this is functionally equivalent to having thread2 lock the list so that thread1 can't add to it - either way thread2 isn't counting the new elements.
I'm having a-bit of trouble with threads in java. Basically Im creating an array of threads and starting them. the point of the program is to simulate a race, total the time for each competitor ( i.e. each thread ) and pick the winner.
The competitor moves one space, waits ( i.e. thread sleeps for a random period of time between 5 and 6 seconds ) and then continues. The threads don't complete in the order that they started as expected.
Now for the problem. I can get the total time it takes for a thread to complete; what I want is to store all the times from the threads into a single array and be able to calculate the fastest time.
To do this should I place the array in the main.class file? Would I be right in assuming so because if it was placed in the Thread class it wouldn't work. Or should I create a third class?
I'm alittle confused :/
It's fine to declare it in the method where you invoke the threads, with a few notes:
each thread should know its index in the array. Perhaps you should pass this in constructor
then you have three options for filling the array
the array should be final, so that it can be used within anonymous classes
the array can be passed to each thread
the threads should notify a listener when they're done, which in turn will increment an array.
consider using Java 1.5 Executors framework for submitting Runnables, rather than working directly with threads.
EDIT: The solution below assumes you need the times only after all competitors have finished the race.
You can use a structure that looks like below, (inside your main class). Typically you want to add a lot of you own stuff; this is the main outline.
Note that concurrency is not an issue at all here because you get the value from the MyRunnable instance once its thread has finished running.
Note that using a separate thread for each competitor is probably not really necessary with a modified approach, but that would be a different issue.
public static void main(String[] args) {
MyRunnable[] runnables = new MyRunnable[NUM_THREADS];
Thread[] threads = new Thread[NUM_THREADS];
for (int i = 0; i < NUM_THREADS; i++) {
runnables[i] = new MyRunnable();
threads[i] = new Thread(runnables[i]);
}
// start threads
for (Thread thread : threads) {
thread.start();
}
// wait for threads
for (Thread thread : threads) {
try {
thread.join();
} catch (InterruptedException e) {
// ignored
}
}
// get the times you calculated for each thread
for (int i = 0; i < NUM_THREADS; i++) {
int timeSpent = runnables[i].getTimeSpent();
// do something with the time spent
}
}
static class MyRunnable implements Runnable {
private int timeSpent;
public MyRunnable(...) {
// initialize
}
public void run() {
// whatever the thread should do
// finally set the time
timeSpent = ...;
}
public int getTimeSpent() {
return timeSpent;
}
}