I'm having some troubles with a variable while in a multi-threaded environment.
I have 3 components/threads: Controller, Handler, ExternalAgent.
Controller hands off raw data to Handler. Handler spins off a new ExternalAgent thread for further processing. ExternalAgent comes back to Handler once it's done doing its thing and adds messages to a linked list for further processing by the Handler.
The problem is, the messages added by the ExternalAgent are not visible to some methods inside the Handler. Here's the Handler code:
public class InboundHandler implements InboundController.Handler {
private Thread receiver;
private volatile LinkedList<Message> bufferedMsgs = new LinkedList<Message>();
private volatile boolean isAlive = true;
private final static Logger logger = Logger.getLogger(Logger.GLOBAL_LOGGER_NAME);
public InboundHandler() {
receiver = new Thread(this,"receiver");
receiver.start();
}
#Override
public void run() {
while (isAlive) {
processBuffer();
}
}
#Override
public void process(String msg) {
Message packet = (Message) new Parser<Message>(Message.class,msg).getObject();
new ExternalAgent(this,packet).start();
}
public void processBuffer() {
logger.info("Processing buffersize .." + bufferedMsgs.size());
synchronized (bufferedMsgs) {
Iterator<Message> it = bufferedMsgs.iterator();
while (it.hasNext()) {
//PROCESSING
}
}
}
public void addMsgToBuffer(Message m) {
synchronized (bufferedMsgs) {
bufferedMsgs.addLast(m);
}
}
Controller talks to Handler through the process(msg) interface
ExternalAgent talks to the Handler through the addMsgToBuffer(msg) interface.
In the method processBuffer(), the LinkedList bufferedMsgs always shows empty although if I print it in the method addMsgToBuffer(), it gives me the expected contents. I have tried everything in my knowledge to understand why this is happening but I am unable to figure it out.
Related
I am trying to access variables from multiple running threads.
I have a main class that starts up 2 threads, a producer and a consumer.
the PRODUCER thread reads a binary file. For each line in that binary file the producer thread creates an object from it, and passes that object to the consumer thread through a blockingqueue.
the CONSUMER then takes that object passed in through a blocking queue and outputs the values of the fields in that object to a text file.
sometimes there are errors in the binary file that the producer thread is reading from.
when there are too many errors in the binary file, I want the consumer thread to change the extension of the txt file that it outputs to .err
My problem: I dont know how to modify a value from a producer thread in a consumer thread. Ive been reading that I could use a volatile field. But I dont know what is the proper way to use it between threads.
Here is a very shortened and less complex example of my code :
public class Main
{
private volatile static boolean tooManyErrors= false;
public static void main(String[] args)
{
BlockingQueue<binaryObject> queue = new LinkedBlockingQueue<>(null);
binaryObject poison = null;
new Thread(new Producer(tooManyErrors, queue, poison)).start();
new Thread(new Consumer(tooManyErrors, queue, poison)).start();
}
}
public class Producer implements Runnable
{
private final BlockingQueue<binaryObject> queue;
private final binaryObject POISON;
private boolean tooManyErrors;
private int errorsCounter = 0;
public Producer(boolean tooManyErrors, BlockingQueue<binaryObject> queue,
binaryObject POISON)
{
this.tooManyErrors = tooManyErrors;
this.queue = queue;
this.POISON = POISON;
}
#Override
public void run()
{
try
{
process();
}
catch (InterruptedException e)
{
Thread.currentThread().interrupt();
}
finally
{
while (true)
{
try
{
queue.put(POISON);
break;
}
catch (InterruptedException e)
{
//...
}
}
}
}
private void process() throws InterruptedException
{
//here is where all the logic to read the file and create
//the object goes in. counts the number of errors in the file
//if too many errors, want to change the tooManyErrors to true
if(errorsCounter > 100)
{
tooManyErrors = true;
}
}
}
public class Consumer implements Runnable
{
private final BlockingQueue<binaryObject> queue;
private final binaryObject POISON;
private boolean tooManyErrors;
//variable with extension name
private String extension;
public Consumer(boolean tooManyErrors, BlockingQueue<Integer> queue,
binaryObject POISON)
{
this.tooManyErrors = tooManyErrors;
this.queue = queue;
this.POISON = POISON;
}
#Override
public void run()
{
try
{
while (true)
{
binaryObject take = queue.take();
process(take);
// if this is a poison pill, break, exit
if (take == POISON)
{
break;
}
}
}
catch (InterruptedException e)
{
Thread.currentThread().interrupt();
}
}
private void process(Integer take) throws InterruptedException
{
//this is where all the logic goes that takes the binaryObject
//grabs all the fields from it such as binaryObject.getFileName,
//happens. It then outputs a text file with all the fields grabbed
//from the object. If the producer thread found too many errors
// I want the extension changed to .err instead of .txt
// I dont know how to do that
if(tooManyErrors == false)
{
extension = ".txt";
createFile(extension);
}
else
{
extension = ".err";
createFile(extension);
}
}
private void createFile(String extension)
{
//...
}
}
OK, a couple of things here. I assume there is only one producer and one consumer. If that is the case you can mark the classes producer and consumer as static. If you mark the class as static, only one instance of its fields will exist - it will be a singleton. You can mark any of the fields of the producer and consumer as not private, and the simply access the fields. somethingINeed = Producer.fieldThatIsNotPrivate from inside the consumer and vice versa.
Another alternative would be to keep a handle to the object you want and pass that into the constructor.
Producer p = new Producer(tooManyErrors, queue, poison);
Consumer c = new Consumer(tooManyErrors, queue, poison);
p.setConsumer(c);
c.setProducer(p);
new Thread(p).start();
new Thread(c).start();
And you could create accessors for whatever fields need to share information.
My main spawns 2 threads and they both need to access the same list. I am not sure what the best way to do this is. Here is what I have, but I still run into concurrentModificationException.
class Parent {
private List<String> data;
public List<String> getData() {
return data;
}
public static void main(String args[]) {
Parent p = new Parent();
p.start();
}
public void start() {
Thread a = new Thread(new A(this)).start();
Thread b = new Thread(new B(this)).start();
}
public A implements Runnable {
private Parent parent;
public A(Parent p) {
parent = p;
}
public void run() {
while (true) {
parent.getData().add("data");
}
}
}
public B implements Runnable {
private Parent parent;
public B(Parent p) {
parent = p;
}
public void run() {
Iterator<String> i = parent.getData().iterator();
while(i.hasNext()) {
// do more stuff with i
i.remove();
}
}
}
}
my A class is basically a producer of data and B is the consumer. I accept the possibility that I am going about this the wrong way. So all help is welcome. I just need to safely be able to add to a list from one thread and remove an item from the list from another thread. thanks in advance.
Well, for a producer/consumer, I'd recommend LinkedBlockingQueue or ConcurrentLinkedQueue. This will handle concurrent reads and writes (or pushes/polls in this case).
You'll probably want your consumer to run until some shutdown condition is sent to it. If you were using a blocking queue, this means you will want to send a queued item that indicates that the consumer should stop consuming. This would be a blocking queue implementation with a shutdown.
public enum QueueItemType {
CONSUMABLE,
SHUTDOWN
}
public class QueueItem {
public final QueueItemType type;
public final String payload;
public QueueItem(QueueItemType type, String payload) {
this.type = type;
this.payload = payload;
}
}
public class B implements Runnable {
private Parent parent;
public B(Parent p) {
parent = p;
}
public void run() {
while(true) {
QueueItem data = parent.getData().poll();
if (data.type == QueueItemType.SHUTDOWN) {
break;
} else {
// do more stuff with data.payload
}
}
}
}
Note that there is no null check for a blocking queue's poll result. This is because, by definition, blocking queues block the running thread until something is there.
If you prefer to have a consumer that does not contend with the producer, then you will need to poll periodically and sleep the consumer thread. Here's an example if you used the ConcurrentLinkedQueue:
public class B implements Runnable {
private Parent parent;
public B(Parent p) {
parent = p;
}
public void run() {
while(parent.isStillRunning()) {
String data = parent.getData().poll();
if (data != null) {
// do more stuff with data
} else {
Thread.sleep(10 /*10 ms, but you can make this whatever poll interval you want*/);
}
}
}
}
The least impactful change, could be to use a setter that is synchronized.
That way a thread would have to wait on the lock to be released before being able to add to the Collection.
I'm trying to call a method in another class, however, I'm getting an error. I initialize the class to a variable, but I do not get the option to call that method. not even the variable shows up in the auto-complete when I start typing.
this is the class I'm calling:
public class aMessageBuffer {
private Collection<Message> messageBuffer = new ArrayList<Message>();
private Collection<Integer> responseBuffer = new ArrayList<Integer>();
private boolean messageBufferFull = false;
private boolean responseBufferFull = false;
//Called by aProducerTask
public void sendMsg(String op, int val){
//place message in messageBuffer
Message msg = new Message();
msg.setValues(op,val);
messageBuffer.add(msg);
messageBufferFull = true;
//todo: notify
while (responseBufferFull == false){
//wait...
}
//remove response from responseBuffer
responseBuffer.clear();
responseBufferFull = false;
//todo: Out Response
}
}
and this is where I'm calling it from:
public class aProducerTask extends Thread {
//TODO: Send Messages
private aMessageBuffer msgbuf = new aMessageBuffer();
msgbuf.sendMsg("add",3);
//TODO: Print Results
}
I'm getting the error in the msgbuf.sendMsg("add",3);
it is not allowing me to call the method with that variable
This is my Main:
public class Main{
public static void main(String args[]){
new aConsumerTask().start();
new aProducerTask().start();
}
public void run() {
}
}
You should define run method in your class aProducerTask.
public class aProducerTask extends Thread {
//TODO: Send Messages
private aMessageBuffer msgbuf = new aMessageBuffer();
public void run() {
msgbuf.send("add",3);
}
//TODO: Print Results
}
Don't call non-assignment methods naked in the class and outside of any method, constructor or similar block.
Here you'll want to call the non-assignment method in the run method.
Don't extend Thread, implement Runnable and give it a run method which will hold your method call. Then create a new Thread when needed, pass in the Runnable and call start() on the Thread.
You will want to learn and use Java naming conventions. Variable names should all begin with a lower letter while class names with an upper case letter. Learning this and following this will allow us to better understand your code, and would allow you to better understand the code of others.
In the future if you have similar questions about a compilation error, post the full error message with your question, and please proof-read your code for correctness. 90% of our comments to questions are about clarification and correction.
So not:
public class aProducerTask extends Thread {
//TODO: Send Messages
private aMessageBuffer msgbuf = new aMessageBuffer();
msgbuf.sendMsg("add",3);
//TODO: Print Results
}
but rather this:
// improve class name and have it implement Runnable
public class AProducerTask implements Runnable {
private aMessageBuffer msgbuf = new aMessageBuffer();
#Override // don't forget this!
public void run() {
msgbuf.sendMsg("add",3);
}
}
Then to use:
public new Thread(new AProducerTask()).start();
I have this 'Worker' class, which uses a resource 'Client'.
There may be any number of threads, running the 'Worker' at any given time.
The 'Client' is not thread-safe, thus I'm using 'ThreadLocal' for it.
The 'Client' connects to some server and executes a HTTP 'Request' that the worker feeds the 'Client'.
public class Worker {
// Client is NOT thread-safe !!!
private static ThreadLocal<Client> client = new ThreadLocal<Client>();
#Override
protected void onGet(Request req) {
handleRequest(req);
}
private void handleRequest(Request req) {
someRunnableExecutor(new Runnable() {
#Override
public void run() {
get_client().send_req(req);
}
});
}
private Client get_client() {
Client c = client.get();
if (c == null) {
c = new Client();
client.set(c);
}
return c;
}
At the current implementation (above), stripped down for clarity, there are as many "active" 'Clients' as there are running 'Workers'.
This is a problem because the server is being exhausted.
What I can do is only fix the 'Worker'. Have no access to the 'Client', server or the executor that runs the workers.
What I want to do is to have a Queue of 'Client'(s) and a piece of a synchronized code, in the 'Worker', that takes a 'Client' off the Queue, if the Queue is empty the 'Worker' should wait till there is one in the Queue for him to take. Then put the 'Client' back into the Queue - synchronized as well.
I really want to keep it as simple as possible, with the possible minimum changes made to the code.
No new classes, no factories, just some data structure to hold the 'Client'(s) and synchronization.
I am a bit puzzled with how to achieve that generally, as well as by the fact that the 'Client' is not thread-safe and that I have to 'ThreadLocal'(ize) it. Is this how do I put that in a Queue?
private static Queue<ThreadLocal<CLient>> queue =
new LinkedList<ThreadLocal<CLient>>();
Also, how/where do I initialize that Queue, once, with say 5 clients?
Please share your thoughts.
You don't need ThreadLocal here, as you want to have less Clients than Workers. All you need in BlockingQueue.
Notice! I supposed that Client's send_req is synchronous, if it's not - the code needs some changes in run() method
public class Worker {
private static final int CLIENTS_NUMBER = 5;
private static final BlockingQueue<Client> queue = new LinkedBlockingQueue<>(CLIENTS_NUMBER);
static {
for (int i = 0; i < CLIENTS_NUMBER; i++)
queue.put(new Client());
}
#Override
protected void onGet(Request req) {
handleRequest(req);
}
private void handleRequest(Request req) {
someRunnableExecutor(new Runnable() {
#Override
public void run() {
try {
Client client = takeClient();
client.send_req(req);
putClient(client);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
});
}
private Client takeClient() throws InterruptedException {
return queue.take();
}
private void putClient(Client client) throws InterruptedException {
queue.put(client);
}
}
I have a thread inside a class like this-
import java.util.Observable;
public class Download extends Observable {
private int state = 0;
private final Thread myThread = new Thread(() -> {
/*
some work to do here
*/
setChanged();
notifyObservers(state);
});
public void download(int state) {
if (!myThread.isAlive()) {
this.state = state;
myThread.start();
}
}
public Thread getThread() {
return myThread;
}
public static void MyMethod() throws InterruptedException {
Download down = new Download();
down.addObserver((Observable ob, Object dat) -> {
System.out.println(ob);
if ((int) dat == 1) {
down.download(2);
} else {
System.out.println("success");
}
});
down.download(1);
down.getThread().join();
}
public static void main() throws InterruptedException {
MyMethod();
}
}
The problem is I never get it to print the "success" message.
I assume, it is because all observers are being notified from inside of MyThread. So when down.download(2) is called from the observer inside MyMethod(), the previous thread is still running and the call is ignored.
How can I notify all observers from the main thread, not from the myThread?
You are calling down.download(2) from within the execution of MyThread, therefore the thread is still alive which means that your download method does nothing because of if(!myThread.isAlive()).
I would recommend you to use the Executor framework and Listenable Futures from Guava instead of creating threads manually. Example code from the Guava wiki:
ListeningExecutorService service =
MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(10));
ListenableFuture<Explosion> explosion = service.submit(new Callable<Explosion>() {
public Explosion call() {
return pushBigRedButton();
}
});
Futures.addCallback(explosion, new FutureCallback<Explosion>() {
// we want this handler to run immediately after we push the big red button!
public void onSuccess(Explosion explosion) {
walkAwayFrom(explosion);
}
public void onFailure(Throwable thrown) {
battleArchNemesis(); // escaped the explosion!
}
});
Note that Futures.addCallback(..) also has an overload which allows you to determine which executor should execute the callback, this seems to be what you want.