I would like to create a runnable class which contain a list inside. While running, thread will listen to any list changes and process all elements inside. That's my idea.
Below is my code but it still doesn't execute process() when I add new element to list or list changed by . Please give me some help.
public class QueueService implements Runnable {
private List<Request> requestQueue;
public synchronized void add(Request request) {
if (requestQueue == null) {
requestQueue = new LinkedList<Request>();
}
requestQueue.add(request);
LOGGER.info("Added new request");
}
#Override
public void run() {
LOGGER.info("Queue started...");
while (true) {
if (requestQueue != null) {
process();
}
}
}
private synchronized void process() {
try {
Iterator<VMRequest> requests = requestQueue.iterator();
while(requests.hasNext()) {
Request req = requests.next();
// process each element and remove when done
}
} catch (Exception e) {
LOGGER.info(e.getMessage());
}
}
}
Related
I have feign service:
#FeignClient(url = "email-service")
public interface EmailFeign {
#RequestMapping(method = POST, path = "/")
void postEmail(#RequestBody Email email);
#RequestMapping(method = PUT, path = "/")
void putEmail(#RequestBody Email email);
}
Requirements:
No need to wait for the method execute (async).
When email-service is down it is required to save all mails and send them after enabling email-service.
Email-service is considered down after the first unsuccessful
attempt to send mail.
If email-service is down is required to check email-service each
DELAY seconds.
It is allowed to send messages in one thread but it is preferable to
send messages in several threads.
I resolved it with BlockingDeque and DemonThread:
public void run() {
while (true) {
try {
if (queue.isEmpty()) {
synchronized(this) {
wait();
}
}
Runnable poll = queue.poll();
if (!tryRun(poll)) {
queue.addFirst(poll);
//I use notify when queue is not empty so I can't use wait() in this case
Thread.sleep(delay);
}
} catch (Exception e) {}
}
}
I want to find smarter solution. Perhaps with one of the ServiceExecutor implementations.
Thanks.
Edit:
One more solution
public class PausableThreadPoolExecutor extends ThreadPoolExecutor {
private final long delay;
private volatile boolean isPaused = false;
private ReentrantLock pauseLock = new ReentrantLock();
private Condition unpaused = pauseLock.newCondition();
public PausableThreadPoolExecutor(long delay, int threadCount) {
super(1, threadCount, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>());
this.delay = delay;
}
#Override
protected void beforeExecute(Thread t, Runnable r) {
super.beforeExecute(t, r);
pauseLock.lock();
pauseLock.unlock();
}
#Override
protected void afterExecute(Runnable r, Throwable t) {
super.afterExecute(r, t);
//If exception has occurred
if (t != null) {
isPaused = true;
pauseLock.lock();
// Double Checked Locking
if (isPaused) {
try {
unpaused.await(delay, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
log.error("Pause error", e);
} finally {
isPaused = false;
pauseLock.unlock();
}
}
execute(r);
}
}
}
I am writing a job queue using BlockingQueue and ExecutorService. It basically waiting new data in the queue, if there are any data put into the queue, executorService will fetch data from queue. But the problem is that i am using a loop that loops to wait the queue to have data and thus the cpu usage is super high.
I am new to use this api. Not sure how to improve this.
ExecutorService mExecutorService = Executors.newSingleThreadExecutor();
BlockingQueue<T> mBlockingQueue = new ArrayBlockingQueue();
public void handleRequests() {
Future<T> future = mExecutorService.submit(new WorkerHandler(mBlockingQueue, mQueueState));
try {
value = future.get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
if (mListener != null && returnedValue != null) {
mListener.onNewItemDequeued(value);
}
}
}
private static class WorkerHandler<T> implements Callable<T> {
private final BlockingQueue<T> mBlockingQueue;
private PollingQueueState mQueueState;
PollingRequestHandler(BlockingQueue<T> blockingQueue, PollingQueueState state) {
mBlockingQueue = blockingQueue;
mQueueState = state;
}
#Override
public T call() throws Exception {
T value = null;
while (true) { // problem is here, this loop takes full cpu usage if queue is empty
if (mBlockingQueue.isEmpty()) {
mQueueState = PollingQueueState.WAITING;
} else {
mQueueState = PollingQueueState.FETCHING;
}
if (mQueueState == PollingQueueState.FETCHING) {
try {
value = mBlockingQueue.take();
break;
} catch (InterruptedException e) {
Log.e(TAG, e.getMessage(), e);
break;
}
}
}
Any suggestions on how to improve this would be much appreciated!
You don't need to test for the queue to be empty, you just take(), so the thread blocks until data is available.
When an element is put on the queue the thread awakens an value is set.
If you don't need to cancel the task you just need:
#Override
public T call() throws Exception {
T value = mBlockingQueue.take();
return value;
}
If you want to be able to cancel the task :
#Override
public T call() throws Exception {
T value = null;
while (value==null) {
try {
value = mBlockingQueue.poll(50L,TimeUnit.MILLISECONDS);
break;
} catch (InterruptedException e) {
Log.e(TAG, e.getMessage(), e);
break;
}
}
return value;
}
if (mBlockingQueue.isEmpty()) {
mQueueState = PollingQueueState.WAITING;
} else {
mQueueState = PollingQueueState.FETCHING;
}
if (mQueueState == PollingQueueState.FETCHING)
Remove these lines, the break;, and the matching closing brace.
I'm using an asyncronus XML-RPC-Client (https://github.com/gturri/aXMLRPC) in my Project and wrote some methods using the asyncronous Callback-Methods of this Client like this this:
public void xmlRpcMethod(final Object callbackSync) {
XMLRPCCallback listener = new XMLRPCCallback() {
public void onResponse(long id, final Object result) {
// Do something
if (callbackSync != null) {
synchronized (callbackSync) {
callbackSync.notify();
}
}
}
public void onError(long id, final XMLRPCException error) {
// Do something
if (callbackSync != null) {
synchronized (callbackSync) {
callbackSync.notify();
}
}
}
public void onServerError(long id, final XMLRPCServerException error) {
Log.e(TAG, error.getMessage());
if (callbackSync != null) {
synchronized (callbackSync) {
callbackSync.notifyAll();
}
}
}
};
XMLRPCClient client = new XMLRPCClient("<url>");
long id = client.callAsync(listener, "<method>");
}
In other methods I like to call this method (here "xmlRpcMethod") and wait until it finished. I wrote methods like this:
public void testMethod(){
Object sync = new Object();
xmlRpcMethod(sync);
synchronized (sync){
try{
sync.wait();
}catch(Interrupted Exception e){
e.printStackTrace();
}
}
// Do something after xmlRcpFinished
}
But this way of waiting and synchronizing get's ugly when the projects grows larger and I need to wait for many requests to finish.
So is this the only possible / best way? Or does someone knows a better solution?
My first shot to create blocking RPC calls would be:
// Little helper class:
class RPCResult<T>{
private final T result;
private final Exception ex;
private final long id;
public RPCResult( long id, T result, Exception ex ){
// TODO set fields
}
// TODO getters
public boolean hasError(){ return null != this.ex; }
}
public Object xmlRpcMethod() {
final BlockingQueue<RPCResult> pipe = new ArrayBlockingQueue<RPCResult>(1);
XMLRPCCallback listener = new XMLRPCCallback() {
public void onResponse(long id, final Object result) {
// Do something
pipe.put( new RPCResult<Object>(id, result, null) );
}
public void onError(long id, final XMLRPCException error) {
// Do something
pipe.put( new RPCResult<Object>(id, null, error) );
}
public void onServerError(long id, final XMLRPCServerException error) {
Log.e(TAG, error.getMessage());
pipe.put(new RPCResult<Object>(id, null, error));
}
};
XMLRPCClient client = new XMLRPCClient("<url>");
long id = client.callAsync(listener, "<method>");
RPCResult result = pipe.take(); // blocks until there is an element available
// TODO: catch and handle InterruptedException!
if( result.hasError() ) throw result.getError(); // Relay Exceptions - do not swallow them!
return result.getResult();
}
Client:
public void testMethod(){
Object result = xmlRpcMethod(); // blocks until result is available or throws exception
}
Next step would be to make a strongly typed version public T xmlRpcMethod().
Problem statement
I have a JMS listener running as a thread listening to a topic. As soon a message comes in, I spawn a new Thread to process the in-bounded message. So for each incoming message I spawn a new Thread.
I have a scenario where duplicate message is also being processed when it is injected immediately in a sequential order. I need to prevent this from being processed. I tried using a ConcurrentHashMap to hold the process times where I add in the entry as soon as Thread is spawn and remove it from the map as soon Thread completes its execution. But it did not help when I tried with the scenario where I passed in same one after the another in concurrent fashion.
General Outline of my issue before you plunge into the actual code base
onMessage(){
processIncomingMessage(){
ExecutorService executorService = Executors.newFixedThreadPool(1000);
//Map is used to make an entry before i spawn a new thread to process incoming message
//Map contains "Key as the incoming message" and "value as boolean"
//check map for duplicate check
//The below check is failing and allowing duplicate messages to be processed in parallel
if(entryisPresentInMap){
//return doing nothing
}else{
//spawn a new thread for each incoming message
//also ensure a duplicate message being processed when it in process by an active thread
executorService.execute(new Runnable() {
#Override
public void run() {
try {
//actuall business logic
}finally{
//remove entry from the map so after processing is done with the message
}
}
}
}
Standalone example to mimic the scenario
public class DuplicateCheck {
private static Map<String,Boolean> duplicateCheckMap =
new ConcurrentHashMap<String,Boolean>(1000);
private static String name=null;
private static String[] nameArray = new String[20];
public static void processMessage(String message){
System.out.println("Processed message =" +message);
}
public static void main(String args[]){
nameArray[0] = "Peter";
nameArray[1] = "Peter";
nameArray[2] = "Adam";
for(int i=0;i<=nameArray.length;i++){
name=nameArray[i];
if(duplicateCheckMap.get(name)!=null && duplicateCheckMap.get(name)){
System.out.println("Thread detected for processing your name ="+name);
return;
}
addNameIntoMap(name);
new Thread(new Runnable() {
#Override
public void run() {
try {
processMessage(name);
} catch (Exception e) {
System.out.println(e.getMessage());
} finally {
freeNameFromMap(name);
}
}
}).start();
}
}
private static synchronized void addNameIntoMap(String name) {
if (name != null) {
duplicateCheckMap.put(name, true);
System.out.println("Thread processing the "+name+" is added to the status map");
}
}
private static synchronized void freeNameFromMap(String name) {
if (name != null) {
duplicateCheckMap.remove(name);
System.out.println("Thread processing the "+name+" is released from the status map");
}
}
Snippet of the code is below
public void processControlMessage(final Message message) {
RDPWorkflowControlMessage rdpWorkflowControlMessage= unmarshallControlMessage(message);
final String workflowName = rdpWorkflowControlMessage.getWorkflowName();
final String controlMessageEvent=rdpWorkflowControlMessage.getControlMessage().value();
if(controlMessageStateMap.get(workflowName)!=null && controlMessageStateMap.get(workflowName)){
log.info("Cache cleanup for the workflow :"+workflowName+" is already in progress");
return;
}else {
log.info("doing nothing");
}
Semaphore controlMessageLock = new Semaphore(1);
try{
controlMessageLock.acquire();
synchronized(this){
new Thread(new Runnable(){
#Override
public void run() {
try {
lock.lock();
log.info("Processing Workflow Control Message for the workflow :"+workflowName);
if (message instanceof TextMessage) {
if ("REFRESH".equalsIgnoreCase(controlMessageEvent)) {
clearControlMessageBuffer();
enableControlMessageStatus(workflowName);
List<String> matchingValues=new ArrayList<String>();
matchingValues.add(workflowName);
ConcreteSetDAO tasksSetDAO=taskEventListener.getConcreteSetDAO();
ConcreteSetDAO workflowSetDAO=workflowEventListener.getConcreteSetDAO();
tasksSetDAO.deleteMatchingRecords(matchingValues);
workflowSetDAO.deleteMatchingRecords(matchingValues);
fetchNewWorkflowItems();
addShutdownHook(workflowName);
}
}
} catch (Exception e) {
log.error("Error extracting item of type RDPWorkflowControlMessage from message "
+ message);
} finally {
disableControlMessageStatus(workflowName);
lock.unlock();
}
}
}).start();
}
} catch (InterruptedException ie) {
log.info("Interrupted Exception during control message lock acquisition"+ie);
}finally{
controlMessageLock.release();
}
}
private void addShutdownHook(final String workflowName) {
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
disableControlMessageStatus(workflowName);
}
});
log.info("Shut Down Hook Attached for the thread processing the workflow :"+workflowName);
}
private RDPWorkflowControlMessage unmarshallControlMessage(Message message) {
RDPWorkflowControlMessage rdpWorkflowControlMessage = null;
try {
TextMessage textMessage = (TextMessage) message;
rdpWorkflowControlMessage = marshaller.unmarshalItem(textMessage.getText(), RDPWorkflowControlMessage.class);
} catch (Exception e) {
log.error("Error extracting item of type RDPWorkflowTask from message "
+ message);
}
return rdpWorkflowControlMessage;
}
private void fetchNewWorkflowItems() {
initSSL();
List<RDPWorkflowTask> allTasks=initAllTasks();
taskEventListener.addRDPWorkflowTasks(allTasks);
workflowEventListener.updateWorkflowStatus(allTasks);
}
private void clearControlMessageBuffer() {
taskEventListener.getRecordsForUpdate().clear();
workflowEventListener.getRecordsForUpdate().clear();
}
private synchronized void enableControlMessageStatus(String workflowName) {
if (workflowName != null) {
controlMessageStateMap.put(workflowName, true);
log.info("Thread processing the "+workflowName+" is added to the status map");
}
}
private synchronized void disableControlMessageStatus(String workflowName) {
if (workflowName != null) {
controlMessageStateMap.remove(workflowName);
log.info("Thread processing the "+workflowName+" is released from the status map");
}
}
I have modified my code to incorporate suggestions provided below but still it is not working
public void processControlMessage(final Message message) {
ExecutorService executorService = Executors.newFixedThreadPool(1000);
try{
lock.lock();
RDPWorkflowControlMessage rdpWorkflowControlMessage= unmarshallControlMessage(message);
final String workflowName = rdpWorkflowControlMessage.getWorkflowName();
final String controlMessageEvent=rdpWorkflowControlMessage.getControlMessage().value();
if(controlMessageStateMap.get(workflowName)!=null && controlMessageStateMap.get(workflowName)){
log.info("Cache cleanup for the workflow :"+workflowName+" is already in progress");
return;
}else {
log.info("doing nothing");
}
enableControlMessageStatus(workflowName);
executorService.execute(new Runnable() {
#Override
public void run() {
try {
//actual code
fetchNewWorkflowItems();
addShutdownHook(workflowName);
}
}
} catch (Exception e) {
log.error("Error extracting item of type RDPWorkflowControlMessage from message "
+ message);
} finally {
disableControlMessageStatus(workflowName);
}
}
});
} finally {
executorService.shutdown();
lock.unlock();
}
}
private void addShutdownHook(final String workflowName) {
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
disableControlMessageStatus(workflowName);
}
});
log.info("Shut Down Hook Attached for the thread processing the workflow :"+workflowName);
}
private synchronized void enableControlMessageStatus(String workflowName) {
if (workflowName != null) {
controlMessageStateMap.put(workflowName, true);
log.info("Thread processing the "+workflowName+" is added to the status map");
}
}
private synchronized void disableControlMessageStatus(String workflowName) {
if (workflowName != null) {
controlMessageStateMap.remove(workflowName);
log.info("Thread processing the "+workflowName+" is released from the status map");
}
}
This is how you should add a value to a map. This double checking makes sure that only one thread adds a value to a map at any particular moment of time and you can control the access afterwards. Remove all the locking logic afterwards. It is as simple as that
public void processControlMessage(final String workflowName) {
if(!tryAddingMessageInProcessingMap(workflowName)){
Thread.sleep(1000); // sleep 1 sec and try again
processControlMessage(workflowName);
return ;
}
System.out.println(workflowName);
try{
// your code goes here
} finally{
controlMessageStateMap.remove(workflowName);
}
}
private boolean tryAddingMessageInProcessingMap(final String workflowName) {
if(controlMessageStateMap .get(workflowName)==null){
synchronized (this) {
if(controlMessageStateMap .get(workflowName)==null){
controlMessageStateMap.put(workflowName, true);
return true;
}
}
}
return false;
}
Read here more for https://en.wikipedia.org/wiki/Double-checked_locking
The issue is fixed now. Many thanks to #awsome for the approach. It is avoiding the duplicates when a thread is already processing the incoming duplicate message. If no thread is processing then it gets picked up
public void processControlMessage(final Message message) {
try {
lock.lock();
RDPWorkflowControlMessage rdpWorkflowControlMessage = unmarshallControlMessage(message);
final String workflowName = rdpWorkflowControlMessage.getWorkflowName();
final String controlMessageEvent = rdpWorkflowControlMessage.getControlMessage().value();
new Thread(new Runnable() {
#Override
public void run() {
try {
if (message instanceof TextMessage) {
if ("REFRESH".equalsIgnoreCase(controlMessageEvent)) {
if (tryAddingWorkflowNameInStatusMap(workflowName)) {
log.info("Processing Workflow Control Message for the workflow :"+ workflowName);
addShutdownHook(workflowName);
clearControlMessageBuffer();
List<String> matchingValues = new ArrayList<String>();
matchingValues.add(workflowName);
ConcreteSetDAO tasksSetDAO = taskEventListener.getConcreteSetDAO();
ConcreteSetDAO workflowSetDAO = workflowEventListener.getConcreteSetDAO();
tasksSetDAO.deleteMatchingRecords(matchingValues);
workflowSetDAO.deleteMatchingRecords(matchingValues);
List<RDPWorkflowTask> allTasks=fetchNewWorkflowItems(workflowName);
updateTasksAndWorkflowSet(allTasks);
removeWorkflowNameFromProcessingMap(workflowName);
} else {
log.info("Cache clean up is already in progress for the workflow ="+ workflowName);
return;
}
}
}
} catch (Exception e) {
log.error("Error extracting item of type RDPWorkflowControlMessage from message "
+ message);
}
}
}).start();
} finally {
lock.unlock();
}
}
private boolean tryAddingWorkflowNameInStatusMap(final String workflowName) {
if(controlMessageStateMap.get(workflowName)==null){
synchronized (this) {
if(controlMessageStateMap.get(workflowName)==null){
log.info("Adding an entry in to the map for the workflow ="+workflowName);
controlMessageStateMap.put(workflowName, true);
return true;
}
}
}
return false;
}
private synchronized void removeWorkflowNameFromProcessingMap(String workflowName) {
if (workflowName != null
&& (controlMessageStateMap.get(workflowName) != null && controlMessageStateMap
.get(workflowName))) {
controlMessageStateMap.remove(workflowName);
log.info("Thread processing the " + workflowName+ " is released from the status map");
}
}
I work on a wizard for creation of a java project and get a invalid thread access exception if I run it in the empty workspace for the first time. I try to implement my wizard similar to JavaProjectWizard, but I don't need the second page, so I try to perform finish from the first page and to initialize the second page in advance:
import org.eclipse.jdt.ui.wizards.NewJavaProjectWizardPageTwo;
import org.eclipse.jface.wizard.Wizard;
import org.eclipse.ui.INewWizard;
public class SomeNewWizard
extends Wizard
implements INewWizard {
private SomeWizardPageTwo javaWizardPageTwo;
#Override
public void addPages() {
if (javaWizardPageTwo == null)
someWizardPageTwo = new SomeWizardPageTwo(newSeeAppWizardPageOne);
}
#Override
public boolean performFinish() {
/*line 109*/someWizardPageTwo .createProvisonalProject();
final IWorkspaceRunnable op = new IWorkspaceRunnable() {
public void run(IProgressMonitor monitor)
throws CoreException, OperationCanceledException {
try {
someWizardPageTwo.performFinish(new SubProgressMonitor(monitor, 1));
}
catch (InterruptedException e) {
throw new OperationCanceledException(e.getMessage());
}
finally {
monitor.done();
}
}
};
try {
rule = null;
Job job = Job.getJobManager().currentJob();
if (job != null)
rule = job.getRule();
IRunnableWithProgress runnable = new IRunnableWithProgress() {
#Override
public void run(IProgressMonitor monitor)
throws InvocationTargetException, InterruptedException {
try {
JavaCore.run(op, rule, monitor);
}
catch (OperationCanceledException e) {
throw new InterruptedException(e.getMessage());
}
catch (CoreException e) {
throw new InvocationTargetException(e);
}
}
};
getContainer().run(true, true, runnable);
}
catch (InvocationTargetException e) {
handleFinishException(getShell(), e);
return false;
}
catch (InterruptedException e) {
return false;
}
return true;
}
public class SomeWizardPageTwo
extends NewJavaProjectWizardPageTwo
{
private NewJavaProjectWizardPageOne mainPage;
public SomeWizardPageTwo(NewJavaProjectWizardPageOne mainPage)
{
super(mainPage);
this.mainPage = mainPage;
}
#Override
public IProject createProvisonalProject()
{
return super.createProvisonalProject();
}
#Override
protected IWizardContainer getContainer()
{
if (mainPage == null)
return null;
return mainPage.getWizard().getContainer();
}
}
The stacktrace can be found here.
The root of the issue seems that ImageDescriptorRegistry is created from the wrong thread because the corresponding IRunnableWithProgress runs forked. But I wonder why does it work for the normal JavaProjectWizard then? And the main question is: how to make it work for my wizard?
getContainer().run(true, true, runnable); causes the runnable to be executed in a separate thread. The first parameter fork is responsible therefore.
The call to javaWizardPageTwo.performFinish() atempts to access the UI thread and causes the invalid thread access exception.
If you set the fork parameter to false, the code will be executed on the current thread.
Why don't you call javaWizardPageTwo.performFinish() directly?
There is not a bug in Display.checkDevice.
Your call to NewJavaProjectWizardPageTwo.performFinish is running in a background thread, but the code is using ImageDescriptorRegistry which needs to be initialized on the User Interface thread.
This is intermittent because sometimes something else that you do will have already initialized the registry.