We Keep Coding

Using multithreading get response from different urls

I am making an ajax call to a Java servlet which gets response from different urls and manipulate the data before sending back. Now as there are many urls, to speed up the process rather than sequential processing I have used CountDownLatch and made a class for each url response(which I will use for multithreading). I am not able to understand how to send data back from each response class to the main servlet so that I can manipulate and send back the data to ajax call.
Here's the doget() of my servlet which is called through ajax call.
private static CountDownLatch _latch;
private static int N = 2;
_latch = new CountDownLatch(N);
try {
new Thread(new FirstUrl(_latch)).start();
new Thread(new SecondUrl(_latch)).start();
_latch.await();
}
catch (InterruptedException e) {
e.printStackTrace();
}
Here are the url response classes.
public class SecondUrl implements Runnable {
private CountDownLatch _latch = null;
public SecondUrl(CountDownLatch _latch) {
this._latch = _latch;
}
public void run() {
URI uri;
try {
uri = new URI("http://url1.com");
JSONTokener tokener = new JSONTokener(uri.toURL().openStream());
JSONObject root = new JSONObject(tokener);
_latch.countDown();
} catch (URISyntaxException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
Similarly, there is another class with a different url. How to send the JSONObject result back to servlet, so that I can manipulate it.
I am new to multithreading so I might be wrong about any approach. Feel free to point out. Thanks.

Make use of the Callable instead of Runnable. Look at Future interface.

One (of the many) way (to pass result back from your threads) is to pass to your threads a reference to a ArrayBlockingQueue that is initialized in your main thread. The result can be added to the queue. In the main thread, you can either process as the data becomes available or process at the end (after you finish you _latch.await()).

try using a synchronized method to 'join' result in each individual thread, Using Runnable is fine.
And you don't need a second class for SecondUrl. you can use a single class and you can pass the url as a parameter to the class.

Why does my Client freeze when awaiting a response from the Server? [duplicate]

I have a small bit of code that runs in an applet that contains SWING controls and is used to write information to a socket on a certain port and then listens for a response. This works fine, but there is a problem with it. The port listener is essentially in a loop until null is received by the server. I want users to be able to perform other actions in the GUI instantiated by the applet while waiting for the server to respond (this could take minutes to occur). I also need to worry about the connection between the server and the client disconnecting. But the way the code is written, the applet appears to freeze (its really in a loop) until the server responds. How can I allow the listener to do its listening in the background, allowing other things to occur in the program. I assume I need to use threads and I'm sure for this application, it is easy to implement, but my lack of a solid thread foundation is hampering me. Below is the code (you can see how simple it is). How can I improve it to make it do what I need it to do>
public String writePacket(String packet) {
/* This method writes the packet to the port - established earlier */
System.out.println("writing out this packet->"+packet+"<-");
out.println(packet);
String thePacket = readPacket(); //where the port listener is invoked.
return thePacket;
}
private String readPacket() {
String thePacket ="";
String fromServer="";
//Below is the loop that freezes everything.
try {
while ((fromServer = in.readLine()) != null) {
if (thePacket.equals("")) thePacket = fromServer;
else
thePacket = thePacket+newLine+fromServer;
}
return thePacket; //when this happens, all listening should stop.
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return null;
}
}
Thanks,
Elliott

There lots of different means of getting the IO performed on a different thread, but in this case you probably want to use SwingWorker.
Your code would look something like:
private final Executor executor = Executors.newSingleThreadExecutor();
public void writePacket(final String packet)
{
// schedules execution on the single thread of the executor (so only one background operation can happen at once)
//
executor.execute(new SwingWorker<String, Void>()
{
#Override
protected String doInBackground() throws Exception
{
// called on a background thread
/* This method writes the packet to the port - established earlier */
System.out.println("writing out this packet->"+packet+"<-");
System.out.println(packet);
String thePacket = readPacket(); //where the port listener is invoked.
return thePacket;
}
#Override
protected void done()
{
// called on the Swing event dispatch thread
try
{
final String thePacket = get();
// update GUI with 'thePacket'
}
catch (final InterruptedException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
catch (final ExecutionException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
}
private String readPacket()
{
String thePacket ="";
String fromServer="";
//Below is the loop that freezes everything.
try
{
while ((fromServer = in.readLine()) != null)
{
if (thePacket.equals(""))
thePacket = fromServer;
else
thePacket = thePacket+newLine+fromServer;
}
return thePacket; //when this happens, all listening should stop.
}
catch (IOException e)
{
e.printStackTrace();
return null;
}
}

All the network I/O should be in a separate thread.
BTW readLine() returns null when the server closes the connection, not when it has finished sending data for the moment.

Why does Java send needs Thread.Sleep

I am implementing a Server/Client system using Java. the server listens for incoming connection from clients and after a client connects, the server creates a new socket and passes it to a new thread which will only be used for receiving data:
while (true){
clientSocket=serverSocket.accept();
new ClientReceiver(clientSocket,this.clientsManager).start();
}
the clientReceiver class is as follows:
public class ClientReceiver extends Thread {
private Socket clientSocket=null;
private Client client=null;
private ClientsManager clientsManager;
private ClientActionParser clientActionParser=new ClientActionParser();
ClientHandlerState clientHandlerState;
PrintWriter outputStream=null;
BufferedReader inputStream=null;
public ClientReceiver(Socket clientSocket, ClientsManager clientsManager){
this.clientSocket=clientSocket;
this.clientsManager=clientsManager;
this.setClientHandlerState(ClientHandlerState.Connected);
}
public void run(){
String actionString;
try{
//define output and input stream to client
outputStream =new PrintWriter(clientSocket.getOutputStream(),true);
inputStream = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
//while client is connected read input to actionString
while((actionString=inputStream.readLine()) != null){
AbstractClientAction clientAction= this.clientActionParser.parse(actionString);
if(this.getClientHandlerState()==ClientHandlerState.Connected){
if(clientAction instanceof ClientLoginAction){
ClientLoginAction clientLoginAction=(ClientLoginAction) clientAction;
if(this.authenticate(clientLoginAction)){
}
else{
throw new AuthenticationException();
}
}
else{
throw new AuthenticationException();
}
}
}
if(this.getClientHandlerState()==ClientHandlerState.Authorized){
//receive other client actions: transfer barge ....
}
try {
Thread.sleep(400);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
catch(IOException e){
}
catch (AuthenticationException e) {
// TODO: handle exception
}
//clean up the resources
try{
outputStream.close();
inputStream.close();
clientSocket.close();
}
catch(Exception e){
}
}
private boolean authenticate(ClientLoginAction clientLoginAction){
//perform authentication. If authentication successful:
this.client=this.clientsManager.authenticateClient(clientLoginAction.getUsername(), clientLoginAction.getPassword());
if(this.client==null){
return false;
}
else{
ClientSender clientSender=new ClientSender(this.outputStream, this.client);
this.clientsManager.addClientSender(clientSender);
this.setClientHandlerState(ClientHandlerState.Authorized);
clientSender.start();
return true;
}
}
public ClientHandlerState getClientHandlerState(){
return this.clientHandlerState;
}
public void setClientHandlerState(ClientHandlerState clientHandlerState){
this.clientHandlerState=clientHandlerState;
}
after successful authentication in the receiver thread, a new thread is created for sending data to client and socket's outputStream is passed to the new thread.
the clientSender class contains a queue as a buffer which contains the data that should be sent to the client. here is class clientSender:
public class ClientSender extends Thread {
private Client client=null;
private final Log logger = LogFactory.getLog(getClass());
PrintWriter outputStream=null;
private Queue<String> clientEventsQueue= new LinkedList<String>();
public ClientSender(PrintWriter outputStream, Client client){
this.outputStream=outputStream;
this.client=client;
}
public void run(){
//System.out.println("ClientSender run method called.");
while(true){
try {
Thread.sleep(10);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
if(!this.clientEventsQueue.isEmpty()){
this.outputStream.println(this.clientEventsQueue.remove());
}
}
}
public Client getClient(){
return this.client;
}
public void insertClientEvent(String clientEvent){
this.clientEventsQueue.add(clientEvent);
}
whenever I want to send something to the client I use:
clientSender.insertClientEvent("some text");
the problem is that if I remove Thread.sleep(10) I will not receive anything in the client side. Since TCP sockets are blocking I think this should not happen. Is this normal or am I doing something wrong?
EDIT:
there is no "termination" for the sender thread. the server should send proper information to all clients whenever it receives an event from another system. so I think the best scenario is to stop the thread when there is no data to send and start it whenever there is. so I tried this in the clientSender class:
public void run(){
while(true){
if(this.clientEventsQueue.isEmpty()){
break;
}
else{
try {
this.outputStream.println(this.clientEventsQueue.take());
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
but now the problem is when to start the thread? I tried starting it whenever I want to send data but as I expected it does not work properly and only sends the fist package:
clientSender.insertClientEvent(clientEvent.getEventString());
clientSender.start();
EDIT2
I came up with this idea. It is very simple and I think it consumes so much less CPU time.
while(true){
while(this.clientEventsQueue.isEmpty()){
try {
Thread.sleep(300);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
try {
this.outputStream.println(this.clientEventsQueue.take());
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
as much as I tested it, it worked just fine. what do you think about it?

I see that you are using a LinkedList as a queue accessed by multiple threads and that you are doing a busy wait on it in the ClientSender.
This implementation is not thread safe and may cause problems like clientEvents posted on it not being made visible to the ClientSender thread, CPU being wasted etc.
You could use a BlockingQueue instead and call take() on it to block on the queue until an event is posted.
I also see that you are relying on sleep(400) to wait for the communication. That will also cause issues. The thread that is using the socket resource can close it when it's done, instead of this.
EDIT:
There are number of techniques to deal with terminating the thread. I think a "poison pill" will work well in this case. Basically you do:
String stuff = queue.take();
if (stuff == null) break;
and post null on the queue when you want to terminate it (doesn't have to be null, can be anything e.g. "terminate" etc.
EDIT2:
Your way of terminating won't work, as it will terminate immediately before anyone can post an event on it. Theoretically you could be spawning and then immediately killing the thread over and over etc. The easiest way to cope with this is to use a special message (aka "poison pill") for the termination condition.
As for having a thread only when there is an event, at that point I'd recommend using a thread pool. You could encapsulate the event sending into a Runnable, and hold the sockets in a Map. This is however quite complicated to implement and requires good understanding of multithreading to get it right. Multithreading is hard and can introduce severe headache when done wrong. Tbh I wouldn't recommend to try doing this without studying more about multithreaded programming.
EDIT3:
#user2355734: Polling the queue with an interval as you did is done by many people, but is discouraged. The take() method will in effect "sleep" and only wake up if there is something on the queue, so in theory by removing the "sleep" loop you should get even lower CPU usage and shorter latency. In general, you should try to avoid using "sleep" altogether in multithreaded code. It's rare that you genuinely need it and it's frequently a sign of broken/suboptimal code. As for tests, although they are useful it's hard or even pretty much impossible to guarantee correctness of multithreaded code through tests. Your code may run fine in your tests yet fail in production, under high load, under different environment etc. Therefore it's important to review the code and make sure it's theoretically correct.

java: read and write on a file at the same time ,concurrency on file [duplicate]

I have read many a posts where-in they speak about reading and writing into the file NOT simultaneously using JavaME. I have a special use case scenarios where-in my log file (maybe full file or just portion of the file) is uploaded to the server on regular basis. This must continue without hampering the current logging of the application in this same file.
The code sample is a under:
boolean writing = true;
boolean reading = true;
void main() {
new Thread("THREAD-FILE-READ") {
public void run() {
InputStream instream = getFileInStream();
if (null != instream) {
while (reading) {
try {
try {
synchronized(READ_LOCK) {
READ_LOCK.wait();
}
} catch (InterruptedException ex) {
ex.printStackTrace();
}
if (writtenCharsLen > 0) {
byte[] bytes = new byte[writtenCharsLen];
instream.read(bytes, 0, writtenCharsLen);
System.out.println("Read="+new String(bytes));
bytes = null;
writtenCharsLen = 0;
}
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
}
closeStream(instream);
}
}.start();
new Thread("THREAD-FILE-WRITE") {
public void run() {
OutputStream outstream = getFileOutStream();
if (null != outstream) {
while (writing) {
try {
byte[] str = randomString();
if (null != str) {
writtenCharsLen = str.length;
System.out.println("Write=" + new String(str));
outstream.write(str);
str = null;
}
} catch (IOException ex) {
ex.printStackTrace();
} finally {
notifyReadStream();
}
try {
synchronized(WRITE_LOCK) {
WRITE_LOCK.wait();
}
} catch (InterruptedException ex) {
ex.printStackTrace();
}
}
}
closeStream(outstream );
}
}.start();
}
void notifyReadStream() {
try {
synchronized (READ_LOCK) {
READ_LOCK.notify();
}
} catch (Exception e) {
e.printStackTrace();
}
}
void notifyWriteStream() {
try {
synchronized (WRITE_LOCK) {
WRITE_LOCK.notify();
}
} catch (Exception e) {
e.printStackTrace();
}
}
In the above code I will replace sop-read and sop-write with proper calls to network IO methods.
PS: Since this piece of code will run of multiple files and multitude of devices i need the modification as compressed as possible to keep my runtime heap as low as possible. Also this piece of code will run till the application life cycle hence closing and opening the file in middle is out of consideration.
Present Undesired Result:
The read and write threads are showing running sop's for read and write. The read thread is reading from the position the writing thread has written. I am not facing any exception in this code but the result is undesired. I have also tried synchronizing read and write streams but that is throwing IllegalMonitorStateException
Expected Result:
Reading of the stream must be triggered after writing into the stream is completed, also the read thread must be able to read from any position in the file.
Any help / pointers is useful?
EDIT: I was able to synchronize the read and the write streams using different monitors but i still feel, i could have done better using single monitor. Will try it sometime later.

I will attack this problem:
Present Undesired Result: The read and write threads are showing running sop's for read and write. The read thread is reading from the position the writing thread has written. I am not facing any exception in this code but the result is undesired. I have also tried synchronizing read and write streams but that is throwing IllegalMonitorStateException.
If you have synchronized the access using monitors i.e. the reader calls someObject.wait() and the writer calls someObject.notify(), remember that you have to wrap these calls in a synchronized block on someObject:
synchronized(someObject) {
someObject.wait();
}
synchronized(someObject) {
someObject.notify();
}
This is the cause for IllegalMonitorStateException.

Your first problem is that you are setting writtenCharsLen before you write the data. If your read thread sees it being non-zero before the write thread actually writes it, you have a problem. Move writtenCharsLen = str.length after the write.
Another problem I see in your sample is that the threads never yield control. They will be CPU hogs.

How to interrupt a task in progress with a timeout mechanism?

I'm currently developping a ftp client based on the package commons.net, in order to run tests to see the speed of the connection.
Basically my ftp test consists in connect to the server, logging onto it, and then start a cycle of download/upload as long as necessary, until the user decides to stop it via a button, then the current cycle will end and so will the test.
However, while running those tests, a situation requiering a timout mechanism has occured. the server was transmitting the file, and send the return code 226 (transfer complete) before it was indeed completed.
So my thread remains stuck, trying to empty the inputStream when it is not possible anymore.
My idea was to start a threaded timer with the downloading process, that will be reset each time a byte is transferred to my client.
When the timeout occurs, then an exception or so would be raised, and my client would react to it, abording the download.
I have read and try many solutions, among them:
- raising an exception from a thread -> the thread catches the exception and not the client;
- interrupt the client from the thread, so the client raises itself an interruptedException -> doesn't seem to work;
- using an executor with a timeout -> since I can't know the "normal" duration of a download, I can't give it to the executor when I start the task, moreover, the timer has to be reset when I receive data.
I read a lot about it on many forums, and didn't find any solution that seem to be adapted AND work in this case. If anyone has an idea of another way to do it?
This is the code of the action I am performing:
public double get(String fileName) {
[...]
org.apache.commons.net.io.Util.copyStream(stO,stD,client.getBufferSize(),
this.localSize,
new org.apache.commons.net.io.CopyStreamAdapter() {
public void bytesTransferred(long totalBytesTransferred,
int bytesTransferred,long streamSize) {
setProgressDL(totalBytesTransferred);
//reset the timer here
}
});
[...]
}
Here is some of the code of my test, launching my client:
public class TestFtp extends Thread {
[...]
public void run() {
System.out.println("Launching FTP test");
FtpClient client = new FtpClient(this.model, this, this.model.getFtpServer());
try {
//Attempting connection on the server
client.connect();
try {
// Attempting login
client.login(this.model.getUsername(), this.model.getPassword());
do {
client.changeDirectory("get");
// start timer
client.get(this.model.getDistantFileName());
// stop timer
client.changeToParentDirectory();
client.changeDirectory("put");
client.set(this.model.getDistantFileName(),
this.model.getNewFileName());
client.changeToParentDirectory();
try {
// Little pause between each test
Thread.sleep(500);
} catch (InterruptedException e) {}
// Continue test until the user stops it
} while (this.continuous);
// Once the test is over, logout
client.logout();
} catch (FTPLoginException e) {
// If login fails, the test ends
System.out.println("Unable to login to the server.");
}
} catch (FTPConnectException e) {
// If connection error, the test ends
System.out.println("Unable to connect to the server.");
}
}
Thank you by advance if anyone can help, and if you need further information on my actual code, I can put more of it in here.

If you do not want to throw unecessary Exceptions, you should use a boolean flag that controls the execution of the thread (or runnable):
public class TestFtp extends Thread {
[...]
boolean abort;
public void run() {
[...]
do{
[...]
} while (this.continuous && !abort);
if (abort){
// You might want to do something here
}else{
// The stuff you normally do
}
}
}
And then simply set the abort flag to false from outside.
This way you can better control how you thread will terminate, as thread.interrupt(); will have an undefined behavior.

Well, I'm sorry but I admit I haven't read all your code, but if you want to interrupt a running thread, do two things:
run the thread code inside a try/catch block like this:
Example:
public void run() {
try {
// code to run
} catch (InterruptedException ie) {
// thread interrupted, may want to do some clean up
// but must return as quickly as possible to avoid halting external code
}
}
Call the interrupt() method of the thread above externally when the need arises.
Example:
thread.interrupt();
This will tell the VM to throw the InterruptedException in your thread no matter what it's doing, giving you a chance to do some stuff.
I hope this is what you're looking for...
EDIT
Ok, a concrete example that works:
public class Driver {
private static int count = 0;
public static void main(String[] args) {
Thread t = new Thread(new Runnable() {
#Override
public void run() {
try {
bigTask();
} catch (InterruptedException ie) {
System.out.println("Interrupted thread! Count is " + count);
}
}
});
t.start();
try {
Thread.sleep(1000);
System.out.println("Trying to interrupt thread");
t.interrupt();
} catch (InterruptedException e) {}
}
private static void bigTask() throws InterruptedException {
List<BigDecimal> bigs = new ArrayList<BigDecimal>();
for (int i = 0; i < 10000000; i++) {
bigs.add(BigDecimal.valueOf(i));
if (Thread.interrupted()) {
throw new InterruptedException();
}
count = i;
}
System.out.println("Ok, added ten million items, count is " + count);
}
}