I'm reading messages from a socket (trough a TCP protocol), but I note that the CPU spend a lot of time to call the method available() of my BufferedInputStream. This is my code:
#Override
public void run()
{
Socket socket;
Scanner scanner;
BufferedInputStream buffer = null;
try
{
socket = new Socket(SERVER_HOST, SERVER_PORT);
System.out.println("Connection Completed");
InputStream inputStream = socket.getInputStream();
buffer = new BufferedInputStream(inputStream);
StringBuilder readCharacter;
while (true)
{
readCharacter = new StringBuilder();
try
{
while (buffer.available() > 0)
{
readCharacter.append((char) buffer.read());
}
}
catch (IOException e)
{
e.printStackTrace();
buffer.close();
}
String array[] = separe(new String(readCharacter));
... //parsing the message
I've also tried to use int read=buffer.read() and check if (read!=-1) instead of using the available function, but in this case I'm not able to recognize the end of the message...in my StringBuilder 'readCharacter' I have more than one message, one after the other..and it cause the fail of my parsing process...
Instead using the available() check, into the readCharacter I have only one message at a time..and the parsing works...
Can you help me to understand why, and how avoid the eating of CPU?
This loop:
while (buffer.available() > 0)
{
readCharacter.append((char) buffer.read());
}
can be replaced with simple:
readCharacter.append((char) buffer.read());
Instead of calling non-blocking available() over and over again (which consumes a lot of CPU) just call read() which will block not consuming CPU until something is available. Looks like this is what you want to achieve with less code and complexity.
The available() itself does not eat CPU. What does it is your loop:
while (buffer.available() > 0) {
readCharacter.append((char) buffer.read());
}
While bytes are unavailable you are actually calling available() multiple times (probably thousands of times). Since read() method of streams is blocking you do not have to call available() at all. The following code does the same but does not eat CPU.
String line = null;
while ((line = buffer.read()) != null) {
readCharacter.append(line);
}
Related
Saying my requirement is:
Either user type something in console (from system.in) or socket receive something, proceed to next step.
So I have a scanner
Scanner sc = new Scanner(System.in);
Have an Udp client. (a different input source)
DatagramSocket clientSocket = new DatagramSocket();
My code is
while (true) {
if (sc.hasNext()) {
String str = sc.next();
proceed(str)
} else {
clientSocket.receive(pack);
proceed(pack)
}
}
Obviously this code will not work. Because when checking sc.hasNext(), java is waiting user to type some input in console. Currently what I can do is open an thread for Udp client. If I change the order,
while (true) {
clientSocket.receive(pack);
if (not receive) read from system.in
}
It doesn't make sense, since receive() will keep waiting util receive something, it will never read from system.in concurrently.
So how can i achieve my requirement without using a thread?
Inspired by #Andriy Kryvtsun's answer, i did a quick test
As he said, this is somehow using non-blocking read, and keep letting socket timeout, to emulate
InputStream ins = System.in;
byte buffer[] = new byte[512];
BufferedReader reader = new BufferedReader(new InputStreamReader(ins));
DatagramSocket clientSocket = new DatagramSocket();
System.out.println("From buffer:" + clientSocket.getLocalPort());
while (true) {
try {
if (ins.available() > 0) {
String line = reader.readLine();
System.out.println("Read:" + line);
} else {
DatagramPacket pack = new DatagramPacket(buffer,buffer.length);
clientSocket.setSoTimeout(2000);
clientSocket.receive(pack);
System.out.println("Receive: " + new String(pack.getData()));
}
} catch (IOException e1) {
}
}
Use unblocking method call InputStream#available() to get info if something ready for reading before using blocking Scanner#hasNext() call.
Also you can call DatagramSocket#setSoTimeout(int timeout) before call blocking receive(DatagramPacket p) method. Thus you can break infinity waiting inside receive method after timeout period emulating unblocking reading.
I am developing a tool to get client information, send to a server, and receive the information again (a proxy). I'm also trying to dump the data being received from the server. I can read the Integer representation of the inputStream, but I am not able to read the String format. I've tried the below example, but it hangs and never connects to the server. Also, System.out.println(inputStream.nextLine()) displays only one line and hangs.
public void run() {
try {
int i;
while ((i = inputStream.read()) != -1){
System.out.println(IOUtils.toString(inputStream));
outputStream.write(i);
}
} catch (IOException e) {
System.out.println("Lost connection to the client.");
}
}
My guess at this is that you're reading from the input stream, and then using the IOUtils library to read from the stream too. My suspicion is that your application is reading the first byte from the input stream, then reading the remainder of the inputstream with the IOUtils library, and then printing out the initial byte that was read.
It doesn't make any sense to call IOUtils.toString(inputstream) from within a loop. That method call will put all the data from the inputstream into a string. Why have the loop at all in this case?
You might want to try not using the IOUtils library for this. Just read a byte of data, push it into a StringBuilder, and then print that byte. In this approach, the loop would be necessary, and you'll probably get what you're looking for.
Try something like this, but modify it as necessary to print the data at the same time to your output stream:
public static String inputStreamToString(final InputStream is, final int bufferSize)
{
final char[] buffer = new char[bufferSize];
final StringBuilder out = new StringBuilder();
try {
final Reader in = new InputStreamReader(is, "UTF-8");
try {
for (;;) {
int rsz = in.read(buffer, 0, buffer.length);
if (rsz < 0)
break;
out.append(buffer, 0, rsz);
}
}
finally {
in.close();
}
}
catch (UnsupportedEncodingException ex) {
/* ... */
}
catch (IOException ex) {
/* ... */
}
return out.toString();
}
The code you posted doesn't attempt to connect to the server, but if any of it executes you must already have connected.
If your program is hanging in this code, either the server hasn't sent any data yet, or the IOUtils.toString() method probably tries to read to EOS, so if the peer doesn't close the connection you will block here forever.
If your program hangs at a readLine() call it means the peer hasn't sent a line to read.
I have the following infinite loop which listens for incoming messages:
public static void listenForMessages(){
while (true) {
dsocket.receive(receivepacket);
byte[] rcvMsg = receivepacket.getData();
MessageCreator tmc = new MessageCreator();
TrafficMessage message = tmc.constructMessageFromBinary(rcvMsg);
System.out.println("message: "+message);
}
}
This calls a method that reads the byte array into a string and populates a message object.
public Message constructMessageFromBinary(byte[] rcvMsg)
throws IOException {
ByteArrayInputStream bais = new ByteArrayInputStream(rcvMsg);
DataInputStream dis = new DataInputStream(bais);
StringBuffer inputLine = new StringBuffer();
String tmp;
while ((tmp = dis.readLine()) != null) {
inputLine.append(tmp);
}
dis.close();
Message message = new Message();
message.setDescriptions(tmp);
return message;
}
This simple process slowly leaks memory over a few hours and I receive an out of memory exception.
Is there anything wrong with this logic?
The problem was that I left a database connection open. I wanted to leave it open to pass data with out having to worry about stopping and starting connections. I now open and close connections each time and all is good.
The best bet here would be to move all possible object instantiations outside the loops. For example, in the first code snippet, every iteration creates a
MessageCreator tmc.
On your second snippet, each call to the method creates a
StringBuffer inputLine.
This instantiation process may be eating away your memory slowly.
I know that there is a good variant to use Scanner object when you need to get data from server during connetion. But I have question about the following code snippet:
public void sendMessage(String message) {
try {
OutputStream os = socket.getOutputStream();
try {
byte[] buffer;
buffer = message.getBytes();
os.write(buffer);
} finally {
os.close();
}
InputStream is = socket.getInputStream();
try {
StringBuffer data = new StringBuffer();
Scanner in = new Scanner(is);
while (in.hasNext()) {
data.append(in.next());
}
System.out.println(data.toString());
} finally {
is.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
I'm confused by the snippet where Scanner gets data from InputStream, because it starts just after I send a message to the Server. Is it fair to suppose that data from the Server won't be in InputStream immediatelly after sending message to it?
Please, give me an advice, what is the best way to make reading data from InputStream in such case and what I should to take into consideration?
The InputStream.read() method called by Scanner blocks until there is some data available. So you don't have to worry about the response time of the server.
See: http://download.oracle.com/javase/6/docs/api/java/net/Socket.html#getInputStream()
The code is invalid. All it does is read as much input as can be read without blocking. There is no implication that what has been read is a complete message, or corresponds to a single write() invocation at the sender, etc. If you want messages in TCP/IP you must implement them yourself, with a length word prefix, a self-describing protocol such as Object Serialization or XML, etc. etc.
I have a thread in which the read() method of an InputStream is called in a loop. When there are no more bytes to read, the stream will block until new data arrives.
If I call close() on the InputStream from a different thread, the stream gets closed, but the blocked read() call still remains blocked. I would assume that the read() method should now return with a value of -1 to indicate the end of the stream, but it does not. Instead, it stays blocked for several more minutes until a tcp timeout occurs.
How do I unblock the close() call?
Edit:
Apparently, the regular JRE will throw a SocketException immediately when the stream or socket the blocking read() call corresponds to is close()'d. The Android Java runtime which I am using, however, will not.
Any hints on a solution for the Android environment would be greatly appreciated.
Only call read() when there is data available.
Do something like that:
while( flagBlock )
{
if( stream.available() > 0 )
{
stream.read( byteArray );
}
}
set the flagBlock to stop the reading.
See Java Concurrency In Practice for a really good system to cancel a thread when working with sockets. It uses a special executor (CancellingExecutor) and a special Callable (SocketUsingTask).
When the other end closes the connection your stream will return -1 on a read(). If you cannot trigger the other end to close the connection e.g. by closing your output stream, you can close the socket which will cause an IOException in the blocking read() thread.
Can you provide a short example which reproduces your problem?
ServerSocket ss = new ServerSocket(0);
final Socket client = new Socket("localhost", ss.getLocalPort());
Socket server = ss.accept();
Thread t = new Thread(new Runnable() {
public void run() {
int ch;
try {
while ((ch = client.getInputStream().read()) != -1)
System.out.println(ch);
} catch (SocketException se) {
System.out.println(se);
} catch (IOException e) {
e.printStackTrace();
}
}
});
t.start();
server.getOutputStream().write("hi\n".getBytes());
Thread.sleep(100);
client.close();
t.join();
server.close();
ss.close();
prints
104
105
10
java.net.SocketException: Socket closed
We were having the same issue: no exception when switching network (e.g. switching from 3G to WiFi while downloading).
We are using the code from http://www.androidsnippets.com/download-an-http-file-to-sdcard-with-progress-notification, which is working perfectly except in some cases when the network connection was lost.
The solution was specifying a timeout value, this is set standard to 0 (meaning: wait infinitely).
HttpURLConnection c = (HttpURLConnection) u.openConnection();
c.setRequestMethod("GET");
c.setDoOutput(true);
c.setReadTimeout(1000);
c.connect();
Experiment with a timeout value appropriate for you.
I had such issue on Samsung 2.3. When switching from 3G to Wifi InputStream.read() method blocks. I tried all tips from this topic. Nothing helped. From my prospective this is device specific issue because it should throw IOException due to javadoc. My solution is to listen for android broadcast android.net.conn.CONNECTIVITY_CHANGE and close connection from another thread it will cause IOException in blocked thread.
Here is code example:
DownloadThread.java
private volatile boolean canceled;
private volatile InputStream in;
private boolean downloadFile(final File file, final URL url, long totalSize) {
OutputStream out = null;
try {
Log.v(Common.TAG, "DownloadThread: downloading to " + file);
in = (InputStream) url.getContent();
out = new FileOutputStream(file);
return copy(out, totalSize);
} catch (Exception e) {
Log.e(Common.TAG, "DownloadThread: Exception while downloading. Returning false ", e);
return false;
} finally {
closeStream(in);
closeStream(out);
}
}
public void cancelDownloading() {
Log.e(Common.TAG, "DownloadThread: cancelDownloading ");
canceled = true;
closeStream(in); //on my device this is the only way to unblock thread
}
private boolean copy(final OutputStream out, long totalSize) throws IOException {
final int BUFFER_LENGTH = 1024;
final byte[] buffer = new byte[BUFFER_LENGTH];
long totalRead = 0;
int progress = 0;
int read;
while (!canceled && (read = in.read(buffer)) != -1) {
out.write(buffer, 0, read);
totalRead += read;
}
return !canceled;
}
You could use java.nio package. NIO stands for Non-blocking IO. Here the calls (to say read & write) aren't blocked. This way you can close the stream.
There is a sample program you can look at here. Method: processRead