I'm writing a client/server pair of applications. The server runs multiple threads that collect data and adds it to a BlockingQueue. The socket code loops over the queue and sends whatever data it finds to the client. The data is a string and I append a line separator so that the client can read it using BufferedReader.readLine().
My problem is that instead of readLine() returning on each line that's available it waits until the entire buffer is full before spitting out all the complete lines in the buffer. With the default 8K buffer this means I get data via the client in 8K chunks, which is highly undesirable. I've attached MRE code that represents this. I have confirmed via logging in my actual application that the BufferedWriter is writing the data as soon as it's available from the queue, but to be honest I don't know if the delay is coming after this on the sending side, or is truly on the reading side. If you run the MRE you'll see that the data is displayed approximately 170 lines at a time by the client.
I've searched online for this phenomenon for a couple of days and the one snippet that I could find of a similar issue suggests that maybe it's something to do with the underlying InputStreamReader and/or StreamDecoder, but that is starting to get beyond my expertise. (See this link)
So my question is whether I'm implementing the BufferedReader correctly and how can I resolve the issue I'm seeing so that I get each incoming line without unnecessary delays.
package serverTest;
import java.util.concurrent.BlockingQueue;
public class ServerTest {
public static void main(String[] args) {
int port = 54321;
ServerSocketComms server = new ServerSocketComms(port);
BlockingQueue<String> queue = server.getQueue();
new Thread(server).start();
ClientSocketComms client = new ClientSocketComms("localhost", port);
new Thread(client).start();
for(int i = 0; i < 1000; i++) { // should give about 10 seconds of output
try {
queue.put("" + i + " - All work and no play makes Jack a dull boy");
// Slow things down enough to show what's happening
Thread.sleep(10);
// 48 characters should fill the 8K buffer in approximately 2 seconds
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
package serverTest;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
public class ServerSocketComms implements Runnable {
private final BlockingQueue<String> queue = new LinkedBlockingQueue<>();
private final int port;
public ServerSocketComms(int port) {
this.port = port;
}
#Override
public void run() {
// Open server socket and wait for connection
try {
ServerSocket serverSocket = new ServerSocket(port);
Socket socket = serverSocket.accept();
// Continually loop over blocking data queue until stopped
BufferedWriter dataOut = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
while(socket.isConnected()) {
dataOut.write(queue.take());
dataOut.newLine(); // delimit strings with a line separator
}
// Loop never exits because client socket never completes because of BufferedReader issue
// so sockets never close and application never terminates
socket.close();
serverSocket.close();
} catch (IOException | InterruptedException e) {
e.printStackTrace();
}
}
public BlockingQueue<String> getQueue() {
// Return a reference to the sending queue to be populated by other threads
return this.queue;
}
}
package serverTest;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.Socket;
public class ClientSocketComms implements Runnable {
private final String server;
private final int port;
public ClientSocketComms(String server, int port) {
this.server = server;
this.port = port;
}
#Override
public void run() {
// Open socket to server and wait for incoming data
try {
Socket socket = new Socket(server, port);
BufferedReader dataIn = new BufferedReader(new InputStreamReader(socket.getInputStream()));
// Continually loop over incoming data until stopped
String data;
while((data = dataIn.readLine()) != null) {
// Should print out every line as it's received,
// but instead waits until buffer is full
// (outputs about 170 lines at a time)
System.out.println(data);
}
// Close socket and thread will die
// (but loop never ends because buffer doesn't get completely refilled)
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Your server is using a BufferedWriter:
BufferedWriter dataOut = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
This one does the buffering that you do not like. It seems the default buffer size is the 8k that you are seeing although that is not documented in the API and could change. Try flushing the buffer using dataOut.flush() if at some point in time you want to ensure everything stored in the buffer so far is sent out to the client immediately. Have a look at the BufferedWriter API for details.
BTW, I have not checked whether there are any other problems in your code. But the above is definitely one.
Related
So I'm writing two ServerSockets. One that listens for HTTP requests on port 8085 and saves the byte input into a static LinkedList, and a second that listens on port 8086 and returns all the results in the static LinkedList.
The issue is that when saving the data from ServerSocket:8085 to the LinkedList, the thread hangs and I'm not sure why.
This is the main Listener class:
import java.net.ServerSocket;
import java.nio.charset.Charset;
import java.util.*;
public class Listener {
public static LinkedList<byte[]> Calls = new LinkedList<>();
public static void main(String[] args) {
Thread callback = new Thread(new ThreadListener());
callback.start();
while (true) {
try (var listener = new ServerSocket(8086)) {
System.out.println("Listening on 8086...");
try (var client = listener.accept()) {
StringBuffer response = new StringBuffer();
response.append("HTTP/1.1 200 OK\r\n\r\n");
Iterator<byte[]> iterator = Calls.iterator();
while (iterator.hasNext()) {
response.append(new String(iterator.next(), Charset.forName("UTF-8")) + "\r\n");
iterator.remove();
}
client.getOutputStream().write(response.toString().getBytes("UTF-8"));
client.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
This is the ThreadListener class:
import java.io.IOException;
import java.net.ServerSocket;
import java.util.Date;
public class ThreadListener implements Runnable {
#Override
public void run() {
while (true) {
try (var listener = new ServerSocket(8085)) {
System.out.println("Listening on 8085...");
try (var socket = listener.accept()) {
if (!socket.getInetAddress().getHostAddress().equals("127.0.0.1")) {
System.out.println("Not localhost");
} else {
System.out.println("Its us!");
}
Listener.Calls.add(socket.getInputStream().readAllBytes());
System.out.println("Result collected");
Date today = new Date();
String httpResponse = "HTTP/1.1 200 OK\r\n\r\n" + today;
socket.getOutputStream().write(httpResponse.getBytes("UTF-8"));
socket.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
As for my test, I tried calling 127.0.0.1:8085, and I get an ERR_CONNECTION_RESET, and all I have in the console is the following:
Listening on 8085...
Listening on 8086...
Its us!
Process finished with exit code -1 (I killed the app after 2 mins)
The fact that the "Its us!" message got printed, but the "Results collected!" right after the LinkedList.add didn't is what leads me to assume the LinkedList.add is the one hanging the thread.
Regards
EDIT: No one is calling to 8085 (or 8086), I'm doing it manually on my browser. I solved the Syncronization issue by creating a method to call instead of calling the LinkedList.add directly:
public static synchronized void addElementsToList(byte[] bytes) {
Calls.add(bytes);
}
This does work, but calling the 8085 socket gives the connection reset every time.
Your test mechanism of using a browser to create the request is probably also not helping here, as the InputStream.readAllBytes()
blocks until all remaining bytes have been read and end of stream is detected, or an exception is thrown. This method does not close the input stream.
From the documentation. Specifically, the browser is keeping the connection open, because it's expecting some response. Your server is trying to read everything from the connection until the connection is closed. Catch 22 (aka deadlock).
Try making the connection to localhost:8085 using telnet and closing the connection from the client end.
telnet 127.0.0.1 8085
^D
where ^D is literally the [CTRL] and [D] keys (eg: logout)
A LinkedList is not synchornized, as highlighted in the documentation.
You can either handle synchronisation manually, use a synchronised list, or a concurrent list. Probably some other methods too, but for now, just keep it simple.
public static LinkedList<byte[]> Calls = Collections.synchronizedList(new LinkedList<>());
// or
public static LinkedList<byte[]> Calls = new CopyOnWriteArrayList<>();
I am trying to improve the speed at which the sockets transfer information but i am unsure how to do so. the pourpose of the code is to transfer a number, the date, and a short xml which is being sent in the form of a string.
this is the server code
import java.net.*;
import java.io.*;
public class SSocket extends Thread
{
private ServerSocket serverSocket;
public SSocket(int port) throws IOException
{
serverSocket = new ServerSocket(port);
serverSocket.setSoTimeout(100000);
}
public void run()
{
System.out.println("Waiting for client on port " + serverSocket.getLocalPort() + "...");
while(true)
{
try
{
Socket server = serverSocket.accept();
DataInputStream in = new DataInputStream(server.getInputStream());
int cor=in.readInt();
int i=0;
String transaccion = in.readUTF();
String fecha = in.readUTF();
System.out.println(cor);
System.out.println(transaccion);
System.out.println(fecha);
DataOutputStream out =
new DataOutputStream(server.getOutputStream());
if(transaccion!=null && fecha != null && cor>0){
out.writeInt(cor);
}
else {
out.writeInt(-1);
}
if (i==100){
out.flush();
i=0;
}
i++;
server.close();
}catch(SocketTimeoutException s)
{
System.out.println("Socket timed out!");
break;
}catch(IOException e)
{
e.printStackTrace();
break;
}
}
}
public static void main(String [] args)
{
int port = 1337;
try
{
Thread t = new SSocket(port);
t.start();
}catch(IOException e)
{
e.printStackTrace();
}
}
}
the code for the client is
import java.net.*;
import java.io.*;
public class ClientSocket
{
public static void send(int correl, String transaccion, String fecha)
{
String serverName = "localhost";
int port = 1337;
try
{
Socket client = new Socket(serverName, port);
int i=0;
OutputStream outToServer = client.getOutputStream();
DataOutputStream out =
new DataOutputStream(outToServer);
out.writeInt(correl);
out.writeUTF(transaccion);
out.writeUTF(fecha);
InputStream inFromServer = client.getInputStream();
DataInputStream in =
new DataInputStream(inFromServer);
int corin=in.readInt();
if(corin>0){
Envio.updater(corin);
}
else {
}
if (i==100){
out.flush();
i=0;
}
i++;
client.close();
}catch(IOException e)
{
e.printStackTrace();
}
}
}
i have done some reading on the mater and it seems that posible solutions are to use either a buffer or swich to a datagram. however my experience on working with sockets is rather limited and i am unsure which would be best to use for this situation or if there is another option i havent yet considered. this code will be moving many transactions and i wish to do it in as short time as posible.
thanks in advance
ps. sorry for my bad english it is not my first language
Datagrams imply UDP, which is an unreliable delivery protocol so you're not guaranteed to get all content. That's probably not what you want; I'd stay with plain Sockets (which use TCP, which has reliable delivery).
Will the same client be calling send() repeatedly and connecting to the same server each time? That is, will there be many messages going across a single connection, or will each message be to a different server, with only a single message (or only a few) going to each of the many servers? If there's just one server that a client is going to connect to and if a given client is going to send lots of messages, you should keep the Socket open between send() calls; setting up and tearing down Sockets is expensive, so you're paying a high price for making a new connection each time.
Also, your server appears to only be able to handle a single connection at a time: you accept a connection, read from it, and then close it and accept a new one. So to make this work for more than one client, you'll need to separate the logic for accepting connections onto a different thread from the logic that reads data. If you'll only have a few clients at a time, you can just start a new thread to read from each socket as you create it for a new client; if you'll have lots of clients (thousands), you'll probably need to look at NIO for its ability to service multiple sockets from a single thread. But I suspect you're a long way from having that problem, if you ever do, so I'd just spawn a new thread for each socket.
I'm trying to play around with the concurrent package and for this reason I tried to write a simple socket handler. Here is the code:
import java.io.*;
import java.net.*;
import java.util.*;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class MultiThreadedServer{
private final static int number_of_threads = 4;
private final static int port = 1134;
public static void main(String[] args){
try{
ServerSocket ss = new ServerSocket(port);
ExecutorService pool = Executors.newFixedThreadPool(number_of_threads);
for(;;){
pool.execute(new SocketHandler(ss.accept()));
}
}catch(Exception e){
System.out.println(e.toString());
}
}
}
class SocketHandler implements Runnable {
private Socket socket;
SocketHandler(Socket s){
this.socket = s;
System.out.println("-- Socket has connected -- ");
}
public void run(){
try{
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
String s = "";
while((s = reader.readLine()) != null ){
System.out.println(s);
}
}catch(Exception e){
System.out.println(e.toString());
}
}
}
This code ^^ simply waits for sockets and reads whatever the socket sends. It works fine to an extent. ( I will explain a bit later what bothers me).
The following code is the sender class which "sends" a socket
import java.io.*;
import java.net.*;
public class Sender{
public static void main(String args[]){
try{
int port = Integer.parseInt(args[0]);
Socket socket = new Socket(InetAddress.getByName(null),port);
PrintWriter writer = new PrintWriter(socket.getOutputStream());
writer.write("Wohoo!");
writer.close();
}catch(Exception e){
System.out.println(e.toString());
}
}
}
The code compiles fine and it even runs fine when I run my sender. However if I type
java Sender
about 3 times a second and I try to run it into my console the following thing gets printed:
java.net.BindException: Address already in use
However the whole point of this code was to not block the connections to my port and to queue the tasks. How can I tackle this?
See my comment. But I think what you are seeing is exhausting your tcp ports. Are you running this thing for awhile before it starts printing port already in use? If so read ahead.
So after closing a socket there is a state of TIME_WAIT. Until it passes, you cannot reuse same port unless you have set the socket option SO_REUSEADDR.
So you need to use that option perhaps. See this answer for more insight:
https://stackoverflow.com/a/14388707/520567
I'm trying to write a simple console program that allows me to send and receive String messages. The problem I am encountering though, is that I don't know how to run the receiving code and the sending code simultaneously.
Individually, the classes are working. I can receive packets and send packets, but making them run at once seems impossible to me.
I've looked into multi-threading but since my knowledge is still very basic, I can't seem to understand how it really works.
This is the code I'm currently using. I wrote the Dialog class myself and found the other two classes on the internet.
Dialog class:
import java.util.Scanner;
public class Dialog {
Scanner scanner = new Scanner(System.in);
User user = new User();
Network net = new Network();
ThreadReceive tr = new ThreadReceive();
ThreadSend ts = new ThreadSend();
public void run() {
System.out.println("WELCOME");
System.out.print("Port: ");
while(!user.setPort(giveInput())) {
System.out.println("Enter a valid port.");
}
System.out.print("IP: ");
user.setIP(giveInput());
System.out.println();
System.out.println("--- CONVERSATION STARTED ---");
tr.receive(user.getIP(), user.getPort()); // Starts receiving loop (within ThreadReceive class).
while (true) { // Starts sending loop.
ts.sendMessage(giveInput(), user.getIP(), user.getPort()); // Sends packet when input is given.
}
}
private String giveInput() {
String input = scanner.nextLine();
return input;
}
}
Receiving class:
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
public class ThreadReceive extends Thread {
public void receive(String ip, int port) {
try {
// Create a socket to listen on the port.
DatagramSocket dsocket = new DatagramSocket(port);
// Create a buffer to read datagrams into. If a
// packet is larger than this buffer, the
// excess will simply be discarded!
byte[] buffer = new byte[2048];
// Create a packet to receive data into the buffer
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
// Now loop forever, waiting to receive packets and printing them.
while (true) {
// Wait to receive a datagram
dsocket.receive(packet);
// Convert the contents to a string, and display them
String msg = new String(buffer, 0, packet.getLength());
System.out.println(packet.getAddress().getHostName() + ": " + msg);
// Reset the length of the packet before reusing it.
packet.setLength(buffer.length);
}
}
catch (Exception e) {
System.err.println(e);
}
}
}
Sending class:
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
public class ThreadSend extends Thread {
public void sendMessage(String message, String ip, int port) {
try {
byte[] data = message.getBytes();
InetAddress address = InetAddress.getByName(ip);
DatagramPacket packet = new DatagramPacket(data, data.length, address, port);
DatagramSocket datagramSocket = new DatagramSocket();
datagramSocket.send(packet);
}
catch (Exception e) {
e.printStackTrace();
}
}
}
Also, is there any way to test if I can receive packets? I've been testing it with a friend but it would be much more convenient to do it myself.
Thanks!
You are not using Threads correctly.
The logic should be in the run method.
I suggest you use a queue such as an ArrayBlockingQueue to pass parameters to your threads. For instance, you could have a method to add elements to this queue
public void addMessage(String message) {
synchronized(inputQueue) {
inputQueue.offer(r);
inputQueue.notify();
}
}
And the run method will use these elements as so :
public void run() {
try {
while(!running)
synchronized (inputQueue) {
inputQueue.wait(); // you can have a timeout also...
String message = this.inputQueue.poll();
// use the message item....
// in your case send it to the other user.
}
}
}
} catch (Exception e) {
/////// your exception handler
}
}
Also Remember to start your threads :
Thread t = new MyThread();
t.start(); /// Start the thread !!!
PS : The messages can be any object here is used strings as i based this on some of my code where i'm using a
Queue<String>
Check out Beej's Guide to Network Programming: http://beej.us/guide/bgnet/output/html/singlepage/bgnet.html -- This will give you some more examples to take a look at. As far as testing goes, you could set up a virtual machine or use another computer you own. Back when I had to learn networking in school we would ssh into two separate Linux boxes to test our code.
EDIT: Also to make sure you are receiving correctly you could make the sender and receiver both print out the packet data they receive. Or you could just have them print a simple string to say the packet was received.
You also might want to check out TCP rather than UDP if you are wanting a continuous stream. UDP just creates a datagram packet and sends it out on the network, whereas TCP creates a persistant connection between two hosts.
I am currently developing a prototype for a game and I need a simple server to run it.
At this stage, I don't want to invest the time learning about all the different full-featured multiplayer game servers already there (smartfox, etc...)
I know how to develop a basic Server with Threads listening on Sockets but I have ran into a roadblock. Here's the run() function of the Thread
public void run() {
try {
out = new PrintWriter(mSocket1.getOutputStream(), true);
in = new BufferedReader( new InputStreamReader( mSocket1.getInputStream() ) );
String inputLine1 = null, outputLine;
out.println("hello");
out.flush();
while( (inputLine1 = in.readLine()) != null) {
outputLine = mGameControl.processInput(mPlayerNum, inputLine1);
out.println(outputLine);
out.flush();
if(outputLine.contentEquals("bye"))
break;
}
Terminate();
}
catch(IOException e) { e.printStackTrace(); }
}
Now my problem is that the thread is blocked waiting for input. I do have other similar Threads connected to other client which may result in information being dispatched to all clients...
How can I modify it so that a different Thread can interact with it and push info to the client?
Just write a synchronised public method which writes to your PrintWriter, and allow other threads to use it to send messages to your client. Call the same method from your read loop to avoid two threads writing at the same time.
Here's a tested example:
import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;
public class SocketTest {
public static class Client implements Runnable {
private final BufferedReader in;
private final PrintWriter out;
public Client(Socket clientSocket) throws IOException {
out = new PrintWriter(clientSocket.getOutputStream(), true);
in = new BufferedReader( new InputStreamReader( clientSocket.getInputStream() ) );
}
public void run() {
send("Hello");
String inputLine1 = null, outputLine;
try {
while( (inputLine1 = in.readLine()) != null) {
outputLine = inputLine1.toLowerCase();
System.out.println(inputLine1);
send(outputLine);
if(outputLine.contentEquals("bye"))
break;
}
} catch (IOException e) {
e.printStackTrace();
}
}
public synchronized void send(String message) {
out.println(message);
out.flush();
}
}
public static void main(String[] args) throws IOException, InterruptedException {
ServerSocket s = new ServerSocket(5050);
Socket clientSocket = s.accept();
Client client = new Client(clientSocket);
Thread clientThread = new Thread(client);
clientThread.start();
int i = 1;
while (true) {
Thread.sleep(1000);
client.send("Tick " + (i++));
}
}
}
Use Netty to handle your connections and query treatments. Since I discovered that project, I never touched sockets directly anymore (except when writing C programs, etc.)
There are actually some examples to look at and the documentation is quite extensive. The project is very well alive since a couple of years already, and is not soon to die! There's a fairly large user base behind it.
If you only need to perform a non-blocking read on the socket, one of the simplests ways is to use available(). Call available and if there is data to read (bytes pending > 0), perform the read, any way, wait using Thread.sleep() and try to read again. This allow the thread to do while waiting for input data and cat react to external signals.
The use of selectors is encouraged when you need to perform high-performance non-blocking read using one thread and several sockets (java.nio.channels.Selector).