We Keep Coding

How to properly stream data from a socket with Java

I am attempting stream data over a socket with Java in an attempt to write a Kafka producer. I've written a class to pull the data in but I'm not getting the results I'd expect. I've got it set up so the data is being streamed from a Linux box. The source of the data is a csv file that I'm using the nc utility to stream. The class is running on a Windows 10 machine from Eclipse. When I run the class I see two weird things.
The column headers don't get transmitted.
I can only run the class once. If I want to run it again, I have to stop nc and restart it.
Below is my code. Am I missing anything? At this point I'm just trying to connect to the socket and pull the data over.
I run nc with the following command:
$ nc -kl 9999 < uber_data.csv
Below is my class
import java.net.*;
import java.io.*;
public class Client
{
static String userInput;
public static void main(String [] args)
{
try
{
InetAddress serverAddress = InetAddress.getByName("servername");
Socket socket = new Socket(serverAddress, 9999);
BufferedReader input = new BufferedReader(new InputStreamReader(socket.getInputStream()));
while ((userInput = input.readLine()) != null) {
System.out.println(input.readLine());
}
input.close();
socket.close();
}
catch(UnknownHostException e1)
{
System.out.println("Unknown host exception " + e1.toString());
}
catch(IOException e2)
{
System.out.println("IOException " + e2.toString());
}
catch(IllegalArgumentException e3)
{
System.out.println("Illegal Argument Exception " + e3.toString());
}
catch(Exception e4)
{
System.out.println("Other exceptions " + e4.toString());
}
}
}

You're throwing away every odd-numbered line. It should be:
while ((userInput = input.readLine()) != null) {
System.out.println(userInput);
}
Secondly, you aren't closing the socket. Use a try-with-resources:
try
{
InetAddress serverAddress = InetAddress.getByName("servername");
try (
Socket socket = new Socket(serverAddress, 9999);
BufferedReader input = new BufferedReader(new InputStreamReader(socket.getInputStream()));
) {
while ((userInput = input.readLine()) != null) {
System.out.println(input.readLine());
}
}
}
catch (...)

First, each call readLine() tries to read line from input stream.
In userInput = input.readLine() you read header, but println(input.readLine()) read body and print in console.
while ((userInput = input.readLine()) != null) {
System.out.println(userInput); //instead input.readLine()
}
Second, I didn't use nc, but I think problem will solve if you will close socket (and reader) in finally statement.

I hope it would be helpful.
For the first question: you were trying to print userInput string. But it's printing the result of another readline() call.
For the second: after the file has been transferred, you have to stop and restart nc; no matter what you do from your side. It's from nc side.
See the nc documentation.

How to automatically update server and client side in java

I'm learning distributed systems basics and currently I'm trying to do a simple yet realistic messenger between one server and one client. What I do intend is that on each endpoint socket side (Server and Client) text automatically updates (like a real "messaging app"). In other words, I want that the moment I write and "send" the message, it automatically appears on recipient side. What I have now follows this schema:
I send a message (let's assume from client)
To see that message on Server's side I need to reply first (because Server's BufferedReader / Client's PrintWriter is only read after asking for the answer)
My code:
public class ClientSide {
public static void main(String [] args){
String host_name = args[0];
int port_number = Integer.parseInt(args[1]);
try {
Socket s = new Socket(host_name, port_number);
PrintWriter out =
new PrintWriter(s.getOutputStream(), true);
BufferedReader in =
new BufferedReader(
new InputStreamReader(s.getInputStream()));
BufferedReader stdIn =
new BufferedReader(
new InputStreamReader(System.in));
String answer;
while ((answer = stdIn.readLine()) != null) {
out.println(answer);
System.out.println("\nlocalhost said\n\t" + in.readLine());
}
} catch (IOException ex) {
Logger.getLogger(ClientSide.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
public class ServerSide {
public static void main(String [] args){
int port_number = Integer.parseInt(args[0]);
try {
ServerSocket ss = new ServerSocket(port_number);
Socket tcp = ss.accept();
PrintWriter out =
new PrintWriter(tcp.getOutputStream(), true);
BufferedReader in =
new BufferedReader(
new InputStreamReader(tcp.getInputStream()));
BufferedReader stdIn =
new BufferedReader(
new InputStreamReader(System.in));
String answer;
while ((answer = stdIn.readLine()) != null){
out.println(answer);
System.out.println("\nClient said\n\t" + in.readLine());
}
} catch (IOException ex) {
Logger.getLogger(ServerSide.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
How can I do this? Does it involve advanced knowledge on the matter?
Thanks in advance.

The core problem is that you want to wait for two events concurrently -- either a message from the socket, or input from the user!
You want to wait on both at the same time -- you don't want to be stuck waiting for a message in the socket if the user types a message; nor to be waiting for the user message while you have a new message from the network.
To 'wait' for messages from multiple streams, you have java.nio. I believe it is the most correct way of doing it.
But if you want to keep using the BufferedReader, there is a ready() method that returns true if and only if there is a message waiting to be read.
Your code after the in and stdIn declarations would then look something like (I didn't test it!!):
while(true) {
if(stdIn.ready()) {
System.out.println("I said " + stdIn.readLine());
}
if(in.ready()) (
System.out.println("He said " + in.readLine());
}
}
A few somewhat useful random links:
Java - Reading from a buffered reader (from a socket) is pausing the thread
Is there epoll equivalent in Java?

Send a string command to a socket

I am not very familiar with telnet so I would appreciate the help from any willing.
I have smart plugs which can be switch on or off through a telnet interface.
I always use telnet via command prompt to connect to the server Digi X4 connect port (via >telnet ). If I want to switch the socket on/off, I have to now type: "12 set pow=on/off" and press enter.
I would like to implement this through java using the telnet client. I am now able to connect to the port (thanks to the answers posted on this platform), but to send the command to switch devices on/off is proving difficult for me. I still have to type "12 set pow=on/off" and press enter. I would like Java to send this command.
Below is my java code. I would appreciate your assistance. Bab
public class TelnetConnection {
static TelnetClient tc = null;
public static void main(String[] a) throws Exception
{
String[] args = {"122.1222.181.45","8085"};
System.out.println("arg value: "+args);
if(args.length < 1)
{
System.err.println("Usage: Error <remote-ip> [<remote-port>]");
System.exit(1);
}
String remoteip = args[0];
int remoteport;
if (args.length > 1)
{
remoteport = (new Integer(args[1])).intValue();
}
else
{
remoteport = 7000;
}
tc = new TelnetClient();
while (true)
{
boolean end_loop = false;
try
{
tc.connect(remoteip, remoteport);
Thread reader = new Thread (new TelnetClientExample());
tc.registerNotifHandler(new TelnetClientExample());
System.out.println("TelnetClientExample");
reader.start();
OutputStream outstr = tc.getOutputStream();
PrintWriter out = new PrintWriter(outstr);
String buff = "11 set pow=on";
//int ret_read = 0;
do
{
try
{
out.print(buff);
outstr.flush();
}
catch (IOException e)
{
System.err.println("Error");
end_loop = true;
}
}
while((true) && (end_loop == false));
try
{
tc.disconnect();
}
catch (IOException e)
{
System.err.println("Error");
}
}
catch (IOException e)
{
System.err.println("Exception while connecting:" + e.getMessage());
System.exit(1);
}
}
}
}

Try tring buff = "11 set pow=on\n"; the server may need the newline to detect end-of-command.
By the way, the loop that infinitely sends that to the server looks worrisome.

You need to send a line terminator corresponding to 'and press Enter'.
The line terminator in Telnet is defined as \r\n.

ArrayList<String> size not persisting between server and client

I'm writing a server/client chat application in Java. In order to receive messages and then print the total dialog of the conversation, I add each message from the client to an ArrayList of Strings, and then send the whole ArrayList back to the client, to print out as the entire conversation.
My problem is that even though I am constantly adding elements to the ArrayList in the server, whenever I send it to the client, the size does not change, and only the first element is stored.
Server Program:
public class ArrayListServer {
public static void main(String[] args) {
int port = 8000;
String me = "Server: ";
ArrayList<String> convo = new ArrayList<String>();
try {
ServerSocket server = new ServerSocket(port);
System.out.println("Waiting for connection...");
Socket client = server.accept();
System.out.println("Established connection.");
ObjectInputStream in = new ObjectInputStream(client.getInputStream());
ObjectOutputStream out = new ObjectOutputStream(client.getOutputStream());
int i = 0;
// receive messages from client
while (true) {
String msgFromClient = (String)in.readObject();
convo.add(msgFromClient);
System.out.println(me + "size: " + convo.size());
out.writeObject(convo);
}
} catch (IOException ioex) {
System.out.println("IOException occurred.");
} catch (ClassNotFoundException cnfex) {
System.out.println("ClassNotFoundException occurred.");
}
}
}
Client Program:
public class ArrayListClient {
#SuppressWarnings("unchecked")
public static void main(String[] args) {
int port = 8000;
String me = "Client: ";
Scanner input = new Scanner(System.in);
ArrayList<String> convo = new ArrayList<String>();
try {
Socket client = new Socket("localhost", port);
ObjectOutputStream toServer = new ObjectOutputStream(client.getOutputStream());
ObjectInputStream fromServer = new ObjectInputStream(client.getInputStream());
while (true) {
System.out.print("> ");
String msg = input.nextLine().trim();
toServer.writeObject(msg);
convo = (ArrayList<String>)fromServer.readObject();
System.out.println(me + "size: " + convo.size());
}
} catch (UnknownHostException uhex) {
System.out.println("UnknownHostException occurred.");
} catch (IOException ioex) {
System.out.println("IOException occurred.");
} catch (ClassNotFoundException cnfex) {
System.out.println("ClassNotFoundException occurred.");
}
}
}
When I run the server and client, my output is:
> hi
Client: size: 1
> hi
Client: size: 1
> hi
Client: size: 1
> hi
Client: size: 1
> hi
Client: size: 1
> hi
Client: size: 1
>
Server: Waiting for connection...
Server: Established connection.
Server: size: 1
Server: size: 2
Server: size: 3
Server: size: 4
Server: size: 5
Server: size: 6
I know the ArrayList<> and String class are both serializable, so I have no clue why this isn't working. I think it may be something to do with my Input/Output streams, but when I declared them local to the while loop and closed them at the end of it, my program would throw an IOException and halt.
What am I doing wrong?

I solved it. You use the same ObjectOutput and ObjectInput instances. ObjectOutput tries to not rewrite instances it has seen before. So the first time the server writes the convo array list it keeps a reference to it. When you add to it the reference does not change even though the contents of the list do. If you make the one line change below in your server this will prove to you that this is the problem:
out.writeObject( new ArrayList( convo ) );
The above is not a great solution because the ObjectOutputStream's reference map will keep growing one very iteration. But it will show you what the problem is so you can create a better solution.
The better solution is to just move the getInputStream and getOutputStream lines within the loop on both sides.
Also in case it's not obvious the strings work because on each iteration they are a different object reference. The List does not because on the server side you write the same reference each time even though it changes.

Great, i tried to solve this issue several Days..thanks!
You can use the reset() method to 'forgot' known references as shown below:
out.writeObject(object);
out.reset();
out.flush();

java.lang.Exception: Error while transmitting data how to solve this error

I am trying to send a file from server side to client side upon request. The file that is sent is encrypted and the client shpuld decrypt it. the encryption process works fine but while decrypting i need to have the DerIOBuffer objetc which I have using serializing. what should i do..please help
server:
import java.net.*;
import java.io.*;
import java.math.BigInteger;
import com.dragongate_technologies.borZoi.*;
public class FileServer {
static final int LISTENING_PORT = 3210;
public static void main(String[] args) {
File directory; // The directory from which the gets the files that it serves.
ServerSocket listener; // Listens for connection requests.
Socket connection; // A socket for communicating with a client.
/* Check that there is a command-line argument.
If not, print a usage message and end. */
if (args.length == 0) {
System.out.println("Usage: java FileServer <directory>");
return;
}
/* Get the directory name from the command line, and make
it into a file object. Check that the file exists and
is in fact a directory. */
directory = new File(args[0]);
if ( ! directory.exists() ) {
System.out.println("Specified directory does not exist.");
return;
}
if (! directory.isDirectory() ) {
System.out.println("The specified file is not a directory.");
return;
}
/* Listen for connection requests from clients. For
each connection, create a separate Thread of type
ConnectionHandler to process it. The ConnectionHandler
class is defined below. The server runs until the
program is terminated, for example by a CONTROL-C. */
try {
listener = new ServerSocket(LISTENING_PORT);
System.out.println("Listening on port " + LISTENING_PORT);
while (true) {
connection = listener.accept();
new ConnectionHandler(directory,connection);
}
}
catch (Exception e) {
System.out.println("Server shut down unexpectedly.");
System.out.println("Error: " + e);
return;
}
} // end main()
static class ConnectionHandler extends Thread {
// An object of this class is a thread that will
// process the connection with one client. The
// thread starts itself in the constructor.
File directory; // The directory from which files are served
Socket connection; // A connection to the client.
TextReader incoming; // For reading data from the client.
PrintWriter outgoing; // For transmitting data to the client.
ConnectionHandler(File dir, Socket conn) {
// Constructor. Record the connection and
// the directory and start the thread running.
directory = dir;
connection = conn;
start();
}
void sendIndex() throws Exception {
// This is called by the run() method in response
// to an "index" command. Send the list of files
// in the directory.
String[] fileList = directory.list();
for (int i = 0; i < fileList.length; i++)
outgoing.println(fileList[i]);
outgoing.flush();
outgoing.close();
if (outgoing.checkError())
throw new Exception("Error while transmitting data.");
}
void ecies_ex(String fileName) throws Exception {
// This function encrypts the file that has been requested
// by the client.
String at1,dc1,der1;
OutputStream os = connection.getOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(os);
ECDomainParameters dp = ECDomainParameters.NIST_B_163();
ECPrivKey skA = new ECPrivKey(dp, BigInteger.valueOf(123));
ECPubKey pkA = new ECPubKey(skA);
ECPrivKey skB = new ECPrivKey(dp, BigInteger.valueOf(230));
ECPubKey pkB = new ECPubKey(skB);
File file = new File(directory,fileName);
if ( (! file.exists()) || file.isDirectory()) {
// (Note: Don't try to send a directory, which
// shouldn't be there anyway.)
outgoing.println("error");
}
else {
outgoing.println("ok");
String pt1 = new String();
BufferedReader br = null;
try {
String sCurrentLine;
br = new BufferedReader(new FileReader(fileName));
while ((sCurrentLine = br.readLine()) != null) {
pt1=pt1+"\n"+sCurrentLine;
}
} catch (IOException e) {
e.printStackTrace();
}
ECIES crypt = new ECIES(skA, pkB, pt1.getBytes()); // encrypt the data
try {
DerIOBuffer der = new DerIOBuffer(crypt);
oos.writeObject(der);
ECIES decrypt = der.toECIES();
dc1=decrypt.toString2(); //cipher text
//at1=decrypt.toString3(); //authentication tag
FileWriter fstream = new FileWriter("encrypted.txt");
BufferedWriter out = new BufferedWriter(fstream);
out.write(dc1);
//Close the output stream
out.close();
} catch (Exception e) {
System.out.println(e.toString());
}
TextReader fileIn = new TextReader( new FileReader("encrypted.txt") );
while (fileIn.peek() != '\0') {
// Read and send lines from the file until
// an end-of-file is encountered.
String line = fileIn.getln();
outgoing.println(line);
}
}
outgoing.flush();
// oos.close();
// os.close();
outgoing.close();
if (outgoing.checkError())
throw new Exception("Error while transmitting data.");
}
public void run() {
// This is the method that is executed by the thread.
// It creates streams for communicating with the client,
// reads a command from the client, and carries out that
// command. The connection is logged to standard output.
// An output beginning with ERROR indicates that a network
// error occurred. A line beginning with OK means that
// there was no network error, but does not imply that the
// command from the client was a legal command.
String command = "Command not read";
try {
incoming = new TextReader( connection.getInputStream() );
outgoing = new PrintWriter( connection.getOutputStream() );
command = incoming.getln();
if (command.equals("index")) {
sendIndex();
}
else if (command.startsWith("get")){
String fileName = command.substring(3).trim();
ecies_ex(fileName);
//sendFile(fileName);
}
else {
outgoing.println("unknown command");
outgoing.flush();
}
System.out.println("OK " + connection.getInetAddress()
+ " " + command);
}
catch (Exception e) {
System.out.println("ERROR " + connection.getInetAddress()
+ " " + command + " " + e);
}
finally {
try {
connection.close();
}
catch (IOException e) {
}
}
}
} // end nested class ConnectionHandler
} //end class FileServer
client :
import java.net.*;
import java.io.*;
import java.math.BigInteger;
import com.dragongate_technologies.borZoi.*;
public class FileClient {
static final int LISTENING_PORT = 3210;
public static void main(String[] args) {
String computer; // Name or IP address of server.
Socket connection; // A socket for communicating with that computer.
PrintWriter outgoing; // Stream for sending a command to the server.
TextReader incoming; // Stream for reading data from the connection.
String command; // Command to send to the server.
String pt3;
ECDomainParameters dp = ECDomainParameters.NIST_B_163();
ECPrivKey skB = new ECPrivKey(dp, BigInteger.valueOf(230));
//ECPrivKey skB = new ECPrivKey (dp);
ECPubKey pkB = new ECPubKey(skB);
/* Check that the number of command-line arguments is legal.
If not, print a usage message and end. */
if (args.length == 0 || args.length > 3) {
System.out.println("Usage: java FileClient <server>");
System.out.println(" or java FileClient <server> <file>");
System.out.println(" or java FileClient <server> <file> <local-file>");
return;
}
/* Get the server name and the message to send to the server. */
computer = args[0];
if (args.length == 1)
command = "index";
else
command = "get " + args[1];
/* Make the connection and open streams for communication.
Send the command to the server. If something fails
during this process, print an error message and end. */
try {
connection = new Socket( computer, LISTENING_PORT );
incoming = new TextReader( connection.getInputStream() );
outgoing = new PrintWriter( connection.getOutputStream() );
outgoing.println(command);
outgoing.flush();
}
catch (Exception e) {
System.out.println(
"Can't make connection to server at \"" + args[0] + "\".");
System.out.println("Error: " + e);
return;
}
/* Read and process the server's response to the command. */
try {
if (args.length == 1) {
// The command was "index". Read and display lines
// from the server until the end-of-stream is reached.
System.out.println("File list from server:");
while (incoming.eof() == false) {
String line = incoming.getln();
System.out.println(" " + line);
}
}
else {
// The command was "get <file-name>". Read the server's
// response message. If the message is "ok", get the file.
String message = incoming.getln();
if (! message.equals("ok")) {
System.out.println("File not found on server.");
return;
}
PrintWriter fileOut; // For writing the received data to a file.
if (args.length == 3) {
// Use the third parameter as a file name.
fileOut = new PrintWriter( new FileWriter(args[2]) );
}
else {
// Use the second parameter as a file name,
// but don't replace an existing file.
File file = new File(args[1]);
if (file.exists()) {
System.out.println("A file with that name already exists.");
System.out.println("To replace it, use the three-argument");
System.out.println("version of the command.");
return;
}
fileOut = new PrintWriter( new FileWriter(args[1]) );
}
while (incoming.peek() != '\0') {
// Copy lines from incoming to the file until
// the end of the incoming stream is encountered.
String line = incoming.getln();
fileOut.println(line);
}
InputStream is = connection.getInputStream();
ObjectInputStream ois = new ObjectInputStream(is);
DerIOBuffer der = (DerIOBuffer)ois.readObject();
ECIES decrypt = der.toECIES();
byte[] pt2 = decrypt.decrypt(skB); // decrypt the data
pt3=new String(pt2);
if (fileOut.checkError()) {
System.out.println("Some error occurred while writing the file.");
System.out.println("Output file might be empty or incomplete.");
}
}
}
catch (Exception e) {
System.out.println("Sorry, an error occurred while reading data from the server.");
System.out.println("Error: " + e);
}
} // end main()
} //end class FileClient

If you care about errors, you should not use PrintWriter. Why? Because if an error does occur on output via a PrintWriter, you have no way to find out what it was. This is what makes it difficult to figure out what the real problem is in this case. I recommend that you fix this so that you can get to the real cause of the problem.
The real problem could be related to to the following issues:
If the stuff you are trying to write could be binary, you shouldn't use PrintWriter ... or Readers / Writers at all.
You seem to be using Object serialization unnecessarily ... and on a class that looks like it may not be serializable.
Based on the difficulty I had in finding documentation for the "borZoi" library ... and other things ... I think you may have made a poor choice of library for doing crypto work.