I'm trying to make a simple transfer of a text .txt file from client to server, and no matter how much I think I know, and understand what I'm doing, and what exactly happening, I always get it wrong. I can really use some help here please.
So, this is the code, two function that transfer a .txt file from one to another:
Client side:
private void sendFileToServer(String file_name) throws IOException {
File file=new File(file_name);
int file_size=(int)file.length();
byte[] bytes=new byte[file_size];
FileInputStream os=null;
try {
os = new FileInputStream(file);
} catch (FileNotFoundException e) {
System.out.println("The file "+file+" wasn't found");
return;
}
BufferedInputStream bos=new BufferedInputStream(os);
bos.read(bytes);
output.write(bytes,0,bytes.length);
/* 'output' is a PrintStream object, that holds the output stream
* for the client's socket, meaning:
* output=new PrintStream(client_socket.getOutputStream()); */
output.flush();
bos.close();
}
this will buffer everything into BufferedInputStream, will copy it to bytes and will then send it to the other side - the server.
Server side:
public static String receiveFileFromClient(Client client) throws IOException {
int buffer_size=client.getSocket().getReceiveBufferSize();
byte[] bytes=new byte[buffer_size];
FileOutputStream fos=new FileOutputStream("transfered_file.txt");
BufferedOutputStream bos=new BufferedOutputStream(fos);
DataInputStream in=client.getInputStream();
int count;
System.out.println("this will be printed out");
while ((count=in.read(bytes))>0) { // execution is blocked here!
bos.write(bytes, 0, count);
}
System.out.println("this will not be printed");
bos.flush();
bos.close();
return "transfered_file.txt";
}
My intention here is to keep reading bytes from the client (the while loop), until the other side (the client) have no more bytes to send, and this is where in.read(bytes) should return 0 and the loop should break, but this is never happens, it just get blocked, even though all the bytes from the client's input-stream were successfully transferred!
Why doesn't the loop breaks?
From Javadoc:
If no byte is available because the stream is at end of file, the
value -1 is returned
doesn't the last byte is considered "end of file"? I made sure that the function sendFileToServer properly writes the entire file to the output instance (PrintStream object) and returns.
Any help would be appreciated.
As i understand it, the read() method will block until either it read[bytes] OR the socket is closed. So there is nothing for the read() what would indicate that it should stop reading, because it does not "understand" the file, its just some data.
A solution...
You could determine the number of bytes the client will send (on the client side) and then send the NUMBER over to the server. Now the server can process this number and knows how many bytes to read before the file is complete. So you can break the loop (or even don't use a loop) when the transfer is completed.
You could also process the data the server receives, and let the client send some "flag" after the file is complete, so the server knows when it is done. But this is more difficult, because you have to find something, that is not contained in the file-byte data
read() method will block for further input if you dont close the stream. So eather close the stream, or remove the loop and only read the number of bytes, you receive from the client
Related
Env: Windows 7, java 1.8, default OS encodings
I'm trying to read a byte stream of currency market data from a socket to a file, and then play that file back to simulate the market over a fixed period; however, the file has a few malformed bytes, seemingly at random.
Below, I outline the problem with metacode, where the notation "..." indicates skipped irrelevant or boilerplate code.
Bytes are coming over the socket, and I'm reading them with a non-blockingNIO selector, then writing to disk via BufferedOutputStream:
class SocketReadDiskWrite implements Runnable{
...
blobWriter = new BufferedOutputStream(new FileOutputStream(blobFileName));
sc = SocketChannel.open(addr)
sc.configureBlocking(false);
And then in the run() method, when the selector deems the socket readable,
public void run(){
...
while(keyIterator.hasNext())
{
SelectionKey key = keyIterator.next();
if (key.isReadable()) {
if(bytesRead == -1)
{
connected = false;
logger.warn("no bytes to read");
break;
}
readBuffer.flip();
// Write bytes from socket to file, then rewind and process data
while (readBuffer.hasRemaining()){
byte[] b = new byte[readBuffer.remaining()];
readBuffer.get(b);
blobWriter.write(b);
}
readBuffer.rewind();
processData(readBuffer); //<-- Further processing
...
}
The processData method works fine when reading from a live stream of the market. For example, maybe processData reads a list of currencies and prints them, and the output is,
`EUR.USD.SPOT, EUR.AUD.SPOT, ..<thousands more>.. AUD.CAD.SPOT`
However, if I instead try to play back the captured bytestream (ie. Read in the contents of the file that was just previously created), on occasion, a corrupt symbol appears,
`EUR.USD.SPOT, EUR.AUD.SPOT, ..<thousands more>.. AUD.C##$###X`
Looking at the file in notepad++, indeed I find incorrect bytes (blue = correct symbols, red = malformed).
Subsequently, when the application points to the bytefile reader (instead of live market), the app fails at exactly these lines, throwing errors like Invalid symbol: EUR.-XD##O##$.
For what it's worth, this is how I playback the file by reading it from disk and streaming to socket:
class FilePlayer implements runnable (Socket clientSocket) {
clientWriter= clientSocket.getOutputStream();
blobReader = new FileInputStream(blobFileName);
byte[] dataArray = new byte[1024]; //<-- Store 1024 bytes data at a time
...
}
public void run() {
while(true){
blobReader.read(dataArray); //<-- Read 1024 bytes of data from disk
clientWriter.write(dataArray); //<-- Write 1024 bytes of data to socket
}
}
Note, I recently opened a related thread similar thread, but that was in regard to FileChannels, which were actually not the culprit. Figured that discussion had deviated enough to warrant a fresh post.
I have a DataOutputStream I would like to copy into a string. I've found a lot of tutorials on converting DataOutputStreams by setting it to a new ByteArrayOutputStream, but I just want to read the string it sends when it flushes, and my DataOutputStream is already assigned to an output stream though a socket.
output.writeUTF(input.readLine());
output.flush();
If the context is helpful, I'm trying to read the output stream of a server and compare it to a string.
the flush method will flush, i.e. force write, anything buffered, but not yet written.
In the code below, try putting a breakpoint on the second call to writeUTF - if you navigate to your file system you should see the created file, and it will contain "some string". If you put the break point on flush, you can verify that the content has already been written to file.
public static void test() throws IOException {
File file = new File("/Users/Hervian/tmp/fileWithstrings.txt");
DataOutputStream dos = null;
try {
dos = new DataOutputStream(new FileOutputStream(file));
dos.writeUTF("some string");
dos.writeUTF("some other string");
dos.flush();//Flushes this data output stream. This forces any buffered output bytes to be written out to the stream.
} finally {
if (dos!=null) dos.close();
}
}
As such, you cannot extract the data from the DataOutputStream object, but in the example above we off course have those strings in the write calls.
I'm creating a Server application in Java but when client connects to a server and opens a stream, the stream come to and end the connection is lost. What I need is to keep that connection alive even when the stream has ended. Here is a code example to better explain what I'm saying:
diSTR = new DataInputStream(Conexao.getInputStream());
doSTR = new DataOutputStream(Conexao.getOutputStream());
conectado = true;
while (diSTR.available() > 0)
{
byte[] buffer = new byte[size];
diSTR.readFully(buffer);
String str = new String(buffer, "UTF-8");
log(str);
}
So when diSTR.available() returns 0 the method returns and the connection is over, how can I solve this problem?
So when diSTR.available() = 0 the method returns and the connection is over, how can I solve this problem?
The solution is to NOT use available().
That method tells you how many bytes are available to read right now without blocking. If you use this to tell you "the connection is over", then you will get a premature end if the other end or the network cannot keep up with the rate at which you can read and process the data. Even if the other end can keep up, all it takes is a brief networking disruption for the reader to catch up, and the connection to be "over" ... according to your criterion.
The correct way to do this is to just read on the input stream until the read call returns -1. That means "end of stream" and indicates that the other end has closed, and there won't be any more data.
You should probably use the java.net package, here's documentation for socket connections:
http://docs.oracle.com/javase/tutorial/networking/sockets/
You are misusing InputStream.available(). The available() call only tells you how many bytes you can read without blocking. It doesn't tell you if you have reached the end of the stream. It is common that an inputstream may have 0 bytes to read immediately but still be open.
Your while loop can be reconstructed like this
int count;
byte[] buffer = new byte[4096];
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while((count = diSTR.read(buffer)) != -1){
baos.write(buffer, 0, count);
}
String str = new String(baos.toByteArray(), "UTF-8")
log(sb.toString());
InputStream.read(byte[]) will read bytes and return the number of bytes read or -1 when the end of stream is reached. Each time read() returns, the contents of the buffer are written to a ByteArrayOutputStream. Once all the bytes have been read (read(byte[]) returns -1) the contents of the stream can then be interpreted as a UTF-8 encoded String.
Is there a way to ask a DataInputStream, if it has content to read? .readByte() will just hang it, waiting for a byte to be read :( Or do I always have to send a Dummy-Byte, to make sure it always sees something?
dis.available();
Returns:
an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking.
Is this what you looking for?
also check answers here. You might get even more informations. "available" of DataInputStream from Socket
Look at
public int available() throws IOException
according to docs it "Returns an estimate of the number of bytes that can be read"
so you should call dis.available()
When reading past the end of the file, an EOFException is thrown. So you can tell there's no more data to read. For examle:
DataInputStream inputStream = new DataInputStream(new FileInputStream(file));
int data = 0;
try {
while (true) {
data += inputStream.readInt();
}
} catch (EOFException e) {
System.out.println("All data were read");
System.out.println(data);
}
I'm trying to write an upload system for a fairly complex java server. I have reproduced the error in the two small programs listed below. Basically, I am using an ObjectOutputStream/ObjectInputStream to communicate via the client/server. This is a requirement; I have thousands of lines of code working perfectly fine around this ObjectOutputStream/ObjectInputStream setup, so I must be able to still use these streams after an upload is complete.
To access the files(the one being read on the client and the one being written on the server), FileInputStream and FileOutputStream is used. My client appears to be functioning perfectly; it reads in the file and sends a different byte array each iteration(it reads in 1MB at a time, so large files can be handled without overflowing the heap). However, on the server it appears as though the byte array is ALWAYS just the first array sent(the first 1MB of the file). This does not conform to my understanding of ObjectInputStream/ObjectOutputStream. I am seeking either a working solution to this issue or enough education on the matter to form my own solution.
Below is the client code:
import java.net.*;
import java.io.*;
public class stupidClient
{
public static void main(String[] args)
{
new stupidClient();
}
public stupidClient()
{
try
{
Socket s = new Socket("127.0.0.1",2013);//connect
ObjectOutputStream output = new ObjectOutputStream(s.getOutputStream());//init stream
//file to be uploaded
File file = new File("C:\\Work\\radio\\upload\\(Op. 9) Nocturne No. 1 in Bb Minor.mp3");
long fileSize = file.length();
output.writeObject(file.getName() + "|" + fileSize);//send name and size to server
FileInputStream fis = new FileInputStream(file);//open file
byte[] buffer = new byte[1024*1024];//prepare 1MB buffer
int retVal = fis.read(buffer);//grab first MB of file
int counter = 0;//used to track progress through upload
while (retVal!=-1)//until EOF is reached
{
System.out.println(Math.round(100*counter/fileSize)+"%");//show current progress to system.out
counter += retVal;//track progress
output.writeObject("UPACK "+retVal);//alert server upload packet is incoming, with size of packet read
System.out.println(""+buffer[0]+" "+buffer[1]+" "+buffer[2]);//preview first 3 bytes being sent
output.writeObject(buffer);//send bytes
output.flush();//make sure all bytes read are gone
retVal = fis.read(buffer);//get next MB of file
}
System.out.println(Math.round(100*counter/fileSize)+"%");//show progress at end of file
output.writeObject("UPLOAD_COMPLETE");//let server know protocol is finished
output.close();
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
The following is my server code:
import java.net.*;
import java.io.*;
public class stupidServer
{
Socket s;
ServerSocket server;
public static void main(String[] args)
{
new stupidServer();
}
public stupidServer()
{
try
{
//establish connection and stream
server = new ServerSocket(2013);
s = server.accept();
ObjectInputStream input = new ObjectInputStream(s.getInputStream());
String[] args = ((String)input.readObject()).split("\\|");//args[0] will be file name, args[1] will be file size
String fileName = args[0];
long filesize = Long.parseLong(args[1]);
String upack = (String)input.readObject();//get upload packet(string reading UPACK [bytes read])
FileOutputStream outStream = new FileOutputStream("C:\\"+fileName.trim());
while (!upack.equalsIgnoreCase("UPLOAD_COMPLETE"))//until protocol is complete
{
int bytes = Integer.parseInt(upack.split(" ")[1]);//get number of bytes being written
byte[] buffer = new byte[bytes];
buffer = (byte[])input.readObject();//get bytes sent from client
outStream.write(buffer,0,bytes);//go ahead and write them bad boys to file
System.out.println(buffer[0]+" "+buffer[1]+" "+buffer[2]);//peek at first 3 bytes received
upack = (String)input.readObject();//get next 'packet' - either another UPACK or a UPLOAD_COMPLETE
}
outStream.flush();
outStream.close();//make sure all bytes are in file
input.close();//sign off
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
As always, many thanks for your time!
Your immediate problem is that ObjectOutputStream uses an ID mechanism to avoid sending the same object over the stream multiple times. The client will send this ID for the second and subsequent writes of buffer, and the server will use its cached value.
The solution to this immediate problem is to add a call to reset():
output.writeObject(buffer);//send bytes
output.reset(); // force buffer to be fully written on next pass through loop
That aside, you're misusing object streams by layering your own protocol on top of them. For example, writing the filename and filesize as a single string delimited by "|"; just write them as two separate values. Ditto for the number of bytes on each write.