I'm trying to write an upload system for a fairly complex java server. I have reproduced the error in the two small programs listed below. Basically, I am using an ObjectOutputStream/ObjectInputStream to communicate via the client/server. This is a requirement; I have thousands of lines of code working perfectly fine around this ObjectOutputStream/ObjectInputStream setup, so I must be able to still use these streams after an upload is complete.
To access the files(the one being read on the client and the one being written on the server), FileInputStream and FileOutputStream is used. My client appears to be functioning perfectly; it reads in the file and sends a different byte array each iteration(it reads in 1MB at a time, so large files can be handled without overflowing the heap). However, on the server it appears as though the byte array is ALWAYS just the first array sent(the first 1MB of the file). This does not conform to my understanding of ObjectInputStream/ObjectOutputStream. I am seeking either a working solution to this issue or enough education on the matter to form my own solution.
Below is the client code:
import java.net.*;
import java.io.*;
public class stupidClient
{
public static void main(String[] args)
{
new stupidClient();
}
public stupidClient()
{
try
{
Socket s = new Socket("127.0.0.1",2013);//connect
ObjectOutputStream output = new ObjectOutputStream(s.getOutputStream());//init stream
//file to be uploaded
File file = new File("C:\\Work\\radio\\upload\\(Op. 9) Nocturne No. 1 in Bb Minor.mp3");
long fileSize = file.length();
output.writeObject(file.getName() + "|" + fileSize);//send name and size to server
FileInputStream fis = new FileInputStream(file);//open file
byte[] buffer = new byte[1024*1024];//prepare 1MB buffer
int retVal = fis.read(buffer);//grab first MB of file
int counter = 0;//used to track progress through upload
while (retVal!=-1)//until EOF is reached
{
System.out.println(Math.round(100*counter/fileSize)+"%");//show current progress to system.out
counter += retVal;//track progress
output.writeObject("UPACK "+retVal);//alert server upload packet is incoming, with size of packet read
System.out.println(""+buffer[0]+" "+buffer[1]+" "+buffer[2]);//preview first 3 bytes being sent
output.writeObject(buffer);//send bytes
output.flush();//make sure all bytes read are gone
retVal = fis.read(buffer);//get next MB of file
}
System.out.println(Math.round(100*counter/fileSize)+"%");//show progress at end of file
output.writeObject("UPLOAD_COMPLETE");//let server know protocol is finished
output.close();
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
The following is my server code:
import java.net.*;
import java.io.*;
public class stupidServer
{
Socket s;
ServerSocket server;
public static void main(String[] args)
{
new stupidServer();
}
public stupidServer()
{
try
{
//establish connection and stream
server = new ServerSocket(2013);
s = server.accept();
ObjectInputStream input = new ObjectInputStream(s.getInputStream());
String[] args = ((String)input.readObject()).split("\\|");//args[0] will be file name, args[1] will be file size
String fileName = args[0];
long filesize = Long.parseLong(args[1]);
String upack = (String)input.readObject();//get upload packet(string reading UPACK [bytes read])
FileOutputStream outStream = new FileOutputStream("C:\\"+fileName.trim());
while (!upack.equalsIgnoreCase("UPLOAD_COMPLETE"))//until protocol is complete
{
int bytes = Integer.parseInt(upack.split(" ")[1]);//get number of bytes being written
byte[] buffer = new byte[bytes];
buffer = (byte[])input.readObject();//get bytes sent from client
outStream.write(buffer,0,bytes);//go ahead and write them bad boys to file
System.out.println(buffer[0]+" "+buffer[1]+" "+buffer[2]);//peek at first 3 bytes received
upack = (String)input.readObject();//get next 'packet' - either another UPACK or a UPLOAD_COMPLETE
}
outStream.flush();
outStream.close();//make sure all bytes are in file
input.close();//sign off
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
As always, many thanks for your time!
Your immediate problem is that ObjectOutputStream uses an ID mechanism to avoid sending the same object over the stream multiple times. The client will send this ID for the second and subsequent writes of buffer, and the server will use its cached value.
The solution to this immediate problem is to add a call to reset():
output.writeObject(buffer);//send bytes
output.reset(); // force buffer to be fully written on next pass through loop
That aside, you're misusing object streams by layering your own protocol on top of them. For example, writing the filename and filesize as a single string delimited by "|"; just write them as two separate values. Ditto for the number of bytes on each write.
Related
I wrote a piece of Java code to send PDF-turned postscript scripts to a network printer via Socket.
The files were printed in perfect shape but every job comes with one or 2 extra pages with texts like ps: stack underflow or error undefined offending command.
At beginning I thought something is wrong with the PDF2PS process so I tried 2 PS files from this PS Files. But the problem is still there.
I also verified the ps files with GhostView. Now I think there may be something wrong with the code. The code does not throw any exception.
The printer, Toshiba e-studion 5005AC, supports PS3 and PCL6.
File file = new File("/path/to/my.ps");
Socket socket = null;
DataOutputStream out = null;
FileInputStream inputStream = null;
try {
socket = new Socket(printerIP, printerPort);
out = new DataOutputStream(socket.getOutputStream());
DataInputStream input = new DataInputStream(socket.getInputStream());
inputStream = new FileInputStream(file);
byte[] buffer = new byte[8000];
while (inputStream.read(buffer) != -1) {
out.write(buffer);
}
out.flush();
} catch (IOException e) {
e.printStackTrace();
}
You are writing the whole buffer to the output stream regardless of how much actual content there is.
That means that when you write the buffer the last time it will most probably have a bunch of content from the previous iteration at the end of the buffer.
Example
e.g. imagine you have the following file and you use a buffer of size 10:
1234567890ABCDEF
After first inputStream.read() call it will return 10 and in the buffer you will have:
1234567890
After second inputStream.read() call it will return 6 and in the buffer you will have:
ABCDEF7890
After third inputStream.read() call it will return -1 and you will stop reading.
A printer socket will receive these data in the end:
1234567890ABCDEF7890
Here the last 7890 is an extra bit that the printer does not understand, but it can successfully interpret the first 1234567890ABCDEF.
Fix
You should consider the length returned by inputStream.read():
byte[] buffer = new byte[8000];
for (int length; (length = inputStream.read(buffer)) != -1; ){
out.write(buffer, 0, length);
}
Also consider using try-with-resources to avoid problems with unclosed streams.
I have written a program to transfer a file from client side to server side. The program runs without any errors. Also the file is created on the destination path. But when i try to open the file the gedit crashes. The program should transfer audio,video,text file.I know UDP should not be used to transfer file as it may be corrupted but i have to do it as assignment. Following code is for text file transfer.
Server.java
import java.net.*;
import java.io.*;
public class server
{
public static void main(String args[])throws IOException
{
DatagramSocket dsoc=new DatagramSocket(8008);
byte b[]=new byte[787734];
byte c[];
String pac ;
File file = new File("/home/san_16398/ip2.java");
FileOutputStream f = new FileOutputStream(file);
while(true)
{
DatagramPacket dp=new DatagramPacket(b,b.length);
dsoc.receive(dp);
pac =new String(dp.getData(),0,dp.getLength());
System.out.println(pac);
}
pac.getBytes();
f.write(b);
f.flush();
f.close();
}
}
Client.java
import java.net.*;
import java.io.*;
public class client
{
private static final int BUFFER_SIZE = 4096;
public static void main(String args[])throws Exception
{
byte b[]=new byte[787734];
FileInputStream f=new FileInputStream("/home/san_16398/ip.java");
DatagramSocket dsoc=new DatagramSocket(2000);
int i=0;
byte[] buffer = new byte[BUFFER_SIZE];
while(f.read(buffer) != -1)
{
b[i]=(byte)f.read();
i++;
}
f.close();
dsoc.send(new DatagramPacket(b,i,InetAddress.getLocalHost(),8008));
}
}
byte b[]=new byte[787734];
// ...
while(f.read(buffer) != -1){
b[i]=(byte)f.read();
i++;
}
dsoc.send(new DatagramPacket(b,i,InetAddress.getLocalHost(),8008));
This is complete and utter nonsense. You're reading up to 787734 bytes and throwing them away, and then reading one byte, if there is one, and then apparently attempting to send a datagram of size up to 787734, which is impossible. It should be:
byte b[]=new byte[534];
// ...
int count;
while((count = f.read(b)) != -1){
dsoc.send(new DatagramPacket(b,count,InetAddress.getLocalHost(),8008));
}
and then you have to take care of duplicate datagrams, datagrams not received, datagrams received out of order, ...
And calling File.exists() and File.createNewFile() when you're using a FileOutputStream is a complete waste of time at best, but when you call them after creating the FileOutputStream you are liable to clobber the file that is being written to, depending on the platform. Remove.
Here:
while(f.read(buffer) != -1){
b[i]=(byte)f.read();
first you are reading into your buffer object - to then never make use of the buffer array
and then you read a single byte again
to then read into the buffer again
Long story short: either use the read(buffer) or use the single byte read() consistently.
And beyond that: you also forgot to check that the file you intend to read fits into the buffer that you want to transmit to the server!
Finally: you are also trying to do *too many** things at once. Instead: focus on the different parts of this work, like:
first write code that reads a file
then enhance that to create a local copy of that file
and then, when that works, instead of creating a local copy, send to a remote server.
Env: Windows 7, java 1.8, default OS encodings
I'm trying to read a byte stream of currency market data from a socket to a file, and then play that file back to simulate the market over a fixed period; however, the file has a few malformed bytes, seemingly at random.
Below, I outline the problem with metacode, where the notation "..." indicates skipped irrelevant or boilerplate code.
Bytes are coming over the socket, and I'm reading them with a non-blockingNIO selector, then writing to disk via BufferedOutputStream:
class SocketReadDiskWrite implements Runnable{
...
blobWriter = new BufferedOutputStream(new FileOutputStream(blobFileName));
sc = SocketChannel.open(addr)
sc.configureBlocking(false);
And then in the run() method, when the selector deems the socket readable,
public void run(){
...
while(keyIterator.hasNext())
{
SelectionKey key = keyIterator.next();
if (key.isReadable()) {
if(bytesRead == -1)
{
connected = false;
logger.warn("no bytes to read");
break;
}
readBuffer.flip();
// Write bytes from socket to file, then rewind and process data
while (readBuffer.hasRemaining()){
byte[] b = new byte[readBuffer.remaining()];
readBuffer.get(b);
blobWriter.write(b);
}
readBuffer.rewind();
processData(readBuffer); //<-- Further processing
...
}
The processData method works fine when reading from a live stream of the market. For example, maybe processData reads a list of currencies and prints them, and the output is,
`EUR.USD.SPOT, EUR.AUD.SPOT, ..<thousands more>.. AUD.CAD.SPOT`
However, if I instead try to play back the captured bytestream (ie. Read in the contents of the file that was just previously created), on occasion, a corrupt symbol appears,
`EUR.USD.SPOT, EUR.AUD.SPOT, ..<thousands more>.. AUD.C##$###X`
Looking at the file in notepad++, indeed I find incorrect bytes (blue = correct symbols, red = malformed).
Subsequently, when the application points to the bytefile reader (instead of live market), the app fails at exactly these lines, throwing errors like Invalid symbol: EUR.-XD##O##$.
For what it's worth, this is how I playback the file by reading it from disk and streaming to socket:
class FilePlayer implements runnable (Socket clientSocket) {
clientWriter= clientSocket.getOutputStream();
blobReader = new FileInputStream(blobFileName);
byte[] dataArray = new byte[1024]; //<-- Store 1024 bytes data at a time
...
}
public void run() {
while(true){
blobReader.read(dataArray); //<-- Read 1024 bytes of data from disk
clientWriter.write(dataArray); //<-- Write 1024 bytes of data to socket
}
}
Note, I recently opened a related thread similar thread, but that was in regard to FileChannels, which were actually not the culprit. Figured that discussion had deviated enough to warrant a fresh post.
I'm trying to make a simple transfer of a text .txt file from client to server, and no matter how much I think I know, and understand what I'm doing, and what exactly happening, I always get it wrong. I can really use some help here please.
So, this is the code, two function that transfer a .txt file from one to another:
Client side:
private void sendFileToServer(String file_name) throws IOException {
File file=new File(file_name);
int file_size=(int)file.length();
byte[] bytes=new byte[file_size];
FileInputStream os=null;
try {
os = new FileInputStream(file);
} catch (FileNotFoundException e) {
System.out.println("The file "+file+" wasn't found");
return;
}
BufferedInputStream bos=new BufferedInputStream(os);
bos.read(bytes);
output.write(bytes,0,bytes.length);
/* 'output' is a PrintStream object, that holds the output stream
* for the client's socket, meaning:
* output=new PrintStream(client_socket.getOutputStream()); */
output.flush();
bos.close();
}
this will buffer everything into BufferedInputStream, will copy it to bytes and will then send it to the other side - the server.
Server side:
public static String receiveFileFromClient(Client client) throws IOException {
int buffer_size=client.getSocket().getReceiveBufferSize();
byte[] bytes=new byte[buffer_size];
FileOutputStream fos=new FileOutputStream("transfered_file.txt");
BufferedOutputStream bos=new BufferedOutputStream(fos);
DataInputStream in=client.getInputStream();
int count;
System.out.println("this will be printed out");
while ((count=in.read(bytes))>0) { // execution is blocked here!
bos.write(bytes, 0, count);
}
System.out.println("this will not be printed");
bos.flush();
bos.close();
return "transfered_file.txt";
}
My intention here is to keep reading bytes from the client (the while loop), until the other side (the client) have no more bytes to send, and this is where in.read(bytes) should return 0 and the loop should break, but this is never happens, it just get blocked, even though all the bytes from the client's input-stream were successfully transferred!
Why doesn't the loop breaks?
From Javadoc:
If no byte is available because the stream is at end of file, the
value -1 is returned
doesn't the last byte is considered "end of file"? I made sure that the function sendFileToServer properly writes the entire file to the output instance (PrintStream object) and returns.
Any help would be appreciated.
As i understand it, the read() method will block until either it read[bytes] OR the socket is closed. So there is nothing for the read() what would indicate that it should stop reading, because it does not "understand" the file, its just some data.
A solution...
You could determine the number of bytes the client will send (on the client side) and then send the NUMBER over to the server. Now the server can process this number and knows how many bytes to read before the file is complete. So you can break the loop (or even don't use a loop) when the transfer is completed.
You could also process the data the server receives, and let the client send some "flag" after the file is complete, so the server knows when it is done. But this is more difficult, because you have to find something, that is not contained in the file-byte data
read() method will block for further input if you dont close the stream. So eather close the stream, or remove the loop and only read the number of bytes, you receive from the client
I am working on a TFTP server application. I managed to process a successful file transfer from server to client however the other way around is bugged.
Client instead of transmitting the entire file simply terminated whit compiler returning no errors. Debugger shows IOBE exception on the marked code referring that the array is out of range.
The whole transfer process goes like so:
Client transmits a file name and requested operation WRQ - Write Request
Server received the packet and determines the operation if WRQ is gives the new file appropriate name.
Server now starts executing receiveData() until it gets a packet < 512 indicationg EOT
Client keeps transferring data it read from the file.
Key code:
Client:
private void sendWRQ() throws Exception
{
String rrq = "WRQ-" + data;
outgoingData = rrq.getBytes();
DatagramPacket output = new DatagramPacket(outgoingData, outgoingData.length, serverAddress, serverPort);
clientSocket.send(output);
//Thread.sleep(50);
sendData();
}
byte outgoingData = new byte[512];
private void sendData() throws Exception
{
DatagramPacket dataTransfer = new DatagramPacket(outgoingData, outgoingData.length, serverAddress, serverPort);
InputStream fis = new FileInputStream(new File(data));
int x;
while((x = fis.read(outgoingData,0,512)) != -1) // << Debugged gives IOBE
{
dataTransfer.setLength(x);
clientSocket.send(dataTransfer);
Thread.sleep(5);
}
fis.close();
}
Server:
private void listen() throws Exception
{
DatagramPacket incTransfer = new DatagramPacket(incomingData, incomingData.length);
serverSocket.receive(incTransfer);
clientAddress = incTransfer.getAddress();
clientPort = incTransfer.getPort();
String output = new String(incTransfer.getData());
if(output.substring(0, 3).equals("RRQ"))
{
File test = new File(output.substring(4));
responseData = output.substring(4);
if(test.exists())
{
sendResponse("Y");
} else {
sendResponse("N");
}
} else if (output.substring(0, 3).equals("WRQ"))
{
File test = new File(output.substring(4));
if(test.exists())
{
Calendar cal = Calendar.getInstance();
SimpleDateFormat prefix = new SimpleDateFormat(date_format);
String date = prefix.format(cal.getTime()).toString();
responseData = date + output.substring(4);
receiveData();
} else {
responseData = output.substring(4);
receiveData();
}
}
}
private void receiveData() throws Exception
{
DatagramPacket receiveData = new DatagramPacket(incomingData, incomingData.length);
OutputStream fos = new FileOutputStream(new File(responseData));
while(true)
{
serverSocket.receive(receiveData);
if(receiveData.getLength() == 512)
{
fos.write(receiveData.getData());
} else {
fos.write(receiveData.getData(), receiveData.getOffset(), receiveData.getLength());
break;
}
}
fos.close();
}
The only way that can happen is if the offset or length parameters violate the constraints specified for InputStream.read(byte[], int, int); in this case probably the buffer isn't 512 bytes long. There's no need to specify the 2nd nd third parameters in this case, just omit them, then it becomes read(buffer, 0, buffer.length) internally, which can't be wrong.
Okay, the way this is coded, the 'outgoingData' field is:
1) Initialized to a length of 512
2) Then, in sendWRQ(), 'outgoingData' is re-initialized to whatever rrq.getBytes() sends back.
3) Then, in sendData(), 'outgoingData' is used as the intermediate buffer to read data from file and put it in the dataTransfer object.
However, since 'outgoingData' is re-initialized in step #2, the assumption in step #3 that 'outgoingData' is still 512 bytes in length is false.
So while EJP was correct in saying that using read(outgoingData, 0, outgoingData.length()) will work, there are some architecture issues that if you address, you'll clean up a lot of potential errors.
For instance:
WIth the code provided, there is seemingly no reason to have outgoingData declared at the class level and shared among two functions. Depending on the rest of the app, this could end up being a Threading issue.
Perhaps byte[] buffer = rrq.getBytes(); in sendWRQ() and byte[] buffer = new byte[1024]; in sendData().
Also, the 'data' parameter is at the class level.... for what reason? Might be better able to be controlled if its a passed in parameter.
Lastly, I've had good luck using the do{} while() loop in network situations. Ensures that the send() gets at least one chance to send the data AND it keeps the code a bit more readable.