I am working on a TFTP server application. I managed to process a successful file transfer from server to client however the other way around is bugged.
Client instead of transmitting the entire file simply terminated whit compiler returning no errors. Debugger shows IOBE exception on the marked code referring that the array is out of range.
The whole transfer process goes like so:
Client transmits a file name and requested operation WRQ - Write Request
Server received the packet and determines the operation if WRQ is gives the new file appropriate name.
Server now starts executing receiveData() until it gets a packet < 512 indicationg EOT
Client keeps transferring data it read from the file.
Key code:
Client:
private void sendWRQ() throws Exception
{
String rrq = "WRQ-" + data;
outgoingData = rrq.getBytes();
DatagramPacket output = new DatagramPacket(outgoingData, outgoingData.length, serverAddress, serverPort);
clientSocket.send(output);
//Thread.sleep(50);
sendData();
}
byte outgoingData = new byte[512];
private void sendData() throws Exception
{
DatagramPacket dataTransfer = new DatagramPacket(outgoingData, outgoingData.length, serverAddress, serverPort);
InputStream fis = new FileInputStream(new File(data));
int x;
while((x = fis.read(outgoingData,0,512)) != -1) // << Debugged gives IOBE
{
dataTransfer.setLength(x);
clientSocket.send(dataTransfer);
Thread.sleep(5);
}
fis.close();
}
Server:
private void listen() throws Exception
{
DatagramPacket incTransfer = new DatagramPacket(incomingData, incomingData.length);
serverSocket.receive(incTransfer);
clientAddress = incTransfer.getAddress();
clientPort = incTransfer.getPort();
String output = new String(incTransfer.getData());
if(output.substring(0, 3).equals("RRQ"))
{
File test = new File(output.substring(4));
responseData = output.substring(4);
if(test.exists())
{
sendResponse("Y");
} else {
sendResponse("N");
}
} else if (output.substring(0, 3).equals("WRQ"))
{
File test = new File(output.substring(4));
if(test.exists())
{
Calendar cal = Calendar.getInstance();
SimpleDateFormat prefix = new SimpleDateFormat(date_format);
String date = prefix.format(cal.getTime()).toString();
responseData = date + output.substring(4);
receiveData();
} else {
responseData = output.substring(4);
receiveData();
}
}
}
private void receiveData() throws Exception
{
DatagramPacket receiveData = new DatagramPacket(incomingData, incomingData.length);
OutputStream fos = new FileOutputStream(new File(responseData));
while(true)
{
serverSocket.receive(receiveData);
if(receiveData.getLength() == 512)
{
fos.write(receiveData.getData());
} else {
fos.write(receiveData.getData(), receiveData.getOffset(), receiveData.getLength());
break;
}
}
fos.close();
}
The only way that can happen is if the offset or length parameters violate the constraints specified for InputStream.read(byte[], int, int); in this case probably the buffer isn't 512 bytes long. There's no need to specify the 2nd nd third parameters in this case, just omit them, then it becomes read(buffer, 0, buffer.length) internally, which can't be wrong.
Okay, the way this is coded, the 'outgoingData' field is:
1) Initialized to a length of 512
2) Then, in sendWRQ(), 'outgoingData' is re-initialized to whatever rrq.getBytes() sends back.
3) Then, in sendData(), 'outgoingData' is used as the intermediate buffer to read data from file and put it in the dataTransfer object.
However, since 'outgoingData' is re-initialized in step #2, the assumption in step #3 that 'outgoingData' is still 512 bytes in length is false.
So while EJP was correct in saying that using read(outgoingData, 0, outgoingData.length()) will work, there are some architecture issues that if you address, you'll clean up a lot of potential errors.
For instance:
WIth the code provided, there is seemingly no reason to have outgoingData declared at the class level and shared among two functions. Depending on the rest of the app, this could end up being a Threading issue.
Perhaps byte[] buffer = rrq.getBytes(); in sendWRQ() and byte[] buffer = new byte[1024]; in sendData().
Also, the 'data' parameter is at the class level.... for what reason? Might be better able to be controlled if its a passed in parameter.
Lastly, I've had good luck using the do{} while() loop in network situations. Ensures that the send() gets at least one chance to send the data AND it keeps the code a bit more readable.
Related
I'm trying to serialize Object between NIO SocketChannel and blocking IO Socket. Since I can't use Serializable/writeObject on NIO, I thought to write code to serialize object into an ByteArrayOutputStream then send array length followed by array.
Sender function is
public void writeObject(Object obj) throws IOException{
ByteArrayOutputStream serializedObj = new ByteArrayOutputStream();
ObjectOutputStream writer = new ObjectOutputStream(serializedObj);
writer.writeUnshared(obj);
ByteBuffer size = ByteBuffer.allocate(4).putInt(serializedObj.toByteArray().length);
this.getSocket().write(size);
this.getSocket().write(ByteBuffer.wrap(serializedObj.toByteArray()));
}
and receiver is:
public Object readObject(){
try {
//Leggi dimensione totale pacchetto
byte[] dimension = new byte[4];
int byteRead = 0;
while(byteRead < 4) {
byteRead += this.getInputStream().read(dimension, byteRead, 4 - byteRead);
}
int size = ByteBuffer.wrap(dimension).getInt(); /* (*) */
System.out.println(size);
byte[] object = new byte[size];
while(size > 0){
size -= this.getInputStream().read(object);
}
InputStream in = new ByteArrayInputStream(object, 0, object.length);
ObjectInputStream ois = new ObjectInputStream(in);
Object res = ois.readUnshared();
ois.close();
return res;
} catch (IOException | ClassNotFoundException e) {
return null;
}
}
The problem is that size (*) is always equals to -1393754107 when serializedObj.toByteArray().length in my test is 316.
I don't understand why casting not works properly.
this.getSocket().write(size);
this.getSocket().write(ByteBuffer.wrap(serializedObj.toByteArray()));
If the result of getSocket() is a SocketChannel in non-blocking mode, the problem is here. You aren't checking the result of write(). In non-blocking mode it can write less than the number of bytes remaining in the ByteBuffer; indeed it can write zero bytes.
So youu aren't writing all the data you think you're writing, so the other end overruns and reads the next length word as part of the data being written, and reads part of the next data as the next length word, and gets a wrong answer. I'm surprised it didn't barf earlier. In fact it probably did, but your deplorable practice of ignoring IOExceptions masked it. Don't do that. Log them.
So you need to loop until all requested data has been written, and if any write() returns zero you need to select on OP_WRITE until it fires, which adds a considerable complication into your code as you have to return to the select loop while remembering that there is an outstanding ByteBuffer with data remaining to be written. And when you get the OP_WRITE and the writes complete you have to deregister interest in OP_WRITE, as it's only of interest after a write() has returned zero.
NB There is no casting in your code.
The problem was write() returned 0 always. This happens because the buffer wasn't flipped before write().
I'm trying to write an upload system for a fairly complex java server. I have reproduced the error in the two small programs listed below. Basically, I am using an ObjectOutputStream/ObjectInputStream to communicate via the client/server. This is a requirement; I have thousands of lines of code working perfectly fine around this ObjectOutputStream/ObjectInputStream setup, so I must be able to still use these streams after an upload is complete.
To access the files(the one being read on the client and the one being written on the server), FileInputStream and FileOutputStream is used. My client appears to be functioning perfectly; it reads in the file and sends a different byte array each iteration(it reads in 1MB at a time, so large files can be handled without overflowing the heap). However, on the server it appears as though the byte array is ALWAYS just the first array sent(the first 1MB of the file). This does not conform to my understanding of ObjectInputStream/ObjectOutputStream. I am seeking either a working solution to this issue or enough education on the matter to form my own solution.
Below is the client code:
import java.net.*;
import java.io.*;
public class stupidClient
{
public static void main(String[] args)
{
new stupidClient();
}
public stupidClient()
{
try
{
Socket s = new Socket("127.0.0.1",2013);//connect
ObjectOutputStream output = new ObjectOutputStream(s.getOutputStream());//init stream
//file to be uploaded
File file = new File("C:\\Work\\radio\\upload\\(Op. 9) Nocturne No. 1 in Bb Minor.mp3");
long fileSize = file.length();
output.writeObject(file.getName() + "|" + fileSize);//send name and size to server
FileInputStream fis = new FileInputStream(file);//open file
byte[] buffer = new byte[1024*1024];//prepare 1MB buffer
int retVal = fis.read(buffer);//grab first MB of file
int counter = 0;//used to track progress through upload
while (retVal!=-1)//until EOF is reached
{
System.out.println(Math.round(100*counter/fileSize)+"%");//show current progress to system.out
counter += retVal;//track progress
output.writeObject("UPACK "+retVal);//alert server upload packet is incoming, with size of packet read
System.out.println(""+buffer[0]+" "+buffer[1]+" "+buffer[2]);//preview first 3 bytes being sent
output.writeObject(buffer);//send bytes
output.flush();//make sure all bytes read are gone
retVal = fis.read(buffer);//get next MB of file
}
System.out.println(Math.round(100*counter/fileSize)+"%");//show progress at end of file
output.writeObject("UPLOAD_COMPLETE");//let server know protocol is finished
output.close();
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
The following is my server code:
import java.net.*;
import java.io.*;
public class stupidServer
{
Socket s;
ServerSocket server;
public static void main(String[] args)
{
new stupidServer();
}
public stupidServer()
{
try
{
//establish connection and stream
server = new ServerSocket(2013);
s = server.accept();
ObjectInputStream input = new ObjectInputStream(s.getInputStream());
String[] args = ((String)input.readObject()).split("\\|");//args[0] will be file name, args[1] will be file size
String fileName = args[0];
long filesize = Long.parseLong(args[1]);
String upack = (String)input.readObject();//get upload packet(string reading UPACK [bytes read])
FileOutputStream outStream = new FileOutputStream("C:\\"+fileName.trim());
while (!upack.equalsIgnoreCase("UPLOAD_COMPLETE"))//until protocol is complete
{
int bytes = Integer.parseInt(upack.split(" ")[1]);//get number of bytes being written
byte[] buffer = new byte[bytes];
buffer = (byte[])input.readObject();//get bytes sent from client
outStream.write(buffer,0,bytes);//go ahead and write them bad boys to file
System.out.println(buffer[0]+" "+buffer[1]+" "+buffer[2]);//peek at first 3 bytes received
upack = (String)input.readObject();//get next 'packet' - either another UPACK or a UPLOAD_COMPLETE
}
outStream.flush();
outStream.close();//make sure all bytes are in file
input.close();//sign off
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
As always, many thanks for your time!
Your immediate problem is that ObjectOutputStream uses an ID mechanism to avoid sending the same object over the stream multiple times. The client will send this ID for the second and subsequent writes of buffer, and the server will use its cached value.
The solution to this immediate problem is to add a call to reset():
output.writeObject(buffer);//send bytes
output.reset(); // force buffer to be fully written on next pass through loop
That aside, you're misusing object streams by layering your own protocol on top of them. For example, writing the filename and filesize as a single string delimited by "|"; just write them as two separate values. Ditto for the number of bytes on each write.
I have the following infinite loop which listens for incoming messages:
public static void listenForMessages(){
while (true) {
dsocket.receive(receivepacket);
byte[] rcvMsg = receivepacket.getData();
MessageCreator tmc = new MessageCreator();
TrafficMessage message = tmc.constructMessageFromBinary(rcvMsg);
System.out.println("message: "+message);
}
}
This calls a method that reads the byte array into a string and populates a message object.
public Message constructMessageFromBinary(byte[] rcvMsg)
throws IOException {
ByteArrayInputStream bais = new ByteArrayInputStream(rcvMsg);
DataInputStream dis = new DataInputStream(bais);
StringBuffer inputLine = new StringBuffer();
String tmp;
while ((tmp = dis.readLine()) != null) {
inputLine.append(tmp);
}
dis.close();
Message message = new Message();
message.setDescriptions(tmp);
return message;
}
This simple process slowly leaks memory over a few hours and I receive an out of memory exception.
Is there anything wrong with this logic?
The problem was that I left a database connection open. I wanted to leave it open to pass data with out having to worry about stopping and starting connections. I now open and close connections each time and all is good.
The best bet here would be to move all possible object instantiations outside the loops. For example, in the first code snippet, every iteration creates a
MessageCreator tmc.
On your second snippet, each call to the method creates a
StringBuffer inputLine.
This instantiation process may be eating away your memory slowly.
I'm trying to read a webpage using following code :
URL url = new URL("somewebsitecomeshere");
URLConnection c = url.openConnection();
if(getHttpResponseCode(c) == 200)
{
if (isContentValid(c))//accept html/xml only!
{
InputStream is = c.getInputStream();
Reader r = new InputStreamReader(is);
System.out.println(r.toString());
//after commenting this everything works great!
setHTMLString(getStringFromReader(r));
System.out.println(getHTMLString());
ParserDelegator parser = new ParserDelegator();
parser.parse(r, new Parser(url), true);
r.close();
is.close();
try {
Thread.sleep(500);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
else
log("content is not valid!");
}
else
{
System.out.println("ERROR" + c.getContentType() + c.getURL());
}
//---------------------------------------------------
private String getStringFromReader(Reader reader) throws IOException {
char[] arr = new char[8*1024]; // 8K at a time
StringBuffer buf = new StringBuffer();
int numChars;
while ((numChars = reader.read(arr, 0, arr.length)) > 0) {
buf.append(arr, 0, numChars);
}
//Reset position to 0
reader.reset();
return buf.toString();
}
if try to read string using getStringFromReader() the rest of the code will be ignored due to changing position of Reader to EOF so I tried to reset the position to 0 but I got the following error :
java.io.IOException: reset() not supported
at java.io.Reader.reset(Unknown Source)
at sample.getStringFromReader(Spider.java:248)
at default(sample.java:286)
at default.main(sample.java:130)
How can I reset the Reader position to 0?
Short answer, your stream doesn't support reset or mark methods. Check the result of:
is.markSupported()
Long answer, an InputStream is a flow of bytes. Bytes can come from a file, a network resource, a string, etc. So basically, there are streams that don't support resetting the reader position to the start of the stream, while others do (random access file).
A stream from a web site will normally use underlying network connection to provide the data. It means that it's up to the underlying network protocol (TCP/IP for example) to support or not resetting the stream, and normally they don't.
In order to reset any stream you would have to know the entire flow, from start to end. Network communications send a bunch of packages (which may be in order or not) to transfer data. Packages may get lost or even be duplicated, so normally information is buffered and interpreted as it is received. It would be very expensive to reconstruct all messages at network level. So that is normally up to the receiver, if it wants to do that.
In your case If what you want is print the input stream I would recommend creating a custom InputStream, which receives the original InputStream and whenever it is read it prints the read value and returns it at the same time. For example:
class MyInputStream extends InputStream {
InputStream original = null;
public MyInputStream(InputStream original) {
this.original = original;
}
#Override
public int read() throws IOException {
int c = original.read();
System.out.printf("%c", c);
return c;
}
}
Then wrap your original InputStream with that:
.
.
.
InputStream myIs = new MyInputStream(is);
Reader r = new InputStreamReader(myIs);
.
.
.
Hope it helps.
InputStreamReader does not support reset(). Also, you did not call mark(0) before.
What you could do is wrap your reader in a BufferedReader of a sufficient size so that reset is supported. If you cannot do that, then you should try to open a new connection to your URL.
For my homework assignment, I have a network of Nodes that are passing messages to each other. Each Node is connected to a set amount of other Nodes (I'm using 4 for testing). Each Link has a weight, and all the Nodes have computed the shortest path for how they want their messages sent. Every Packet that is sent is composed of the message protocol (a hard-coded int), an integer that tells how many messages have passed through the sending Node, and the routing path for the Packet.
Every Node has a Thread for each of its Links. There is an active Socket in each Link. The Packets are sent by adding a 4-byte int to the beginning of the message telling the message's length.
Everything works fine until I stress the network. For my test, there are 10 Nodes, and I get 5 of them to send 10000 packets in a simple while() loop with no Thread.sleep(). Without exception, there is always an error at some point during execution at the if(a!=len) statement.
Please let me know if I can clarify anything. Thanks in advance! Here is the code (from the Link Thread; send() and forward() are called from the Node itself):
protected void listen(){
byte[] b;
int len;
try{
DataInputStream in = new DataInputStream(sock.getInputStream());
while(true){
len = in.readInt();
b = new byte[len];
int a = in.read(b,0,len);
if(a!=len){
System.out.println("ERROR: " + a + "!=" + len);
throw new SocketException(); //may have to fix...this will happen when message is corrupt/incomplete
}
Message m = new Message(b);
int p = m.getProtocol();
switch (p){
case CDNP.PACKET:
owner.incrementTracker();
System.out.print("\n# INCOMMING TRACKER: " + m.getTracker() + "\n>>> ");
owner.forward(m);
}
}
}catch (IOException e){
e.printStackTrace();
}
}
public void send(int tracker){
String[] message = { Conv.is(CDNP.PACKET), Conv.is(tracker), owner.getMST().toString() };
Message m = new Message(message);
forward(m);
}
public synchronized void forward(Message m){
try{
OutputStream out = sock.getOutputStream();
//convert length to byte array of length 4
ByteBuffer bb = ByteBuffer.allocate(4+m.getLength());
bb.putInt(m.getLength());
bb.put(m.getBytes());
out.write(bb.array());
out.flush();
}catch (UnknownHostException e){
System.out.println("ERROR: Could not send to Router at " + sock.getRemoteSocketAddress().toString());
return;
}catch (IOException e1){
}
}
int a = in.read(b,0,len);
if(a!=len){
That won't work. The InputStream may not read all the bytes you want, it may read only what is available right now, and return that much without blocking.
To quote the Javadocs (emphasis mine):
Reads up to len bytes of data from the input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read, possibly zero. The number of bytes actually read is returned as an integer.
You need to continue reading in a loop until you have all the data you want (or the stream is finished).
Or, since you are using a DataInputStream, you can also use
in.readFully(b, 0, len);
which always reads exactly len bytes (blocking until those have arrived, throwing an exception when there is not enough data).