I am writing a Java program using sockets to receive data from a C program.
The C program mallocs an integer array to fit the number of elements to be put inside. For example: {111,2,2,2,3,3} (111 is just a message header), so the int[] size is 6. (We free the array after sending is completed with a return value of 0 which is success). We print out the contents of the array, everything is displayed as expected: 111,2,2,2,3,3
However, we realize that in Java, we need to add a minor delay before reading from input stream, otherwise we can't get the correct values. E.g. if we don't put a Thread.sleep(2000) before input stream available, the value Java receive is like 111,0,0,0,3,3 (e.g. the value 2 is lost)
1) Does it affect Java if I send int array in C and ask Java to read int by int?
2) Why does the delay make the data accurate? We try playing around with 1000,1500 but only 2000 gives the most stable result
3) If in SomeAction.class, i put busy wait:
while (!pcClient.readMessage());
or
while (!pcClient.readMessage()) {}
it only go the while loop once and just break out? Whereas if I do this below , it works as intended:
while (!pcClient.readMessage()) {System.out.print("");}
1) Does it affect Java if I send int array in C and ask Java to read int by int?
When you ask Java to read it int by int, it basically waits for next 4 bytes available and builds an int value out of these 4 bytes
Looks like your Java client starts reading data before your C server starts writing it - that's why a delay of 2 seconds solve your problem
You can simply remove the if(dataIn.available() > 0) { } condition - dataIn.readInt() will block until the data is available on the wire
You're allocating and sending the wrong length. msg_queue is a pointer, and its size is irrelevant to the message length. sizeof msg_queue should be sizeof int throughout. You're sending too much data, so the receiver is getting out of sync.
Remove the pointless sleep and available() test. The subsequent reads will block for exactly the right length of time, unlike your sleep.
You don't need to clear the dynamic message befor freeing it. You don't even need to allocate the message dynamically, as its size is known at compile time.
Related
Right now I am using a socket and a datagram packet. This program is for a LAN network and sends at least 30 packets a second at 500 bytes maximum.
this is how I am receiving my data
payload = new String(incomingPacket.getData(), incomingPacket.getOffset(), incomingPacket.getLength(), "UTF-8");
Currently I am using no offset and I parse one by one through each character. I use the first 2 characters right now to determine what type of message it is but that is subject to change, then I break down variables and seperate the data with an exclamation mark to tell me when the next variable begins. At the end I parse it and apply it to my program. Is there a faster way to break down and interpret datagram packets? Will there be a performance difference if I put the length of the variables in the offset. Maybe an example would be useful. Also I think my variables are too small to use StringBuilder so I use normal concatenation.
What you are talking about here is setting up your own protocol for communication. While I have this as the fourth part of my socket tutorial (I'm currently working on part 3, non-blocking sockets) I can explain some things here already.
There are several ways of setting up such a protocol, depending on your needs.
One way of doing it is having a byte in front of each piece of data declaring the size, in bytes. That way, you know the length of the byte array containing the next variable value. This makes it easy to read out whole variables in one go via the System.arraycopy method. This is a method I've used before. If the object being sent is always the same, this is all you need to do. Write the variables in a standardized order, add the size of the variable value and you're done.
If you have to send multiple types of objects throught the stream you might want to add a bit of meta data. This meta data can then be used to tell what kind of object is being sent and the order of the variables. This meta data can then be put in a header which you add before the actual message. Once again, the values in the header are preceded by the size byte.
In my tutorial I'll write up a complete code example.
Don't use a String at all. Just process the underlying byte array directly: scan it for delimiters, counts, what have you. You can use a DataInputStream wrapped around a ByteArrayInputStream wrapped around the byte array if you want an API oriented to Java primitive datatypes.
I have an assignment that requires use of a socket. On the client and server side I have
char[] value= "END STREAM".toCharArray(), which signals the stream to shutdown.
Since I have both these arrays in the 2 different files, my intention is that the client sends the message value > the server. Then the server does the function
while(!Arrays.equals(clientSentence, value))
{
...
inFromClient.read(clientSentence, 0, length); //to read in from client side
.....
}
In essence, while it does not send the END STREAM message, keep reading. My issue is that the array equality does not work as intended. I even test this by doing
System.out.println(Arrays.equals(value, clientSentence));
System.out.println(new String(value));
System.out.println(new String(clientSentence));
and it prints
false
END STREAM
END STREAM
How can it be false when it is printing the same values. I have made sure that both arrays initialize to the same length so where is it going wrong? I have been stuck on this for hours and searched for answers but cannot find a solution. Thanks
EDIT: added my read function. I use a BufferedReader
On my box:
char[] x = "END STREAM".toCharArray();
char[] y = "END STREAM".toCharArray();
System.out.println(Arrays.equals(x, y));
System.out.println(Arrays.toString(x));
System.out.println(Arrays.toString(y));
works which makes me think a few things:
The arrays, as declared in your code, are not equal.
There is a character set incompatibility, you said you were using a BufferedReader and, if you're using the Files.newBufferedReader() functionality of Java 1.7, you need specify a charset when using it which may be causing a problem.
End of line issues either from cross platform systems or something else e.g. \r vs \n
Thinking a little more about it...it's probably #2, check this out for further information: http://docs.oracle.com/javase/tutorial/i18n/text/string.html
I'm working on a Java program where I'm reading from a file in dynamic, unknown blocks. That is, each block of data will not always be the same size and the size is determined as data is being read. For I/O I'm using a MappedByteBuffer (the file inputs are on the order of MB).
My goal:
Find an efficient way to store each complete block during the input phase so that I can process it.
My constraints:
I am reading one byte at a time from the buffer
My processing method takes a primitive byte array as input
Each block gets processed before the next block is read
What I've tried:
I played around with dynamic structures like Lists but they don't have backing arrays and the conversion time to a primitive array concerns me
I also thought about using a String to store each block and then getBytes() to get the byte[], but it's so slow
Reading the file multiple times in order to find the block size first, and then grab the relevant bytes
I am trying to find an approach that doesn't defeat the purpose of fast I/O. Any advice would be greatly appreciated.
Additional Info:
I'm using a rolling hash to decide where blocks should end
Here's a bit of pseudo-code:
circular_buffer[] = read first 128 bytes
rolling_hash = hash(buffer[])
block_storage = ??? // this is the data structure I'd like to use
while file has more text
b = next byte
add b to block_storage
add b to next index in circular_buffer (if reached end, start adding/overwriting front)
shift rolling_hash one byte to the right
if hash has a certain characteristic
process block_storage as a byte[] //should contain entire block of data
As you can see, I'm reading one byte at a time, and storing/overwriting that one byte repeatedly. However, once I get to the processing stage, I want to be able to access all of the info in the block. There is no predetermined max size of a block either, so I can't pre-allocate.
It seems to me, that you reqire a dynamically growing buffer. You can use the built in BytaArrayOutputStream to achieve that. It will automatically grow to store all data written to it. You can use write(int b) and toByteArray() to realize add b to block_storage and process block_storage as a byte[].
But take care - this stream will grow unbounded. You should implement some sanity checks around it to avoid using up all memory (e.g. count bytes written to it and break by throwing an exception, when it exceeds an reasonable amount). Also make sure to close and throw away the reference to a stream after consuming the block, to allow the GC to free up memory.
edit: As #marcman pointed out, the buffer can be reset().
OK, so I've been trying to read a ("any") binary file to a byte[] array using FileInputReader.read()...But no matter the file length it only reads 5 bytes at a time...
(btw im using udp to send the array/file)
byte[] array_bytes = new byte[1024];
while((nread=file.read(array_bytes))!=-1){
number_bytesread += array_bytes.length;
socket_udp.send(send_package);
count += 1;
}
-send_package is a datagrampacket using array_bytes to send the message
I've tried to use the function read(byte[], offset, lenght), but if I put the lenght over 5, it occurs this error, at the first time reading, even thought the file is surely bigger than 5 bytes:
nread=file.read(array_bytes, 0, 1024);
no need for offset since i send immediatly after reading.
Exception in thread "main" java.lang.IndexOutOfBoundsException
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:272)
at udp_server2.UDP_Server2.Send_Udp(UDP_Server2.java:122)
at udp_server2.UDP_Server2.main(UDP_Server2.java:77)
Java Result: 1
Thanks in advance for the help,
André
It sounds like your array length isn't actually 1024. I know you've shown code creating it with 1024 bytes, but I suspect your actual code either changes the value somewhere else, or that it's created in a different way.
It would be helpful if you could show a short but complete program demonstrating the problem, but failing that, the first diagnostic step I'd take would be to look at array_bytes.length, either in the debugger or via logging. I strongly suspect that you'll find it's 5. Once you've established that, you need to work out why it's 5, which has nothing to do with the calls to reading from the input stream.
Also note that this line is fundamentally wrong:
number_bytesread += array_bytes.length;
It should be:
number_bytesread += nread;
... because you've just read nread bytes. You're also sending send_package which appears to have nothing to do with the data you've just read.
Additionally, I suggest you start following Java naming conventions, with things like arrayBytes rather than array_bytes. (I'd just use buffer or data, mind you - but I'm pointing out that variable and names in Java are typically camelCased rather than using_underscores_for_word_breaks.)
I have encountered an interesting C sockets problem.
I am receiving incoming strings and noticed that I will, randomly, receive 3 strings at a same time for the first 2 ~ 4 strings.
For example, I am receiving the following incoming strings.
1~message~i love you\r\n
2~message~do you love me?\r\n
3~message~when are we going to meet again?\r\n
4~message~How about now?\r\n
5~message~Oh! I'm pregnant!\r\n
I added a counter to track the number of messages received and noticed that the counter sometimes does not count the first 3 strings. For example
1~message~i love you\r\n
->Line 1 received
2~message~do you love me?\r\n
3~message~when are we going to meet again?\r\n
4~message~How about now?\r\n
->Line 2 received
5~message~Oh! I'm pregnant!\r\n
->Line 3 received
The following is my code for printing the line number
int lineNo = 1;
while ((recvBytes = recv(clntSockfd, buffer, sizeof(buffer), 0)) > 0) {
printf("%s", buffer);
memset(&buffer, 0, sizeof(buffer));
printf("Line %d received\n", lineNo++);
}
I'm not sure why is this happening since this problem did not appear when i coded in Java nio.
Any ideas, folks?
Assuming you are using TCP, relating recv() calls to "messages" (or "lines") in your case is flawed. TCP, conceptually, is a stream of bytes. The sending operating system is free to group multiple send() calls into a single IP packet, as is the receiving operating system free to report multiple incoming packets as a single recv() call (assuming the buffer is large enough). It may even choose to split an incoming packet across recv calls.
So you really need to put a message structure in the data itself, eg. by scanning for line breaks in the data received.
That this didn't occur in Java was pure luck.
You are not reading till end of line. The buffer can contain more than one line.
what connection type are you using?
UDP is unreliable most of the time.
TCP is much better than UDP in terms of reliability.