The problem is when i send up to 40 KB everything is okay when i send more sometime half of the data received some time nothing ,is there a limit of the networkstream.Read ,even though i cunked the data ,i can't determine if the problem form the java or the c# from the network stream or the Output stream
C# SERVER
private void ReadData(){
if (networkStream.DataAvailable)
{
int size = GetBufferSize();
Thread.Sleep(340);
byte[] myReadBuffer = new byte[size];
int numberOfBytesRead = 0;
while (true)
{
numberOfBytesRead = networkStream.Read(myReadBuffer, 0, myReadBuffer.Length);
if (numberOfBytesRead >= myReadBuffer.Length)
{
break;
}
}
string str = Encoding.UTF8.GetString(myReadBuffer, 0, size);
dynamic Message = Newtonsoft.Json.JsonConvert.DeserializeObject<dynamic>(str);
// Android Message , JSON String
if (OnAndroidMessage != null)
{
OnAndroidMessage(Message);
}
}
}
private int GetBufferSize()
{
byte[] myReadBuffer = new byte[4];
int numberOfBytesRead = 0;
do
{
numberOfBytesRead = networkStream.Read(myReadBuffer, 0, myReadBuffer.Length);
} while (networkStream.DataAvailable && numberOfBytesRead < myReadBuffer.Length);
if (numberOfBytesRead > 0)
{
// reverse the byte array.
if (BitConverter.IsLittleEndian)
{
Array.Reverse(myReadBuffer);
}
return BitConverter.ToInt32(myReadBuffer, 0);
}
else
{
return 0;
}
}
Java Client // i tested this also without cutting the data to smaller paces ,half of the data received not all of them
mBufferOut = socket.getOutputStream();
private void sendMessage(final String message) {
if (mBufferOut != null && message != null) {
try {
byte[] data = message.getBytes("UTF-8");
Log.d("_TAG", "Sending: " + message);
Log.d("_TAG", "Message length: " + Integer.toString(data.length));
mBufferOut.write(toByteArray(data.length));
mBufferOut.flush();
List<byte[]> divideArray = divideArray(data, 10000);
for (byte[] dataChunk : divideArray) {
Log.e("_TAG","CHUNK SIZE > " + Integer.toString(dataChunk.length));
mBufferOut.write(dataChunk, 0, dataChunk.length);
mBufferOut.flush();
}
} catch (Exception e1) {
e1.printStackTrace();
}
}
}
private List<byte[]> divideArray(byte[] source, int chunksize) {
List<byte[]> result = new ArrayList<byte[]>();
int start = 0;
while (start < source.length) {
int end = Math.min(source.length, start + chunksize);
result.add(Arrays.copyOfRange(source, start, end));
start += chunksize;
}
return result;
}
Any ideas ?
Solution from this post NetworkStream is reading data that should not be there
static void ReadExact(Stream stream, byte[] buffer, int offset, int count)
{
int read;
while(count > 0 && (read = stream.Read(buffer, offset, count)) > 0) {
offset += read;
count -= read;
}
if(count != 0) throw new EndOfStreamException();
}
the problem is the Read it takes size and want to get that size you need to give it chunks and check each chunk
And also read does not restart from where it stopped until it reads the amount is set to read meaning if i set to read 10 then if it not find the 10 then it will read what it find as example it reads 6 ,it will return 6 and when to loop another time ti read the rest it dose not start from 6 it start from 0 and read until 4 so you overwrite your data ,and if it read 10 from the first try then it set the read to finish so it dose not start from 0 ,it needs to read the amount the has been set to it to re set the read to new buffer location.
Related
So I've been trying to read the content of a text file and write the content chunk by chunk alternately into e.g. 2 new files.
I already tried multiple ways to do that but it won't work (OutputStream and FileOutputStream seems to be the most suitable).
Before i tried to part the file in e.g. 3 Parts and wrote the first part in one file, the second part in another and so on. Which worked perfectly fine with OutputStream and FileOutputStream.
But it won't work when i want to do it alternately.
To do it alternately i use the round robin algorithm, which on its own works fine.
I would be really thankful if you could show me some examples to do it!
public void splitFile(String filePath, int numberOfParts, long sizeOfParts[]) throws FileNotFoundException, IOException, SQLException {
long bytes = 8;
OutputStream partsPath[] = new OutputStream[numberOfParts];
long bytePositition[] = new long[numberOfParts];
long copy_size[] = new long[numberOfParts];
for (int i = 0; i < numberOfParts; i++) {
copy_size[i] = sizeOfParts[i];
partsPath[i] = new FileOutputStream(path); //Gets Path from my Database (works)
//System.out.println(cloudsTable.getCloudsPathsFromDatabase(i) + '\\' + name + (i + 1) + fileType);
}
InputStream file = new FileInputStream(filePath);
while (true) {
boolean done = true;
for (int i = 0; i < numberOfParts; i++) {
if (copy_size[i] > 0) {
done = false;
if (copy_size[i] > bytes) {
copy_size[i] -= bytes;
bytePositition[i] += bytes;
System.out.println("file " + i + " " + bytePositition[i]);
readWrite(file, bytePositition[i], partsPath[i]);
} else {
bytePositition[i] += copy_size[i];
System.out.println("rest file " + i + " " + bytePositition[i]);
readWrite(file, bytePositition[i], partsPath[i]);
copy_size[i] = 0;
}
}
}
if (done == true) {
break;
}
}
file.close();
for (int i = 0; i < partsPath.length; i++) {
partsPath[i].close();
}
}
private void readWrite(InputStream file, long bytes, OutputStream path) throws IOException {
byte[] buf = new byte[(int) bytes];
while (file.read(buf) != -1) {
path.write(buf);
path.flush();
}
}
What the code does is, it only write the content of the Originalfile in the first-copied file and the following files are empty
EDIT:
To clarify what the code should do is write the first 8 bytes to go to file 1, second 8 bytes to go to file 2, third 8 bytes to go to file 3, fourth 8 bytes to go to file 1, and so on, round robin, until file 1 is sizeOfParts[0] long, file 2 is sizeOfParts[1] long, and file 3 is sizeOfParts[2] long.
The main problem is that the readWrite() method is only supposed to copy one 8-byte block of bytes, but has a loop that makes it copy all the remaining bytes in the input file.
In addition, the code should be enhanced to use try-finally to close the files, and to correctly handle end-of-file, in case the input file is shorter than the sum of parts.
I would eliminate the readWrite() method, and consolidate the logic to prevent duplicate code, like this:
public void splitFile(String inPath, long[] sizeOfParts) throws IOException, SQLException {
final int numberOfParts = sizeOfParts.length;
String[] outPath = new String[numberOfParts];
// Gets Paths from Database here
InputStream in = null;
OutputStream[] out = new OutputStream[numberOfParts];
try {
in = new BufferedInputStream(new FileInputStream(inPath));
for (int part = 0; part < numberOfParts; part++)
out[part] = new BufferedOutputStream(new FileOutputStream(outPath[part]));
byte[] buf = new byte[8];
long[] remain = sizeOfParts.clone();
for (boolean done = false; ! done; ) {
done = true;
for (int part = 0; part < numberOfParts; part++) {
if (remain[part] > 0) {
int len = in.read(buf, 0, (int) Math.min(remain[part], buf.length));
if (len == -1) {
done = true;
break;
}
remain[part] -= len;
System.out.println("file " + part + " " + (sizeOfParts[part] - remain[part]));
out[part].write(buf, 0, len);
done = false;
}
}
}
} finally {
if (in != null)
in.close();
for (int part = 0; part < out.length; part++)
if (out[part] != null)
out[part].close();
}
}
I am trying to skip a negative number of bytes with AudioInputStream
skip(long bytes) method .
The problem is trying to (let's say a small number of bytes...) :
int skipped = audioInputStream.skip(-bytes);
always returns 0 as described on this answer Java AudioInputStream skip with negative number of bytes always returns 0
I need to create an implementation which supports also negative number of bytes or something like backwards.
Here is the full code of the library on github .
What I do is recreating the line every time the user skips audio
which is extremely slow when i can of course do much better ... by just going backward or forward . Now it supports only forward ...
/**
* Skip bytes in the File input stream. It will skip N frames matching to bytes, so it will never skip given bytes len
*
* #param bytes
* the bytes
* #return value bigger than 0 for File and value = 0 for URL and InputStream
* #throws StreamPlayerException
* the stream player exception
*/
public long seek(long bytes) throws StreamPlayerException {
long totalSkipped = 0;
//If it is File
if (dataSource instanceof File) {
//Check if the requested bytes are more than totalBytes of Audio
long bytesLength = getTotalBytes();
System.out.println("Bytes: " + bytes + " BytesLength: " + bytesLength);
if ( ( bytesLength <= 0 ) || ( bytes >= bytesLength )) {
generateEvent(Status.EOM, getEncodedStreamPosition(), null);
return totalSkipped;
}
logger.info(() -> "Bytes to skip : " + bytes);
Status previousStatus = status;
status = Status.SEEKING;
try {
synchronized (audioLock) {
generateEvent(Status.SEEKING, AudioSystem.NOT_SPECIFIED, null);
initAudioInputStream();
if (audioInputStream != null) {
long skipped;
// Loop until bytes are really skipped.
while (totalSkipped < ( bytes )) { //totalSkipped < (bytes-SKIP_INACCURACY_SIZE)))
//System.out.println("Running");
skipped = audioInputStream.skip(bytes - totalSkipped);
if (skipped == 0)
break;
totalSkipped += skipped;
logger.info("Skipped : " + totalSkipped + "/" + bytes);
if (totalSkipped == -1)
throw new StreamPlayerException(StreamPlayerException.PlayerException.SKIP_NOT_SUPPORTED);
logger.info("Skeeping:" + totalSkipped);
}
}
}
generateEvent(Status.SEEKED, getEncodedStreamPosition(), null);
status = Status.OPENED;
if (previousStatus == Status.PLAYING)
play();
else if (previousStatus == Status.PAUSED) {
play();
pause();
}
} catch (IOException ex) {
logger.log(Level.WARNING, ex.getMessage(), ex);
}
}
return totalSkipped;
}
You can create your own buffer, It could be ByteArrayOutputStream but that is a bloated thing - always gives me Out of memory after a couple of minutes - or have your own Vector or other ArrayList.
I tried with a 10 min .wav file and it runs fine - as far as playing and adding the bytes to the buffer.
e.g.
Vector v=new Vector();
byte[] data=new byte[basicU];
while(true) {
k=audioInputStream.read(data, 0, data.length);
v.add(data);
if(k<0) break;
tot+=k;
}
--
Here is my method for playing a file with seeks. I have a thread for generating seek signals. The problem is complicated when we have multiple seeks. I use a variable K to check whether we need to add data to the buffer. I don't use skip but normal read; just don't play it in the line.
public void play() {
boolean seekingBack=false;
int i, j, k=0, seekPos=0, basicU=1024;
AudioFormat targetFormat=null;
int tot=0;
new Thread() {
public void run() {
while(true) {
numBytes=(Math.random()>0.5?1:-1)*500000;
try { Thread.sleep(5000); } catch (Exception e) {}
seekSignal=true;
}
}}.start();
try {
File fileIn=new File("........");
AudioInputStream audioInputStream=AudioSystem.getAudioInputStream(fileIn);
targetFormat=audioInputStream.getFormat();
DataLine.Info dinfo=new DataLine.Info(SourceDataLine.class, targetFormat);
SourceDataLine line=null;
line=(SourceDataLine)AudioSystem.getLine(dinfo);
if(line==null) return;
line.open(targetFormat);
line.start();
Vector v=new Vector();
byte[] data=new byte[basicU];
int K=0;
while(true) {
if(seekingBack) { // seeking backwards
K=seekPos;
k=data.length;
for(j=0; j<data.length; j++)
if(seekPos+j<v.size()) data[j]=((Byte)v.get(seekPos+j)).byteValue();
else { k=j; break; }
line.write(data, 0, k);
seekPos+=k;
K+=k;
if(seekPos>v.size()-1) seekingBack=false;
}
else { // normal playing
k=audioInputStream.read(data, 0, data.length);
if(k<0) break;
line.write(data, 0, k);
if(K>=v.size()) for(j=0; j<k; j++) v.add(data[j]);
K+=k;
}
if(seekSignal) { // received a seek signal
if(seekingBack) { // we are on a previous back seek - reading from the buffer
if(numBytes<0) {
seekPos+=numBytes;
if(seekPos<0) seekPos=0;
}
else { // depending on where the seek will go (in the buffer or actual audio stream)
if(numBytes+seekPos<v.size())
seekPos+=numBytes;
else { // actual stream
int rem=numBytes-(v.size()-seekPos);
K=v.size();
while(rem>0) {
k=audioInputStream.read(data, 0, data.length);
if(k<0) break;
if(K>=v.size()) for(j=0; j<k; j++) v.add(data[j]);
rem-=k;
K+=k;
}
}
}
}
else { // we are not processing a previous back seek
if(numBytes>=0) { // forward
while(numBytes>0) {
k=audioInputStream.read(data, 0, data.length);
if(k<0) break;
if(K>=v.size()) for(j=0; j<k; j++) v.add(data[j]);
numBytes-=k;
K+=k;
}
}
else { // backward
seekingBack=true; seekPos=v.size()+numBytes; if(seekPos<0) seekPos=0; }
}
seekSignal=false;
}
}
line.stop();
line.close();
}
catch(Exception ex) { ex.printStackTrace(); System.out.println("audio problem "+ex); }
}
Use your own buffer which holds a rolling window of history. I'd build a helper class that does this by allocating a List<byte[]> to manage history in blocks of for example 8192 bytes. Then you need some simple overflowing mechanism that throws out the oldest block, in combination with some pointer manipulation to keep track of where you actually are in the stream. Good luck!
My C++ program which is the client connects with the Java server . From the time of connection establishment the C++ client sends a block of data of size ~1MB to 3MB in a fixed frequency( say 10 sec).
My Java server opens a socket
Socket client = new ServerSocket(14001, 10).accept();//blocking
ReceiveThread st = new ReceiveThread(client);
and it receives the data from client as below.
private String getDataFromSocket(BufferedReader reader) throws IOException
{
int byteLimit = 1024*1024*2; //2 MB
String output = "";
char[] charArray = null;
int availableSize = is.available();
if(availableSize < 1) // If available size is 0 just return empty
{
return output;
}
while(availableSize > byteLimit) // Reads 2MB max if available size is more than 2 MB
{
charArray = new char[byteLimit];
reader.read(charArray,0,charArray.length);
output += new String(charArray);
availableSize = is.available();
}
charArray = new char[availableSize];
reader.read(charArray,0,charArray.length);
output = output +new String(charArray);
return output;
}
The above GetDataFromSocket keeps on checking for available data till the socket is closed gracefully.
the C++ connects with the Java server
void CreateSocket()
{
int err, nRet = 0;
sockfd = 0;
WORD wVersionRequested;
WSADATA wsaData;
//WSACleanup();// Is this needed?
wVersionRequested = MAKEWORD(1, 1);
while (1)
{
err = WSAStartup(wVersionRequested, &wsaData);
if (err != 0)
{
Sleep(50);
continue;
}
else
{
break;
}
}
while (1)
{
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd == -1 || sockfd == INVALID_SOCKET)
{
Sleep(50);
continue;
}
else
{
nRet = 1;
break;
}
}
}
void ConnectWithServer()
{
int nRet = 0;
char myname[256] = { 0 };
int wsaErr = 0, portNum = 0, retryCount=0;
struct hostent *h = NULL;
struct sockaddr_in server_addr;
gethostname(myname, 256);
portNum = 1401;
while (1)
{
if ((h = gethostbyname(myname)) != NULL)
{
memset(&server_addr, 0, sizeof(struct sockaddr_in));
memcpy((char *)&server_addr.sin_addr, h->h_addr, h->h_length);
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(portNum);
server_addr.sin_addr = *((struct in_addr*) h->h_addr);
}
if (0 == connect(sockfd, (struct sockaddr *)&server_addr, sizeof(server_addr)))
{
nRet = 1;
break;
}
else
{
}
Sleep(50);
}
}
The connection establishment to the server is done by the above two functions and it returns success. After these steps i am sending the data buffer to the Java server once in every 10 seconds.
while(index<retryCount)
{
string toSend = wstring_to_utf8(sRequestData);
nRet = send(sockfd, toSend.c_str(), toSend.length(), 0);
if (nRet == SOCKET_ERROR)
{
wsaErr = WSAGetLastError();
Sleep(DEFAULT);
index++;
}
else if(nRet == toSend.length())
{
break;
}
else
{
index = 0;
}
}
The problem here is, after some hours of send and receive from C++ to Java server , the send gets hanged for infinite time. The execution never comes out from the send() function. But after the hang if i abruptly close the Java server , then the send returns socket error and again works well for some hours and the hang still occurs.
As i mentioned i keep on sending data to server of size varied from 1 MB to 3 MB ten seconds once. What could be the issue here? How can i sort this out?
In my current project, I am trying to transmit a string from one computer to another, and after finding and learning from numerous examples I have managed to get a basic form of communication working.
The issue I am having is if one computer tries sending a message that is too long, it seems to get broken up into multiple parts (roughly 3700 characters), and my parsing method fails.
I am using a Selector to iterate through all of the channels. Here is the relevant code:
if(key.isReadable()) {
// Get the channel and read in the data
SocketChannel keyChannel = (SocketChannel)key.channel();
ByteBuffer buffer = buffers.get(keyChannel);
int length = 0;
try {
length = keyChannel.read(buffer);
} catch ( IOException ioe) {
closeChannel(keyChannel);
}
if(length > 0) {
buffer.flip();
// Gather the entire message before processing
while( buffer.remaining() > 0) {
byte[] data = new byte[buffer.remaining()];
buffer.get(data);
fireReceiveEvent(keyChannel, data);//Send the data for processing
}
buffer.compact();
} else if (length < 0) {
closeChannel(keyChannel);
}
}
How can I guarantee that the entire message (regardless of length) is read at once before passing it along?
After talking to numerous people that know more about this than I do. The issue turns out to be that with TCP it is impossible to know when an entire "message" has arrived because there is no such thing as a message since TCP works on a two-way byte stream. The solution is to create your own protocol and implements your own definition of "message".
For my project, every message either starts with a [ or { and ends with a ] or } depending on the starting character. I search through the received data, and if there is a complete message, I grab it and pass it along to the handler. Otherwise skip the channel, and wait for more to arrive.
Here is the final version of my code that handles the message receiving.
if(key.isReadable()) {
// Get the channel and read in the data
SocketChannel keyChannel = (SocketChannel)key.channel();
ByteBuffer buffer = buffers.get(keyChannel);
int length = 0;
try {
length = keyChannel.read(buffer);
} catch ( IOException ioe) {
key.cancel();
closeChannel(keyChannel);
}
if (length > 0) {
buffer.flip();
// Gather the entire message before processing
if (buffer.remaining() > 0) {
byte[] data = new byte[buffer.remaining()];
buffer.get(data);
buffer.rewind();
int index = 0;
int i = 0;
// Check for the beginning of a packet
//[ = 91
//] = 93
//{ = 123
//} = 125
if (data[0] == 91 || data[0] == 123) {
// The string we are looking for
byte targetByte = (byte) (data[0] + 2);
for (byte b : data) {
i += 1;
if (b == targetByte) {
index = i;
break;
}
}
if (index > 0) {
data = new byte[index];
buffer.get(data, 0, index);
fireReceiveEvent(keyChannel, data);
}
} else {
for (byte b : data) {
i += 1;
if (b == 91 || b == 123) {
index = i;
break;
}
}
if (index > 0) {
data = new byte[index];
buffer.get(data, 0, index); // Drain the data that we don't want
}
}
}
buffer.compact();
} else if (length < 0) {
key.cancel();
closeChannel(keyChannel);
}
}
I'm trying to connect to a WebSocket via Weberknecht Java library. When the Java client tries to create the connection, I get the following error message:
W/System.err(805): java.lang.ArrayIndexOutOfBoundsException: length=1000; index=1000
W/System.err(805): at ...WebSocketConnection.connect(WebSocketConnection.java:86)
Here is the part which causes the problem:
public void connect() throws WebSocketException {
try {
if (connected) {
throw new WebSocketException("already connected");
}
socket = createSocket();
input = new DataInputStream(socket.getInputStream());
output = new PrintStream(socket.getOutputStream());
output.write(handshake.getHandshake());
boolean handshakeComplete = false;
int len = 10000;
byte[] buffer = new byte[len];
int pos = 0;
ArrayList<String> handshakeLines = new ArrayList<String>();
while (!handshakeComplete) {
int b = input.read();
buffer[pos] = (byte) b; // THIS LINE CAUSES THE EXCEPTION
pos += 1;
if (buffer[pos - 1] == 0x0A && buffer[pos - 2] == 0x0D) {
String line = new String(buffer, "UTF-8");
if (line.trim().equals("")) {
handshakeComplete = true;
} else {
handshakeLines.add(line.trim());
}
buffer = new byte[len];
pos = 0;
}
}
What could cause this issue? Is it that this client library may not comply with the WebSocket version that the server uses?
The issue over there at Weberknecht has not been solved yet. I want to ask this question here because it seems to be quite a general problem (reading WebSocket headers and handshake data).
The code needs to check for -1 from input.read() to denote the connection hitting EOF.
Also I will assume based on the same bit of code that if a handshake hasn't been reaching after 10000 bytes it is an invalid handshake.
while (!handshakeComplete) {
int b = input.read();
if (b == -1)
throw new WebSocketException ("Connected closed.");
if (pos == buffer.length)
throw new WebSocketException ("Invalid handshake.");
buffer[pos] = (byte) b; // THIS LINE CAUSES THE EXCEPTION
pos += 1;
if (buffer[pos - 1] == 0x0A && buffer[pos - 2] == 0x0D) {
String line = new String(buffer, "UTF-8");
if (line.trim().equals("")) {
handshakeComplete = true;
} else {
handshakeLines.add(line.trim());
}
buffer = new byte[len];
pos = 0;
}
}
Update
Here is a better implementation:
public void connect() throws WebSocketException {
try {
if (connected) {
throw new WebSocketException("already connected");
}
socket = createSocket();
input = new DataInputStream(socket.getInputStream());
output = new PrintStream(socket.getOutputStream());
output.write(handshake.getHandshake());
boolean handshakeComplete = false;
ArrayList<String> handshakeLines = new ArrayList<String>();
BufferedReader reader = new BufferedReader(input);
String handshakeData;
while ((handshakeData = reader.readLine()) != null) {
if (handshakeData.trim().equals("")) {
handshakeComplete = true;
break;
}
handshakeLines.add(handshakeData);
}
if (!handshakeComplete)
throw new WebSocketException ("Failed to establish handshake.");
I am just thinking, why don't you use int read(byte[] b) which reads the the bytes for you. int read(byte[] b) --> Reads some number of bytes from the contained input stream and stores them into the buffer array b.
while (!handshakeComplete) {
int bytesRead = input.read(buffer);//returns number of bytes read
if(bytesRead < 10000 || bytesRead == -1 ||
(buffer[bytesRead - 1] == 0x0A && buffer[bytesRead - 2] == 0x0D)){
//done with reading
handshakeComplete = true;
}
if(bytesRead != -1){
String line = new String(buffer, "UTF-8");
if (!line.trim().equals("")) {
handshakeLines.add(line.trim());
}
}
}
Just by looking briefly, you make no guarantees of stopping before buffer.length == pos. That would be one thing to double check - pos has to stay strictly less than the length of the buffer.