I tried to solve the problem in many ways but without success and I have also looked for information in this forum but with same results, so here we go.
I am actually doing a server daemon that accepts client requests and then it (the server) transfers all the files contained in a specific folder. I'm going to post the code of the sendFileData (on the server) and the receiveFileData (on the client).
The server uses:
public static void sendFileData(File file, Socket socket) throws FileNotFoundException, IOException, SocketException {
byte[] auxiliar = new byte[8192];
byte[] mybytearray = new byte[(int) file.length()];
int longitud = mybytearray.length;
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file));
bis.read(mybytearray, 0, longitud);
DataOutputStream os = new DataOutputStream(socket.getOutputStream());
int paquetes = longitud / 8187;
int resto = longitud % 8187;
int i = 0;
while(i<paquetes){//The length goes on the first 4 bytes and the 5th tells if there are more packets to send (8192 bytes or less).
byte[] bytes = ByteBuffer.allocate(4).putInt(8187).array();
auxiliar[0] = bytes[0];
auxiliar[1] = bytes[1];
auxiliar[2] = bytes[2];
auxiliar[3] = bytes[3];
auxiliar[4] = 1;
for(int j = 5; j < 8192; j++){
auxiliar[j] = mybytearray[i*8187+(j-5)];
}
os.write(auxiliar, 0, 8192);
i+=1;
}
if(resto > 0){
byte[] bytes = ByteBuffer.allocate(4).putInt(resto).array();
auxiliar[0] = bytes[0];
auxiliar[1] = bytes[1];
auxiliar[2] = bytes[2];
auxiliar[3] = bytes[3];
auxiliar[4] = 0;
for(int j = 5; j < resto+5; j++){
auxiliar[j] = mybytearray[i*8187+(j-5)];
}
os.write(auxiliar, 0, resto+5);
}
os.flush();
}
And in the client side:
public static void receiveFileData(String nombreFichero, Socket s) throws IOException{
File monitored = new File(nombreFichero);
if(monitored.exists() == false){
monitored.createNewFile();
}
byte[] mybytearray;
DataInputStream is = new DataInputStream(s.getInputStream());
FileOutputStream fos = new FileOutputStream(monitored);
BufferedOutputStream bos = new BufferedOutputStream(fos);
int bytesRead = 0;
int hasNext = 1;
do {
bytesRead = is.readInt();//Leo longitud
try {
Thread.sleep(1);// HERE!!!!
} catch (InterruptedException e) {
}
// System.out.println("Bytes read "+bytesRead );
if(bytesRead <= 8187 && bytesRead > 0){
// System.out.println("Bytes leídos "+bytesRead );
hasNext = is.readByte();//Leo si hay más datos por enviar
mybytearray = new byte[bytesRead];
is.read(mybytearray);
if(monitored.exists()){
synchronized(monitored){
bos.write(mybytearray, 0, mybytearray.length);
}
}
mybytearray = null;
}else{
System.out.println("Fuera de rango "+bytesRead);
}
}while(hasNext == 1);
bos.close();
mybytearray = null;
System.out.println("Fichero recibido: "+monitored.getAbsolutePath());
}
In the receiveFileData code, if I do not put a Thread.sleep(1) or a System.out.println() or whatever who takes time to execute, I am not receiving the data in the correct way on the client, because readInt() returns a very high number randomly negative or positive (which implies Heap out of memory and other exceptions).
Sure it's something about synchronization but I think the transfering schema between the two methods is correct (maybe the client is too slow and server too fast).
What is happening?? Because I do not want to put a Thread.sleep, this is not good programming here I think.
Thank you so much!
is.read(bytes) is not guaranteed to fill the supplied byte array. You need to check its return value to see how many bytes were read or (better) use readFully().
The sleep() probably just allows time for all bytes to have been returned from the socket.
Related
Note: I'm currently learning UDP and how effective a VoIP system would be in comparison with a TCP system, I've already done TCP so please no one comment TCP is better etc..
So I'm trying to add a sequence number to the packets so I can order them on the server end and prepare for any lost packets, by repeating the previous for example
Problem: I read a stackoverflow that said using DataOutputStreams is a good way to go about this, so I implemented it. however when using the code dos.writeInt(sequenceNumber++); I get a horrible repetitive crackle. I was thinking maybe the problem is the amount of bytes that im sending.
Thank you in advance, any pointers will be great
boolean running = true;
try {
AudioRecorder recorder = new AudioRecorder();
int sequenceNumber = 0;
while (running) {
byte[] tempBuffer = recorder.getBlock();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(baos);
dos.write(tempBuffer);
dos.writeInt(sequenceNumber++);
dos.flush();
DatagramPacket sendPacket = new DatagramPacket(baos.toByteArray(), baos.size(), clientIP, PORT);
sending_socket.send(sendPacket);
}
recorder.close();
} catch (Exception e) {
System.out.println("Error" + e);
Server end:
byte[] buffer = new byte[512];
byte[] temp = new byte[512];
try {
// CODE TO RECEIVE THE AUDIO
AudioPlayer player = new AudioPlayer();
int expectedValue = 0;
while (running) {
//Vector used to store audio blocks (32ms/512bytes each)
Vector<byte[]> voiceVector = new Vector<>();
// creates a new udp packet to receive the audio
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
receiving_socket.receive(packet);
// creates a byte array from the data
byte[] udpPacketBytes = packet.getData();
ByteArrayInputStream baos = new ByteArrayInputStream(udpPacketBytes);
DataInputStream dos = new DataInputStream(baos);
int receivedValue = dos.readInt();
if (receivedValue == expectedValue) {
byte[] filteredByteArray = Arrays.copyOfRange(udpPacketBytes, 4, udpPacketBytes.length - 4);
voiceVector.add(filteredByteArray);
Iterator<byte[]> voiceItr = voiceVector.iterator();
while (voiceItr.hasNext()) {
player.playBlock(voiceItr.next());
}
} else {
// play the previous again
byte[] filteredByteArray = Arrays.copyOfRange(temp, 4, temp.length - 4);
voiceVector.add(filteredByteArray);
Iterator<byte[]> voiceItr = voiceVector.iterator();
while (voiceItr.hasNext()) {
player.playBlock(voiceItr.next());
}
// play the current one
byte[] fba = Arrays.copyOfRange(udpPacketBytes, 4, udpPacketBytes.length - 4);
voiceVector.add(fba);
Iterator<byte[]> vItr = voiceVector.iterator();
while (vItr.hasNext()) {
player.playBlock(vItr.next());
}
}
System.out.println(receivedValue + " " + expectedValue);
expectedValue = receivedValue + 1;
temp = packet.getData();
}
} catch (Exception e) {
System.out.println("yo");
}
//Close the socket
receiving_socket.close();
You are writing the sequence number but you are not reading or removing it so your sequence number ends up in the audio.
I suggest you read the same format you write.
I'm having trouble running these two threads in Java. I have two methods in the client class and in each method they both have a socket of different ports but when I run the client, i see the error for a split second of one of the threads but the other one that sends the file over works.
Any help?
ClientApp.java
public static void main(String[] args) throws UnknownHostException, IOException, InterruptedException {
Thread getFileThread = new Thread() {
public void run() {
Client client = new Client();
try {
client.getTheFile("girlwithmask.jpg");
} catch (UnknownHostException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
Thread getListOfFilesThread = new Thread() {
public void run() {
Client client = new Client();
ArrayList<String> listOfFiles = null;
try {
listOfFiles = client.getFileList();
System.out.println(listOfFiles.get(1));
notify();
} catch (ClassNotFoundException | IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
getListOfFilesThread.start();
getFileThread.start();
}
Client.java
public class Client {
private static final int PORT = 2665;
private static String HOST = "localhost";
Client() {
}
public void getTheFile(String filename) throws UnknownHostException, IOException {
filename = "girlwithmask.jpg"; ///this is temporary
int filesize = 5000000; //buffer size 5mb
int bytesRead;
int currentTotalNumberOfBytes = 0;
//connect to port on server - server waits for this after running socket.accept() in the Server class
Socket socket = new Socket(HOST, PORT);
byte[] byteArray = new byte[filesize]; //create a byte array of 5mb
InputStream inputStream = socket.getInputStream(); //channel to to server
FileOutputStream fileOutStream = new FileOutputStream("myClientFiles/" + filename);
BufferedOutputStream bufferOutStream = new BufferedOutputStream(fileOutStream);
bytesRead = inputStream.read(byteArray, 0, byteArray.length);
currentTotalNumberOfBytes = bytesRead;
do { //read till the end and store total in bytesRead and add it to currentTotalNumberOfBytes
bytesRead = inputStream.read(byteArray, currentTotalNumberOfBytes, (byteArray.length - currentTotalNumberOfBytes));
if (bytesRead >= 0) {
currentTotalNumberOfBytes += bytesRead;
}
} while (bytesRead > -1); // when bytesRead == -1, there's no more data left and we exit the loop
bufferOutStream.write(byteArray, 0, currentTotalNumberOfBytes); //write the bytes to the file
bufferOutStream.flush();
bufferOutStream.close();
socket.close();
}
public ArrayList<String> getFileList() throws UnknownHostException, IOException, ClassNotFoundException {
Socket socket = new Socket("localhost", 9999);
ArrayList<String> titleList = new ArrayList<String>();
ObjectInputStream objectInput = new ObjectInputStream(socket.getInputStream());
Object object = objectInput.readObject();
titleList = (ArrayList<String>) object;
// System.out.println(titleList.get(2));
return titleList;
}
}
I'm not sure what is going on here. Been working with this for a couple of hours.
In the absence of an actual error, or indeed question, all we can do is critique your code:
byte[] byteArray = new byte[filesize]; //create a byte array of 5mb
You don't know what filesize is. You've hardcoded a guess of 5000000. This will not work for any case where the filesize is bigger than 5000000 which might be very often. There is an approach where you don't need to know the filesize: nor do you need a buffer the size of the whole file in the first place. You're assuming the file fits into memory and that the file length fits into an int. Both assumptions may be wrong. Use a smaller buffer size of 8192 or some such reasonable number which is usually a multiple of 1024 to get a good memory alignment. Hard-coding a big size of 5000000 has the drawbacks mentioned.
InputStream inputStream = socket.getInputStream(); //channel to to server
FileOutputStream fileOutStream = new FileOutputStream("myClientFiles/" + filename);
BufferedOutputStream bufferOutStream = new BufferedOutputStream(fileOutStream);
You don't really need the BufferedOutputStream with this code, or at least with this code as it's going to be, but let it pass for now.
bytesRead = inputStream.read(byteArray, 0, byteArray.length);
currentTotalNumberOfBytes = bytesRead;
do { //read till the end and store total in bytesRead and add it to currentTotalNumberOfBytes
bytesRead = inputStream.read(byteArray, currentTotalNumberOfBytes, (byteArray.length - currentTotalNumberOfBytes));
if (bytesRead >= 0) {
currentTotalNumberOfBytes += bytesRead;
}
} while (bytesRead > -1); // when bytesRead == -1, there's no more data left and we exit the loop
bufferOutStream.write(byteArray, 0, currentTotalNumberOfBytes); //write the bytes to the file
To make this code shorter you might want to change it to the canonical form:
int count;
byte[] buffer = new byte[8192];
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
substituting variable names as appropriate. You will:
Save memory
Reduce latency
Have clear well-tested code that's been working for 18 years.
bufferOutStream.flush();
flush() before close() is redundant.
bufferOutStream.close();
socket.close();
Closing the socket after closing its output stream (or input stream) is redundant. Just close the output stream. In a finally block.
I wish to send a JPG-image from a client to a server over a TCP socket connection. I convert the file to a byte array in the client then send it to the server after having sent the array length. The server reads the data one byte at a time from the socketinputstream to a new bytearray which is then written to a jpg-file. The data however is corrupted and this image cannot be viewed. Below is core java code for client and server.
Client:
Socket s = new Socket("localhost", 666);
DataOutputStream dos = new DataOutputStream(s.getOutputStream());
String imgPath = "C:/Users/huehuehue/Documents/Uni/D0036D/prick1.JPG";
File file = new File(imgPath);
byte[] b = new byte[(int) file.length()];
FileInputStream fis = new FileInputStream(file);
fis.read(b);
fis.close();
dos.writeInt((int) file.length());
dos.flush();
dos.write(b, 0, b.length);
dos.flush();
Server:
ServerSocket serverSocket = new ServerSocket(666);
Socket connect = serverSocket.accept();
DataInputStream dis = new DataInputStream(connect.getInputStream());
File file = new File("C:/Users/huehuehue/Documents/Uni/D0036D/PLAYERprick.JPG");
FileOutputStream fos = new FileOutputStream(file);
int arrlen = dis.readInt();
byte[] b = new byte[arrlen];
int i = 0;
for(i = 0; i < arrlen; i++) {
b[i] = dis.readByte();
i++;
}
fos.write(b, 0 , b.length);
I can't see why this wouldn't work and any help or suggestions is greatly appreciated.
int arrlen = dis.readInt();
byte[] b = new byte[arrlen];
int i = 0;
for(i = 0; i < arrlen; i++) {
b[i] = dis.readByte();
i++;
}
You're incrementing i twice so you're ignoring half the input, and writing what you don't ignore into the wrong slots. Fortunately there is an easier way, which is also much more efficient:
int arrlen = dis.readInt();
byte[] b = new byte[arrlen];
dis.readFully(b);
NB fis.read(b); isn't valid. You can't assume it fills the buffer. You should use DataInputStream.readFully() here too.
You need to match both ends of the socket and your client doesn't fill your byte[] fully; instead your client could send each byte as it reads it. And, since Java provides BufferedInputStream you don't have to manually use and tweak a byte[] buffer directly. Instead you could do something like,
Socket s = new Socket("localhost", 666);
String imgPath = "C:/Users/huehuehue/Documents/Uni/D0036D/prick1.JPG";
File file = new File(imgPath);
try (InputStream is = new BufferedInputStream(new FileInputStream(file));
DataOutputStream dos = new DataOutputStream(s.getOutputStream());) {
dos.writeLong(file.length()); // <-- remember to read a long on server.
int val;
while ((val = is.read()) != -1) {
dos.write(val);
}
dos.flush();
}
Also, your server could use the similar BufferedOutputStream to get the benefit of a buffer like
ServerSocket serverSocket = new ServerSocket(666);
File file = new File("C:/Users/huehuehue/Documents/Uni/D0036D/PLAYERprick.JPG");
try (Socket s = serverSocket.accept();
DataInputStream dis = new DataInputStream(
s.getInputStream());
OutputStream fos = new BufferedOutputStream(
new FileOutputStream(file));) {
long arrlen = dis.readLong();
for (long i = 0; i < arrlen; i++) {
fos.write(dis.read());
}
fos.flush();
}
without having to manually manage it. Notice the above use BufferedInputStream and BufferedOutputStream to manage the buffering as an optimization. You could remove them and read and write directly from the File streams.
I have another problem.
This is a part of my client:
Socket socket = new Socket("127.0.0.1", 3000);
OutputStream out = socket.getOutputStream();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutput oo = null;
try {
oo = new ObjectOutputStream(bos);
oo.writeObject(mp3dataStrings);
byte[] serializedMP3 = bos.toByteArray();
out.write(serializedMP3);
out.flush();
} finally {
oo.close();
bos.close();
}
this is a part of my server:
ServerSocket clientConnect = new ServerSocket(port);
System.out.println("SimpleServer running on port" + port);
Socket clientSock = clientConnect.accept();
InputStream is = clientSock.getInputStream();
byte[] buffer = new byte[1024];
for (int i = 0; i < buffer.length; i++) {
int b = is.read();
buffer[i] = (byte) b;
if (b == -1 | b == 0) break;
}
ObjectInputStream stream = new ObjectInputStream(new ByteArrayInputStream(buffer));
String[][] songs = (String[][]) stream.readObject();
stream.close();
When I send my object (a String[][]) I get the exception invalid stream header: ACED0000.
I can't find what this means and what I have to do.
greets
Alex
You have made it far more complicated than you need to.
Socket socket = new Socket("127.0.0.1", 3000);
try {
ObjectOutputStream oo = new ObjectOutputStream(socket.getOutputStream());
oo.writeObject(mp3dataStrings);
oo.close();
} finally {
socket.close();
}
and
ServerSocket clientConnect = new ServerSocket(port);
System.out.println("SimpleServer running on port" + port);
Socket clientSock = clientConnect.accept();
try {
ObjectInputStream stream = new ObjectInputStream(clientSock.getInputStream());
String[][] songs = (String[][]) stream.readObject();
} finally {
clientSock.close();
}
I agree with Peter Lawrey's answer, however the problem in your original code stems from the exit condition in the byte buffer population code
byte[] buffer = new byte[1024];
for (int i = 0; i < buffer.length; i++) {
int b = is.read();
// THIS ARE PROBLEM LINES
buffer[i] = (byte) b;
if (b == -1 | b == 0) break;
}
ObjectInputStream stream =
new ObjectInputStream(new ByteArrayInputStream(buffer));
You should only exit this loop when you've detected End-Of-Stream condition. In other words you should never consider b==0, as it is a valid part of the ObjectInputStream.
Second, you should not assign the byte to the buffer before checking for break condition.
Third, if you initialize ByteArrayInputStream, you should pass only the number of bytes that contain input, not the whole buffer itself.
Corrected block should be like this:
// How do you know if 1024 is enough to get all data?
// For the sake of this example, assume it's enough
byte[] buffer = new byte[1024];
int count = 0;
for (; count < buffer.length; count++) {
int b = is.read();
if ( b == -1 )
{
// exit only on End-Of-Stream, and do not record
// this result into the buffer
break;
}
buffer[count] = (byte) b;
}
ObjectInputStream stream =
new ObjectInputStream(
// Note, that we are now passing the number of 'active' bytes in the buffer
new ByteArrayInputStream(buffer, 0, count)
);
I'm trying to transfer files between client-server if the file is bigger than 12MB than it sends by blocks otherwise it sends normally. my main problem is that everytime I transfer something it gets more bytes than the original one, and I need to use digest messages after I complete this so it wont work the way it is, and the other one is when I try to send a file by network the client reads the file to send faster than the server writes it so client closes the program closing the connection, corrupting the file. my transfer code is below:
this is the client transfer code:
if(fSize>maxfileSize){
totbLidos = 0;
byte[] fBytes = new byte[fBsize];
while(totbLidos < fSize){
int bRemain = (int) f.length() - totbLidos;
if(bRemain < fBsize){
fBsize = bRemain;
}
int bRead = tFile.read(fBytes, 0, fBsize);
tServidor.write(fBytes, 0, fBsize);
tServidor.flush();
if(bRead>0){
totbLidos+=bRead;
}
System.out.println("Total Bytes Lidos: " + totbLidos);
}
tFile.close();
System.out.println("Ficheiro enviado");
cliente.close();
}
else{
totbLidos = 0;
byte[] fBytes = new byte[fSize];
while(totbLidos < fSize){
int bRead = tFile.read(fBytes,0,fSize);
if(bRead>0){
totbLidos+=bRead;
}
tServidor.write(fBytes, 0, fSize);
System.out.println("Total Bytes Lidos: " + totbLidos);
tServidor.flush();
}
tFile.close();
System.out.println("Ficheiro enviado");
cliente.close();
}
}
server transfer code:
if(fSize > maxfileSize){
totbLidos = 0;
DataInputStream tFile = new DataInputStream(cliente.getInputStream());
BufferedOutputStream fos = new BufferedOutputStream(new FileOutputStream(fName));
byte[] fBytes = new byte[fBsize];
while(totbLidos < fSize){
int bRemain = size - totbLidos;
if(bRemain < fBsize){
fBsize = bRemain;
}
int bRead = tFile.read(fBytes, 0, fBsize);
fos.write(fBytes);
fos.flush();
if(bRead>0){
totbLidos+=bRead;
}
System.out.println("Bytes lidos: " + bRead);
System.out.println("Total Bytes Escritos: " + totbLidos);
}
System.out.println("Ficheiro recebido");
fos.close();
tFile.close();
cliente.close();
servidor.close();
}
else if(fSize < maxfileSize){
totbLidos = 0;
DataInputStream tFile = new DataInputStream(cliente.getInputStream());
BufferedOutputStream fos = new BufferedOutputStream(new FileOutputStream(fName));
byte[] fBytes = new byte[fSize];
while(totbLidos < fSize){
int bRead = tFile.read(fBytes,0,fSize);
fos.write(fBytes);
fos.flush();
if(bRead>0){
totbLidos+=bRead;
}
System.out.println("Total Bytes Escritos: " + totbLidos);
}
System.out.println("Ficheiro recebido");
fos.close();
tFile.close();
cliente.close();
servidor.close();
}
}
You are not writing the same number of bytes you read here.
fos.write(fBytes);
try using
fos.write(fBytes, 0, bRead);
In addition to #Peter Lawrey answer I suggest you to reuse Apache Commons IO API to avoid this kind of bugs in future:
IOUtils.copyLarge(InputStream input, OutputStream output, long inputOffset, long length)