We Keep Coding

Transfer Audio-File from Client to Http Server via URLConnection

i am currently working on a programming-project in my school. I need to send an audio file (MIDI format) from the Client successfully to a Http Server. I already tried to do this myself and did much research on the internet and in the Stackoverflow forum. Currently it is possible to send the file from the client to the server, but on the server side, the audio file is not playable.
The following is the client-side-code:
private static void sendPOST() throws IOException{
final int mid = 1;
final String POST_URL = "http://localhost:8080/musiker/hörprobe?mid="+mid;
final File uploadFile = new File("C://Users//Felix Ulbrich//Desktop//EIS Prototype MIDIs//Pop//BabyOneMoreTime.mid");
String boundary = Long.toHexString(System.currentTimeMillis());
String CRLF = "\r\n";
String charset = "UTF-8";
URLConnection connection = new URL(POST_URL).openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
try (
OutputStream output = connection.getOutputStream();
PrintWriter writer = new PrintWriter(new OutputStreamWriter(output, charset), true);
){
writer.append("--" + boundary).append(CRLF);
writer.append("Content-Disposition: form-data; name=\"binaryFile\"; filename=\"" + uploadFile.getName() + "\"").append(CRLF);
writer.append("Content-Type: " + URLConnection.guessContentTypeFromName(uploadFile.getName())).append(CRLF);
writer.append("Content-Transfer-Encoding: binary").append(CRLF);
writer.append(CRLF).flush();
Files.copy(uploadFile.toPath(), output);
output.flush();
writer.append(CRLF).flush();
writer.append("--" + boundary + "--").append(CRLF).flush();
int responseCode = ((HttpURLConnection) connection).getResponseCode();
System.out.println(responseCode);
}
}
The following is the server-side-code:
int FILE_SIZE = Integer.MAX_VALUE-2;
int bytesRead = 0;
int current = 0;
FileOutputStream fos = null;
BufferedOutputStream bos = null;
byte[] mybytearray = new byte[FILE_SIZE];
String FILE_TO_RECEIVED = "C://root//m"+musikerid+"hp"+(hörprobenzaehler+1)+".mid";
File f = new File(FILE_TO_RECEIVED);
if(!f.exists()){
f.createNewFile();
}
InputStream input = t.getRequestBody();
fos = new FileOutputStream(FILE_TO_RECEIVED);
bos = new BufferedOutputStream(fos);
bytesRead = input.read(mybytearray,0,mybytearray.length);
current = bytesRead;
do{
bytesRead = input.read(mybytearray, current, mybytearray.length-current);
if(bytesRead >= 0){
current += bytesRead;
}
}while(bytesRead>-1);
bos.write(mybytearray,0,current);
bos.flush();
fos.close();
bos.close();
t.sendResponseHeaders(200, 0);
input.close();
I am pretty desperate right now, because i couldn't find any solution to this problem. I need to use an HTTP server, but i don't need to use the TCP protocol (which is used right now via streams). I thought about a solution via ftp so i don't need to convert the file to a byte-array first. I assume that the problem lies exactly there. The server can't create the audio-file (midi-file) correctly from the byte-array. If anyone of you knows of a solution. Pls, i need your help :D
Greetings, Gizpo

So I've dug deeper into the matter. I've found several problems:
You're mixing binary and character based I/O. While you get away with it on the client side, the server has a hard time dealing with this.
You forgot to specify the size of the file you're sending over to the server. On the server side you have no way of knowing (unless some one tells you beforehand) what the size (of the incoming file) will be.
I've edited your code and came up with this:
Client:
private static void sendPOST() throws IOException{
final int mid = 1;
final String POST_URL = "http://localhost:8080/musiker/hörprobe?mid="+mid;
final File uploadFile = new File("C://Users//Felix Ulbrich//Desktop//EIS Prototype MIDIs//Pop//BabyOneMoreTime.mid");
String boundary = Long.toHexString(System.currentTimeMillis());
String CRLF = "\r\n";
String charset = "UTF-8";
URLConnection connection = new URL(POST_URL).openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
try (
OutputStream output = connection.getOutputStream();
PrintWriter writer = new PrintWriter(new OutputStreamWriter(output, charset), true);
) {
writer.append("--" + boundary).append(CRLF);
writer.append("Content-Disposition: form-data; name=\"binaryFile\"; filename=\"" + uploadFile.getName() + "\"").append(CRLF);
writer.append("Content-Length: " + uploadFile.length()).append(CRLF);
writer.append("Content-Type: " + URLConnection.guessContentTypeFromName(uploadFile.getName())).append(CRLF);
writer.append("Content-Transfer-Encoding: binary").append(CRLF);
writer.append(CRLF).flush();
Files.copy(uploadFile.toPath(), output);
output.flush();
int responseCode = ((HttpURLConnection) connection).getResponseCode();
System.out.println("Response code: [" + responseCode + "]");
}
}
Server:
#Override
public void handle(HttpExchange t) throws IOException {
String CRLF = "\r\n";
int fileSize = 0;
String FILE_TO_RECEIVED = "C://root//m"+musikerid+"hp"+(hörprobenzaehler+1)+".mid";
File f = new File(FILE_TO_RECEIVED);
if (!f.exists()) {
f.createNewFile();
}
InputStream input = t.getRequestBody();
String nextLine = "";
do {
nextLine = readLine(input, CRLF);
if (nextLine.startsWith("Content-Length:")) {
fileSize =
Integer.parseInt(
nextLine.replaceAll(" ", "").substring(
"Content-Length:".length()
)
);
}
System.out.println(nextLine);
} while (!nextLine.equals(""));
byte[] midFileByteArray = new byte[fileSize];
int readOffset = 0;
while (readOffset < fileSize) {
int bytesRead = input.read(midFileByteArray, readOffset, fileSize);
readOffset += bytesRead;
}
BufferedOutputStream bos =
new BufferedOutputStream(new FileOutputStream(FILE_TO_RECEIVED));
bos.write(midFileByteArray, 0, fileSize);
bos.flush();
t.sendResponseHeaders(200, 0);
}
private static String readLine(InputStream is, String lineSeparator)
throws IOException {
int off = 0, i = 0;
byte[] separator = lineSeparator.getBytes("UTF-8");
byte[] lineBytes = new byte[1024];
while (is.available() > 0) {
int nextByte = is.read();
if (nextByte < -1) {
throw new IOException(
"Reached end of stream while reading the current line!");
}
lineBytes[i] = (byte) nextByte;
if (lineBytes[i++] == separator[off++]) {
if (off == separator.length) {
return new String(
lineBytes, 0, i-separator.length, "UTF-8");
}
}
else {
off = 0;
}
if (i == lineBytes.length) {
throw new IOException("Maximum line length exceeded: " + i);
}
}
throw new IOException(
"Reached end of stream while reading the current line!");
}

Java - Files get corrupted when sent over socket, except when sent to localhost

I'm trying to build a file server and file client program in Java using sockets, and I've been running into some issues when trying to send files from the server to the client. Below is the code that I use to send and receive files respectively:
private void sendFile(String filePath) {
try (BufferedInputStream fileInputStream = new BufferedInputStream(new FileInputStream(filePath))) {
BufferedOutputStream outputStream = new BufferedOutputStream(socket.getOutputStream());
byte[] buffer = new byte[4096];
while (fileInputStream.read(buffer) != -1) {
outputStream.write(buffer);
outputStream.flush();
}
}
catch (IOException e) {e.printStackTrace();}
}
private void downloadFile(String fileName, long fileSize) {
try (BufferedOutputStream fileOutputStream = new BufferedOutputStream(new FileOutputStream(downloadDir + "/" + fileName));
BufferedInputStream inputStream = new BufferedInputStream(socket.getInputStream());
OutputStreamWriter writer = new OutputStreamWriter(new BufferedOutputStream(socket.getOutputStream()), "UTF-8");) {
writer.write("GET" + fileName + System.lineSeparator());
writer.flush();
long totalReceived = 0;
int bufferSize = 4096;
byte[] buffer = new byte[bufferSize];
while (totalReceived < fileSize) {
inputStream.read(buffer);
int numOfBytesToWrite = fileSize - totalReceived > bufferSize ? buffer.length : (int)(fileSize % bufferSize);
fileOutputStream.write(buffer, 0, numOfBytesToWrite);
fileOutputStream.flush();
totalReceived += numOfBytesToWrite;
}
}
catch (IOException e) {}
}
The downloaded file does get created and seems to be of the right size, but always gets corrupted and cannot be opened by any program. However, this issue does not show itself when I run the client on the same machine and connect it to "localhost" or "127.0.0.1", then there are no problems and downloaded files are not corrupted. See any issues with my code?

In your sendFile() you need to consider the return value from the read() which may be less than 4096... This value should then be used in the write call to only write out the portion of the array that has been populated...
int bytesRead = 0;
while ((bytesRead = fileInputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, bytesRead);
outputStream.flush();
}
A similar problem occurs in downloadFile(), return from read() is the actual number of bytes read, some value less than or equal to 4096...
long totalReceived = 0;
int bufferSize = 4096;
byte[] buffer = new byte[bufferSize];
while (totalReceived < fileSize) {
int bytesRead = inputStream.read(buffer);
fileOutputStream.write(buffer, 0, bytesRead);
fileOutputStream.flush();
totalReceived += bytesRead;
}
Why does your code work on localhost, but not over a network?
Atypical physical layer of a network is Ethernet, this will have a MTU of 1500 bytes. So you'll probably be seeing successive read() calls only filling 1500, or fewer bytes of your buffer...
However, localhost is optimized in the stack to bypass the physical layer which will not have this limitation. It is likely in this case successive calls will fill the full 4096 buffer, apart from the last, unless your file size is exact multiple of 4096.

Can't get my java server to accept a file transfer from my client

I have two files: a chat server and a chat client. The chat client is supposed to say that it wants to upload a file to the server. And then it uploads. However, right now, all of the messages are being sent / received properly, but when I try to get the file transfer, the only thing I get is a file with 0 bytes (which is at the path I specify inside of the server class.
Broken part of the chatclient class:
/**
* Sends a broadcast to the server
*/
public static void broadcast() throws IOException {
if (UserInput.getText() == "/upload") {
File myFile = new File (FILE_TO_SEND);
byte [] mybytearray = new byte [(int)myFile.length()];
fis = new FileInputStream(myFile);
bis = new BufferedInputStream(fis);
bis.read(mybytearray,0,mybytearray.length);
os = Socket.getOutputStream();
System.out.println("Sending " + FILE_TO_SEND + "(" + mybytearray.length + " bytes)");
os.write(mybytearray,0,mybytearray.length);
os.flush();
System.out.println("Done.");
}
System.out.println("" + UserInput.getText());
outputStream.println(UserInput.getText());
outputStream.flush();
}
Broken part of the server class:
if (input.contains("/upload")) {
byte [] mybytearray = new byte [FILE_SIZE];
InputStream is = csocket.getInputStream();
fos = new FileOutputStream(FILE_TO_RECEIVED);
bos = new BufferedOutputStream(fos);
bytesRead = is.read(mybytearray,0,mybytearray.length);
current = bytesRead;
do {
bytesRead = is.read(mybytearray, current, (mybytearray.length-current));
if (bytesRead >= 0) current += bytesRead;
}
while(bytesRead > -1);
bos.write(mybytearray, 0 , current);
bos.flush();
System.out.println("File " + FILE_TO_RECEIVED + " downloaded (" + current + " bytes read)");
}

Your copy loop is nonsense. The canonical way to copy a stream in Java is as follows:
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
where 'count' is an int, and 'buffer' is a byte[] array of length > 0. I usually use 8192.

You should try surrounding the broken code with try-catch block and print out the error message from the stack. this would give you a better idea of what is not working. It's not a solution, I know, but it's easier to find a solution if you know the exact problem.

Java HTTPConnection retrieve file size before downloading

I am trying to download a large (11MB) JSON file from a web service using this code:
public static void downloadBigFile(final String serverUrl,
final String fileName) throws MalformedURLException, IOException {
System.out.println("Downloading " + serverUrl + " (" + fileName + ")");
URL url = new URL(serverUrl);
URLConnection con = url.openConnection();
con.setConnectTimeout(10000);
con.setReadTimeout(2 * 60 * 1000);
int totalFileSize = con.getContentLength();
System.out.println("Total file size: " + totalFileSize);
InputStream inputStream = con.getInputStream();
FileOutputStream outputStream = new FileOutputStream(fileName);
// Used only for knowing the amount of bytes downloaded.
int downloaded = 0;
final byte[] buffer = new byte[1024 * 8];
int bytesRead;
bytesRead = inputStream.read(buffer);
while (bytesRead != -1) {
downloaded += bytesRead;
outputStream.write(buffer, 0, bytesRead);
bytesRead = inputStream.read(buffer);
System.out.println(String.format("%d/%d (%.2f%%)", downloaded,
totalFileSize,
(downloaded * 1.0 / totalFileSize * 1.0) * 100));
}
System.out
.println(fileName + " downloaded! (" + downloaded + " bytes)");
inputStream.close();
outputStream.close();
}
However the call to con.getContentLength() blocks the thread for several minutes, while it downloads the whole file I presume.
The problem is I need a quick way to discover file size before the download starts so I can notify the user accordingly.
Note: already tried to call con.connect() and con.getHeaderField("Content-Length").

If the server does not specify the Content-Length header, the only way to get the content length is to download the whole file and see how big it is.

File only partially uploaded to server

I'm trying to upload a file to my Spring server running on Tomcat7. It's a simple POST request, the code is below:
#RequestMapping(method = RequestMethod.POST)
public void saveFile(HttpServletRequest request, #RequestParam("file_name") String fileName) {
Logger.getLogger(FileRestAction.class).info("saving file with name " + fileName);
try {
byte[] buf = readFromRequest(request);
String filePath = writeToFile(buf, fileName);
File_ file = new File_(filePath, request.getContentType());
Logger.getLogger(FileRestAction.class).info(request.getContentType() + " " + request.getContentLength());
fService.save(file);
} catch (IOException e) {
Logger.getLogger(FileRestAction.class).error("Failed to upload file. " +
"Exception is: " + e.getMessage());
}
}
private String writeToFile(byte[] buf, String fileName) throws IOException {
String fileBasePath = ConfigurationProvider.getConfig().getString(Const.FILE_SAVE_PATH);
File file = new File(fileBasePath + fileName);
FileOutputStream fos = new FileOutputStream(file);
fos.write(buf);
fos.close();
Logger.getLogger(FileRestAction.class).info("filepath: " + file.getAbsolutePath());
return file.getAbsolutePath();
}
private byte[] readFromRequest(HttpServletRequest request) throws IOException {
InputStream is = request.getInputStream();
byte[] buf = new byte[request.getContentLength()];
is.read(buf);
is.close();
return buf;
}
Now the problem is that the file on the server is only "half-done", it's as if all the bytes aren't there. For example, if I send a 256x256 .png-file with a size of 54kB, the file written on the server is also 54kB and 256x256 in size, but the actual picture cuts off near the beginning (the rest is blank). No exceptions are thrown.
After a bit of testing I've found out that the cutoff is around 15-20Kb (images below that are fully uploaded).
Any ideas as to what might cause this?
EDIT: I changed the readFromRequest method according to what GreyBeardedGeek suggested. It's now as follows:
private byte[] readFromRequest(HttpServletRequest request) throws IOException {
InputStream is = request.getInputStream();
int fileLength = (int) request.getContentLength();
byte[] buf = new byte[fileLength];
int bytesRead = 0;
while (true) {
bytesRead += is.read(buf, bytesRead, fileLength - bytesRead);
Logger.getLogger(FileRestAction.class).info("reading file: " + bytesRead + " bytes read");
if (bytesRead == fileLength) break;
}
is.close();
return buf;
}

InputStream.read is not guaranteed to read the amount of data that you ask for.
The size that you ask for is the maximum number of bytes that it will read.
That's why the read() method returns the number of bytes actually read.
See http://docs.oracle.com/javase/6/docs/api/java/io/InputStream.html#read(byte[])
So, the answer is to read multiple times, until read() returns -1