We Keep Coding

java HttpServer receive large file in HttpHandler

I am building a quick HTTP server for my application and it is required to receive a large file (7z archive - 250MB to 1GB).
The server is based on com.sun.net.httpserver.HttpServer and com.sun.net.httpserver.HttpHandler.
So far I made the following which obviously does not work - it receive 44 bytes and finishes.
#Override
public void handle(HttpExchange httpExchange) throws IOException
{
String requestMethod = httpExchange.getRequestMethod();
if (requestMethod.equalsIgnoreCase("POST"))
{
Headers responseHeaders = httpExchange.getResponseHeaders();
responseHeaders.set("Content-Type", "text/plain");
httpExchange.sendResponseHeaders(200, 0);
InputStream inputStream = httpExchange.getRequestBody();
byte[] buffer = new byte[4096];
int lengthRead;
int lengthTotal = 0;
FileOutputStream fileOutputStream = new FileOutputStream(FILENAME);
while ((lengthRead = inputStream.read(buffer, 0, 4096)) > 0)
{
fileOutputStream.write(buffer, 0, lengthRead);
lengthTotal += lengthRead;
}
fileOutputStream.close();
System.out.println("File uploaded - " + lengthTotal + " bytes total.");
httpExchange.getResponseBody().close();
}
}
The request look like this:
How do I receive a file?

java zip to binary format and then decompress

I have a task that
read a zip file from local into binary message
transfer binary message through EMS as String (done by java API)
receive transferred binary message as String (done by java API)
decompress the binary message and then print it out
The problem I am facing is DataFormatException while decompress the message.
I have no idea which part went wrong.
I use this to read file into binary message:
static String readFile_Stream(String fileName) throws IOException {
File file = new File(fileName);
byte[] fileData = new byte[(int) file.length()];
FileInputStream in = new FileInputStream(file);
in.read(fileData);
String content = "";
System.out.print("Sent message: ");
for(byte b : fileData)
{
System.out.print(getBits(b));
content += getBits(b);
}
in.close();
return content;
}
static String getBits(byte b)
{
String result = "";
for(int i = 0; i < 8; i++)
result = ((b & (1 << i)) == 0 ? "0" : "1") + result;
return result;
}
I use this to decompress message:
private static byte[] toByteArray(String input)
{
byte[] byteArray = new byte[input.length()/8];
for (int i=0;i<input.length()/8;i++)
{
String read_data = input.substring(i*8, i*8+8);
short a = Short.parseShort(read_data, 2);
byteArray[i] = (byte) a;
}
return byteArray;
}
public static byte[] unzipByteArray(byte[] file) throws IOException {
byte[] byReturn = null;
Inflater oInflate = new Inflater(false);
oInflate.setInput(file);
ByteArrayOutputStream oZipStream = new ByteArrayOutputStream();
try {
while (! oInflate.finished() ){
byte[] byRead = new byte[4 * 1024];
int iBytesRead = oInflate.inflate(byRead);
if (iBytesRead == byRead.length){
oZipStream.write(byRead);
}
else {
oZipStream.write(byRead, 0, iBytesRead);
}
}
byReturn = oZipStream.toByteArray();
}
catch (DataFormatException ex){
throw new IOException("Attempting to unzip file that is not zipped.");
}
finally {
oZipStream.close();
}
return byReturn;
}
The message I got is
java.io.IOException: Attempting to unzip file that is not zipped.
at com.sourcefreak.example.test.TibcoEMSQueueReceiver.unzipByteArray(TibcoEMSQueueReceiver.java:144)
at com.sourcefreak.example.test.TibcoEMSQueueReceiver.main(TibcoEMSQueueReceiver.java:54)
After check, the binary message does not corrupted after transmission.
Please help to figure out the problem.

Have you tried using InflaterInputStream? Based on my experience, using Inflater directly is rather tricky. You can use this to get started:
public static byte[] unzipByteArray(byte[] file) throws IOException {
InflaterInputStream iis = new InflaterInputStream(new ByteArrayInputStream(file));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[512];
int length = 0;
while ((length = iis.read(buffer, 0, buffer.length) != 0) {
baos.write(buffer, 0, length);
}
iis.close();
baos.close();
return baos.toByteArray();
}

I finally figure out the problem.
The problem is the original file is a .zip file, so I should use zipInputStream to unzip the file before further processing.
public static byte[] unzipByteArray(byte[] file) throws IOException {
// create a buffer to improve copy performance later.
byte[] buffer = new byte[2048];
byte[] content ;
// open the zip file stream
InputStream theFile = new ByteArrayInputStream(file);
ZipInputStream stream = new ZipInputStream(theFile);
ByteArrayOutputStream output = new ByteArrayOutputStream();
try
{
ZipEntry entry;
while((entry = stream.getNextEntry())!=null)
{
//String s = String.format("Entry: %s len %d added %TD", entry.getName(), entry.getSize(), new Date(entry.getTime()));
//System.out.println(s);
// Once we get the entry from the stream, the stream is
// positioned read to read the raw data, and we keep
// reading until read returns 0 or less.
//String outpath = outdir + "/" + entry.getName();
try
{
//output = new FileOutputStream(outpath);
int len = 0;
while ((len = stream.read(buffer)) > 0)
{
output.write(buffer, 0, len);
}
}
finally
{
// we must always close the output file
if(output!=null) output.close();
}
}
}
finally
{
// we must always close the zip file.
stream.close();
}
content = output.toByteArray();
return content;
}
This code work for zip file containing single file inside.

mkdir function in java

I am trying to create a program that will download resources from a webpage into a file. I created an mkir function that creates a directory whose name is a hexadecimal version of the hash of a given String. I then, created a saveResource function that saves a resource in a file, as well as in a byte array. However, when I try saving the resource into the file I get an error message stating: java.io.FileNotFoundException: 648451a1 (Is a directory)
Here are the functions:
public static File mkdir(String s) throws IOException
{
String dirname = s;
s = Integer.toHexString(dirname.hashCode());
File directory = new File(s);
if (!directory.exists() && !directory.mkdir())
throw new IOException("can't make directory for " + s);
return directory;
}
public static byte[] saveResource(File dir, String urlString,
String argURLString)
throws IOException, URISyntaxException
{
URL u = new URL(urlString);
URLConnection uc = u.openConnection();
urlString = uc.getContentType();
int contentLength = uc.getContentLength();
try (InputStream raw = uc.getInputStream()) {
InputStream in = new BufferedInputStream(raw);
byte[] data = new byte[contentLength];
int offset = 0;
while (offset < contentLength) {
int bytesRead = in.read(data, offset, data.length - offset);
if (bytesRead == -1) break;
offset += bytesRead;
}
if (offset != contentLength) {
throw new IOException("Only read " + offset
+ " bytes; Expected " + contentLength + " bytes");
}
urlString = u.getFile();
urlString = urlString.substring(urlString.lastIndexOf('/') + 1);
try (FileOutputStream fout = new FileOutputStream(dir)) {
fout.write(data);
fout.flush();
}
return data;
}
}

The error is simple and coming because you can't write data in directory.
Try to print dir.isDirectory() to confirm if it's directory. Since its part of argument, check the caller method.

Using ImageIO.read to read image through inputstream [duplicate]

I am sending a bufferedImage over a socket and I am using the example found in this post:
Sender
BufferedImage image = ....;
ImageIO.write(image, "PNG", socket.getOutputStream());
Receiver
BufferedImage image = ImageIO.read(socket.getInputStream());
It works - IF, and ONLY IF, I close the sender's outputStream after this line:
ImageIO.write(image, "PNG", socket.getOutputStream());
Is there anything I can do apart from closing the outputStream?
Also, is there anything else I can do to avoid using ImageIO altogether? It seems to take ages to do anything.
Also note that reading or writing to the hard disk in anyway should be avoided at all costs due to performance issues. I need to make this transfer as fast as possible, (I'm experimenting and trying to create a client similar to VNC and saving each screenshot to the hard disk would greatly slow down everything)..
#Jon Skeet
Edit 3:
Sender: (Note that I am sending a JPG image not a PNG).
int filesize;
OutputStream out = c.getClientSocket().getOutputStream();
ByteArrayOutputStream bScrn = new ByteArrayOutputStream();
ImageIO.write(screenshot, "JPG", bScrn);
byte[] imgByte = bScrn.toByteArray();
bScrn.flush();
bScrn.close();
filesize = bScrn.size();
out.write(new String("#FS " + filesize).getBytes()); //Send filesize
out.write(new String("#<IM> \n").getBytes()); //Notify start of image
out.write(imgByte); //Write file
System.out.println("Finished");
Reciever: (where input is the socket input stream)
Attempt #1:
String str = input.toString();
imageBytes = str.getBytes();
InputStream in = new ByteArrayInputStream(imageBytes);
BufferedImage image = ImageIO.read(in);
in.close();
System.out.println("width=" + image.getWidth());
(failed: Nullpointer exception on getWidth() line)
I understand this error to mean "corrupt image" because it couldn't initialize it. correct?
Attempt #2:
byte[] imageBytes = new byte[filesize];
for (int j = 0; i < filesize; i++)
{
imageBytes[j] = (byte) input.read();
}
InputStream in = new ByteArrayInputStream(imageBytes);
BufferedImage image = ImageIO.read(in);
in.close();
System.out.println("width=" + image.getWidth());
(failed: Nullpointer exception on getWidth() line)
Attempt #3:
if (filesize > 0)
{
int writtenBytes = 0;
int bufferSize = client.getReceiveBufferSize();
imageBytes = new byte[filesize]; //Create a byte array as large as the image
byte[] buffer = new byte[bufferSize];//Create buffer
do {
writtenBytes += input.read(buffer); //Fill up buffer
System.out.println(writtenBytes + "/" + filesize); //Show progress
//Copy buffer to the byte array which will contain the full image
System.arraycopy(buffer, 0, imageBytes, writtenBytes, client.getReceiveBufferSize());
writtenBytes+=bufferSize;
} while ((writtenBytes + bufferSize) < filesize);
// Read the remaining bytes
System.arraycopy(buffer, 0, imageBytes, writtenBytes-1, filesize-writtenBytes);
writtenBytes += filesize-writtenBytes;
System.out.println("Finished reading! Total read: " + writtenBytes + "/" + filesize);
}
InputStream in = new ByteArrayInputStream(imageBytes);
BufferedImage image = ImageIO.read(in);
in.close();
(failed: Reciever gives: Null pointer exception)
Attempt 4:
int readBytes = 0;
imageBytes = new byte[filesize]; //Create a byte array as large as the image
while (readBytes < filesize)
{
readBytes += input.read(imageBytes);
}
InputStream in = new ByteArrayInputStream(imageBytes);
BufferedImage image = ImageIO.read(in);
in.close();
System.out.println("width=" + image.getWidth());
(failed: sender gives: java.net.SocketException: Connection reset by peer: socket write error)
Attempt #5:
Using Jon skeet's code snippet, the image arrives, but only partially. I saved it to a file (1.jpg) to see what was going on, and it actually sends 80% of the image, while the rest of the file is filled with blank spaces. This results in a partially corrupt image. Here is the code I tried: (note that captureImg() is not at fault, saving the file directly works)
Sender:
Socket s = new Socket("127.0.0.1", 1290);
OutputStream out = s.getOutputStream();
ByteArrayOutputStream bScrn = new ByteArrayOutputStream();
ImageIO.write(captureImg(), "JPG", bScrn);
byte imgBytes[] = bScrn.toByteArray();
bScrn.close();
out.write((Integer.toString(imgBytes.length)).getBytes());
out.write(imgBytes,0,imgBytes.length);
Reciever:
InputStream in = clientSocket.getInputStream();
long startTime = System.currentTimeMillis();
byte[] b = new byte[30];
int len = in.read(b);
int filesize = Integer.parseInt(new String(b).substring(0, len));
if (filesize > 0)
{
byte[] imgBytes = readExactly(in, filesize);
FileOutputStream f = new FileOutputStream("C:\\Users\\Dan\\Desktop\\Pic\\1.jpg");
f.write(imgBytes);
f.close();
System.out.println("done");
The sender still gives a Connection reset by peer: socket write error.
Click here for full sized image

One option would be to write the image to a ByteArrayOutputStream so you can determine the length, then write that length to the output stream first.
Then on the receiving end, you can read the length, then read that many bytes into a byte array, then create a ByteArrayInputStream to wrap the array and pass that to ImageIO.read().
I'm not entirely surprised that it doesn't work until the output socket is closed normally - after all, a file which contains a valid PNG file and then something else isn't actually a valid PNG file in itself, is it? So the reader needs to read to the end of the stream before it can complete - and the "end" of a network stream only comes when the connection is closed.
EDIT: Here's a method to read the given number of bytes into a new byte array. It's handy to have as a separate "utility" method.
public static byte[] readExactly(InputStream input, int size) throws IOException
{
byte[] data = new byte[size];
int index = 0;
while (index < size)
{
int bytesRead = input.read(data, index, size - index);
if (bytesRead < 0)
{
throw new IOException("Insufficient data in stream");
}
index += bytesRead;
}
return data;
}

for other StackOverflow users like me.
In "Jon Skeet's" answer. Modify the following line of readExactly method.
<<original Line>>
index += size;
<<modified Line>>
index += bytesRead;
To get the full image data.

public static void main(String[] args) {
Socket socket = null;
try {
DataInputStream dis;
socket = new Socket("192.168.1.48",8000);
while (true) {
dis = new DataInputStream(socket.getInputStream());
int len = dis.readInt();
byte[] buffer = new byte[len];
dis.readFully(buffer, 0, len);
BufferedImage im = ImageIO.read(new ByteArrayInputStream(buffer));
jlb.setIcon(new ImageIcon(im));
jfr.add(jlb);
jfr.pack();
jfr.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
jfr.setVisible(true);
System.gc();
}
} catch (Exception e) {
e.printStackTrace();
}
finally {
try {
socket.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
In 192.168.1.48:8000 machine python server running and i got stream in java code

Convert File object to byte array

I am developing an application which I capture videos in it. I am saving the recorded videos to the phone. What I want to do is to convert the saved files to byte arrays.

// Serialize to a byte array
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutput out = new ObjectOutputStream(bos);
out.writeObject(yourObject);
out.close();
// Get the bytes of the serialized object
byte[] buf = bos.toByteArray();
//write bytes to private storage on filesystem
FileOutputStream fos = new FileOutPutStream("/....your path...");
fos.write(buf);
fos.close();

You can use this code which may help you:
public static byte[] getBytesFromFile(File file) throws IOException {
InputStream is = new FileInputStream(file);
System.out.println("\nDEBUG: FileInputStream is " + file);
// Get the size of the file
long length = file.length();
System.out.println("DEBUG: Length of " + file + " is " + length + "\n");
/*
* You cannot create an array using a long type. It needs to be an int
* type. Before converting to an int type, check to ensure that file is
* not loarger than Integer.MAX_VALUE;
*/
if (length > Integer.MAX_VALUE) {
System.out.println("File is too large to process");
return null;
}
// Create the byte array to hold the data
byte[] bytes = new byte[(int)length];
// Read in the bytes
int offset = 0;
int numRead = 0;
while ((offset < bytes.length) && ((numRead=is.read(bytes, offset, bytes.length-offset)) >= 0)) {
offset += numRead;
}
// Ensure all the bytes have been read in
if (offset < bytes.length) {
throw new IOException("Could not completely read file " + file.getName());
}
is.close();
return bytes;}