We Keep Coding

since HttpExchange require that sendResponseHeaders(int,long) must be before getResponseBody(), how do I handle transmitting file?

The following code is no longer feasible
server.createContext("/", exchange -> {
URL url = new URL("");
try (final BufferedInputStream in = new BufferedInputStream(url.openStream()); final BufferedOutputStream out = new BufferedOutputStream(exchange.getResponseBody())) {
} catch (IOException e) {
}
});
server.start();
The reason is that sendResponseHeaders(int,long) must be before getResponseBody() and the long in sendResponseHeaders(int,long) is the length of content to be written into ResponseBody...but now I cannot figure out the value.
so what should I do?
The size of the file is unknown: it can be smaller than 1MB but also can be bigger than 200MB, so creating a big enough byte array cannot be accepted. To convert the file to String or bytes directly without buffer cannot be accepted too.
What can I do?
Thanks a lot.

Had the same issue - in my case, helped to read the documentation in more detail...
responseLength - if > 0, specifies a fixed response body length and
that exact number of bytes must be written to the stream acquired from
getResponseBody(), or else if equal to 0, then chunked encoding is
used, and an arbitrary number of bytes may be written. if <= -1, then
no response body length is specified and no response body may be
written.
So zero should work here as well

Edit, merge and return InputStream

I'm working on a Java plug-in that takes two variables of bespoke type and returns one of the same type. This type can be convertet from and to InputStream. I will need to crop the first one at the end and the second one at the beginning and then merge the two before I return them. What is the best intermediate type to use here that will make all the croping and merging simple and easy to maintain? I don't want to go via string beacause I have tried that and it messed up the encoding.

After some more diffing around and testing I found a solution myself:
public Document concat(final Document base, final Document addOn) throws IOException
{
// Convert Documents to InputStreams
InputStream isBase = base.getInputStream();
InputStream isAddOn = addOn.getInputStream();
// Create new variable as base minus last 33 bytes
int baseLength = isBase.available();
byte[] baBase = IOUtils.toByteArray(isBase);
byte[] baEndLessBase = Arrays.copyOf(baBase, baseLength-33);
// Create new variable as addOn minus first 60 bytes
int addOnLength = isAddOn.available();
byte[] baAddOn = IOUtils.toByteArray(isAddOn);
byte[] baHeadLessAddOn = Arrays.copyOfRange(baAddOn, 60, addOnLength);
// Combine the two new variables
byte[] baResult = new byte[baEndLessBase.length + baHeadLessAddOn.length];
System.arraycopy(baEndLessBase, 0, baResult, 0, baEndLessBase.length);
System.arraycopy(baHeadLessAddOn, 0, baResult, baEndLessBase.length, baHeadLessAddOn.length);
// Debug
// FileUtils.writeByteArrayToFile(new File("baEndLessBase.pcl"), baEndLessBase);
// FileUtils.writeByteArrayToFile(new File("baHeadLessAddOn.pcl"), baHeadLessAddOn);
// FileUtils.writeByteArrayToFile(new File("baResult.pcl"), baResult);
// Convert to Document
Document result = new Document(baResult);
result.passivate();
return result;
}
It uses a simple byte Array and then the Arrays and IOUtils classes does most of the heavy lifting.

ByteArrayOutputStream: Odd behavior

I'm writing a simple client-server application and I wanted to be able to take the attributes of a Header class, turn them into a byte[], send them to the other host, and then convert them back into an easily parsed Header. I was using a ByteArrayOutputStream to do this, but the results were not what I expected. For example, just to test it in main() I had:
Header h = Header();
h.setSource(111);
h.setDest(222);
h.setSeq(333);
h.setAck(444);
byte[] header = Header.convertHeaderToByteArray();
Header newHeader = new Header(headerArray);
Where convertHeaderToByteArray() looked like:
public byte[] convertHeaderToByteArray() {
byte[] headerArray;
ByteArrayOutputStream byteStream = new ByteArrayOutputStream(44);
byteStream.write(srcPort);
byteStream.write(dstPort);
byteStream.write(seqNum);
byteStream.write(ackNum);
byteStream.write(controlBits);
headerArray = byteStream.toByteArray();
return headerArray;
}
And the Header(headerArray) constructor:
public Header(byte[] headerArray) {
ByteArrayInputStream header = new ByteArrayInputStream(headerArray);
srcPort = header.read();
dstPort = header.read();
seqNum = header.read();
ackNum = header.read();
}
This definitely did not behave as expected. When I looked at those values at the end, srcPort was correct (111), dstPort was correct (222), seqNum was not correct (77), and ackNum was not correct (188).
After hours of reading and tinkering I couldn't get it right, so I tried to use ByteBuffer instead. Viola, correct results.
What is going on here? I read the documentation for both and although I spotted some differences I'm not seeing what the source of my error is.

OutputStream.write(int) writes a single byte. See the Javadoc. If you want to write wider values, you will have to use the writeXXX() methods of DataOutputStream, and the corresponding readXXX() methods of DataInputStream to read them.

Why CodedInputStream.readRawVarint64() is reading all the bytes from underlying stream?

Here is a sample code demonstrating the problem.
ByteArrayOutputStream bos = new ByteArrayOutputStream();
CodedOutputStream cos = CodedOutputStream.newInstance(bos);
cos.writeRawVarint64(25);
cos.flush();
bos.write("something else".getBytes());
System.out.println("size(bos) = " + bos.size()); // This gives 15
ByteArrayInputStream bis = new ByteArrayInputStream(bos.toByteArray());
CodedInputStream cis = CodedInputStream.newInstance(bis);
System.out.println("size(bis) = " + bis.available()); // This gives 15
long l = cis.readRawVarint64();
System.out.println(cis.getTotalBytesRead()); // This gives 1, which is correct
System.out.println("Raw varint64 = " + l); // This gives 25, which is correct
System.out.println("size(bis) = " + bis.available()); // This now gives 0!!
All I am trying to do is to encode a 64 bit integer and add some more data to the payload. I can read the encoded data correctly. But for some reason, it clears the underlying stream after that. Any one know why this is happening? How can I read the varint from stream and read the remaining bytes as indicated by the varint?
Any help would be great

I have no idea what codedinputstream does but it could very well buffer the input meaning it reads e.g. 100 bytes a time.
Either way you should not wrap an inputstream B around an inputstream A and continue reading from A specifically because you don't know what B does.
For instance maybe B must look ahead in the data to form some conclusion or it uses buffering or...
Additional note: available() is usually a bad idea though it should work correctly specifically on a bytearrayinputstream.
EDIT:
In conclusion: just continue reading from the codedinputstream, don't try to read from the underlying one.

Java equivalent of the VB Request.InputStream

I have a web service that I am re-writing from VB to a Java servlet. In the web service, I want to extract the body entity set on the client-side as such:
StringEntity stringEntity = new StringEntity(xml, HTTP.UTF_8);
stringEntity.setContentType("application/xml");
httppost.setEntity(stringEntity);
In the VB web service, I get this data by using:
Dim objReader As System.IO.StreamReader
objReader = New System.IO.StreamReader(Request.InputStream)
Dim strXML As String = objReader.ReadToEnd
and this works great. But I am looking for the equivalent in Java.
I have tried this:
ServletInputStream dataStream = req.getInputStream();
byte[] data = new byte[dataStream.toString().length()];
dataStream.read(data);
but all it gets me is an unintelligible string:
data = [B#68514fec
Please advise.

You need to use a ByteArrayOutputStream, like this:
ServletInputStream dataStream = req.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int r;
byte[] buffer = new byte[1024*1024];
while ((r = dataStream.read(data, 0, buffer.length)) != -1) {
baos.write(buffer, 0, r);
}
baos.flush();
byte[] data = baos.toByteArray();

You are confusing with printing of java arrays. When you print any java object it is transformed to its string representation by implicit invocation of toString() method. Array is an object too and its toString() implementation is not too user friendly: it creates string that contains [, then symbolic type definition (B for byte in your case, then the internal reference to the array.
If you want to print the array content use Arrays.toString(yourArray). This static method creates user-friendly string representation of array. This is what you need here.
And yet another note. You do not read your array correctly. Please take a look on #Petter`s answer (+1) - you have to implement a loop to read all bytes from the stream.