We Keep Coding

java.net.Http.HttpClient - LineSubscriber not triggered before Body InputStream is closed

I'm using the following setup to send data to a http server:
Method: Post
Headers:
Transfer-Encoding: Chunked
Expect: continue-100
The server responds with a 100 and I continue to send data to the body of the request
The server sends a response back when it has received a data fragment (that is a line of text)
I want to read the server's data fragment response when it arrives
I'm using the java.net.http.HttpClient in the following way:
val inputStream: PipedInputStream = PipedInputStream()
val outputStream = PipedOutputStream(inputStream)
val request = HttpRequest.newBuilder()
.POST(BodyPublishers.ofInputStream {inputStream})
.expectContinue(true)
.uri(url)
.build()
httpClient.sendAsync(request, BodyHandlers.fromLineSubscriber(MyResponseSubscriber()))
....
outputStream!!.write(msg.toByteArray())
This is the code for the LineSubscriber:
private class MyResponseSubscriber : Subscriber<String> {
private var subscription: Subscription? = null
override fun onSubscribe(subscription: Subscription) {
this.subscription = subscription
subscription.request(1)
}
override fun onNext(item: String?) {
println("-> Received response from server: $item")
subscription!!.request(1)
}
override fun onError(throwable: Throwable?) {
}
override fun onComplete() {
}
}
The problem: The LineSubscriber is not triggered until I close the outputStream, so I will only receive the server responses when the client is finished with sending data. What I really want is to react to the server's messages as soon as they come in. I can see that the onSubscribe method is only called, when outputStream.close was called.
// as soon as this is called, the LineSubscriber's subscription is initiated
outputStream.close()
Is there a configuration option in the HttpClient that I'm missing for this use case?

Response header has duplicated values

I have two simple microservices through which the download file flows, but for some unknown reason some header values are duplicated.
First service where is Response build:
#POST
#Path(PRINT_PREVIEW)
fun print(#Valid printRequest: PrintRequest): Response {
// printPreview is Pair<String, ByteArrayInputStream>
val printPreview = printService.printPreview(printRequest)
return Response.ok(printPreview.second)
.header("Content-Disposition", "attachment; filename=${printPreview.first}")
.type("application/pdf")
.build()
}
And in the second microservice the response is just reused:
CLIENT interface:
#POST
#Path(PRINT_PREVIEW)
fun printPreview(request: PrintRequest): Response
CONTROLLER:
#POST
#Path(PRINT_PREVIEW)
fun printPreview(#Valid request: NewAdhocContractRequest): Response {
log.info("printPreview: request=$request")
return contractService.printPreview(request)
}
And as a result, the response header contains duplicate values:
However, when I println what header the Response has, it looks ok, no duplicates.
println(printPreview.headers)
[Content-Disposition=attachment; filename=Test_KNZ_1BB__modelace_2022722_104020.pdf,Content-Type=application/pdf,transfer-encoding=chunked]
Any idea how to avoid it? Thanks.

Kotlin/Java Sending Gzip streaming response

I am trying to send a Response containing compressed (gzip) streamingOutput. My current code is :
#Path("/")
#Get
#Produces(MediaType.APPLICATION_JSON)
fun testRessource() : Response {
val streamingOutput = TestOutputStream()
val gzipStreamingOutput = CompressedHttpOutputStream(streamingOutput)
val response = Response.ok(gzipStreamingOutput)
response.setHeader("Content-Encoding", "gzip")
return response
}
class TestOutputStream() : StreamingOutput {
override fun write(outputStream: OutputStream) {
val writer = BufferedWriter(OutputStreamWriter(outputStream))
writer.write("{ "id" : 5 }")
writer.flush()
}
}
class CompressedHttpOutputStream(private val streamingOutput: StreamingOutput) : StreamingOutput {
override fun write(outputStream: OutputStream) {
val os = GZIPOutputStream(outputStream)
streamingOutput.write(os)
os.finish()
}
}
When I do request this service, I get gibberish data in my browser.
It seems like I am missing something even though my response have the following headers correctly set : Content-Encoding : gzip and Transfer-encoding : chunked.
In my unit tests with rest-assured, if I extract the body and read it through a GzipInputStream(), I am able to retrieve the json body.
When I replace :
val os = GZIPOutputStream(outputStream)
with
val os = DeflaterOutputStream(outputStream) and Content-Encoding : deflate
The output is correctly decompressed into json.
I am using Quarkus 2.6.0.Final.
Thank you for your help and insights !

Nevermind, There is no need of CompressedHttpOutputStream. Quarkus implements gzip support for rest endpoints.
I just need to add #GZIP annotation to the endpoint and quarkus.resteasy.gzip.enabled=true in the application.properties.

org.springframework.web.client.RestClientException: Could not extract response no suitable HttpMessageConverter found

I was fetching data from a CouchDB view using spring boot and everything worked fine, I then copy-and-pasted the json content provided by the view in a .json file and provided it in a URL using NodeJS and I know get the followin error message:
org.springframework.web.client.RestClientException: Could not extract response: no suitable HttpMessageConverter found for response type [class MyClass] and content type [application/octet-stream]
Here is the line of code where the error occurs:
RestTemplate myRestTemplate = new RestTemplate();
ResponseEntity<MyClass> loadRecordResponse = myRestTemplate.getForEntity("http://localhost:4000/", MyClass.class);
Here is the code I use in NodeJS to create my REST web service:
var fs = require('fs');
var express = require('express');
var app = express();
app.get('/', function (req, res) {
fs.readFile('D:\\Userfiles\\x\\Desktop\\Local CouchDB\\loads.json', 'utf8', function (err, data) {
console.log( data );
res.end( data );
});
})
var server = app.listen(4000, function () {
var host = server.address().address
var port = server.address().port
console.log("Example app listening at http://%s:%s", host, port)
})

How to Unit Test handling of incoming Jersey MultiPart requests

We have a REST service that accepts MultiPart POST requests containing BodyParts that hold InputStreams. Inside the REST service a file might be created based on the provided data.
Task
We want to unit test the class that does the file operations based on its MultiPart input. Note: Wo do NOT want to use Jersey-Test! Grizzly does not load our spring application context which we need to inject DAO and fileHandler services into our REST service class. We explicitly want to test how our fileHandler service processes multiPart data.
The problem however is that the MultiPart that is sent out from the REST Client is not the same as the one received by the REST Server as jersey probably does something with the data to stream it or whatever. Trying to test (see below) the following setup will result in an
IllegalArgumentException [B cannot be cast to com.sun.jersey.multipart.BodyPartEntity
REST Client - sending a MultiPart
(just snippets, I omitted the obvious stuff):
byte[] bytes = FileManager.readImageFileToArray(completePath, fileType);
MultiPart multiPart = new MultiPart().
bodyPart(new BodyPart(bytes, MediaType.APPLICATION_OCTET_STREAM_TYPE)).
bodyPart(new BodyPart(fileName, MediaType.APPLICATION_XML_TYPE)).
bodyPart(new BodyPart(senderId, MediaType.APPLICATION_XML_TYPE));
ClientConfig cc = new DefaultClientConfig();
cc.getClasses().add(MultiPartWriter.class);
Client client = Client.create(cc);
WebResource webResource = client.resource(requestUrl);
Builder builder = webResource.type(MediaType.MULTIPART_FORM_DATA_TYPE);
builder = addHeaderParams(builder, headerParams);
ClientResponse response = builder.post(ClientResponse.class, multiPart);
Server Side - receiving a MultiPart
REST:
#POST
#Consumes(MediaType.MULTIPART_FORM_DATA)
#Produces(MediaType.APPLICATION_JSON)
#Transactional
public Response create(MultiPart multiPart) {
try {
multiPartReader.saveFile(multiPart);
Server Side MultiPartReader to save file from multipart
public class MultiPartReader {
public void saveFile(MultiPart multiPart) throws IOException {
BodyPartEntity bpe = (BodyPartEntity) multiPart.getBodyParts().get(0).getEntity();
InputStream inputStream = bpe.getInputStream();
// ...
BufferedImage bi = ImageIO.read(inputStream);
String fileName = getFileNameFromMultiPart(multiPart);
File file = new File(filename);
if (file.isDirectory()) {
ImageIO.write(bi, formatName, file);
} else {
file.mkdirs();
ImageIO.write(bi, formatName, file);
}
bpe.close();
}
Test - handling an incoming MultiPart in isolation
Now I want to test the MultiPartReader:
#Test
public void saveFile_should_Create_file() throws IOException {
byte[] bytes = IOUtils.toByteArray(this.getClass().getResourceAsStream(fileResource));
MultiPart multiPart = new MultiPart().
bodyPart(new BodyPart(bytes, MediaType.APPLICATION_OCTET_STREAM_TYPE)).
bodyPart(new BodyPart(fileName, MediaType.APPLICATION_XML_TYPE)).
bodyPart(new BodyPart(senderId, MediaType.APPLICATION_XML_TYPE));
multiPartReader.saveFile(multiPart);
file = new File(fileName);
Assert.assertNotNull(file);
Assert.assertTrue(file.getTotalSpace() > 0);
file.delete();
}
But, like I said I get a
IllegalArgumentException [B cannot be cast to com.sun.jersey.multipart.BodyPartEntity
at
BodyPartEntity bpe = (BodyPartEntity) multiPart.getBodyParts().get(0).getEntity();
So what can I do to emulate the send/receive handled by jersey so that my test will get the same data as my REST service does deployed on a server and requested by a REST client?
EDIT
Using
BodyPartEntity bpe = multiPart.getBodyParts().get(0).getEntityAs(BodyPartEntity.class);
will throw a
IllegalStateException: Entity instance does not contain the unconverted content
Further pointer, I think, towards having to convert the test-generated MultiPart in some way, before calling my MultiPartReader..
There has to be some method in jersey, I can call that will do this converting just the way it does, when it sends out a MultiPart request on a deployed system or maybe it is the receiving end that does some parsing when receiving the HTTP request..?

Looking at the jersey-multipart docs I see:
"It is not currently possible to know ahead of time what Java class the application would prefer to use for each individual body part, so an appropriate Provider cannot be selected. Currently, the unparsed content of each body part is returned (as a byte array) in the entity property of the returned BodyPart} instance, and the application can decide what further steps are needed based on the headers included in that body part. The simplest technique is to examine the received BodyPart, and then call the getEntityAs() method once you know which implementation class you would prefer."
It looks like you need to follow that suggestion. Examine the byte array returned in your Server Side MultiPartReader code:
multiPart.getBodyParts().get(0).getEntity();
...and call getEntityAs() on the BodyPart.