I recently encountered some problems with java.net.http.HttpClient that comes with JDK 11. I don't know how to use file upload. Found the ofInputStream() in java.net.http.BodyPublishers. I don't know if I using this method file upload.
Here are the examples I wrote.
public HttpResponse<String> post(String url, Supplier<? extends InputStream> streamSupplier, String... headers) throws IOException, InterruptedException {
HttpRequest.Builder builder = HttpRequest.newBuilder()
.uri(URI.create(url))
.headers(headers)
.POST(null == streamSupplier ?
HttpRequest.BodyPublishers.noBody() : HttpRequest.BodyPublishers.ofInputStream(streamSupplier));
HttpRequest request = builder.build();
log.debug("Execute HttpClient Method:『{}』, Url:『{}』", request.method(), request.uri().toString());
return client.send(request, HttpResponse.BodyHandlers.ofString());
}
The HttpRequest type provide factory method for creating request publisher for handling body type such as file:
HttpRequest.BodyPublishers::ofFile(Path)
You can update your method:
public HttpResponse<String> post(String url, Path file, String... headers) throws IOException, InterruptedException {
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(url))
.headers(headers)
.POST(null == file ? HttpRequest.BodyPublishers.noBody() :
HttpRequest.BodyPublishers.ofFile(file))
.build();
log.debug("Execute HttpClient Method:『{}』, Url:『{}』", request.method(),
request.uri().toString());
return client.send(request, HttpResponse.BodyHandlers.ofString());
}
The java.net.http.HttpClient handles bytes supplied through the BodyPublisher as raw body data, without any interpretation. Whether you use HttpRequest.BodyPublishers::ofFile(Path) or HttpRequest.BodyPublishers::ofByteArray(byte[]) is therefore semantically irrelevant: what changes is simply how the bytes that will be transmitted are obtained.
In case of file upload - your server probably expects that the request body will be formatted in certain ways. It might also expect some specific headers to be transmitted with the request (such as Content-Type etc). The HttpClient will not do that magically for you. This is something you need to implement at the caller level.
you may use the method by:
public void uploadLocalFileToRemote(String notEncodedUrlStr, String remoteFilename, String localSourceDir, String localFilename) {
Path sourcePath = Path.of(localSourceDir, localFilename);
if(!sourcePath.toFile().canRead())
{
System.err.println("please check the local file existance/readability: " + sourcePath.toAbsolutePath());
return;
}
FileInputStream ins = null;
try {
ins = new FileInputStream(sourcePath.toFile());//FileNotFoundException extends IOException
BufferedInputStream buf_ins = new BufferedInputStream(ins);
Supplier<? extends InputStream> streamSupplier = new Supplier<BufferedInputStream>() {
#Override
public BufferedInputStream get() {
return buf_ins;
}
};
//HttpResponse<String> response = post(notEncodedUrlStr, streamSupplier,
HttpResponse<String> response = post(notEncodedUrlStr, () -> buf_ins,
"User-Agent", "Java 11 HttpClient Bot", "Content-type", "application/octet-stream",
"accept", "*/*", "fileName", remoteFilename);
// print response:
System.out.println(response.version().name() + " " + response.statusCode());
// print response headers
HttpHeaders headers = response.headers();
headers.map().forEach((k, v) -> System.out.println(k + ":" + v));
// print response body
String body = response.body();
System.out.println(body);
} catch (IOException | InterruptedException e) {
e.printStackTrace();
}
}
another consideration is how your server side is implemented. here assume the server side will using http 1.1 "chunked". and configured a directory for remoteFilename.
Related
Suffered a lot in finding how to mock http response . Mocking http request for the same library was easy . Thought to create a thread here , to save your time should you need it .
Requirement ->
Wanted to mock a HttpResponse that is returned when any HttpRequest object is executed . (Note - this is specifically for google client api library)
//creating mockContent for httpRequest
MockHttpContent mockHttpContent = new MockHttpContent();
String content = new String("requestBody");
mockHttpContent.setContent(str.getBytes());
//mocking httpResponse and linking to httpRequest's execution
HttpTransport transport =
new MockHttpTransport() {
#Override
public LowLevelHttpRequest buildRequest(String method, String url) throws IOException {
return new MockLowLevelHttpRequest() {
#Override
public LowLevelHttpResponse execute() throws IOException {
MockLowLevelHttpResponse result = new MockLowLevelHttpResponse();
result.setContent("responseBody");
result.setContentEncoding("UTF-8");//this is very important
result.setHeaderNames(List.of("header1","header2"));
result.setHeaderValues(List.of("header1","header2"));
return result;
}
};
}
};
HttpRequest httpRequest = transport.createRequestFactory().buildPostRequest(HttpTesting.SIMPLE_GENERIC_URL,mockHttpContent);
//getting httpResponse from httpRequest
httpResponse = httpRequest.execute();
//condition to verify the content (body) of the response
assertEquals("responseBody",IOUtils.toString(httpResponse.getContent()));
I am using Micronaut #Client to call external service which returns me response
of type FullNettyClientHttpResponse and it has body in the form of CompositeByteBuf(freed, components=1); I want to convert CompositeByteBuf to a human readable toString message but it has failing with IllegalReferenceCountException. Please provide suggestion how I can get the text message here.
#Client(value = "url")
public interface MyClient {
#Post(consumes = MediaType.APPLICATION_XML, produces = MediaType.APPLICATION_XML)
HttpResponse call(String body);
}
class service{
void method(){
HttpResponse httpResponse = client.call(request);// returns FullNettyClientHttpResponse with body "Optional[CompositeByteBuf(freed, components=1)]"
Optional<CompositeByteBuf> reBody = httpResponse.getBody(CompositeByteBuf.class);
if(reBody.isPresent()){
CompositeByteBuf b=reBody.get();
byte[] req = new byte[b.readableBytes()];
b.readBytes(req);
String body = new String(req, CharsetUtil.UTF_8).substring(0, req.length -
System.getProperty("line.separator").length());
System.out.println("server receive order : " + body);
}
}
I tried to get the message using toString but failed with IllegalReferenceCountException.
b.toString(Charset.defaultCharset()); // Method threw 'io.netty.util.IllegalReferenceCountException' exception.
toString returns CompositeByteBuf.
b.toString(); //CompositeByteBuf(freed, components=1);
You must specify the body type in the client if you want micronaut to keep the body of the response.
For example:
HttpResponse<String> call(String body);
I need to create a rest service in java which will in turn connect to another rest service for file download. For now, I just need to transfer the file from the other backend to client but in future some processing/transformations would be done.
For all the web services in my project, we are using spring rest (for providing as well as consuming the services).
My question is what would be the appropriate way of doing it considering that the files would be large and I don't want to run into OutOfMemory errors.
People in some other posts have suggested to use streams on both the ends but is that really possible? For this, do I need to write the file on disk first?
My current code for file download (consumer) -
public BackendResponse<byte[]> callBackendForFile(BackendRequest request) {
String body = null;
ResponseEntity<byte[]> responseEntity = null;
URI uri = createURI(request);
MultiValueMap<String, String> requestHeaders = getHeadersInfo(request.getHttpRequest());
if (HttpMethod.GET.equals(request.getMethod())) {
responseEntity = restTemplate.exchange(uri, request.getMethod(),
new HttpEntity<String>(body, requestHeaders), byte[].class);
} else {
LOG.error("Method:{} not supported yet", request.getMethod());
}
BackendResponse<byte[]> response = new BackendResponse<>();
response.setResponse(responseEntity);
return response;
}
My client code (provider):
#RequestMapping(value = "/file", method = RequestMethod.GET, produces = "application/xml")
#ResponseBody
public void downloadFileWithoutSpring(HttpMethod method, HttpServletRequest httpRequest,
HttpServletResponse httpResponse) {
BackendRequest request = new BackendRequest(method,
httpRequest.getRequestURI(), httpRequest.getQueryString(), httpRequest);
BackendResponse<byte[]> backendResponse = dutyplanService.getFile(request);
ResponseEntity<byte[]> response = backendResponse.getResponse();
httpResponse.addHeader("Content-Disposition", "attachment; filename=\"" + "attachment.zip" + "\"");
httpResponse.getOutputStream().write(response.getBody());
httpResponse.flushBuffer();
}
Note: The code above doesn't work somehow as the attachment downloaded is a corrupt file
I don't think you will need to create that file on server as long as you are having the bytearray content of it received from another server.
You can try changing value of produces annotation to the value application/zip (or application/octet-stream, depending on the target browser) instead of 'application/xml'
you can pass HttpServletResponse#getOutputStream() directly in restTemplate and write it without save file in server.
public void getFile(HttpServletResponse response) throws IOException {
restTemplate.execute(
"http://ip:port/temp.csv",
HttpMethod.GET,
null,
clientHttpResponse -> {
StreamUtils.copy(clientHttpResponse.getBody(), response.getOutputStream());
return null;
}
);
}
note that after call getFile(), you should close outputStream like this
response.getOutputStream().close()
I'm in the midst of testing my application which is using an HTTP-server. Instead of mocking I decided to go with a HTTP server fixture. Meaning that I do not have to mock any productional code. To accomplish this goal I currently chose for a free to use 3rd party library fixd.
I was able to successfully create several unit tests - which are working by means of a GET request. Most are quite simple, i.e.:
#Test
public void verifyConnectionTest()
{
try
{
final String body = FileUtils.readFileToString(RESOURCE);
final String path = "/";
this.server.handle(Method.GET, path).with(
new HttpRequestHandler() {
#Override
public void handle(final HttpRequest request,
final HttpResponse response)
{
response.setStatusCode(200);
response.setContentType("text/xml");
response.setBody(body);
}
});
// Setting up my HTTP client
// Execute some tasks
// asserting of everything was valid
}
catch (final IOException e)
{
fail(e.getMessage());
}
}
But I now have to send a POST request with multipart/form-data. Which does not make much of a difference other than changing the method and content-type:
#Test
public void executeStepTest()
{
try
{
final String body = FileUtils.readFileToString(SERVICE_RESPONSE);
final String path = "/";
this.server.handle(Method.POST, path, "multipart/form-data").with(
new HttpRequestHandler() {
#Override
public void handle(final HttpRequest request,
final HttpResponse response)
{
response.setStatusCode(200);
response.setContentType("text/xml");
response.setBody(body);
}
});
// Setting up my HTTP client
// Execute some tasks
// asserting of everything was valid
}
catch (final IOException e)
{
fail(e.getMessage());
}
}
However I get the following error: [ERROR] could not find a handler for POST - / - multipart/form-data; boundary=bqCBI7t-VW1xaJW7BADmTiGMg9w_YM2sHH8ukJYx and my guess is that fixd doesn't recognize the boundary-party. Since the documentation does not show an example I'm quite stuck on this part.
I tried using some wildcards such as '*', no succes. Thus; I need a way to either tell fixd to accept that boundary or use some wildcards I didn't yet discover. Any help would be great, thanks!
I've been making some debug and it seems to be that the problem is in the fixd core.
Basically, fixd indexes every RequestHandlerImpl by a HandlerKey (which includes ContentType as part of the key) in the map handlerMap. See method org.bigtesting.fixd.core.FixtureContainer#resolve.
...
HandlerKey key = new HandlerKey(method, route, contentType);
RequestHandlerImpl handler = handlerMap.get(key);
if (handler == null) {
// Error
}
...
Problem: When the request is multipart/form-data, boundary data (which it's generated dinamically every request) is part of the content type. So, any handler is found in handlerMap because the key changes with every running.
I've made a little test only to check that this is the cause of the problem, passing the contentType to fixd server.handle after the creation of the multipart request, and it works fine.
See the test below:
#Test
public void verifyConnectionTest_multipart() {
try {
// 1. Create multipart request (example with http-commons 3.1)
PostMethod filePost = new PostMethod(url);
Part[] parts = { new StringPart("param", "value") };
MultipartRequestEntity request = new MultipartRequestEntity(parts, filePost.getParams());
filePost.setRequestEntity(request);
// 2. fixd server handle (passing the request content type)
this.server.handle(Method.POST, "/", request.getContentType()).with(
new HttpRequestHandler() {
#Override
public void handle(final HttpRequest request,
final HttpResponse response) {
response.setStatusCode(200);
response.setContentType("text/xml");
}
});
// 3. Execute multipart request
HttpClient client = new HttpClient();
int status = client.executeMethod(filePost);
// 4. Assertions
Assert.assertEquals(200, status);
} catch (Exception e) {
Assert.fail(e.getMessage());
}
}
Hope it helps you to clarify the problem. Cheers
This was a bug in fixd, and has been fixed in version 1.0.3. Your original code should work using this new version of fixd.
I'm currently using Jersey as a proxy REST api to call another RESTful web service. Some of the calls will be passed to and from with minimal processing in my server.
Is there a way to do this cleanly? I was thinking of using the Jersey Client to make the REST call, then converting the ClientResponse into a Response. Is this possible or is there a better way to do this?
Some example code:
#GET
#Path("/groups/{ownerID}")
#Produces("application/xml")
public String getDomainGroups(#PathParam("ownerID") String ownerID) {
WebResource r = client.resource(URL_BASE + "/" + URL_GET_GROUPS + "/" + ownerID);
String resp = r.get(String.class);
return resp;
}
This works if the response is always a success, but if there's a 404 on the other server, I'd have to check the response code. In other words, is there clean way to just return the response I got?
There is no convenience method as far as I am aware. You can do this:
public Response getDomainGroups(#PathParam("ownerID") String ownerID) {
WebResource r = client.resource(URL_BASE + "/" + URL_GET_GROUPS + "/" + ownerID);
ClientResponse resp = r.get(ClientResponse.class);
return clientResponseToResponse(resp);
}
public static Response clientResponseToResponse(ClientResponse r) {
// copy the status code
ResponseBuilder rb = Response.status(r.getStatus());
// copy all the headers
for (Entry<String, List<String>> entry : r.getHeaders().entrySet()) {
for (String value : entry.getValue()) {
rb.header(entry.getKey(), value);
}
}
// copy the entity
rb.entity(r.getEntityInputStream());
// return the response
return rb.build();
}
for me answer from Martin throw:
JsonMappingException: No serializer found for class sun.net.www.protocol.http.HttpURLConnection$HttpInputStream
Change from
rb.entity(r.getEntityInputStream());
to
rb.entity(r.getEntity(new GenericType<String>(){}));
helped.