I want to initiate a byte stream (from reading an InputStream) from one micronaut service to another, using declarative clients. I found this Github issue that deals with the client side of what I'm trying to solve.
The proposed solution, on the client side, is to pass in a #Body Flowable<byte[]> content on the client. I tried that using the sample code, but now I am stuck on how to consume the data on the server side.
For the endpoint's implementation, I similarly take in a Flowable<byte[]> body parameter and I subscribe to it. The problem is that it's not receiving any data. When the client calls the endpoint it ends up idle timing out.
I verified the created flowable has events holding the byte[] data by subscribing to it on the client side.
Does something like the following work:
public class InputStreamWrapper implements byte[] {
private final InputStream inputStream;
private final int chunkSize;
public InputStreamWrapper(InputStream inputStream, int chunkSize) {
this.inputStream = inputStream;
this.chunkSize = chunkSize;
}
#Override
public int read(byte[] b, int off, int len) throws IOException {
return inputStream.read(b, off, len);
}
#Override
public int read() throws IOException {
byte[] chunk = new byte[chunkSize];
int read = inputStream.read(chunk);
if (read < 0) {
return -1;
}
return chunk[0] & 0xff;
}
}
e.g.
#Client("github")
public interface GithubUploadClient {
#Post(value = "/repos/${github.owner}/{github.repo}/releases/{releaseId}/assets", produces = MediaType.APPLICATION_OCTET_STREAM)
Asset uploadAsset(#QueryValue("name") String name, #Positive long releaseId, #Body InputStreamWrapper content);
}
Related
I want to send a request of a file through WebClient by POST method, and I need to send the file as byte[] to get right response.
I made MultipartFile file to byte[], and then I thought I need to use BodyInserters
to make this body contains byte[] but I don't know how to make that request body.
How to send a POST request that contains byte array by WebClient?
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.reactive.function.BodyInserters;
import org.springframework.web.reactive.function.client.WebClient;
#RestController
public class ApiController {
#PostMapping(value = "/update")
public String update(#RequestParam("file") MultipartFile file, #RequestParam("uri") String uri ) {
String result = "{error : error}";
byte[] byteArr;
BodyInserters byteArrInserter;
try {
byteArr = file.getBytes();
A? publisher = B?.C?; // I don't know what will be right for those A?, B?, C?
byteArrInserter = BodyInserters.fromDataBuffers(publisher); // In fact, I'm not sure this method will be good for this situation, either.
} catch (Exception e) {
System.out.println(e);
}
WebClient client = WebClient.builder()
.codecs(configurer -> configurer.defaultCodecs().maxInMemorySize( 1024*1024*1024 * 2)) // 2GB
.build();
try {
result = client
.post()
.uri(uri)
.body(byteArrInserter)
.retrieve().bodyToMono(String.class).block();
} catch (Exception e) {
System.out.println(e);
}
return result;
}
}
I don't know how to send binary content with WebClient, but there are some other options. First there is SpringBoot own RestTemplate class which is an alternative to WebClient. Here is an comparative article about them both: Spring WebClient vs. RestTemplate. If you wish to use third party Http clients that definitely can send binary content you can look at 2 popular choices:
Apache Http Client
OK Http Client
And finally, I wrote my own Http client that is part of Open-source MgntUtils library written and maintained by me. This Http client is very simplistic but does the job. I am actually at the process of adding the feature that allows to upload binary info. I haven't published it yet, but you can download the branch or look at the code how I do it there. The branch is located here The HttpClient class is here. See method public String sendHttpRequest(String requestUrl, HttpMethod callMethod, ByteBuffer data) throws IOException at line 162. The method with which I tested this method looks like this:
private static void testHttpClientBinaryUpload() {
try {
byte[] content = Files.readAllBytes(Paths.get("C:\\Michael\\Personal\\pics\\testPic.jpg"));
HttpClient client = new HttpClient();
Integer length = content.length;
Files.write(Paths.get("C:\\Michael\\tmp\\testPicOrig.jpg"), content);
client.setRequestHeader("Content-Length", length.toString());
String result = client.sendHttpRequest("http://localhost:8080/upload", HttpMethod.POST, ByteBuffer.wrap(content));
System.out.println(result);
System.out.println("HTTP " + client.getLastResponseCode() + " " + client.getLastResponseMessage());
} catch (Exception e) {
System.out.println(TextUtils.getStacktrace(e, "com.mgnt."));
}
}
And server side Springboot rest controller that receives the request looks like this:
#RestController
#RequestMapping("/upload")
public class UploadTestController {
#PostMapping
public ResponseEntity<String> uploadTest(HttpServletRequest request) {
try {
String lengthStr = request.getHeader("content-length");
int length = TextUtils.parseStringToInt(lengthStr, -1);
if(length > 0) {
byte[] buff = new byte[length];
ServletInputStream sis =request.getInputStream();
int counter = 0;
while(counter < length) {
int chunkLength = sis.available();
byte[] chunk = new byte[chunkLength];
sis.read(chunk);
for(int i = counter, j= 0; i < counter + chunkLength; i++, j++) {
buff[i] = chunk[j];
}
counter += chunkLength;
if(counter < length) {
TimeUtils.sleepFor(5, TimeUnit.MILLISECONDS);
}
}
Files.write(Paths.get("C:\\Michael\\tmp\\testPic.jpg"), buff);
}
} catch (Exception e) {
System.out.println(TextUtils.getStacktrace(e));
}
return ResponseEntity.ok("Success");
}
}
I am planning to publish the new version of the library in few days and this feature will be included. In any case here is the link to Maven Artifacts, Github and Javadoc
i want to return array byte inside object on java spring rest.
my expected response object.
public class ResponseByteArray {
private String fileName;
private byte[] file;
public String getFileName() {
return fileName;
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public byte[] getFile() {
return file;
}
public void setFile(byte[] file) {
this.file = file;
}
}
#RequestMapping(value = "/getDataByte", method = RequestMethod.GET)
public ResponseEntity<?> uploadUImageGet() throws IOException {
ResponseByteArray response = new ResponseByteArray();
File fileImage = new File("D://images//download1.jpg");
byte[] fileContent = Files.readAllBytes(fileImage.toPath());
response.setFileName("testFile.jpg");
response.setFile(fileContent);
return new ResponseEntity<ResponseByteArray>(response, HttpStatus.OK);
}
but i the response of rest API is json format and value of file is string encodeBase64 instead if byte[]. what is happend to the process ? Rest can not resturn byte[] inseide object ?:
{
"fileName": "testFile.jpg",
"file": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkGBxIQEg8PEhIPFRIPDw8QEA8QDw8PEBAPFREWFhUVFRUYHSggGBolHRUVITEhJSkrLi4uFx8zODMsNygtLisBCgoKDg0NFQ8PFSsdFR0tLS0tKysrKy0tKy0tNy03LS0rKy0tLSsrKzc3LS0rLSsrKy0rKzctLTcrLSstNy0tK//AABEIAQMAwgMBIgACEQEDEQH/xAAbAAABBQEBAAAAAAAAAAAAAAADAAECBAUGB//EAD8QAAIBAgMFBQUGBQIHAQAAAAECAAMRBCExBRJBUWEGMnGBkRMiUqHRFGJyscHwQpKisuEjkxUXM3OC0vEH/8QAGQEBAQEBAQEAAAAAAAAAAAAAAAECAwUE/8QAIBEBAQEAAwADAAMBAAAAAAAAABEBAgMSITFBEyJRB ...
}
If you're talking about Rest as a protocol, then it can return bynary stream in a sense that HTTP protocol can work with binary streams. Here is an example:
#GetMapping(value = "/images/{id}")
public ResponseEntity<byte[]> get(#PathVariable String id) {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.IMAGE_JPEG);
return new ResponseEntity<>(service.get(id), headers, HttpStatus.OK);
}
However if you're talking about REST as a protocol that returns JSON/XML (which is something that I assumed reading your question), then, in this sense: REST returns textual (not binary) data (because you assume it) in a form of JSON or maybe XML.
In any case how do you represent a byte array as a textual data?
One way is to encode it into String representation which is exactly what Base64 encoding does.
Note that if you check the size of byte array and compare it to the length of the string created by Base64 encoding you'll notice that Base64 has a pretty significant overhead.
I am using the feature of Icecast to add a repeatedly message containing the current metadata of the stream. It is enabled with following header:
request.headers().set("Icy-MetaData", "1");
After this, in the HTTP response there is a header containing the interval, which defines that the server sends metadata again after x bytes of the normal stream content. In addition, I want to store the stream content between the metadata tags.
My question is now, how is the best way to achieve this? One thaught was to use the ByteToMessageDecoder to "filter" the metadata tags, but the bytes were already decoded to an HttpObject. My current handler looks like this:
public class StreamClientHandler extends SimpleChannelInboundHandler<HttpObject> {
private int metadataInterval = 0;
#Override
protected void messageReceived(ChannelHandlerContext handlerContext, HttpObject message) throws Exception {
if(message instanceof HttpResponse) {
HttpResponse response = (HttpResponse) message;
this.metadataInterval = Integer.parseInt(response.headers().get("icy-metaint").toString());
}
if(message instanceof HttpContent) {
HttpContent content = (HttpContent) message;
if(content instanceof LastHttpContent) {
// Close connection
handlerContext.channel().close();
}
}
}
}
Thank you!
I'm using Codename One to upload videos to Vimeo using their respective API's. I'm using a multipart request to actually upload the file, but a response is required to find the upload status. How can I get the response?
public void doFileUpload(String url, String filename) throws IOException {
MultipartRequest req = new MultipartRequest() {
int chr;
StringBuffer sb = new StringBuffer();
public String response = "";
public void readResponse(InputStream input) throws IOException {
response = Util.readToString(input);
Log.p("File Response->" + response);
}
protected void handleErrorResponseCode(int code, String message)
{
Log.p("Error Response->" + message);
}
protected void readHeaders(Object connection) throws IOException {
String val = getHeader(connection, "MyHeaderName");
Log.p("Header Response->" + val);
}
protected void handleException(Exception err) {
Dialog.show(
"Connection Err!!",
"Are you connected to the internet? Check your connection",
"Ok", null);
}
};
req.setUrl(url);
req.setPost(true);
String mime = "application/mp4";
req.addData("file_data", filename, mime);
req.setFilename("file_data", filename);
req.setReadResponseForErrors(true);
req.addResponseCodeListener(new ActionListener() {
public void actionPerformed(ActionEvent ev) {
try {
NetworkEvent event = (NetworkEvent) ev;
Log.p("Err Rsp Code->" + event.getResponseCode());
Log.p("ResponseCodeListener:");
Log.p(ev.toString() );
I suggest you use the addData method that accepts a file URL rather than a byte array since you might exceed device memory with the byte array version (e.g. for a larger video or low memory device).
You can read the response just like any other connection request:
Derive the connection request and override: protected void readResponse(InputStream input)
Use addResponseListener to bind a listener to the users response.
Use addToQueueAndWait to detect that the upload finished then just invoke getResponseData() to get the byte array data.
I had the same problem! on the back-end, I am using Spring boot, my upload controller was simply tagged as #Controller when changed to #RestController, it worked fine and I got readResponse(InputStream input) callback, where I handle the response as JSON.
I'm pretty new to coding with streams but now I have to do it for more efficient Http coding.
Here is code that I wrote(not working) to get ContentProducer for HttpClient:
public static ContentProducer getContentProducer(final Context context, final UUID gId)
{
return new ContentProducer()
{
public void writeTo(OutputStream outputStream) throws IOException
{
outputStream = new Base64.OutputStream(new FileOutputStream(StorageManager.getFileFromName(context, gId.toString())));
outputStream.flush();
}
};
}
I'm using Base64 streaming encoder from here: http://iharder.sourceforge.net/current/java/base64/
My goal is to use this function to provide data that I read from binary file to HttpClient as base64 encoded stream.
This is how I consume content producers:
private MyHttpResponse processPOST(String url, ContentProducer requestData)
{
MyHttpResponse response = new MyHttpResponse();
try
{
HttpPost request = new HttpPost(serviceURL + url);
HttpEntity entity = new EntityTemplate(requestData);
request.setEntity(entity);
ResponseHandler<String> handler = new BasicResponseHandler();
response.Body = mHttpClient.execute(request, handler);
}
catch (HttpResponseException e)
{
}
catch (Throwable e)
{
}
return response;
}
I have another ContentProducer which works with GSON streamer(and it's working):
public ContentProducer getContentProducer(final Context context)
{
return new ContentProducer()
{
public void writeTo(OutputStream outputStream) throws IOException
{
Gson myGson = MyGsonWrapper.getMyGson();
JsonWriter writer = new JsonWriter(new OutputStreamWriter(outputStream, "UTF-8"));
writer.beginObject();
// stuff
writer.endObject();
writer.flush();
}
};
}
My question is: How to make my first example work. Am I doing it correctly? Right now I get empty post on server side, so it seems like no data coming through.
EDIT:
I believe that the issue is that you are being passed an OutputStream in your ContentProviders writeTo() method, and you are overwriting it with your own OutputStream. The contract of that class/method probably requires you to write your data to the OutputStream passed to you.
Based on looking at the Android Documentation, I do not see a way for you to specify the OutputStream to use, so you will probably need to just write out the data of the file to the OutputStream that is passed in.
Instead, you should do something like this:
public void writeTo(OutputStream outputStream) throws IOException
{
byte[] buf = createByteBufferFromFile(StorageManager.getFileFromName(context, gId.toString()));
outputStream.write(buf);
outputStream.flush();
}
Of course you will need to provide an implementation to the createByteBufferFromFile(...) method that I mention. Again, you should note that it is not likely that you will be using the Base64 OutputStream, so if that is a necessity, then you may have to find a different approach.