How to download 10GB file with Angularjs, Java, Tomcat,Spring and REST? - java

When I download small file , everything is OK , but I need some way to download large files . If file large, Blob didn't created , haven't enough memory.
Download file without save on client , directly save to disk with many requests on the server or something like that.
My code on server is :
#RequestMapping(value = "/oneFile/{name}", method = RequestMethod.GET)
public void getOneFile( #PathVariable("name") String name, HttpServletResponse response,HttpServletRequest request) {
....
InputStream in = new FileInputStream(new File(file.getAbsolutePath()));
org.apache.commons.io.IOUtils.copy(in, response.getOutputStream());
response.flushBuffer();
On client and this is work for small size:
backupFileServer.downloadOneFileBrow(data)
.success(function(databack) {
var file = new Blob([ databack ], {
type : 'application/csv'
});
var fileURL = URL.createObjectURL(file);
var a = document.createElement('a');
a.href = fileURL;
a.target = '_blank';
a.download = data;
document.body.appendChild(a);
a.click();
})
.error(function() {
alert($scope.DOWNLOAD_ERROR);
});
I tried something like this but didn't work :
var a = document.createElement('a');
a.href = 'data:attachment/csv;charset=utf-8,' + encodeURI(databack);
a.target = '_blank';
a.download = data;
document.body.appendChild(a);
a.click();
How someone idea how to do this or some example or code ....
Thank you in advance

You need to split your file into multiple files and rebuild the file with your server.
Because it's bad practice to upload a big file without splitting it, the user can be disconnect at 99% of the upload and he need to upload the whole file again.
An example here : https://flowjs.github.io/ng-flow/
A good example here : http://ryansouthgate.com/2015/12/24/upload-amazon-s3-using-angularjs/ with Amazon S3 and their SDK.

Related

Grails. Upload files to temp folder and display them in gsp

My intention is to upload images and store them in a temp folder. Then I want to display these images in the .gsp views. The process I've been trying to make it to work is something like this:
First, upload the file from input:
<input id="inputImg" type="file" accept="image/*">
Create the file:
def saveFile(MultipartFile inputImg) {
def contentType = inputImg.getContentType()
def originalFilename = inputImg.getOriginalFilename()
def extension = FilenameUtils.getExtension(originalFilename)
String tempPath = System.getProperty("java.io.tmpdir") + "/uploads"
File file = new File("$tempPath/$originalFilename")
FileUtils.forceMkdirParent(file)
inputImg.transferTo(file)
if (contentType == 'application/octet-stream') {
contentType = MimeTypeUtils.getContentTypeByFileName(originalFilename)
}
Path filePath = Paths.get(file.toString())
Path path = Paths.get(tempPath)
Path relativePath = path.relativize(filePath)
Avatar avatar = new Avatar(
path: relativePath.toString(),
contentType: contentType,
name: originalFilename,
extension: extension
)
}
Once is stored in the temp folder, I found this solution but I'm not sure if it's the best way to do it. I'm trying to process the image with base64 encoding before sending it to the view:
def filename = user?.avatar?.name
def file = new File("$tempPath/$filename")
def base64file = file?.readBytes()?.encodeBase64()
And finally show it in the gsp:
<img alt="img" src="data:image/*;base64,${base64file}"/>
I would like to know if there is another best way to do this process, I don't know if I'm missing something or if this isn't a good procedure to manage with files and images...
You are using the inline images with Base64 encoding which is good for displaying relatively small images (up to 5k). The advantage of this approach is that you dump the page WITH images in a single HTTP-connection.
If the images grow considerably larger (> 1MB), then you can not use caching and other nice features, so you have to send the data over the line over and over again and that would slow down user experience.
Another way would be to deliver each image in a separate request.
You could define a controller action like:
class ImageController {
def image(String id){
def file = new File("$tempPath/$id")
if( !file.exists() )
render status: 404
else{
response.contentType = 'image/jpeg'
response.withOutputStream{ it << file.readBytes() }
}
}
}
then in your GSP you put:
<img alt="img" src="${g.createLink( controller:'image', action:'image', id:user.avatar.name )}"/>

download a file from rest api is giving me some garbage value

I am trying to download a file from rest API, I am writing code in Java and react. but when i call that rest api it is not downloading that file instead gives me some garbage
#POST
#Path("/{loginId}")
#Produces(MULTIPART_FORM_DATA)
#Consumes(APPLICATION_JSON)
public Response downloadExportedFile(#PathParam("loginId") String loginId, ExportFileDTO fileDetails) {
File exportFolder = new File("C://directory");
File[] listOfFiles = exportFolder.listFiles();
for (File listOfFile : listOfFiles) {
if (listOfFile.getName().equals(fileDetails.getFileName())) {
InputStream is = new FileInputStream(listOfFile.getAbsolutePath());
byte[] buffer = IOUtils.toByteArray(is);
return Response.ok(listOfFile)
.header("content-disposition", "attachment; filename=" + new File(listOfFile.getName()).getName())
.type(MediaType.APPLICATION_OCTET_STREAM_TYPE).build();
}
}
It should download the file instead it is giving me output as
PK!b�h^�[Content_Types].xml �(����N�0E�H�C�-Jܲ#5��Q>�ēƪc[�ii����B�j7���{2��h�nm���ƻR����U^7/���%��rZY�#1__�f��q��R4D�AJ�h>����V�ƹ�Z�9����NV�8ʩ����ji){^��-I�"{�v^�P!XS)bR�r��K�s(�3�`c�0��������7M4�����ZƐk+�|\|z�(���P��6h_-[�#�!���Pk���2n�}�?�L��� ��%���d����dN"m,�ǞDO97�~��ɸ8�O�c|n���E������B��!$}�����;{���[����2���PK!�U0#�L_rels/.rels �(���MO�0��H�����ݐBKwAH�!T~�I����$ݿ'T�G�~����<���!��4��;#�w����qu*&r�Fq���v�����GJy(v��*����K��#F��D��.W ��=��Z�MY�b���BS�����7��ϛז��
?�9L�ҙ�sbgٮ|�l!��USh9i�b�r:"y_dl��D���|-N��R"4�2�G�%��Z�4�˝y�7 ë��ɂ�����PK!
You have to change the associated mimetype by changing the the parameter of the #Produces annotation which basically describes what type of data you transmit in your response.
It should become:
#Produces("application/vnd.ms-excel")
According to this other stackoverflow question you should change the #Produces annotation to #Produces(MediaType.APPLICATION_OCTET_STREAM).
According to this second stackoverflow question you are asking an impossible question.
Out of curiosity I reproduced your problem here : see the full gist
If you change #POST to #GET it starts working
If you keep #POST, it has to be posted from a real form and can't post application/json
Finally, posting application/json means React is doing a programmatic XmlHTTPRequest. The above gist shall convince you there is no user prompt in that case
When you say it 'is giving me output', you're not telling where and how the post was requested . You will have to adapt that part.
actually It is APPLICATION_OCTET_STREAM response for a file. we have to handle download functionality at client side AS per Nate's answer here, the response of Ajax request is not recognized by a browser as a file. It will behave in the same way for all Ajax responses. You need to trigger the download popup manually.
downloadFile(fileDetails) {
let username = getUserName();
return fetch(`/files/${username}`, {
method: 'POST',
body: JSON.stringify(fileDetails)
}).then(response => {
return response.blob();
}).then(response => {
let blob = new Blob([response], {type: 'application/octet-stream'});
let fileUrl = window.URL.createObjectURL(blob);
Files.triggerDownload(fileUrl, fileDetails.fileName);
}).catch((error) => {
//myerror
});
}
static triggerDownload(url, fileName) {
let a = document.createElement('a');
a.setAttribute('href', url);
a.setAttribute('download', fileName);
a.click();
}
This will download the file at client machine

angular2 file download in java

How could download a file using Angular2 and Java?
There is a GET HTTP call which returns the file data:
If receive a byte[] array, the file (in this case ODT, but could be
other format) opens as a document with the literal byte content
written in it: "UEsDBBQAAAgAAPBZjklexjIMJwAAACcAAAA"
If receive a blob object , it shows
"{"binaryStream":{},"wrappedBlob":{"binaryStream":{}}}" in the
document
Angular2 code:
goFile() {
//This service calls #angular/http for a GET request and returs the response
//ends up doing:
// this.http.get(url, {responseType: ResponseContentType.Blob})
// .map(res => res.blob())
this.myService.subscribe(result=> { this.saveFile(result); });
return false;
}
downloadFile(file: any) {
var blob = new Blob([file], {type: 'application/vnd.oasis.opendocument.text'});
var url= window.URL.createObjectURL(blob);
window.open(url);
}
Java code just reads the file and returns a blob or byte array (in each case tried):
byte[] file = FileUtils.getFile("E:/file.odt");
return file;
byte[] file = FileUtils.getFile("E:/file.odt");
Blob blob = Hibernate.createBlob(fichero);
return blob;
UPDATE
I believe the problem comes that data is being received as a json object, something setup in the system i am working.
Have tried intead to return byte[] from java and try to convert to a blob in Angular2 (for a txt file):
//file is returned by a call to http.get which has a map:
// .map(res => res.text())
var blob = new Blob([file], {type: 'text/plain'});
var objectUrl = URL.createObjectURL(blob);
window.open(objectUrl);
but returns a page with the byte content ("YWJj"), obviously the content received is not converted to a proper blob object.
Also tried with same result:
var byteArray = new Uint8Array(file);
var blob = new Blob([byteArray], {type: 'text/plain'});
there is not solution for different mime types without using additional plugins?

how to upload file in java for given SAS URI

Sample C# code:
static void UploadFile(string sasUrl, string filepath)
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("x-ms-version", Version);
client.DefaultRequestHeaders.Add("x-ms-client-request-id", SessionGuid);
StringBuilder sb = new StringBuilder("<?xml version=\"1.0\" encoding=\"utf-8\"?><BlockList>");
foreach (byte[] chunk in GetFileChunks(filepath))
{
var blockid = GetHash(chunk);
HttpRequestMessage chunkMessage = new HttpRequestMessage()
{
Method = HttpMethod.Put,
RequestUri = new Uri(sasUrl + "&timeout=90&comp=block&blockid=" + WebUtility.UrlEncode(blockid)),
Content = new ByteArrayContent(chunk)
};
chunkMessage.Headers.Add("x-ms-blob-type", "BlockBlob");
chunkMessage.Content.Headers.Add("MD5-Content", blockid);
TimeAction("Uploading chunk " + blockid + " took {0} ms", () =>
{
var response = client.SendAsync(chunkMessage).Result;
});
sb.Append("<Latest>");
sb.Append(blockid);
sb.Append("</Latest>");
}
sb.Append("</BlockList>");
Trace.WriteLine(sb.ToString());
HttpRequestMessage commitMessage = new HttpRequestMessage()
{
Method = HttpMethod.Put,
RequestUri = new Uri(sasUrl + "&timeout=90&comp=blocklist"),
Content = new StringContent(sb.ToString())
};
TimeAction("Commiting the blocks took {0} ms", () =>
{
var commit = client.SendAsync(commitMessage).Result;
});
}
}
I am stuck at the point where I've to upload a file. Also want to know what the reason is to commit in given code?
my progress so far is :
public static void uploadFile(String sasUrl , String filepath , String sessionGuid)
{
File file = new File(filepath);
FileInputStream fileInputStream=null;
Response reply = new Response();
HttpClient client = HttpClientBuilder.create().build();
HttpPost request = new HttpPost(sasUrl);
request.setHeader("x-ms-version", "2013-08-15");
request.setHeader("x-ms-client-request-id", sessionGuid);
StringBuilder sb = new StringBuilder("<?xml version=\"1.0\" encoding=\"utf-8\"?><BlockList>");
}
}
Note: I cannot run the code multiple times as I cannot spam the server. Any suggestions will be appreciated
Referring to : https://msdn.microsoft.com/en-us/library/windows/hardware/dn800660(v=vs.85).aspx
According to the reference code in C#, it seems to be using the REST API Put Block List to upload a file as a block blob.
So you can refer to the REST API reference without refering to the C# sample to use httpclient to construct the request for uploading.
However, the simple way is using Azure Storage SDK for Java. To upload a file, you just need to use the class CloudBlockBlob to upload a file with the function upload(InputStream sourceStream, long length), please refer to the tutorial https://azure.microsoft.com/en-us/documentation/articles/storage-java-how-to-use-blob-storage/#upload-a-blob-into-a-container.
The sas url seems like https://myaccount.blob.core.windows.net/mycontainer/myblob?comp=blocklist&...
Here is the code as example.
URL sasUrl = new URL("<sas-url>");
try
{.
CloudBlockBlob blob = new CloudBlockBlob(sasUrl)
File source = new File(filePath);
blob.upload(new FileInputStream(source), source.length());
}
catch (Exception e)
{
// Output the stack trace.
e.printStackTrace();
}
As reference, please see the javadocs for Azure Java Storage SDK.

Sending in-memory generated .docx files from server to client with Spark

I am creating a web application using the Spark Java framework. The front-end is developed using AngularJS.
I want to generate a .docx file on the server (in-memory) and send this to the client for download.
To achieve this I created an angular service with the following function being called after the user clicks on a download button:
functions.generateWord = function () {
$http.post('/api/v1/surveys/genword', data.currentSurvey).success(function (response) {
var element = angular.element('<a/>');
element.attr({
href: 'data:attachment;charset=utf-8;application/vnd.openxmlformats-officedocument.wordprocessingml.document' + response,
target: '_blank',
download: 'test.docx'
})[0].click();
});
};
On the server, this api call gets forwarded to the following method:
public Response exportToWord(Response response) {
try {
File file = new File("src/main/resources/template.docx");
FileInputStream inputStream = new FileInputStream(file);
byte byteStream[] = new byte[(int)file.length()];
inputStream.read(byteStream);
response.raw().setContentType("data:attachment;chatset=utf-8;application/vnd.openxmlformats-officedocument.wordprocessingml.document");
response.raw().setContentLength((int) file.length());
response.raw().getOutputStream().write(byteStream);
response.raw().getOutputStream().flush();
response.raw().getOutputStream().close();
return response;
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
I have tried to solve this in MANY different ways and I always end up with a corrupted 'test.docx' that looks like this:
Solved it by using blobs and specifying the response type as 'arraybuffer' in the $http.post api call. The only bad thing with this solution (as far as I know) is that it doesn't play well with IE, but that's a problem for another day.
functions.generateWord = function () {
$http.post('/api/v1/surveys/genword', data.currentSurvey, {responseType: 'arraybuffer'})
.success(function (response) {
var blob = new Blob([response], {type: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'});
var url = (window.URL || window.webkitURL).createObjectURL(blob);
var element = angular.element('<a/>');
element.attr({
href: url,
target: '_blank',
download: 'survey.docx'
})[0].click();
});
};
I think what went wrong was that the byte stream got encoded as plain text when I tried to create a URL with:
href: 'data:attachment;charset=utf-8;application/vnd.openxmlformats-officedocument.wordprocessingml.document' + response
thus corrupting it.
When using blobs instead, I get a "direct" link to the generated byte stream and no encoding is done on it since the response type is set to 'arraybuffer'.
Note that this is just my own reasoning of why things went wrong with the original code. I might be terribly wrong, so feel free to correct me if that's the case.

Categories