There is such a problem. To transfer files in an Android application, we use a Socket, we encrypt and transfer to the server, there the data is decrypted and transmitted by the usual POST method, but when some data is transferred, the file is not loaded, but returns 500 The error stream terminated unexpectedly. One xlsx file is transferred. but the other is gone. We also saw that if the file is still uploaded to the server and we download it, the file is broken and at the end a piece of header Content-Disposition: form-data; name = "publicAccess" false.
ANDROID CODE
#Override
public void run() {
try {
byte[] request = new byte[4096];
byte[] reply = new byte[8192];
final InputStream inFromClient = sClient.getInputStream();
final OutputStream outToClient = sClient.getOutputStream();
SSLSocket remoteSocket = tlsConnectionService.createSSLSocket(remoteHost, remotePort);
final InputStream inFromServer = remoteSocket.getInputStream();
final OutputStream outToServer = remoteSocket.getOutputStream();
// a new thread for uploading to the server
new Thread() {
public void run() {
int bytes_read;
try {
while ((bytes_read = inFromClient.read(request)) != -1 ) {
String newReq = new String(request);
if (newReq != null) {
outToServer.write(newReq.replace(LOCAL_SOCKET_URL, remoteHost).getBytes(), 0, bytes_read);
outToServer.flush();
}
}
} catch (IOException e) {
if (!(e instanceof SocketException)) {
Log.e(M.CPP, e.toString());
}
}
}
}.start();
// current thread manages streams from server to client (DOWNLOAD)
int bytes_read;
try {
while ((bytes_read = inFromServer.read(reply)) != -1 ) {
outToClient.write(reply, 0, bytes_read);
outToClient.flush();
}
} catch (IOException e) {
Log.e(M.CPP, e.toString());
} finally {
try {
remoteSocket.close();
} catch (IOException e) {
Log.e(M.CPP, e.toString());
}
}
sClient.close();
} catch (IOException e) {
Log.e(M.CPP, e.toString());
}
}
Server CODE
#PostMapping(
value = {"/file", "/file/" },
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
produces = MediaType.APPLICATION_JSON_UTF8_VALUE
)
public ResponseEntity<FileEntryDto> upload(
#PathVariable("source") String source,
#AuthenticationPrincipal JwtUserDetails currentUser,
#Validated FileUploadRequest request
) {
return ResponseEntity.ok(
converters.fileToDto(
fileOperationService.upload(Source.fromName(source), request, currentUser.getUserId()))
);
}
The request that we see when inspecting app
request
I'm trying to block connection download from certain field that I don't want public to know it.
I tried to use some if else method but it didn't work.
This is My Controller:
#RequestMapping(value = "/path/path/{idUser}/{idDocumentType}", method = RequestMethod.GET)
public void downloadDocumentForPublic(#PathVariable("idUser") Integer idUser, #PathVariable("idDocumentType") Integer idDocumentType, HttpServletResponse response)
{
try
{
UserDocumentUpload doc = UserDocumentUploadService.idUserDocumentForPublicDownload(idUser, idDocumentType);
if (doc != null)
{
File file = userFileService.latestUserDocumentUpload(doc);
String contentType = userFileService.getResponseContentTypeByFileExtension(file);
MediaType mediaType = FileUtil.getMediaType(file);
response.setContentType(contentType);
response.setHeader("Content-Disposition", "filename=\"" + dok.getOriginalFilename() + "\"");
InputStream inputStream = new FileInputStream(file);
ServletOutputStream outputStream = response.getOutputStream();
IOUtils.copy(inputStream, outputStream);
inputStream.close();
outputStream.flush();
outputStream.close();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (MimeTypeException e) {
e.printStackTrace();
}
}
I expect to block {idDocumentType} = number 13?
What am I supposed to do?
I have a rest method for downloading files which works. But, it seems that the download doesn't start on the web client until the file is completely copied to the output stream, which can take a while for large files.
#GetMapping(value = "download-single-report")
public void downloadSingleReport(HttpServletResponse response) {
File dlFile = new File("some_path");
try {
response.setContentType("application/pdf");
response.setHeader("Content-disposition", "attachment; filename="+ dlFile.getName());
InputStream inputStream = new FileInputStream(dlFile);
IOUtils.copy(inputStream, response.getOutputStream());
response.flushBuffer();
} catch (FileNotFoundException e) {
// error
} catch (IOException e) {
// error
}
}
Is there a way to "stream" the file such that the download starts as soon as I begin writing to the output stream?
I also have a similar method that takes multiple files and puts them in a zip, adding each zip entry to the zip stream, and the download also only begins after the zip has been created:
ZipEntry zipEntry = new ZipEntry(entryName);
zipOutStream.putNextEntry(zipEntry);
IOUtils.copy(fileStream, zipOutStream);
You can use InputStreamResource to return stream result. I tested and it is started copying to output immediately.
#GetMapping(value = "download-single-report")
public ResponseEntity<Resource> downloadSingleReport() {
File dlFile = new File("some_path");
if (!dlFile.exists()) {
return ResponseEntity.notFound().build();
}
try {
try (InputStream stream = new FileInputStream(dlFile)) {
InputStreamResource streamResource = new InputStreamResource(stream);
return ResponseEntity.ok()
.contentType(MediaType.APPLICATION_PDF)
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + dlFile.getName() + "\"")
.body(streamResource);
}
/*
// FileSystemResource alternative
FileSystemResource fileSystemResource = new FileSystemResource(dlFile);
return ResponseEntity.ok()
.contentType(MediaType.APPLICATION_PDF)
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + dlFile.getName() + "\"")
.body(fileSystemResource);
*/
} catch (IOException e) {
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();
}
}
The second alternative is a partial download method.
#GetMapping(value = "download-single-report-partial")
public void downloadSingleReportPartial(HttpServletRequest request, HttpServletResponse response) {
File dlFile = new File("some_path");
if (!dlFile.exists()) {
response.setStatus(HttpStatus.NOT_FOUND.value());
return;
}
try {
writeRangeResource(request, response, dlFile);
} catch (Exception ex) {
response.setStatus(HttpStatus.INTERNAL_SERVER_ERROR.value());
}
}
public static void writeRangeResource(HttpServletRequest request, HttpServletResponse response, File file) throws IOException {
String range = request.getHeader("Range");
if (StringUtils.hasLength(range)) {
//http
ResourceRegion region = getResourceRegion(file, range);
long start = region.getPosition();
long end = start + region.getCount() - 1;
long resourceLength = region.getResource().contentLength();
end = Math.min(end, resourceLength - 1);
long rangeLength = end - start + 1;
response.setStatus(206);
response.addHeader("Accept-Ranges", "bytes");
response.addHeader("Content-Range", String.format("bytes %s-%s/%s", start, end, resourceLength));
response.setContentLengthLong(rangeLength);
try (OutputStream outputStream = response.getOutputStream()) {
try (InputStream inputStream = new BufferedInputStream(new FileInputStream(file))) {
StreamUtils.copyRange(inputStream, outputStream, start, end);
}
}
} else {
response.setStatus(200);
response.addHeader("Accept-Ranges", "bytes");
response.setContentLengthLong(file.length());
try (OutputStream outputStream = response.getOutputStream()) {
try (InputStream inputStream = new BufferedInputStream(new FileInputStream(file))) {
StreamUtils.copy(inputStream, outputStream);
}
}
}
}
private static ResourceRegion getResourceRegion(File file, String range) {
List<HttpRange> httpRanges = HttpRange.parseRanges(range);
if (httpRanges.isEmpty()) {
return new ResourceRegion(new FileSystemResource(file), 0, file.length());
}
return httpRanges.get(0).toResourceRegion(new FileSystemResource(file));
}
Spring Framework Resource Response Process
Resource response managed by ResourceHttpMessageConverter class. In writeContent method, StreamUtils.copy is called.
package org.springframework.http.converter;
public class ResourceHttpMessageConverter extends AbstractHttpMessageConverter<Resource> {
..
protected void writeContent(Resource resource, HttpOutputMessage outputMessage)
throws IOException, HttpMessageNotWritableException {
try {
InputStream in = resource.getInputStream();
try {
StreamUtils.copy(in, outputMessage.getBody());
}
catch (NullPointerException ex) {
// ignore, see SPR-13620
}
finally {
try {
in.close();
}
catch (Throwable ex) {
// ignore, see SPR-12999
}
}
}
catch (FileNotFoundException ex) {
// ignore, see SPR-12999
}
}
}
out.write(buffer, 0, bytesRead); sends data immediately to output (I have tested on my local machine). When whole data is transferred, out.flush(); is called.
package org.springframework.util;
public abstract class StreamUtils {
..
public static int copy(InputStream in, OutputStream out) throws IOException {
Assert.notNull(in, "No InputStream specified");
Assert.notNull(out, "No OutputStream specified");
int byteCount = 0;
int bytesRead;
for(byte[] buffer = new byte[4096]; (bytesRead = in.read(buffer)) != -1; byteCount += bytesRead) {
out.write(buffer, 0, bytesRead);
}
out.flush();
return byteCount;
}
}
Use
IOUtils.copyLarge(InputStream input, OutputStream output)
Copy bytes from a large (over 2GB) InputStream to an OutputStream.
This method buffers the input internally, so there is no need to use a BufferedInputStream.
The buffer size is given by DEFAULT_BUFFER_SIZE.
or
IOUtils.copyLarge(InputStream input, OutputStream output, byte[] buffer)
Copy bytes from a large (over 2GB) InputStream to an OutputStream.
This method uses the provided buffer, so there is no need to use a BufferedInputStream.
http://commons.apache.org/proper/commons-io/javadocs/api-2.4/org/apache/commons/io/IOUtils.html
You can use "StreamingResponseBody" File download would start immediately while the chunks are written to the output stream. Below is the code snippet
#GetMapping (value = "/download-single-report")
public ResponseEntity<StreamingResponseBody> downloadSingleReport(final HttpServletResponse response) {
final File dlFile = new File("Sample.pdf");
response.setContentType("application/pdf");
response.setHeader(
"Content-Disposition",
"attachment;filename="+ dlFile.getName());
StreamingResponseBody stream = out -> FileCopyUtils.copy(new FileInputStream(dlFile), out);
return new ResponseEntity(stream, HttpStatus.OK);
}
This is my code that saves image in directory:
#POST
#Path("/imagestore")
#Consumes(MediaType.MULTIPART_FORM_DATA)
// #Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public JSONObject uploadFile(#FormDataParam("file") InputStream file) {
String dirPath = servletContext.getContextPath()+"/images";
File imagesDir = new File(dirPath);
boolean dirCreated = true;
if (!imagesDir.exists()) {
try {
dirCreated = imagesDir.mkdirs();
} catch (Exception e) {
e.printStackTrace();
}
}
if (dirCreated) {
String filePath = dirPath + "/1.jpg";
JSONObject obj = new JSONObject();
// save the file to the server
try {
File newFile = new File(filePath);
boolean fileCreated = true;
if (!newFile.exists()) {
fileCreated = newFile.createNewFile();
}
if (fileCreated) {
FileOutputStream outpuStream = new FileOutputStream(newFile);
int read = 0;
byte[] bytes = new byte[1024];
while ((read = file.read(bytes)) != -1) {
outpuStream.write(bytes, 0, read);
}
outpuStream.flush();
outpuStream.close();
}
} catch (IOException e) {
try {
obj.put("error", e.getMessage());
} catch (JSONException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
e.printStackTrace();
}
String output = "File saved to server location : " + filePath;
try {
obj.put("output", output);
return obj;
} catch (JSONException e) {
e.printStackTrace();
return null;
}
}
return obj;
}
Now this works perfectly but i also need to save data to database and for that i need to consume Json data as well but i don't know how to do both these things are the same time because you can only write one consumes.
In Simple words i want to consume both json (containing user info) and Mulipart_form_data (containing image to upload on server) . So how do i do it.I'll appreciate the help :)
Some PHP sites use a page to act as a middle man for handling file downloads.
With a browser this works transparently. There seems to a be a slight pause while the php page processes the request.
However, attempting a download through Java using a URL or HttpURLConnection returns a plain html page. How could I get the file downloads working in the same way?
Edit: Here is an example link:
http://depot.eice.be/index.php?annee_g=jour&cours=poo
Edit: Here is some of the code I've been testing:
// This returns an HTML page
private void downloadURL(String theURL) {
URL url;
InputStream is = null;
DataInputStream dis;
String s;
StringBuffer sb = new StringBuffer();
try {
url = new URL(theURL);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.connect();
if (conn.getResponseCode()!=HttpURLConnection.HTTP_OK)
return;
InputStream in = conn.getInputStream();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
int i;
while ((i = in.read()) != -1) {
bos.write(i);
}
byte[] b = bos.toByteArray();
FileOutputStream fos = new FileOutputStream( getNameFromUrl( theURL ) );
fos.write(b);
fos.close();
conn.disconnect();
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
// This will throw Exceptions if the URL isn't in the expected format
public String getNameFromUrl(String url) {
int slashIndex = url.lastIndexOf('/');
int dotIndex = url.lastIndexOf('.');
System.out.println("url:" + url + "," + slashIndex + "," + dotIndex);
if (dotIndex == -1) {
return url.substring(slashIndex + 1);
} else {
try {
return url.substring(slashIndex + 1, url.length());
} catch (StringIndexOutOfBoundsException e) {
return "";
}
}
}
Considering no other constrains, you can read the redirected URL from the HTTP header and connect to that URL directly from JAVA.
There is an API setting to follow redirects automatically – but it should be true by default. How do you access the URL?
See Java API docs...
I think I've found a solution using HttpUnit. The source of the framework is available if you wish to see how this is handled.
public void downloadURL(String url) throws IOException {
WebConversation wc = new WebConversation();
WebResponse indexResp = wc.getResource(new GetMethodWebRequest(url));
WebLink[] links = new WebLink[1];
try {
links = indexResp.getLinks();
} catch (SAXException ex) {
// Log
}
for (WebLink link : links) {
try {
link.click();
} catch (SAXException ex) {
// Log
}
WebResponse resp = wc.getCurrentPage();
String fileName = resp.getURL().getFile();
fileName = fileName.substring(fileName.lastIndexOf("/") + 1);
System.out.println("filename:" + fileName);
File file = new File(fileName);
BufferedInputStream bis = new BufferedInputStream(
resp.getInputStream());
BufferedOutputStream bos = new BufferedOutputStream(
new FileOutputStream(file.getName()));
int i;
while ((i = bis.read()) != -1) {
bos.write(i);
}
bis.close();
bos.close();
}
System.out.println("Done downloading.");
}