I have developed a REST Service with RESTEasy 3.0.12 deployed on WildFly 9 to upload a file in local file system.
I was trying to upload a zip file (testing with POST MAN sending file as binary not multi-part) I could successfully upload CSV, TXT, file format but when I try to send a ZIP file it saves correctly in the local file system but when I try to unzip it says
Headers error
Unconfirmed start of archive
Warnings: headers error
There are some data after the end of the payload data
Code:
#Path("/uploadZip")
#POST
#Produces(MediaType.APPLICATION_JSON)
public Response uploadZip(#Context HttpServletRequest req, InputStream payload){
// save to filesystem local.
Writer wr = null;
String tempFileName = System.getProperty("java.io.tmpdir");
try {
wr = new BufferedWriter(new FileWriter(tempFileName));
IOUtils.copy(payload, wr, "UTF-8");
} catch (IOException e) {
return errorResponse;
} finally {
// closing writer and stream
IOUtils.closeQuietly(wr);
IOUtils.closeQuietly(payload);
}
...
}
Does anyone know how to save a good zip file?
***General information on your Headers error:
Zip files contain local headers and a central directory at the end of the file. I don't know the gruesome details--and I won't attempt to expand upon them, but if you're getting a headers error then your zip file is corrupt. I'm not sure what you mean by there is some data after the "payload data."
***Thoughts on your code: Let's think conceptually.
Your endpoint is capturing an InputStream object, which is the superclass of all InputStream objects that represent an input stream of bytes. Your file is essentially wrapped in an object that you can read from byte-for-byte. The first problem I see is that you declare a Writer to write those bytes from the InputStream. Remember--Readers and Writers are for writing character streams, Input and Output streams are for byte streams. This explains why your CSV and TXT files are successful. It is important to know that difference and remember it!
#Path("/uploadZip")
#POST
#Produces(MediaType.APPLICATION_JSON)
public Response uploadZip(#Context HttpServletRequest req, InputStream payload){
OutputStream fos = new FileOutputStream(new File("path/to/filename.zip");
try {
byte[] bufferSize = new byte[1024];
int i = 0;
while ((i = payload.read(bufferSize)) != -1) {
fos.write(buf, 0, i);
}
} catch (IOException e) {
return errorResponse;
} finally {
fos.close();
}
...
}
I am confident that it will work. Let me know if this gives you any trouble. Best of luck!
Related
I have a method in the resource class, uploadFile. It calls a method that takes in a file as a parameter and then split the files into parts before writing said parts to disk.
I would like to know how to write a client test for it, assuming it's possible to upload a file without using #FormDataParam InputStream. Because I don't think I need it in this case.
I've seen plenty of examples for uploading files with Jersey that takes in InputStream and FormDataContentDisposition as #FormDataParam but I'm not sure if that's mandatory.
#POST
#Path("/uploadfile/{file}")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFile(#PathParam("file") File file, #FormDataParam("file") FormDataContentDisposition fileMetaData) throws IOException {
FileSplit.splitFile(file); //Write file to disk
String output = "File successfully uploaded";
return Response.status(200).entity(output).build();
}
//Method to write file to disk
public static void splitFile(File f) throws IOException {
int partCounter = 1;
int sizeOfFiles = (int) (f.length() / 3);
byte[] buffer = new byte[sizeOfFiles];
String fileName = f.getName();
try (FileInputStream fis = new FileInputStream(f);
BufferedInputStream bis = new BufferedInputStream(fis)) {
int bytesAmount = 0;
while ((bytesAmount = fis.read(buffer)) != -1) {
String filePartName = String.format("%s.%03d", fileName, partCounter++);
File newFile = new File("D:\\", filePartName);
try (FileOutputStream out = new FileOutputStream(newFile)) {
out.write(buffer, 0, bytesAmount);
}
}
}
}
If u use Socket to sending requests like HttpUrlConnection u can write file bytes in outPutStream and in server side use HttpServletRequest and process request.getInputStream() to write bytes in your storage.
or if u can use multipart-formdata to sending requests and catch file with RequestPart() MultiPartFile file to uploading file.
i hope this will be helpful
After almost 2 workdays of Googling and trying several different possibilities I found throughout the web, I'm asking this question here, hoping that I might finally get an answer.
First of all, here's what I want to do:
I'm developing a client and a server application with the purpose of exchanging a lot of large files between multiple clients on a single server. The client is developed in pure Java (JDK 1.6), while the web application is done in Grails (2.0.0).
As the purpose of the client is to allow users to exchange a lot of large files (usually about 2GB each), I have to implement it in a way, so that the uploads are resumable, i.e. the users are able to stop and resume uploads at any time.
Here's what I did so far:
I actually managed to do what I wanted to do and stream large files to the server while still being able to pause and resume uploads using raw sockets. I would send a regular request to the server (using Apache's HttpClient library) to get the server to send me a port that was free for me to use, then open a ServerSocket on the server and connect to that particular socket from the client.
Here's the problem with that:
Actually, there are at least two problems with that:
I open those ports myself, so I have to manage open and used ports myself. This is quite error-prone.
I actually circumvent Grails' ability to manage a huge amount of (concurrent) connections.
Finally, here's what I'm supposed to do now and the problem:
As the problems I mentioned above are unacceptable, I am now supposed to use Java's URLConnection/HttpURLConnection classes, while still sticking to Grails.
Connecting to the server and sending simple requests is no problem at all, everything worked fine. The problems started when I tried to use the streams (the connection's OutputStream in the client and the request's InputStream in the server). Opening the client's OutputStream and writing data to it is as easy as it gets. But reading from the request's InputStream seems impossible to me, as that stream is always empty, as it seems.
Example Code
Here's an example of the server side (Groovy controller):
def test() {
InputStream inStream = request.inputStream
if(inStream != null) {
int read = 0;
byte[] buffer = new byte[4096];
long total = 0;
println "Start reading"
while((read = inStream.read(buffer)) != -1) {
println "Read " + read + " bytes from input stream buffer" //<-- this is NEVER called
}
println "Reading finished"
println "Read a total of " + total + " bytes" // <-- 'total' will always be 0 (zero)
} else {
println "Input Stream is null" // <-- This is NEVER called
}
}
This is what I did on the client side (Java class):
public void connect() {
final URL url = new URL("myserveraddress");
final byte[] message = "someMessage".getBytes(); // Any byte[] - will be a file one day
HttpURLConnection connection = url.openConnection();
connection.setRequestMethod("GET"); // other methods - same result
// Write message
DataOutputStream out = new DataOutputStream(connection.getOutputStream());
out.writeBytes(message);
out.flush();
out.close();
// Actually connect
connection.connect(); // is this placed correctly?
// Get response
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line = null;
while((line = in.readLine()) != null) {
System.out.println(line); // Prints the whole server response as expected
}
in.close();
}
As I mentioned, the problem is that request.inputStream always yields an empty InputStream, so I am never able to read anything from it (of course). But as that is exactly what I'm trying to do (so I can stream the file to be uploaded to the server, read from the InputStream and save it to a file), this is rather disappointing.
I tried different HTTP methods, different data payloads, and also rearranged the code over and over again, but did not seem to be able to solve the problem.
What I hope to find
I hope to find a solution to my problem, of course. Anything is highly appreciated: hints, code snippets, library suggestions and so on. Maybe I'm even having it all wrong and need to go in a totally different direction.
So, how can I implement resumable file uploads for rather large (binary) files from a Java client to a Grails web application without manually opening ports on the server side?
HTTP GET method have special headers for range retrieval: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35 It's used by most downloaders to do resumable download from server.
As I understand, there are no standard practice for using this headers for POST/PUT request, but it's up to you, right? You can make pretty standard Grails controller, that will accept standard http upload, with header like Range: bytes=500-999. And controller should put this 500 uploaded bytes from client into file, starting at position 500
At this case you don't need to open any socket, and make own protocols, etc.
P.S. 500 bytes is just a example, probably you're using much bigger parts.
Client Side Java Programming:
public class NonFormFileUploader {
static final String UPLOAD_URL= "http://localhost:8080/v2/mobileApp/fileUploadForEOL";
static final int BUFFER_SIZE = 4096;
public static void main(String[] args) throws IOException {
// takes file path from first program's argument
String filePath = "G:/study/GettingStartedwithGrailsFinalInfoQ.pdf";
File uploadFile = new File(filePath);
System.out.println("File to upload: " + filePath);
// creates a HTTP connection
URL url = new URL(UPLOAD_URL);
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();
httpConn.setDoOutput(true);
httpConn.setRequestMethod("POST");
// sets file name as a HTTP header
httpConn.setRequestProperty("fileName", uploadFile.getName());
// opens output stream of the HTTP connection for writing data
OutputStream outputStream = httpConn.getOutputStream();
// Opens input stream of the file for reading data
FileInputStream inputStream = new FileInputStream(uploadFile);
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = -1;
while ((bytesRead = inputStream.read(buffer)) != -1) {
System.out.println("bytesRead:"+bytesRead);
outputStream.write(buffer, 0, bytesRead);
outputStream.flush();
}
System.out.println("Data was written.");
outputStream.flush();
outputStream.close();
inputStream.close();
int responseCode = httpConn.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
// reads server's response
BufferedReader reader = new BufferedReader(new InputStreamReader(
httpConn.getInputStream()));
String response = reader.readLine();
System.out.println("Server's response: " + response);
} else {
System.out.println("Server returned non-OK code: " + responseCode);
}
}
}
Server Side Grails Programme:
Inside the controller:
def fileUploadForEOL(){
def result
try{
result = mobileAppService.fileUploadForEOL(request);
}catch(Exception e){
log.error "Exception in fileUploadForEOL service",e
}
render result as JSON
}
Inside the Service Class:
def fileUploadForEOL(request){
def status = false;
int code = 500
def map = [:]
try{
String fileName = request.getHeader("fileName");
File saveFile = new File(SAVE_DIR + fileName);
System.out.println("===== Begin headers =====");
Enumeration<String> names = request.getHeaderNames();
while (names.hasMoreElements()) {
String headerName = names.nextElement();
System.out.println(headerName + " = " + request.getHeader(headerName));
}
System.out.println("===== End headers =====\n");
// opens input stream of the request for reading data
InputStream inputStream = request.getInputStream();
// opens an output stream for writing file
FileOutputStream outputStream = new FileOutputStream(saveFile);
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = inputStream.read(buffer);
long count = bytesRead
while(bytesRead != -1) {
outputStream.write(buffer, 0, bytesRead);
bytesRead = inputStream.read(buffer);
count += bytesRead
}
println "count:"+count
System.out.println("Data received.");
outputStream.close();
inputStream.close();
System.out.println("File written to: " + saveFile.getAbsolutePath());
code = 200
}catch(Exception e){
mLogger.log(java.util.logging.Level.SEVERE,"Exception in fileUploadForEOL",e);
}finally{
map <<["code":code]
}
return map
}
I have tried with above code it is worked for me(only for file size 3 to 4MB, but for small size files some bytes of code missing or not even coming but in request header content-length is coming, not sure why it is happening.)
I am doing a rest call and getting a json file from the server by this
HttpGet httpget = new HttpGet("someurl");
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
InputStream source = entity.getContent();
now i want to store this into a local file I am using FileOutputStream to do this but the problem is how to effectively convert the inputstream to outputstream if i am using something like this
FileOutputStream fos = openFileOutput(filename,Context.MODE_PRIVATE);
int nextChar;
while ((nextChar = source.read()) != -1) {
fos.write((char) nextChar);
System.out.println((char) nextChar);
fos.flush();
}
it is storing very slow the file which i am getting is upto 100kb is there any other faster method or any other way which i can use to store the json file in my device?
My applications uses this json heavily and i don't want to call the REST each time.
thanks
Pranay
try to use org.apache.commons.io.IOUtils's
IOUtils.copy(is,fos);
lets see, what happening. Thanks,
EDIT: Why not you are use sqlite database? parse JSON result onetime and insert it in database only onetime stuff then always you get fast execution.
EDIT: try android's internal storage for write json file
try {
FileOutputStream fos = openFileOutput(FILENAME, Context.MODE_APPEND);
fos.write(myJSONString.getBytes());
fos.close();
//Log.d(TAG, "Written to file");
}
catch (Exception e)
{
Log.d(TAG, "cought");
e.printStackTrace();
}
I'm having a tough time figuring something out. (I'm pretty new to all this.)
I wrote this java pgm to ftp a large file to a destination server.
Here's the code (codes been modified a bit for display):
public static void ftpUpload(String path, String upfileName, String dirName) throws Exception
{
FTPClient client = new FTPClient();
client.addProtocolCommandListener((ProtocolCommandListener) new PrintCommandListener(new PrintWriter(System.out)));
client.enterLocalPassiveMode();
FileInputStream fis = null;
int reply;
try {
client.connect(ftpserver);
client.login(ftpuserid, ftppasswd);
reply = client.getReplyCode();
if(FTPReply.isPositiveCompletion(reply)){
client.changeWorkingDirectory(ftpdirectoryName + "/" + dirName);
boolean mkDir = client.makeDirectory(getCurrentMMMYY().toLowerCase());
client.changeWorkingDirectory(getCurrentMMMYY().toLowerCase());
//Create an InputStream of the file to be uploaded
fis = new FileInputStream(path + upfileName);
//Store file to server
client.storeFile(upfileName, fis);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (fis != null) {
fis.close();
}
client.logout();
//client.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Something weird is happening on files I'm sending...
One of my files on the origination server is 82575786 in size, and when I ftp this file it almost sends the entire file. It actually sends 82574867. (missing 919)
Another file on the origination server is 717885, and when I ftp this file it almost sends the entire file. It actually sends 717522. (missing 363)
I pulled the log to see if something crashed, but it didn't show anything wrong with the transfer. Here are the 2 log entries showing the transfer.
[08/09/11 20:21:13:618 EDT] 00000043 SystemOut O 221-You have transferred 717522 bytes in 1 files.
221-You have transferred 82574867 bytes in 1 files.
Anyone's help would greatly be appreciated.
Thanks
Dan.
Are you transferring in ASCII mode instead of binary? ASCII mode converts CR/LF to LF and vice-versa depending on server and client settings.
Are you using Apache's FTP client? It says the default is ASCII, you could try setting BINARY_FILE_TYPE with setFileType:
client.setFileType(FTPClient.BINARY_FILE_TYPE);
To upload a binary File you have to use the FTP.BINARY_FILE_TYPE but is not enough.
You are using only an INPUT stream, and you need to use an outputstream too
I hope that this example will help you:
FTPClient client = new FTPClient();
client.connect("192.168.30.20");
client.login("pwd", "pwd");
client.setFileType(FTP.BINARY_FILE_TYPE);
String path_base = "/myPath/";
InputStream fis = new FileInputStream("A.pdf");
OutputStream os = client.storeFileStream(path_base+ "B.pdf");
byte buf[] = new byte[8192];
int bytesRead = fis.read(buf);
while (bytesRead != -1) {
os.write(buf, 0, bytesRead);
bytesRead = fis.read(buf);}
fis.close();
os.close();
client.completePendingCommand();
client.logout();
client.disconnect();
using java code in windows i need to download several files from a directory placed in a server. those files in server are generated separately. so i'll not know the name of those files. is there any way to download it using JAVA and saving it in a specific folder.
i am using apache tomcat.
I read all other threads related to java file download. But none of them satisfy my requirement.
try {
// Get the directory and iterate them to get file by file...
File file = new File(fileName);
if (!file.exists()) {
context.addMessage(new ErrorMessage("msg.file.notdownloaded"));
context.setForwardName("failure");
} else {
response.setContentType("APPLICATION/DOWNLOAD");
response.setHeader("Content-Disposition", "attachment"+
"filename=" + file.getName());
stream = new FileInputStream(file);
response.setContentLength(stream.available());
OutputStream os = response.getOutputStream();
os.close();
response.flushBuffer();
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (stream != null) {
try {
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Hope you got some idea...
Use java.net.URL and java.net.URLConnection classes.
Hi you can use this following code snippet to down the file directly :
URL oracle = new URL("http://www.example.com/file/download?");
BufferedReader in = new BufferedReader(
new InputStreamReader(oracle.openStream()));
String inputLine;
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
in.close();
Kindly refer about openStream in this [URL] : http://docs.oracle.com/javase/tutorial/networking/urls/readingURL.html
You can use HttpURLConnection to download file over HTTP, HTTPS
It is only possible if server lists directory contents. if it does, your can make an HTTP request to:
http://server:port/folder
that would give you list of files.
Once you have that, you can download individual files by parsing output if this http request.
If it is server, then the process must be like using the FTP credentials you have to dosnload the files. This java file download example may help you.