I am using spring MVC where through API I am uploading zip file using MultipartFile. In backend I have to convert uploaded zip file into InputStream for further processing. But my code is giving error intermittently " File has been moved, can not be read again ".
here is the code snippet :
File temp = null;
InputStream stream = null;
try {
InputStream initialStream = inputFile.getInputStream();
byte[] buffer = new byte[initialStream.available()];
initialStream.read(buffer);
temp = File.createTempFile("upload", null);
try (OutputStream outStream = new FileOutputStream(temp)) {
outStream.write(buffer);
}
ZipFile zipFile = new ZipFile(temp);
stream = zipFile.getInputStream(zipFile.getEntries().nextElement());
} catch (Exception e) {
log.error("Exception occurred while processing zip file " + e.getMessage());
throw e;
} finally {
if (temp != null)
temp.delete();
}
return stream;
Here inputFile is MultipartFile.
Could you please suggest what is wrong here?
Your code is returning an input stream from a file that you have deleted - last line is temp.delete().
ZipInputStream has a small internal buffer for decoding, so that may explain why some read calls work after the delete, but it will not be possible to continue reading from a file that you deleted, hence the exception.
Also, the call initialStream.available() is unlikely to be the correct way to determine the size of the input stream file part. Try printing the size / check how to read the actual length of the file in the multipart stream - such as part.getSize(), or transfer the bytes into a new ByteArrayOutputStream() before assigning to buffer.
I would not recommend doing any work with files or multipart streams using direct transfer to byte[] as you risk OutOfMemoryException. However in your case where you are happy to have byte[] for the ZIP and you read the first entry of the ZIP file (and are ignoring other entries) then you could try extracting the first entry as InputStream without writing to a file as follows:
// Read a zip input stream from a zip stored in byte[]:
ZipInputStream zis = new ZipInputStream(new ByteArrayInputStream(buffer));
// Select first entry from ZIP
ZipEntry entry = zis.getNextEntry();
// You should be able to read the entry from zis directly,
// if this is text file you could test with:
// zis.transferTo(System.out);
return zis;
You should ensure that you close the stream after use.
Potential issues I can see in your code:
temp file is used as zip file, yet you delete the temp file prior to
returning. How can you use the zip file as file stream if you have
deleted it?
Do you support concurrent uploads? If yes, then you have concurrent
resource access problem. Multiple calls to create temp file:
"upload" and process it. Why don't you create a different
filename e.g. with datetime suffix + random number suffix.
Related
I have a zip archive that contains several gzip files. But gzip file's extentions are also .zip . I walk through zip archive with ZipInputStream. How can I detect inner file's type with reading its content rather than extentions. I also need not to change (or reset) ZipInputStream position.
So I need;
Read files in zip with using inputStream (ZipInputStream in my case) Because zip in zip is possible.
Find file type from its content.
While finding file type from its content, inputStream position should not change. Because i will continue to read next files.
Example:
root/1.zip/2.zip/3.zip(actually 3 is gzip)/4.txt
Sample Java Code:
public static void main(String[] args) {
//root/1.zip/2.zip/3.zip(actually 3 is gzip)/4.txt
String file = "root/1.zip";
File rootZip = new File(file);
try (FileInputStream fis = new FileInputStream(rootZip)) {
lookupInZip(fis)
.stream()
.forEach(System.out::println);
} catch (IOException e) {
System.out.println("Failed to get files");
}
}
public static List<String> lookupInZip(InputStream inputStream) throws IOException {
Tika tika = new Tika();
List<String> paths = new ArrayList<>();
ZipInputStream zipInputStream = new ZipInputStream(inputStream);
ZipEntry entry = zipInputStream.getNextEntry();
while (entry != null) {
String entryName = entry.getName();
if (!entry.isDirectory()) {
//Option 1
//String fileType = tika.detect(entryName);
//Option 2
String fileType = tika.detect(zipInputStream);
if ("application/zip".equals(fileType)) {
List<String> innerPaths = lookupInZip(zipInputStream);
paths.addAll(innerPaths);
} else {
paths.add(entryName);
}
}
entry = zipInputStream.getNextEntry();
}
return paths;
}
If I use option 1, '3.zip' is evaluated as zip file but it is gzip.
If I use option 2, '2.zip' is evaluated as zip correctly by using its content. But when lookupInZip() is called for '3.zip' recursively, zipInputStream.getNextEntry() returns null. Because in previous step, we use inputStream content to detect type and inputStrem position changed.
Note: tika.detect() uses BufferedInputStream in implementation to reset inputStream position but it does not solve my problem.
The first two bytes are enough to see if it is likely a zip file, likely a gzip file, or certainly something else.
If the first two bytes are 0x50 0x4b, then it is likely a zip file. If the first two bytes are 0x1f 0x8b, then it is likely a gzip file. If it is neither, then the file is something else.
The first two bytes matching is not a guarantee it is that type, but it appears from your structure that it is usually one or the other, and you can use the extension as further corroborating evidence that it is compressed.
As for not changing the position, you need a way to peek at the first two bytes without advancing the position, or a way to get them and then unget them to return the position to where it was.
I'm trying to create a zip file to be able to send multiple files over http.
My issue is that the Zip file that is generated is "corrupted" before and after the file has been send. The issue is i'm not able to find what i did wrong as i'm getting no errors inside the console.
So does someone has an idea file my generated zip file is corrupted ?
This is my code :
OutputStream responseBody = t.getResponseBody();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ZipOutputStream zos = new ZipOutputStream(baos);
int counter = 1;
for (PDDocument doc : documents)
{
ZipEntry zipEntry = new ZipEntry("document" + counter);
zos.putNextEntry(zipEntry);
ByteArrayOutputStream docOs = new ByteArrayOutputStream();
doc.save(docOs);
docOs.close();
zos.write(docOs.toByteArray());
zos.closeEntry();
zos.finish();
zos.flush();
counter++;
}
zos.close();
baos.close();
responseBody.write(baos.toByteArray());
responseBody.flush();
Thank you for your help !
You need to remove zos.finish() from inside the loop as it terminates the ZIP entries, as it is handled by zos.close() at end of the stream.
With very large streams you will be better off sending ZIP directly to responseBody bypassing ByteArrayOutputStream memory buffer.
If you are still having problems check the content type of the output is set. It might be easier to debug by temporarily writing the byte[] to file to check the ZIP format you are sending with:
Files.write(Path.of("temp.zip"), baos.toByteArray());
This outline below shows sending a simple ZIP over http (from a servlet, adjust the first 2 lines to appropriate calls for "t"). This may help you check which step of your code causes the corruption if you work back to adding your own document objects inside the loop:
// MUST set response content type:
// resp.setContentType("application/zip");
OutputStream out = resp.getOutputStream(); // or t.getResponseBody();
try(ZipOutputStream zos = new ZipOutputStream(out))
{
while (counter-- > 0)
{
ZipEntry zipEntry = new ZipEntry("document" + counter+".txt");
zos.putNextEntry(zipEntry);
zos.write(("This is ZipEntry: "+zipEntry.getName()+"\r\n").getBytes());
}
}
I have been created a application which shall extract single files from tar-archive. The application reads the *.tar properly, but when i try to extract files, the application just create new files with correct filename... The files is empty (0kb). So... I probably just create new files instead of extract...
I'm a totally beginner at this point...
for(TarArchiveEntry tae : tarEntries){
System.out.println(tarEntries.size());
try {
fOutput = new FileOutputStream(new File(tae.getFile(), tae.getName()));
byte[] buf = new byte[(int) tae.getSize()];
int len;
while ((len = tarFile.read(buf)) > 0) {
fOutput.write(buf, 0, len);
}
fOutput.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Assuming tarFile is a TarArchiveInputStream you can only read an entry's content right after calling tarFile.getNextTarEntry().
The stream is processed sequentially, so when you invoke getNextTarEntry you skip over the content of the current entry right to the next entry. It looks as if you had read the whole archive in order to fill tarEntries in which case you've already read past the last entry and the stream is exhausted.
From a third party I am retrieving .zip files. I want to unzip these to another folder. To this end I found a method that does exactly that, see code below. It iterates through all files and unzips them to another folder. However, when observing the corresponding compression method I found out that this changes for some files. And for some files it states: "invalid compression method", after which it aborts further unzipping of the zip file.
As the compression method seems to change, I suspect I need to set the compression method to the correct one (however that might be a wrong assumption). So rises my question: how to determine the compression method needed?
The code I am using:
public void unZipIt(String zipFile, String outputFolder){
//create output directory is not exists
File folder = new File(OUTPUT_FOLDER);
if(!folder.exists()){
folder.mkdir();
}
FileInputStream fis = null;
ZipInputStream zipIs = null;
ZipEntry zEntry = null;
try
{
fis = new FileInputStream(zipFile);
zipIs = new ZipInputStream(new BufferedInputStream(fis));
while((zEntry = zipIs.getNextEntry()) != null){
System.out.println(zEntry.getMethod());
try{
byte[] tmp = new byte[4*1024];
FileOutputStream fos = null;
String opFilePath = OUTPUT_FOLDER + "\\" + zEntry.getName();
System.out.println("Extracting file to "+opFilePath);
fos = new FileOutputStream(opFilePath);
int size = 0;
while((size = zipIs.read(tmp)) != -1){
fos.write(tmp, 0 , size);
}
fos.flush();
fos.close();
} catch(IOException e){
System.out.println(e.getMessage());
}
}
zipIs.close();
} catch (FileNotFoundException e) {
System.out.println(e.getMessage());
}
catch(IOException ex){
System.out.println(ex.getMessage());
}
}
Currently I am retrieving the following output:
8
Extracting file to C:\Users\nlmeibe2\Documents\Projects\Output_test\SOPHIS_cptyrisk_tradedata_1192_20140616.csv
8
Extracting file to C:\Users\nlmeibe2\Documents\Projects\Output_test\SOPHIS_cptyrisk_underlying_1192_20140616.csv
0
Extracting file to C:\Users\nlmeibe2\Documents\Projects\Output_test\10052013/
12
Extracting file to C:\Users\nlmeibe2\Documents\Projects\Output_test\MRM_Daily_Position_Report_Package_Level_Underlying_View_EQB_v2_COBDATE_2014-06-16_RUNDATETIME_2014-06-17-04h15.csv
invalid compression method
invalid compression method
Since you only print the exception message and not the stack trace (with line numbers), it is impossible to know exactly where the exception is thrown, but I suppose it is not thrown until you actually try to read from the ZipEntry.
If the numbers in your output is the ZIP method, the last entry you encounter is compressed with method 12 (bzip2), which is not supported by the Java ZIP implementation. PKWare (the maintainers of the ZIP format) regularly add new compression methods to the ZIP specification and there are currently some 12-15 (not sure about the exact number) compression methods specified. Java only supports the methods 0 (stored) and 8 (deflated) and will throw an exception with the message "invalid compression method" if you try to decompress a ZIP file using an unsupported compression method.
Both WinZip and the ZIP functions in Windows may use compression methods not supported by the Java API.
Use zEntry.getMethod() to get the compression method
Returns the compression method of the entry, or -1 if not specified.
It will return an int which will be
public static final int STORED
public static final int DEFLATED
or -1 if it don't know the method.
Docs.
I am having a InputStream Object which is actually a zip file. I want to change it back to zip file and save it. I am using DWR's FileTransfer class object to receive the uploaded data from client.
FileTransfer have 3 methods, getInputStream() is one of them. It returns InputStream from FileTransfer object.
In my case, fileTransfer object holds zip file and as well as InputStream object too.
I have done, lot of searches in google. But i am not able to find one example, that illustrates InputStream to zip conversion.
Update
String zipName = file.getName();
String zipType = file.getMimeType();
InputStream zipStream = file.getInputStream();
ZipInputStream zis = new ZipInputStream(zipStream);
System.out.println("File Name: "+zipName+"\n"+"File Type: "+zipType);
int c;
File f2 = new File(DATA_STORE_LOC+dat+".zip");
path.setPath2(DATA_STORE_LOC+dat+".zip");
FileOutputStream fos = new FileOutputStream(f2);
ZipOutputStream zos = new ZipOutputStream(fos);
c = zis.read();
System.out.println(c);
while ((c = zis.read(BUFFER)) != -1) {
zos.write(BUFFER, 0, c);
}
zos.close();
zis.close();
I tried this code, by thought of a typical file copy program. I know it is false, just tried. It gives me java.util.zip.ZipException: ZIP file must have at least one entry.
Any suggestion would be really appreciative!!!!!
See the examples java2s, input and output. If you have more questions feel free to ask them :)
For clarity, in this input example you should do something like:
// FileInputStream fin = new FileInputStream(args[i]);
ZipInputStream zin = new ZipInputStream(ft.getInputStream());
As Don Roby correctly said, if you just want to copy you need not know the file structure and you could use for example static IOUtils.copy(in, out) to copy the file.
Further, if you do wish to extract the ZIP file contents, you should not plainly copy bytes. The ZIP file has a structure, and you extract Entries from the ZIP file, and not just bytes (see the example). Every Entry is a (compressed) file (or the data thereof) with the original name:
ZipEntry ze = null;
while ((ze = zin.getNextEntry()) != null) {
System.out.println("Unzipping " + ze.getName());
FileOutputStream fout = new FileOutputStream(ze.getName());
for (int c = zin.read(); c != -1; c = zin.read()) {
...
Please note the javadoc of getNextEntry():
Reads the next ZIP file entry and positions the stream at the beginning of the entry data.
This positioning is crucial to get to the zipped file contents, and not the metadata.
And I do believe that you accidentally remove the first int:
c = zis.read(); // removing the first
while ((c = zis.read(BUFFER)) != -1) { // so you start with the second?
I believe you mix 2 idioms:
c = zis.read();
while(c != -1) {
...
c = zis.read();
}
and:
int c;
while ((c = zis.read(BUFFER)) != -1) { // so you start with the second?
...
}
I think you can see the difference :)
If your input is a an InputStream from a zip file and your desired output is still a zip file with the same contents, you're just doing a file copy operation and shouldn't have to worry about zip at all. You just need to read from the InputStream and write to a FileOutputStream, more or less as you're doing, but without worrying about wrapping either stream in a zip-aware stream.
ZipInputStream is useful if you have to extract the contents of the zip file as separate files, i.e., to programmatically unzip. And on the other side, ZipOutputStream is used if your have the contents and need to combine them into a zip file.