I have been created a application which shall extract single files from tar-archive. The application reads the *.tar properly, but when i try to extract files, the application just create new files with correct filename... The files is empty (0kb). So... I probably just create new files instead of extract...
I'm a totally beginner at this point...
for(TarArchiveEntry tae : tarEntries){
System.out.println(tarEntries.size());
try {
fOutput = new FileOutputStream(new File(tae.getFile(), tae.getName()));
byte[] buf = new byte[(int) tae.getSize()];
int len;
while ((len = tarFile.read(buf)) > 0) {
fOutput.write(buf, 0, len);
}
fOutput.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Assuming tarFile is a TarArchiveInputStream you can only read an entry's content right after calling tarFile.getNextTarEntry().
The stream is processed sequentially, so when you invoke getNextTarEntry you skip over the content of the current entry right to the next entry. It looks as if you had read the whole archive in order to fill tarEntries in which case you've already read past the last entry and the stream is exhausted.
Related
I am using spring MVC where through API I am uploading zip file using MultipartFile. In backend I have to convert uploaded zip file into InputStream for further processing. But my code is giving error intermittently " File has been moved, can not be read again ".
here is the code snippet :
File temp = null;
InputStream stream = null;
try {
InputStream initialStream = inputFile.getInputStream();
byte[] buffer = new byte[initialStream.available()];
initialStream.read(buffer);
temp = File.createTempFile("upload", null);
try (OutputStream outStream = new FileOutputStream(temp)) {
outStream.write(buffer);
}
ZipFile zipFile = new ZipFile(temp);
stream = zipFile.getInputStream(zipFile.getEntries().nextElement());
} catch (Exception e) {
log.error("Exception occurred while processing zip file " + e.getMessage());
throw e;
} finally {
if (temp != null)
temp.delete();
}
return stream;
Here inputFile is MultipartFile.
Could you please suggest what is wrong here?
Your code is returning an input stream from a file that you have deleted - last line is temp.delete().
ZipInputStream has a small internal buffer for decoding, so that may explain why some read calls work after the delete, but it will not be possible to continue reading from a file that you deleted, hence the exception.
Also, the call initialStream.available() is unlikely to be the correct way to determine the size of the input stream file part. Try printing the size / check how to read the actual length of the file in the multipart stream - such as part.getSize(), or transfer the bytes into a new ByteArrayOutputStream() before assigning to buffer.
I would not recommend doing any work with files or multipart streams using direct transfer to byte[] as you risk OutOfMemoryException. However in your case where you are happy to have byte[] for the ZIP and you read the first entry of the ZIP file (and are ignoring other entries) then you could try extracting the first entry as InputStream without writing to a file as follows:
// Read a zip input stream from a zip stored in byte[]:
ZipInputStream zis = new ZipInputStream(new ByteArrayInputStream(buffer));
// Select first entry from ZIP
ZipEntry entry = zis.getNextEntry();
// You should be able to read the entry from zis directly,
// if this is text file you could test with:
// zis.transferTo(System.out);
return zis;
You should ensure that you close the stream after use.
Potential issues I can see in your code:
temp file is used as zip file, yet you delete the temp file prior to
returning. How can you use the zip file as file stream if you have
deleted it?
Do you support concurrent uploads? If yes, then you have concurrent
resource access problem. Multiple calls to create temp file:
"upload" and process it. Why don't you create a different
filename e.g. with datetime suffix + random number suffix.
These are the related questions that might cause my question to be closed, even though I specify another question:
Java: How to write binary files? -> Doesn't really cover the point that I am talking about
create a binary file -> Absolutely doesn't cover the point
Editing a binary file in java -> They are talking about offsets and stuff, when I just need to write the data and stop
Binary files in java -> Vague.
And now to the point. I've got a file with a specific extension, to be more exact it's .nbs. I want to create a file and then write the specific data to it.
That might have sounded vague so let me show you the code I have started with.
try {
File bpdn = new File(getDataFolder() + "song.nbs");
if (!bpdn.exists()) {
bpdn.createNewFile();
}
FileOutputStream fos = new FileOutputStream(bpdn);
} catch (IOException e) {
e.printStackTrace();
}
I'll provide you even more details. So I've got a song.nbs file that I have created in the past, for myself. And now, whenever a person runs my application, I want it so there's a new song.nbs file with the exact contents of a file that I have on my PC right now. Therefore, I need to somehow get the bytes of my existing song.nbs and then copy and paste them in my Java application... or is it the way? I neither know how to get the bytes of my own file right now, nor do I know how to write them.
You need to create a resources folder. More info here.
Assuming your project structure is
ProjectName/src/main/java/Main.java
you can create a resources folder inside main/:
ProjectName/src/main/java/Main.java
ProjectName/src/main/resources/
Move your song.nbs you want to read inside resources/:
ProjectName/src/main/java/Main.java
ProjectName/src/main/resources/song.nbs
Now, get the InputStream of song.nbs stored there:
final ClassLoader classloader = Thread.currentThread().getContextClassLoader();
final InputStream is = classloader.getResourceAsStream("song.nbs");
Then, write this input stream to your new file:
final File bpdn = new File(getDataFolder() + "song.nbs");
if (!bpdn.exists()) bpdn.createNewFile();
final FileOutputStream os = new FileOutputStream(bpdn);
byte[] buffer = new byte[1024];
int read;
while ((read = is.read(buffer)) != -1) {
os.write(buffer, 0, read);
}
I think I came up with a solution, but I am not sure if this is works. I'd appreciate if you would take a look.
try {
File bpdn = new File(getDataFolder() + "badpiggiesdrip.nbs");
if (!bpdn.exists()) {
bpdn.createNewFile();
}
FileOutputStream fos = new FileOutputStream(bpdn);
fos.write(new byte[] {
Byte.parseByte(Arrays.toString(Base64.getDecoder().decode(Common.myString)))
});
} catch (IOException e) {
e.printStackTrace();
}
Common.myString is just a string, that contains data of this type:
(byte) 0x21, (byte) 0x5a, .....
and it's encoded in Base64.
I have the following snippet and this is used to check if the given zip entry is a directory or not which works fine.There is an additional requirement given where I need to check the size of each image (its size should not be > 10 MB) inside the folder which is inside the zip file.
I been going through some articles but couldn't get hold of a scenario similar to mine.
The example structure for a zip file would look like the one given below along with the folder and the images size inside them
XYZ.zip>023423>Bat1.jpg ->11MB
XYZ.zip>023423>Bat2.jpg ->5MB
XYZ.zip>023423>Bat3.jpg ->11MB
XYZ.zip>023423>Bat4.jpg ->10MB
Based on the scenario above, at the end of the execution I should able to get the Bat1 & Bat3 as output as their size is >10 MB.Kindly advise.
private void isGivenZipInFolderStructure(ExcelImportCronJobModel
cronJob) {
try {
foldersInZip = new ArrayList<>();
if(cronJob.getReferencedContent() !=null) {
final ZipInputStream zip = new ZipInputStream(this.mediaService.getStreamFromMedia(cronJob.getReferencedContent()));
ZipEntry entry = null;
while ((entry = zip.getNextEntry()) != null) {
LOG.info("Size of the entry {}",entry.getSize());
if(entry.isDirectory()) {
foldersInZip.add(entry.getName().split(BunningsCoreConstants.FORWARD_SLASH)[0]);
}
}
}
} catch (IOException e) {
LOG.error("Error reading zip, e");
}
}
As mentioned in the comments, the value of getSize is not set when reading from a ZipInputStream - unlike when using ZipFile. However you could try to scan the stream contents yourself and monitor the sizes of each entry.
This method scans any ZIP passed in as InputStream which could be derived from a file or other downloaded source:
public static void scan(InputStream is) throws IOException {
System.out.println("==== scanning "+is);
ZipEntry ze;
// Uses ByteArrayOutputStream to determine the size of the entry
ByteArrayOutputStream bout = new ByteArrayOutputStream();
long maxSize = 10_000_000L;
try (ZipInputStream zis = new ZipInputStream(is)) {
while ((ze = zis.getNextEntry()) != null) {
bout.reset();
long size = zis.transferTo(bout);
System.out.println(ze.getName()
+(ze.isDirectory() ? " DIR" : " FILE["+size+"]")
+(size > maxSize ? " *** TOO BIG ***":"")
);
if (size > maxSize) {
// entry which is too big - do something / warning ...
} // else use content: byte[] content = bout.toByteArray();
}
}
}
This approach is not ideal for very large ZIP content, but it may be worth trying for your specific case - better to have a slow solution than none at all.
If there are really big entries in the ZIP you might also consider replacing the line long size = zis.transferTo(bout); with a call to your own method which does not transfer content but still returns the size - similar to implementation of InputStream.transferTo but commenting out the write().
I'm trying to store multiple data files that I have created into one file. Each file has an ID# (0-35) and each holds some data. But what I want is to be able to store all the files in one file called 'data.xx', then be able to access the each of the files data inside the data.xx file.
public static void pack(int id) {
try {
RandomAccessFile raf = new RandomAccessFile("./data/data.xx", "rw");
ByteArrayInputStream bis = null;
try {
byte[] data = toByteArray(id);
bis = new ByteArrayInputStream(data);
raf.seek(raf.length());
System.out.println(raf.length());
while (bis.read(data, 0, data.length) >= 0) {
raf.write(data, 0, data.length);
}
} finally {
bis.close();
raf.close();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
toByteArray(id) calls the separate data files then puts it into byte array. All the files seem to write fine to the data.xx file. The problem I'm having is I'm not really sure how to read the data.xx file so I can get the data from the files that are stored in it. I hope this makes sense. Also I don't need any compression and I don't want to use a library for this.
Thank you
The simplest way is use markup:
<id_0> content of file 0 </id_0>
...
<id_35> content of file 35 </id_35>
You write file like that and read content inside tags with substring
I would prepend the output file with the offsets of the start of each contained file. A special "token" is a nice idea, but files can contain any byte or bytes; making the idea not realistic. That way your "index" will terminate with something you can't confuse with file data because the information occurres before you expect arbitrary data ... say, 0? Please comment if I misunderstood this question.
I get some very odd errors when using org.apache.commons.compress to read embedded archive files and I suspect it's my inexperience that is haunting me.
When running my code I get a variety of truncated zip file errors (along with other truncated file errors). I suspect it's my use of ArchiveInputStream
private final void handleArchive(String fileName, ArchiveInputStream ais) {
ArchiveEntry archiveEntry = null;
try {
while((archiveEntry = ais.getNextEntry()) != null) {
byte[] buffer = new byte[1024];
while(ais.read(buffer) != -1) {
handleFile(fileName + "/" + archiveEntry.getName(), archiveEntry.getSize(), new ByteArrayInputStream(buffer));
} catch(IOException ioe) {
ioe.printStackTrace();
}
}
When I do this archiveEntry = ais.getNextEntry() does this effectively close my ais, and is there any way to read the bytes of embedded archive files using commons compress?
You re doing some wierd stuff it seems? For each archieve entry while your reading your archieve you re recursively calling your read archieve method which results in opening the next archieve while your parent code is still handling your previous archieve.
You should loop entirely through your archieve entry before handling any new archieve entry in your compressed file. Something like
ArArchiveEntry entry = (ArArchiveEntry) arInput.getNextEntry();
byte[] content = new byte[entry.getSize()];
LOOP UNTIL entry.getSize() HAS BEEN READ {
arInput.read(content, offset, content.length - offset);
}
as stated in the examples on the apache site