I'm using the Java API for reading and writing to the Google App Engine Blobstore.
I need to zip files directly into the Blobstore, meaning I have String objects which I want to be stored in the Blobstore when zipped.
My problem is that standard zipping methods are using OutputStream to write, while it seems that GAE doesn't provide one for writing to the Blobstore.
Is there a way to combine those APIs, or are there different APIs I can use (I haven't found such)?
If I am not wrong, you can try to use the Blobstore low level API. It offers a Java Channel (FileWriteChannel), so you could probably convert it to an OutputStream:
Channels.newOutputStream(channel)
And use that output stream with the java.util.zip.* classes you are currently using (here you have a related example that uses Java NIO to zip something to a Channel/OutputStream)
I have not tried it.
Here is one example to write content file and zip it and store it into blobstore:
AppEngineFile file = fileService.createNewBlobFile("application/zip","fileName.zip");
try {
FileWriteChannel writeChannel = fileService.openWriteChannel(file, lock);
//convert as outputstream
OutputStream blobOutputStream = Channels.newOutputStream(writeChannel);
ZipOutputStream zip = new ZipOutputStream(blobOutputStream);
zip.putNextEntry(new ZipEntry("fileNameTozip.txt"));
//read the content from your file or any context you want to get
final byte data[] = IOUtils.toByteArray(file1InputStream);
//write byte data[] to zip
zip.write(bytes);
zip.closeEntry();
zip.close();
// Now finalize
writeChannel.closeFinally();
} catch (IOException e) {
throw new RuntimeException(" Writing file into blobStore", e);
}
The other answer is using BlobStore api, but currently the recommended way is to use App Engine GCS client.
Here is what I use to zip multiple files in GCS :
public static void zipFiles(final GcsFilename targetZipFile,
Collection<GcsFilename> filesToZip) throws IOException {
final GcsFileOptions options = new GcsFileOptions.Builder()
.mimeType(MediaType.ZIP.toString()).build();
try (GcsOutputChannel outputChannel = gcsService.createOrReplace(targetZipFile, options);
OutputStream out = Channels.newOutputStream(outputChannel);
ZipOutputStream zip = new ZipOutputStream(out)) {
for (GcsFilename file : filesToZip) {
try (GcsInputChannel readChannel = gcsService.openPrefetchingReadChannel(file, 0, MB);
InputStream is = Channels.newInputStream(readChannel)) {
final GcsFileMetadata meta = gcsService.getMetadata(file);
if (meta == null) {
log.warn("{} NOT FOUND. Skipping.", file.toString());
continue;
}
final ZipEntry entry = new ZipEntry(file.getObjectName());
zip.putNextEntry(entry);
ByteStreams.copy(is, zip);
zip.closeEntry();
}
zip.flush();
}
}
Related
I have a webapp that allows users to select images and then download them. For a single image, I use HTML5's anchor download and it works beautifully. Now I need to allow them to select multiple images, and download them as a .zip file. I'm using an api to get each image as an InputStream and returning a Jersey Response.
I'm new to zipping and I'm a bit confused with how zipping with InputStream should work.
For single images, it works like so:
try {
InputStream imageInputStream = ImageStore.getImage(imageId);
if (imageInputStream == null) {
XLog.warnf("Unable to find image [%s].", imageId);
return Response.status(HttpURLConnection.HTTP_GONE).build();
}
Response.ResponseBuilder response = Response.ok(imageInputStream);
response.header("Content-Type", imageType.mimeType());
response.header("Content-Disposition", "filename=image.jpg");
return response.build();
}
It's not much, but here's the java I have so far for multiple images
public Response zipAndDownload(List<UUID> imageIds) {
try {
// TODO: instantiate zip file?
for (UUID imageId : imageIds) {
InputStream imageInputStream = ImageStore.getImage(imageId);
// TODO: add image to zip file (ZipEntry?)
}
// TODO: return zip file
}
...
}
I just don't know how to deal with multiple InputStreams, and it seems that I shouldn't have multiple, right?
An InputStream per image is ok. To zip the files you need to create a .zip file for them to live in and get a ZipOutputStream to write to it:
File zipFile = new File("/path/to/your/zipFile.zip");
ZipOutputStream zos = new ZipOutputStream(new BufferedOutputStream(new FileOutputStream(zipFile)));
For each image, create a new ZipEntry, add it to the ZipOutputSteam, then copy the bytes from your image's InputStream to the ZipOutputStream:
ZipEntry ze = new ZipEntry("PrettyPicture1.jpg");
zos.putNextEntry(ze);
byte[] bytes = new byte[1024];
int count = imageInputStream.read(bytes);
while (count > -1)
{
zos.write(bytes, 0, count);
count = imageInputStream.read(bytes);
}
imageInputStream.close();
zos.closeEntry();
After you add all the entries, close the ZipOutputStream:
zos.close();
Now your zipFile points to a zip file full of pictures you can do whatever you want with. You can return it like you do with a single image:
BufferedInputStream zipFileInputStream = new BufferedInputStream(new FileInputStream(zipFile));
Response.ResponseBuilder response = Response.ok(zipFileInputStream);
But the content type and disposition are different:
response.header("Content-Type", MediaType.APPLICATION_OCTET_STREAM_TYPE);
response.header("Content-Disposition", "attachment; filename=zipFile.zip");
Note: You can use the copy method from Guava's ByteStreams helper to copy the streams instead of copying the bytes manually. Simply replace the while loop and the 2 lines before it with this line:
ByteStreams.copy(imageInputStream, zos);
I'm developing my first application in vaadin. Now I'm trying to customize upload component. In summary I have to do the upload of an image.
Now my component is implemented in a standard way:
public OutputStream receiveUpload(String filename,String mimeType) {
// Create upload stream
FileOutputStream fos = null; // Stream to write to
try {
// Open the file for writing.
file = new File("/tmp/uploads/" + filename);
fos = new FileOutputStream(file);
} catch (final java.io.FileNotFoundException e) {
new Notification("Could not open file<br/>",e.getMessage(),Notification.Type.ERROR_MESSAGE).show(Page.getCurrent());
return null;
}
return fos; // Return the output stream to write to
}
I want to ask you, if i can do the upload of the document without use a temp file on server.
How can I do?
Of course,
you just need to provide a OutputStream for the upload component.
This could be a a ByteArrayOutputStream for example, so you have everything as a large bytearray.
Just be aware, when the user uploads a 10 GByte size file, you will also need that much memory on the server for that request
André
I need to create a File object out of a file path to an image that is contained in a jar file after creating a jar file. If tried using:
URL url = getClass().getResource("/resources/images/image.jpg");
File imageFile = new File(url.toURI());
but it doesn't work. Does anyone know of another way to do it?
To create a file on Android from a resource or raw file I do this:
try{
InputStream inputStream = getResources().openRawResource(R.raw.some_file);
File tempFile = File.createTempFile("pre", "suf");
copyFile(inputStream, new FileOutputStream(tempFile));
// Now some_file is tempFile .. do what you like
} catch (IOException e) {
throw new RuntimeException("Can't create temp file ", e);
}
private void copyFile(InputStream in, OutputStream out) throws IOException {
byte[] buffer = new byte[1024];
int read;
while((read = in.read(buffer)) != -1){
out.write(buffer, 0, read);
}
}
Don't forget to close your streams etc
This should work.
String imgName = "/resources/images/image.jpg";
InputStream in = getClass().getResourceAsStream(imgName);
ImageIcon img = new ImageIcon(ImageIO.read(in));
Usually, you can't directly get a java.io.File object, since there is no physical file for an entry within a compressed archive. Either you live with a stream (which is best most in the cases, since every good API can work with streams) or you can create a temporary file:
URL imageResource = getClass().getResource("image.gif");
File imageFile = File.createTempFile(
FilenameUtils.getBaseName(imageResource.getFile()),
FilenameUtils.getExtension(imageResource.getFile()));
IOUtils.copy(imageResource.openStream(),
FileUtils.openOutputStream(imageFile));
You cannot create a File object to a reference inside an archive. If you absolutely need a File object, you will need to extract the file to a temporary location first. On the other hand, most good API's will also take an input stream instead, which you can get for a file in an archive.
Use Case
I need to package up our kml which is in a String into a kmz response for a network link in Google Earth. I would like to also wrap up icons and such while I'm at it.
Problem
Using the implementation below I receive errors from both WinZip and Google Earth that the archive is corrupted or that the file cannot be opened respectively. The part that deviates from other examples I'd built this from are the lines where the string is added:
ZipEntry kmlZipEntry = new ZipEntry("doc.kml");
out.putNextEntry(kmlZipEntry);
out.write(kml.getBytes("UTF-8"));
Please point me in the right direction to correctly write the string so that it is in doc.xml in the resulting kmz file. I know how to write the string to a temporary file, but I would very much like to keep the operation in memory for understandability and efficiency.
private static final int BUFFER = 2048;
private static void kmz(OutputStream os, String kml)
{
try{
BufferedInputStream origin = null;
ZipOutputStream out = new ZipOutputStream(os);
out.setMethod(ZipOutputStream.DEFLATED);
byte data[] = new byte[BUFFER];
File f = new File("./icons"); //folder containing icons and such
String files[] = f.list();
if(files != null)
{
for (String file: files) {
LOGGER.info("Adding to KMZ: "+ file);
FileInputStream fi = new FileInputStream(file);
origin = new BufferedInputStream(fi, BUFFER);
ZipEntry entry = new ZipEntry(file);
out.putNextEntry(entry);
int count;
while((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
}
}
ZipEntry kmlZipEntry = new ZipEntry("doc.kml");
out.putNextEntry(kmlZipEntry);
out.write(kml.getBytes("UTF-8"));
}
catch(Exception e)
{
LOGGER.error("Problem creating kmz file", e);
}
}
Bonus points for showing me how to put the supplementary files from the icons folder into a similar folder within the archive as opposed to at the same layer as the doc.kml.
Update Even when saving the string to a temp file the errors occur. Ugh.
Use Case Note The use case is for use in a web app, but the code to get the list of files won't work there. For details see how-to-access-local-files-on-server-in-jboss-application
You forgot to call close() on ZipOutputStream. Best place to call it is the finally block of the try block where it's been created.
Update: To create a folder, just prepend its name in the entry name.
ZipEntry entry = new ZipEntry("icons/" + file);
Given an InputStream called in which contains audio data in a compressed format (such as MP3 or OGG), I wish to create a byte array containing a WAV conversion of the input data. Unfortunately, if you try to do this, JavaSound hands you the following error:
java.io.IOException: stream length not specified
I managed to get it to work by writing the wav to a temporary file, then reading it back in, as shown below:
AudioInputStream source = AudioSystem.getAudioInputStream(new BufferedInputStream(in, 1024));
AudioInputStream pcm = AudioSystem.getAudioInputStream(AudioFormat.Encoding.PCM_SIGNED, source);
AudioInputStream ulaw = AudioSystem.getAudioInputStream(AudioFormat.Encoding.ULAW, pcm);
File tempFile = File.createTempFile("wav", "tmp");
AudioSystem.write(ulaw, AudioFileFormat.Type.WAVE, tempFile);
// The fileToByteArray() method reads the file
// into a byte array; omitted for brevity
byte[] bytes = fileToByteArray(tempFile);
tempFile.delete();
return bytes;
This is obviously less desirable. Is there a better way?
The problem is that the most AudioFileWriters need to know the file size in advance if writing to an OutputStream. Because you can't provide this, it always fails. Unfortunatly, the default Java sound API implementation doesn't have any alternatives.
But you can try using the AudioOutputStream architecture from the Tritonus plugins (Tritonus is an open source implementation of the Java sound API): http://tritonus.org/plugins.html
I notice this one was asked very long time ago. In case any new person (using Java 7 and above) found this thread, note there is a better new way doing it via Files.readAllBytes API. See:
How to convert .wav file into byte array?
Too late, I know, but I was needed this, so this is my two cents on the topic.
public void UploadFiles(String fileName, byte[] bFile)
{
String uploadedFileLocation = "c:\\";
AudioInputStream source;
AudioInputStream pcm;
InputStream b_in = new ByteArrayInputStream(bFile);
source = AudioSystem.getAudioInputStream(new BufferedInputStream(b_in));
pcm = AudioSystem.getAudioInputStream(AudioFormat.Encoding.PCM_SIGNED, source);
File newFile = new File(uploadedFileLocation + fileName);
AudioSystem.write(pcm, Type.WAVE, newFile);
source.close();
pcm.close();
}
The issue is easy to solve if you prepare class which will create correct header for you. In my example Example how to read audio input in wav buffer data goes in some buffer, after that I create header and have wav file in the buffer. No need in additional libraries. Just copy the code from my example.
Example how to use class which creates correct header in the buffer array:
public void run() {
try {
writer = new NewWaveWriter(44100);
byte[]buffer = new byte[256];
int res = 0;
while((res = m_audioInputStream.read(buffer)) > 0) {
writer.write(buffer, 0, res);
}
} catch (IOException e) {
System.out.println("Error: " + e.getMessage());
}
}
public byte[]getResult() throws IOException {
return writer.getByteBuffer();
}
And class NewWaveWriter you can find under my link.
This is very simple...
File f = new File(exportFileName+".tmp");
File f2 = new File(exportFileName);
long l = f.length();
FileInputStream fi = new FileInputStream(f);
AudioInputStream ai = new AudioInputStream(fi,mainFormat,l/4);
AudioSystem.write(ai, Type.WAVE, f2);
fi.close();
f.delete();
The .tmp file is a RAW audio file, the result is a WAV file with header.