We Keep Coding

Google Drive API not downloading FIles (Java v3)

I'm using the code from here:
https://developers.google.com/drive/api/v3/manage-downloads#downloading_a_file
The code snippet I'm using is the following and placed in the main method:
String fileId = "some file ID";
OutputStream outputStream = new ByteArrayOutputStream();
driveService.files().get(fileId)
.executeMediaAndDownloadTo(outputStream);
I have found no sign of the code actually downloading the file, nor do I know where the file is IF it actually downloads.
I'm not sure if I am using the proper scope to gain permission to download files. I am able to upload, list, and delete files as long as I know the fileID, but downloading seems to not work.
private static final List<String> SCOPES = Collections.singletonList(DriveScopes.DRIVE);
Alternatively, I'm trying to create a method to enact the download protocol like so:
private static void downloadFile(Drive service, File file (or String fileID)){
}
but am not sure on how to do so. I've tried looking for samples online but most are from v1 or v2 apis and don't seem to work for me.
Also, I've read somewhere that it is not possible to download a Folder. Instead, I have to download each item in the folder one by one.
So do I have to make an Arraylist/list/array of the fileIDs and iterate through it after initializing a variable to represent fileID?
Edit: Some progress has been made, but I still have some problems I'm trying to thrash out.
List<File> files = result.getFiles();
File newFile;
if (files == null || files.isEmpty()) {
System.out.println("No files found.");
} else {
System.out.println("Files:");
for (File file : files) {
System.out.printf("%s (%s)\n", file.getName(), file.getId());
String fileId = file.getId();
//System.out.println(fileId);
String fileName = file.getName();
//System.out.println(fileName);
OutputStream outputstream = new FileOutputStream();
service.files().get(fileId)
.executeMediaAndDownloadTo(outputstream);
outputstream.flush();
outputstream.close();
}
What I want:
The above code is in the main method. I don't know if this is the proper way to do it, but as the program fetches each file and executes the System.out.printf, I also want it to download that file (with the same mimeType, and pref the same name too) into the destination set in the OutputStream constructor (C://User//some name//Downloads).
What I've tried:
From what I've tested, it only downloads the first file exactly the way I want, but only because I specify the name and extension in OutputStream. I've initialized variables 'fileId' and 'fileName' so that they will change according to the info as the program fetches the metadata for the next file, but I don't know how to change or set multiple constructors into this code:
OutputStream outputstream = new FileOutputStream();
service.files().get(fileId)
.executeMediaAndDownloadTo(outputstream);
to download all the files.
My folder hierarchy in Google Drive is like this:
Logs
-- bin (folder)
---- bunch of .bin files
-- .xml file
-- .xml file

You are using a ByteArrayOutputStream object as the output of your download. If your program terminates without having saved the contents of this object somewhere, you will not be able to find this information in your computer's disk, as it is not written to it but rather saved in memory as a buffered byte-array (refer to the previous link for more information).
If you want to save the output of the download to the file, I suggest you use instead a FileOutputStream as the destination of your download. In order to do that, you have to modify your code as follows:
Add the appropriate import declaration:
import java.io.FileOutputStream;
Modify your outputStream variable assignment as follows:
OutputStream outputStream = new FileOutputStream('/tmp/downloadedfile');
Where the parameter passed to FileOutputStream should be the desired destination path of your download.
After writing any contents to your file, add the following lines of code:
outputStream.flush();
outputStream.close();
This will ensure that your file is being written to properly.
In regards to downloading a folder, you are completely right - you will first need to fetch the folder you want to download, and each of their children. In order to better understand how to do it, I suggest you check out the following answer: Download folder with Google Drive API
Edit - example downloading a folder
String destinationFolder = "/tmp/downloadedfiles/";
List<File> files = result.getFiles();
File newFile;
if (files == null || files.isEmpty()) {
System.out.println("No files found.");
} else {
System.out.println("Files:");
for (File file : files) {
System.out.printf("%s (%s)\n", file.getName(), file.getId());
String fileId = file.getId();
String fileName = file.getName();
OutputStream outputstream = new FileOutputStream(destinationFolder + fileName);
service.files().get(fileId)
.executeMediaAndDownloadTo(outputstream);
outputstream.flush();
outputstream.close();
}
}

How to download monthly Treasury Files

Up till early this year the US Treasury web site posted monthly US Receipts and Outlays data in txt format. It was easy to write a program to read and store the info. All I use were:
URL url = new URL("https://www.fiscal.treasury.gov/fsreports/rpt/mthTreasStmt/mts1214.txt")
URLConnection connection.openConnection();
InputStream is = connection.getInputStream();
Then I just read the InputStream into a local file.
Now when I try same code, for May, I get an InputStream with nothing in it.
Just clicking on "https://www.fiscal.treasury.gov/fsreports/rpt/mthTreasStmt/mts0415.xlsx" opens an excel worksheet (the download path has since changed).
Which is great if you don't mind clicking on each link separately ... saving the file somewhere ... opening it manually to enable editing ... then saving it again as a real .xlsx file (because they really hand you an .xls file.)
But when I create a URL from that link, and use it to get an InputStream, the is empty. I also tried url.openStream() directly. No different.
Can anyone see a way I can resume using a program to read the new format?
In case its of interest I now use this code to write the stream to the file bit by bit... but there are no bits, so I don't know if it works.
static void copyInputStreamToFile( InputStream in, File file ) {
try {
OutputStream out = new FileOutputStream(file);
byte[] buf = new byte[1024];
System.out.println("reading: " + in.read(buf));
//This is what tells me it is empty, i.e. the loop below is ignored.
int len;
while((len=in.read(buf))>0){
out.write(buf,0,len);
}
out.close();
in.close();
} catch (Exception e) {
e.printStackTrace();
}
}
Any help is appreciated.

Read hidden zip file

I have a jpeg, and on the end of it I wrote a zip file.
Inside this zip file is a single txt file called hidden.txt. I can change the extension to zip and read the file just fine on my laptop (debian) but when I try to read it using either a ZipInputStream or using ZipFile I get an error telling me it's not a zip file.
I tried separating the jpg part out first by reading the whole thing to a Bitmap then writing that to a byte[], however the byte[] encompassed more than just the image.
My method to combine the bitmap and the zipFile (a byte[])
private byte[] combineFiles(Bitmap drawn, byte[] zip) throws
IOException {
InputStream in;
ByteArrayOutputStream out = new ByteArrayOutputStream();
/*write the first file*/
byte[] img;
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
drawn.compress(Bitmap.CompressFormat.JPEG, 100, byteArrayOutputStream);
img = byteArrayOutputStream.toByteArray();
in = new ByteArrayInputStream(img);
IOUtils.copy(in, out);
in.close();
/*add the second (hidden) file*/
in = new ByteArrayInputStream(zip);
IOUtils.copy(in, out);
out.flush();
in.close();
return out.toByteArray();
}
So really I have two questions,
How do I separate the jpg and zip portions of the file?
How do I unzip hidden.txt (preferably into a byte[])
fairly certain I know this one, but what I am doing currently does not work, probably because I am doing #1 wrong

Ok, well here's how I would do this. Although it's very hacky.
The problem is that it's hard to tell the index of the boundary between the image data and the zip data. Assuming that you can write arbitrary data after the image data and still have a working image file, here is something you could try:
write out the image data.
write out a magical string like "BEGIN_ZIP"
write out the zip data.
Now, when you are trying to read things back in:
byte[] data = readAllTheBytes();
int index = searchFor("BEGIN_ZIP", data) + "BEGIN_ZIP".length();
// now you know that the zip data begins at index and goes to the end of the byte array
// so just use a regular zipinputstream to read in the zip data.

In JPEG file 0xFF, 0xD8 sequence of bytes indicates start of image and 0xFF, 0xD9 sequence of bytes indicates end of image JPEG Structure Wikipedia. So simply search for the latter sequence in file and you will be able to separate image and zip parts. Then use ZipInputStream to read (decompress) the data from zip file.

How do I create an external .exe file using Java

I am storing all of the bytes of an external .exe file, and then re-writing them to another .exe file that I am currently creating with FileOutputStream/BufferedOutputStream.
The bytes are written fine, and the second program is created in the location of my choice, but when I come to run the file, it says it's not a valid .exe file or not a valid 32/64bit application.
I'm guessing because it's not packed and generated properly.
How would I make it so it's an executable file and works the same as the first one?
p.s I can't use any copying of the file, because eventually I'm going to be encrypting the bytes and writing them to the file, but I still want it to be usable.

If all the bytes are identical it will run. If the original file runs and the copy doesn't then some of the bytes have to be different. It has nothing to do with packing.
Maybe you are using a byte variable to store the read data. Don't do that. Just use int. If you don't use byte variables correctly you can run into problems due to automatic sign extension.
This works fine for me
import java.io.*;
public class Test {
public static void main(String[] args) throws Exception {
BufferedInputStream in = new BufferedInputStream(new FileInputStream("a.exe"));
BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream("b.exe"));
int data = in.read();
while(data >= 0) {
out.write(data);
data = in.read();
}
in.close();
out.close();
}
}

Java - Reading multiple images from a single zip file and eventually turning them into BufferedImage objects. Good idea?

I'm working on a game, and I need to load multiple image files (png, gif, etc.) that I'll eventually want to convert into BufferedImage objects. In my setup, I'd like to load all of these images from a single zip file, "Resources.zip". That resource file will contain images, map files, and audio files - all contained in various neatly ordered sub-directories. I want to do this because it will (hopefully) make resource loading easy in both applet and application versions of my program. I'm also hoping that for the applet version, this method will make it easy for me to show the loading progress of the game resources zip file (which could eventually amount to 10MB depending on how elaborate this game gets, though I'm hoping to keep it under that size so that it's browser-friendly).
I've included my zip handling class below. The idea is, I have a separate resource handling class, and it creates a ZipFileHandler object that it uses to pull specific resources out of the Resources.zip file.
import java.io.BufferedInputStream;
import java.io.File;
import java.io.IOException;
import java.util.Enumeration;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
public class ZipFileHandler
{
private ZipFile zipFile;
public ZipFileHandler(String zipFileLocation)
{
try
{
zipFile = new ZipFile(zipFileLocation);
}
catch (IOException e) {System.err.println("Unable to load zip file at location: " + zipFileLocation);}
}
public byte[] getEntry(String filePath)
{
ZipEntry entry = zipFile.getEntry(filePath);
int entrySize = (int)entry.getSize();
try
{
BufferedInputStream bis = new BufferedInputStream(zipFile.getInputStream(entry));
byte[] finalByteArray = new byte[entrySize];
int bufferSize = 2048;
byte[] buffer = new byte[2048];
int chunkSize = 0;
int bytesRead = 0;
while(true)
{
//Read chunk to buffer
chunkSize = bis.read(buffer, 0, bufferSize); //read() returns the number of bytes read
if(chunkSize == -1)
{
//read() returns -1 if the end of the stream has been reached
break;
}
//Write that chunk to the finalByteArray
//System.arraycopy(src, srcPos, dest, destPos, length)
System.arraycopy(buffer, 0, finalByteArray, bytesRead, chunkSize);
bytesRead += chunkSize;
}
bis.close(); //close BufferedInputStream
System.err.println("Entry size: " + finalByteArray.length);
return finalByteArray;
}
catch (IOException e)
{
System.err.println("No zip entry found at: " + filePath);
return null;
}
}
}
And I use the ZipFileHandler class like this:
ZipFileHandler zfh = new ZipFileHandler(the_resourceRootPath + "Resources.zip");
InputStream in = new ByteArrayInputStream(zfh.getEntry("Resources/images/bg_tiles.png"));
try
{
BufferedImage bgTileSprite = ImageIO.read(in);
}
catch (IOException e)
{
System.err.println("Could not convert zipped image bytearray to a BufferedImage.");
}
And the good news is, it works!
But I feel like there might be a better way to do what I'm doing (and I'm fairly new to working with BufferedInputStreams).
In the end, my question is this:
Is this even a good idea?
Is there a better way to load a whole bunch of game resource files in a single download/stream, in an applet- AND application-friendly way?
I welcome all thoughts and suggestions!
Thanks!

Taking multiple resources and putting in them in one compressed file is how several web applications work (i.e. GWT) It is less expensive to load one large file than multiple small ones. This assumes that you are going to use all those resources in your app. If not Lazy loading is also a viable alternative.
That being said, it is usually best to get the app working and then to profile to find where the bottlenecks are. If not you will end up with a lot of complicated code and it will take you a lot longer to get your app working. 10%-20% of the code takes 80-90% of the time to execute. You just don;t know which 10-20% that is until the project is mostly complete.
If your goal is to learn the technologies and tinker, then good going - looks good.

If you are using a Java program, it is usually considered good practice to bundle it as a jar file anyway. So why do not put your classes simply inside this jar file (in directories, of course). Then you can simply use
`InputStream stream = MayClass.class.getResourceAsStream(imagePath);`
to load the data for each image, instead of having to handle all the zip by yourself (and it also works for jars not actually on the file system, such as http url in applets).
I also assume the jar will be cached, but you should measure and compare the performance to your solution with an external zip file.