my app download files from server into app. There could be lots of those file. One file is about 100 mb. I need to do something to safely keep them into my app.
Thirst i tried to encrypt files. How ever this is bad solution because to encrypt and decrypt 100 mb file (it's pdf file) take a some time. Also i need at a time to read this file so i need to decrypt and write decrypted file into some other file for reading at this time files is reachable.
Furthermore i can't keep this file in memory, because of file size. So maybe there is the way to encrypt directory in internal storage where file is saved ? Or this is not good idea as i should then encrypt every file in directory.
As my files is pdf, i could put password to int, but then how to do this ? Also i could try to check if device is rooted or not, but i think someone would find workaround.
So what would you suggest ?
Thanks
It seems like you have 3 options: to encrypt your data; to store the pdfs in a private folder; or to not store the files on-device.
1) Encrypt your data: As you've said, there are disadvantages because the pdfs are quite big and if you can't have those stored in memory, you need to write the decrypted files to file anyway before displaying them, so this doesn't really solve your problem.
2) Store the pdfs in a private folder: Alternatively you could store the pdfs in a private folder only accessible through your app. This can be done using
FileOutputStream fos = openFileOutput(FILENAME, Context.MODE_PRIVATE);
as noted here. "MODE_PRIVATE will create the file (or replace a file of the same name) and make it private to your application". The only problem I see with this is if people are using rooted phones and can access your app's private folders. The only way around this (as far as I know) is to use option 3.
3) Don't store the files on device: You could download the data, or parts of it, each time. This will guarantee that people can't copy the files because they never persist on the device. You could use Google Docs to stream only portions of the document to reduce download requirements if you want. The problem with this is the huge data requirement.
I think you need to weigh up the pros and cons and decide which is best for you. I'd personally go with option 2. I don't think you'll find a solution that addresses all the problems.
Related
I'm a beginner and have never dealt with cloud-based solutions yet before, so apologies for the dumb question.
I have an Azure Blob Storage containing PDF files from which I want to extract data using PDFBox. Because PDFbox can't load blobs directly, I currently download these files locally first. However, eventually my project will need to become fully Cloud-based, preferably as an Azure Function.
The main hurdle therefore is figuring out how my Azure Function should access the files. When using the console inside my Azure Function I noticed it comes with a file storage. Can the Function download blobs and store them here before processing it? Does this file storage work the same as a local environment or are there differences to keep in mind?
I'm only looking to store files temporarily here, for only a few minutes at a time.
The main hurdle therefore is figuring out how my Azure Function should
access the files. When using the console inside my Azure Function I
noticed it comes with a file storage.
Yes, all of the information of your deployed azure function is stored in the file storage you set.(It is defined when you create the function app.)
Can the Function download blobs and store them here before processing
it? Does this file storage work the same as a local environment or are
there differences to keep in mind?
Yes, you can. And the root directory is D:/home/site/wwwroot. So if you don't specify, the file you create will be in this directory.
Remember to delete the files, because the storage space is limited. It is based on the plan you selected.
I'm only looking to store files temporarily here, for only a few
minutes at a time.
By the way, if you get a file from blob storage, at this time you have completely got its data. You can process the obtained data directly in the code without temporarily storing it in the current folder. (Of course, if you have special needs, please ignore this one.)
You can use a blob trigger or input binding to load a blob into memory of your function for processing by PDFBox.
With regards to the local file system, you can read about more about it here. From the description of your problem I think a blob trigger or input binding should be sufficient for you.
I'm currently developing an MMO which utilizes numerous sprites (image files), and I plan to store these files in a compressed state on the user's hard drive. I was wondering if there already exists an implementation of an efficient, directory-based cache system, in which I can utilize to store these image files in different folders that can compress into either one file or multiple files. I was also researching LZ4 (de)compression, and I suppose that would be useful as well, but that does not solve the directory issue.
Thanks!
EDIT: For example, one file should hold numerous image files.
If something like this does not exist, what would the fastest way be to compress multiple image files into one file, and then decompress to load them into memory when the program starts?
I know that when I write a new file to a folder that ends in ".zip" it compresses the file. This is when using BufferedOutputStream in JAVA and saving to a windows file system. I'm saving these files to a network drive, so the write time is dependent on network speed.
Will saving to a .zip folder speed up write time? In other words, does it transfer the data uncompressed and then compresses it (so it wouldn't speed up write time) or does it compress then write out the file? Sorry if this is an ignorant question.
There are so many misconceptions in the Question, I think it is worth going through them one at a time.
I know that when I write a new file to a folder that ends in ".zip" it compresses the file.
That is not correct. Creating a file with a ".zip" suffix does not automatically make it compressed. Writing files to a directory that has ".zip" as its filename suffix (?!?) doesn't either. Not in Java. Not in other languages.
In order to get compression, the application needs to take steps to make this happen. In Java you could use ZipOutputStream to write a file in ZIP file format. However, a ZIP file is actually an "archive" format that is designed to hold multiple files in a ZIP file. If you simply trying to compress a single file, there are better alternatives; e.g. GZIPOutputStream.
(It is also possible that this so-called "ZIP folder" you are talking about is a normal ZIP file that has been "mounted" as a loopback file system. You / someone else would have had to set that up explicitly. Anyhow, if this is what is going on here, it is nothing to do with Java. It is all happening in external software and in the operating system where the ZIP is "mounted".)
This is when using BufferedOutputStream in JAVA and saving to a windows file system.
Erm ... no. See above. However you are correct that it may be better to use a BufferedOutputStream to write files, though it only really helps if your application is writing the files in small chunks; e.g. a byte at a time. (Stream compression complicates the issue, so it is difficult to give a simple, general answer on this.)
I'm saving these files to a network drive, so the write time is dependent on network speed.
Correct. It is also dependent on network latency, the protocols used and the load on the remote file server. (If you have a ZIP "mounted", then that is going to add overheads too.)
Will saving to a .zip folder speed up write time?
Maybe. See above. It depends what you mean by a ZIP folder.
Ignoring that, writing the files (the right way) in compressed and / or archive form from Java may speed up writes. There are actually two things to consider:
For plain compression, you are trading off the time it takes the application (!!) to compress and decompress the data against the time (and disk space) you are saving by moving and storing less bytes.
For ZIP files (and similar archive formats) there is a second potential saving. Storing and retrieving lots of individual small files from a file system is slow compared with storing and retrieving a single ZIP file containing those files.
And if you are looking for optimal compression, then ZIP is not the best option.
In other words, does it transfer the data uncompressed and then compresses it (so it wouldn't speed up write time) or does it compress then write out the file?
There are so many variables that it is hard to say for sure. But unless you have done something odd, it is likely that the bytes are sent over the network in compressed form.
Finally, I would advise you NOT to try to combine mounted ZIP files and network shares:
The combination of the two could potentially interact in ways that makes performance worse.
There is a risk that you will end up with a corrupted ZIP or lost files if the network share goes offline at an inconvenient point.
What would be a practical use for temporary files (see code below)?
File temp = File.createTempFile("temp-file-name", ".tmp");
Why can't you store the data you would keep in the file in some variables? If the file is (probably) going to be deleted on the program exit (as "temp" implies), why even create them?
An example can be such as when downloading a file, it often appears as a temporary file while the downloading completes.
The two reasons I know of:
As storage space for large chunks of memory you don't need at the moment, when doing memory-intensive tasks like video editing
A kind of hacky way of interproccess communication
Aside from the ram versus disk comment above. You may use temp files as precusor files or files about to be processed or served. For example, a server may generate a large PDF for a browser. That PDF file would be stored as a temp file while the (possibly slow) browser downloads the file. Once the communication is complete, the temp file can be destroyed.
For our little 'imagefilesystem' project (http://code.google.com/p/imagefilesystem/) we actually use the /tmp directory to store the thumbnails we created based upon the images in the local filesystem. So the thumbs were created 'on demand' and were, as the name of /tmp says it itself' temporary of nature so that it didn't create GBs of permanent data.
For my app I download some resources like images and small mp3 and save them in the external storage (at /mnt/sdcard/Android/data/com.example.packagename/cache for example.
But I don't want that if a user explores that folder finds all the resources in a "common format".
One of my options is to remove the extensions (I know it's easy to guess the file type even if it have not extension but is a basic protection against most users)
I have noticed most of the programs that have their caches at the external storage don't have their cache as raw files.
I wonder if is there any easy way (with some class or something) for "hiding" those files and access them transparently or I must implement my own system
(It is not vital that these files remain hidden but I'd like keep those resources "unknown" unless a user takes a special trouble to see them)
Thanks
You can use ObjectOutputStream, it will save data as binary data, when the user tries to open it, even using Text Editor, it will show corrupted data, and here some sample how to use it Writing objects to file with ObjectOutputStream