Android Caching To Disk - java

I am wondering if there is a way to cache arbitrary data from web requests onto the disk with Android. The flow I am thinking of is as follows:
The data is stored as a key value pair where the key is some identifier and the value is the raw data. Before actually making my web request, I check to see if the key is in the cache, if so, I skip making the web request. If the key does not exist in the cache, then I make the web request and store the data on the disk. I would like the cached data to be accessible across multiple runs of the app so that I don't have to make the web request again every time I start the app.
I was considering using SharedPreferences for this. Would SharedPreferences be the best way to go about this? Is it okay to store 1 megabyte of data in a single key in SharedPreferences?

The best solution to storing cache files is to store them in a cache directory. Luckily, the Android API provides a solution to this problem: Context#getCacheDir. You are able to create files in the directory returned, you can use a map to store an identifier for each file in order to retrieve them.
Although, this solution has a few limitations:
The system will automatically delete files in this directory as disk space is needed elsewhere on the device.
Cache data should only be used for temporary storage of information.

I may be coming late, but a couple years ago I made a library just for this:
https://github.com/fcopardo/EasyRest
The idea is to allow the app to operate with unstable or no connection without having to implement a secondary data layer for persisting data, instead, it keeps the responses for as long as you want, and refresh them without forcing the user to wait. Take a look, you may get some ideas.

Related

Preventing data modification from outer source in SQLite

I recently created a password manager using Java for my college project in OOP. To handle database I picked SQLite since using MySQL or SQL server was getting hectic for a small project. Though I am already done with the submission, I was thinking if I could do any further improvement in the project.
Biggest drawback that I have observed yet is that if anyone manages to find the location of database in the system (which is way too easy) it would be very simple to open the database.
Now here two problem arises -
User's password list will be visible
Anyone would be able to modify the data using SQLite manager.
In order to solve the first problem, I already used AES encryption and it is working just fine. However, the second problem still remains.
So in a nut shell, How can I prevent my SQLite DB to get modified except from the Password Manager itself?
Point to note that my application is just an offline Password Manager used on a household PC. So, you can consider the level of threat accordingly. Moreover, the Password Manager itself would have to modify the database content, so assigning the permission should be such that it should not prevent the application to do so.
Note: I was wondering if we can use the limitation of SQLite that only one connection to write the data can be established at a time. Using this the intruder won't be able to modify it. But, I am not sure how it can be implemented.
Restrict user access
Only the operating system can secure files against access by unauthorized persons. Put the database into a folder, which is only accessible by the current user, and have a separate database for each user.
Encryption
You're already encrypting the passwords, that's good. If you want to encrypt the whole database, you could have a look at the SQLite Encryption Extension.
The SQLite Encryption Extension (SEE) is an add-on to the public domain version of SQLite that allows an application to read and write encrypted database files.
Also have a look at the question SQLite with encryption/password protection.
Attack
What would actually happen if someone has access to the database file?
If the database is secured properly, the attacker is not able to get the plain passwords (at least not in reasonable time). In the worst case a password is replaced by another one, but that would achieve nothing, besides you using the wrong password and maybe resetting it. Therefore the worst case would be that you'll lose your saved passwords.
You can do nothing to prevent a data loss on a single machine. For example hard disks sometimes just stop working, someone could steal the whole PC, format the hard disk, etc.
Backups
If you really want to make sure that the data is not modified, you need to keep backups on different machines to minimize the possiblity that someone has access to all of them. For example you could upload the database file to a cloud service. Then you sign the file, so that you can see if a file was compromise and if so fallback to another version.
Conclusion
Your password manager is good enough for an offline tool. If you want to improve the data integrity you have to transfer the data to other machines.

Is an Android app which uses local JSON files to store data vulnerable?

My Android app uses plain text JSON files to store some data. Such files are saved into the private folder of the app, e.g. Android/data/com.example.app/. I would like to know if my app is vulnerable with such kind of files around. Data in those files are not sensible or secret, and they are not processed by JavaScript (they are parsed with JSON Java methods); I am concerned about some malicious JSON code to be injected and mess with my app or the user's device. Is it possible?
Even if the injected code was not malicious it can cause you problems because:
Others can see and alter the file. (At least with rooted devices)
if the content is altered then you are prone have unexpected results while parsing the file.
You would not want your app related data to be altered by others by any means unless you want it to be (but by using Content Providers.)
I hope it makes sense.
By default, data that you store into the private folder is neither accessible by the user nor by other applications. See the documentation on the Android developers website about this: http://developer.android.com/guide/topics/data/data-storage.html#filesInternal
However, as mentioned by SMR, if a device has been rooted this data is available to the user and might be compromised by any apps that the user has given root permission to.
However it's a minority of users that are actually rooted and those that are should more or less know what they've gotten themselves into and what they're actions can do. It's up to you to decide if it's worth some effort to look out for these special cases.
But by default your data should be safe and sound.
If the data is not that much private then you can put those data in the assets folder and you can access the same. If it is not like that then you can keep it inside the applications data folder

Attachments under File system vs databases?

I need to sore attachments at server side. I can store them either under blob column of database or under file directory.
My question is which one is more reliable, scalable and maintainable?
EDIT:-
if we go for file system, we have to handle synchroniztion yourself. Is n't it ? For example if two users are trying to create/update the File under same directory how will we handle concurrency with filesystem?
Storing data in directory is more reliable due to indexing and data fetch and other operation. Just store the path of the file into DB and store that file into directory.
When there's lot's of data store request came on server it's very hard and complex to handle so much request.
So it's better to store data on directory so accessing of data becomes more faster and when the daily scale of DB storage increase then these become so important so when you start any system first of all study it well and then decide that what to do or which technique will be the best ?
When more data are there in DB then clustering and indexing become more important.
If you want to use it for small data storage then blob it good option but for large data I ll not recommend you because I have made online data store web application and faced this situation so at end I have used to store data in directory and just path in DB.

Reduce data usage in an application supports offline-mode

I am working with an application which has offline mode. In order to do that we store the information in a local SQLite Database and using Content Provider which provides a wrapper around the SQLite, and sync it every once in a while with the data from the web service.
We are also keeping the images which are taken by user on the sdcard and send them to the server during the sync service.
The problem is bandwidth and data usage. In Android 4.0+, we have a section in device setting named Data usage. It is showing too much data usage and it annoyed the users.
My first question is : Do you think using ProGaurd which is a tool to shrink the code, can have any impact on reducing the Data Usage?
I would appreciate if you share any experience and suggestion with me in order to reduce the Data usage in such an app.
Addenda:
1 - User login to the system and during first sync sqlite file generated and transferred from REST (initialization).
2 - We have sync-status flag for entries in database. If record(using json string for data) or picture is not synced, it will transfer to the REST during sync and status-flag get updated.
3 - An updated database file receives from REST and merge with the current database on the phone in the sync service (if initialization is already done).
ProGuard has nothing to do with the amount of data you send/receive from a server. ProGuard can shrink and obfuscate code (thus making your APK smaller).
You need to analyze the data you send and receive. There is no silver bullet here that will magically solve any bandwidth issues you may come across in an app. You need to ask yourself several questions and take action depending on your answers:
What kind of numbers are we talking about?
In 2011 the average bandwidth use of an app was around 10MB per hour. There are probably more recent surveys if you search a bit. Are you far above the average number? If not, then I don't think you have to worry too much.
How often do you send and receive data?
If it's a real-time app that absolutely require live data then there's little you can do. If it's not a real-time app maybe you can reduce the frequency of send/receive or wait and collect more data before sending it to reduce overhead? If you're sending many small chunks of data you'll get a lot of overhead in HTTP headers and so on. Hold on to the small chunks a while longer and send them in one go to change the data to overhead ratio.
Can you change the protocol?
Maybe you can send data over a socket instead of HTTP to reduce overhead? By your description it doesn't sound like this would work in your case.
Can you compress data before sending it?
Make sure that your server GZips data before sending it to the client. There is a lot to gain by doing this.
Can you use another data format (binary, json, xml, custom)?
You mention that you use JSON. JSON usually/always perform better than XML, so you're already good there, but maybe you can send data in another format that is even more compact?

When caching images and data should you use Internal or External Storage?

I've read this Android save app settings/data in Internal/External Storage but I don't think i've really got an answer.
I'm developing an app where I should store some "cache" data, like a dozen of images and some strings (json).
Initially I was storing all that on the sdcard, (external storage) but later i thought that this could be deprecated by the SDCard deprecation in most recent devices.
After reading a bit, I understood that external storage is not only sdcard, but "a removable storage media (such as an SD card) or an internal (non-removable) storage" so it should not be deprecated but...it's shared space, and there is not ownership over the files stored there, so the first problem was that I was unable to delete them when the app was deleted.
So I changed to the Internal Storage, to avoid having the files/images "public" and also having them removed after app deletion.
Is this the recommended approach?
On devices older with low internal storage but with a lot of space in the SDcard is this a good approach?
My application supports from 1.6 to 4.0 (so far) so I have a lot of legacy devices... and I must have the app working (well) on all.
Looking forward for some interesting answers!
It depends on the type of data you are wanting to store.
You mention it's cached data so myassumption is that it should not matter if for some reason it all disappears. This leads me to believe that you should be using the getCacheDir(). In this case the system will remove files if the cache becomes too big so devices with low internal storage shouldn't present a problem (although it is recommended to manage this your self anyway), it's relatively secure and it will be managed by the app so if there is an uninstall it will be removed.
getExternalCacheDir() was introduced in 2.2 so isn't any use to you unless you would like to detect the version and switch between the 2 caching directories getExternalCacheDir() doesn't provide security so data could be accessed by anyway with access to the SD card. The only reason I could think you might want to do this is because of size of cache you desire but from your description the data doesn't seem excessive.
UPDATED from comment:
although this is a specific case where it's cache...but I don't want
it to be deleted whenever the system wants. It's the kind of cache
that I need the to app decide when to purge. What is the main concern
of storing in on "normal" internal storage without being on the cache
dir?
If you get to the stage where the system is cleaning up internal cached data because storage is so low then you should probably leave it to clean up this sort of app data. By using standard internal data storage you are bypassing this safe guard which would probably create more unpleasant problems than having app data deleted.
If your data is of high importance then I would suggest trying to identify specific data that is more important and managing that separately. If this data you identify needs to be secure then internal storage using files or a db (depending on the data type) seems like your only real option but you would have to be wary of this data building up.
UPDATED from comment
What do you think about using SharedPreferences to save string data?
Is there a limit on SharedPreference string saved size? Is it a (good)
possibility?
I have used shared preferences to store relatively big json strings in the past with no problem, I find it's simpler than using the databases for primitive data types (and strings) where there are limited values to save. However when you have images or lot's of values, management becomes more complex. Also you will have the same problem as you would with standard internal storage in terms of storage space.
I would keep images on the external storage, probably in a "hidden" folder (adding a dot at the beggining of the folder's name: .folder) and also a the Media Scanner "avoider" (.nomedia), because, as you pointed, you want the application to work in old devices, and precisely those devices don't have to much internal memory.
You can also add an option in your application to remove that folder, so in case the user wants to uninstall, he can do that before.
I would use internal storage for cache. This will take away the chance of a user being able to access the files used in your app. I would go for internal

Categories