I have an SQLite database which I have to be constantly retrieving data from. Changes may be done to the data between each retrieval.
My goal is to maximize the app performance, so what is the fastest way to do this retrieving?
I can imagine 2:
constantly opening and closing new cursors
query all data at the beginning and store it in an ArrayList. When changing the data, change both SQLite DB and the ArrayList using indexOf.
---- EDITED ----
I need the data to create markers in a google's map.
I have considered using CursorLoader but as I don't need to interact whith other apps I don't want to use Content Providers.
Would creating a custom loader be a good idea?
In short, while it's not always that simple, the fastest way to do things is all at once.
Constantly making calls to and from a database can really make your apps performance bottleneck, especially if it's to a server and not just your devices SQLite database.
Depending on what you're doing with the data, you may be able to look into something like a CursorAdapter which handles the display of rows from the database, and each time you insert/update a row, the CursorAdapter will update the ListView accordingly. It also handles the opening/closing/moving to next of the Cursor, making it very readable and easy for developers to follow.
Again, however, try to do things in as few calls as possible. If you stick to using an ArrayList:
Make one call in the beginning for all items.
Loop through that cursor and add items to an array list.
Use the array list as a cache. Sure, you could update the DB each time you update the list (which might be safest, IMO), or you can just loop through the list and insert/update/delete when the app closes. If you take that approach, make sure you do so in a method like onPause(), as it is one of the earliest methods in which an Activity can be killed.
Perfect use case for a CursorLoader. Given a query, it'll keep your list adapter up to date with the latest data, assuming you notify when changes happen in the DB. It also conveniently handles activity lifecycle events for your (ie. it'll close the cursor when the activity finishes, stop updating when it pauses, etc.).
The fastest way is obviously to not use a database at all. However, that is clearly not a solution unless you find some way of exposing your array to access from elsewhere.
Using a database is a convenient way of centralising the data so many users can access the data and have the data up-to-date at all times. Unfortunately this is the slowest option.
Choosing your middle-ground between speed and availability is a difficult task. You have to find a balance between stale data and throughput.
If, for example, you would be comfortable with a picture of the data that was valid just 5 seconds ago then you could probably cache the data locally in your array and arrange for some mechanism to keep it up-to-date running behind the scenes.
If a 5 minute lag was acceptable you could probably arrange for a regular push to database.
Also, any mechanism you use must also handle parallel changes to the data - perhaps two users change the same datum at the same time.
You just need to decide on where to strike your balance.
Related
I want to know whether it is useful to use ConcurrentHashMaps for user data. I have the user data saved in a mysql database and retrieve them when a user logs in (or someone edits the user). Every time when the user goes on another page, these user data will be refreshed. Should I use a map and save changes from my application there while having a database in background or should I directly download it from the db. I want to make the application as performant as possible.
What you are describing is a cache. Suppose the calls to the database cost a lot because there is a lot of info to load, or the query that is used to extract the data is complex and requires a lot of time. Here comes in play the cache data structure. It is basically an in memory storage, which is really faster w.r.t querying the database, because indeed, it is already loaded in memory.
The process of filling the cache takes the same time as querying the db for the data (generally more but in the same order). So it makes sense to use caches only if it brings benefit in time. There is a compromise though, speed vs freshness of data. Depending on your use-case you must find the right compromise between those two, and you shall afterwards find out if it is really convenient.
As you describe it, i.e user updates that needs to be saved and displayed, using a cache seems a bit an overkill IMO, unless you have lot of registered users, and so many of those are using the system simultaneously. If you decide to use it keep in mind of some concurrency issues that may rise. Concurrent hash maps saves you from many hazards but with performance compromise.
If the performance is the priority I think you should keep the logged users in memory.
That way, the read requests would be fast as you would not need to query the database. However, you would need to update the map if any of the logged users would be somehow edited.
A human cannot tell the difference between a 1ms delay and a 50ms delay. So it is overkill to optimize beyond "good enough".
MySQL already does a flavor of caching; your addition of another cache may actually slow down the response time.
So I am wondering about how necessary it is to store variables in the realtime firebase database if I want all Users at access the same dynamic variable.
So for instance, I have a arraylist that stores the list of open games, and if I want this list to update in realtime for every user should this List in firebase realtime database?
Sorry for the simplicity of the question
Yes, it may be a simple question, but it surely pops in head of everyone, once.
I think for updating any list dynamically in real time, would require you to access any kind of database.
It is not necessary to have it on Firebase database, but any database online, that can tell every open instance of your app that the list has to be updated at a particular instant.
The main reason of why you need it to be on database is updating it in real time and that too dynamically.
If it's not dynamic, meaning the content that you need, can be hardcoded then one way would be placing everything you need in your code and using timer or something like that to fire at particular moments to update things in your app.
Also that aside, sorting, storing and changing data is much simpler using a database, which also becomes one more reason for you to use a database like Firebase to keep content of your app that has to be updated frequently in real time.
You can know more about database in this Google link, I found.
Our team is building a small application wherein a UI has about 10 drop-down list boxes. ( DDLB ).
These list boxes will be populated by selecting data from different tables.
Our JAVA person feels that making separate database call for each list will be very expensive and wants to make a single database call for all lists.
I feel it is impractical to populate all lists in one database call due to following reason
a. Imagine an end user chooses state = 'NY' from one DDLB.
b. The next drop down should be populated with values from ZIP_CODES table for STATE='NY'
Unless we know ahead of time what state a user will be choosing - our only choice is to populate a java structure with all values from ZIP_CODES table. And after the user has selected the state - parse this structure for NY zipcodes.
And imagine doing this for all the DDLB in the form. This will not only be practical but also resource intensive.
Any thoughts ?
If there are not many items in those lists and memory amount allows you could load all values for all drop boxes into memory at application startup and then filter data in memory. It will be better then execute SQL query for every action user makes with those drop boxes.
You could also use some cache engines (like EhCache) that could offload data to disk and store only some fraction in memory.
You can run some timings to see, but I suspect you're sweating something that might take 100th of a second to execute. UI design wise I never put zip codes in selection menus because the list is too long and people already know it well enough to just punch in. When they leave the zip code field I will query the city and state and pre-fill those fields if they're not already set.
I am working on a trade's appointments agenda. The question is that what is better.
When application initialize, dump data from database to an ArrayList and iterate over ArrayList whenever that I want appointments of a day OR execute one query everytime that I want appointments of a day? The database is on localhost.
Java querys are PreparedStatement.
I ask this because when appointments will increase, Iterate over ArrayList is going to be less and less efficient.
You have different options based on how big your tables and how scalable you need your system to be.
If you have a pattern in showing the daily appointments, then I suggest you extract that pattern to have a generic solution to show the appointments of a day.
On top of this if you have validations to be running for a selected day, keep it simple and safe in your database, that way the whole point of persistence is not lost also.
Have a look at in memory databases as well.
Generally speaking I would leave database processing and the cache the database. I have done some measurements, and these show that processing records in a JVM is quite slow (filter, sort, distinct, group).
This is especially the case, if you are using higher level collections like ArrayList in contrast to arrays (like Customer[]).
I would not even worry with "remote" databases; in case of localhost databases just give the database more RAM for its cache or work memory, and it's hard to beat the database access planning/cache of a database with Java.
But you should follow the rule to query your database with a single SQL statement only. It doesn't really matter how complex that query is, but it's important that your request is done in a single query.
There is another adavantage: You have no cache to deal with explicitly. It's not hard to build caches in Java, but cache synchronization/invalidation can be difficult. So the codebase you have to maintain is much smaller.
I have a customer with a very small set of data and records that I'd normally just serialize to a data file and be done but they want to run extra reports and have expandability down the road to do things their own way. The MySQL database came up and so I'm adapting their Java POS (point of sale) system to work with it.
I've done this before and here was my approach in a nutshell for one of the tables, say Customers:
I setup a loop to store the primary key into an arraylist then setup a form to go from one record to the next running SQL queries based on the PK. The query would pull down the fname, lname, address, etc. and fill in the fields on the screen.
I thought it might be a little clunky running a SQL query each time they click Next. So I'm looking for another approach to this problem. Any help is appreciated! I don't need exact code or anything, just some concepts will do fine
Thanks!
I would say the solution you suggest yourself is not very good not only because you run SQL query every time a button is pressed, but also because you are iterating over primary keys, which probably are not sorted in any meaningful order...
What you want is to retrieve a certain number of records which are sorted sensibly (by first/last name or something) and keep them as a kind of cache in your ArrayList or something similar... This can be done quite easily with SQL. When the user starts iterating over the results by pressing "Next", you can in the background start loading more records.
The key to keep usability is to load some records before the user actually request them to keep latency small, but keeping in mind that you also don't want to load the whole database at once....
Take a look at indexing your database. http://www.informit.com/articles/article.aspx?p=377652
Use JPA with the built in Hibernate provider. If you are not familiar with one or both, then download NetBeans - it includes a very easy to follow tutorial you can use to get up to speed. Managing lists of objects is trivial with the new JPA and you won't find yourself reinventing the wheel.
the key concept here is pagination.
Let's say you set your page size to 10. This means you select 10 records from the database, in a certain order, so your query should have an order by clause and a limit clause at the end. You use this resultset to display the form while the users navigates with Previous/Next buttons.
When the user navigates out of the page then you fetch an other page.
https://www.google.com/search?q=java+sql+pagination