I've created an android app with Java and I use a Firebase realtime-database. Via REST, I get JSON-data for cities. "Data" means:
base data like the city-id, name or population. They rarely change. This is represented in my model class "BaseData".
(movement) data which changes every day. The class is called "CityData".
BaseData is currently part of the CityData:
public class CityData
{
private BaseData baseData;
private int cases;
//...
}
At this time, I can only request data for today. When I save a CityData-object in my db, it looks like this:
My next step: Save the data for the last 7 days. But: Firebase would also save the base data every day. Can I prevent this (=> only save when not exists or update when something changes)? Otherwise, I have to change the data struct. But which is the best possibility?
My first idea: Create two seperate paths like this:
But then, I have to extract BaseData from CityData and join them every time, for example when I need the city name, which is often. Is this the best way? It's my first time with NoSql, so every help would be nice!
Related
My problem is more or less asked here Spring Batch : Compare Data Between Database however I still cannot get my head around it. Maybe it's a bit different.
I have A datasource and I want to write into database B.
I have full trust in A datasource, so if;
A Does contain the record that B does not, I have to add B.
A Does not contain the record that B does, I have to delete from B
A does contain, B does contain, I check and update the record in B accordingly.
I thought my approach would be simple as;
Read Person from A datasource
Read Person from B datasource
(Those two Person can be having different entities)
Compare and find the ones to Add,Update,Delete.
Update the database.
However since I am pretty newbie to Spring Batch, the implementation is kind of ending up to a spaggetti code which I don't want and want to learn the right way for it.
So;
I created this job below
#Bean
public Job job() {
return jobBuilderFactory
.get("myNewbieJob")
.start(populateARepository())
.next(populateBRepository())
.next(compareAndSubmitCountryRepositoriesTasklet())
.build();
}
To explain;
populateARepository() populateARepository() : I have a Repository object just contains a list. This step just does add records to the list.
The part that I don't like is that compareAndSubmitCountryRepositoriesTasklet() is basically comparing those repositories... and then I don't know what to do.
If I create a DB access and push from that class, I won't like it, because I just wanted it to be a step where I find the differences.
If I create another class which contains 3 separate lists for toUpdate,toDelete,toInsert, and then in the next step somehow use that repository... that sounded wrong to me as well.
So, here I am. Any kind of guidance is appreciated. How would you deal in this situation?
Thank you in advance.
Before talking about Spring Batch, I would first look for an algorithm to solve this problem. If I understand correctly, you basically need to replicate the same state of records in database A into database B. What you can do is:
Read Person items from database A
Use an item processor to do the comparison with table B. Here, you would mark the item accordingly to be inserted, updated or deleted
Use an item writer that checks the type of record and do the necessary operation. Here, you can create a custom writer or use a ClassifierCompositeItemWriter (see this example)
This approach works well with small/medium datasets but not for large datasets (due to the additional query for each item, but this is inherent to the algorithm itself and not the implementation with Spring Batch).
I was thinking about fetching data from Firebase and storing it on my phone using room library. But, I have a doubt in my mind. I want to use Firebase ChildEventListener because I want to fetch data only when a child is changed or added. But, I also know that ChildEventListener fetches all data when the app is started for the first time. So,my question is that in what cases firebase ChildEventListener will fetch whole data and in what cases a single child?. Actually, knowing this is important because if ChildEventListener fetches same data twice, room will give unique id duplication error.
If you cannot control when ChildEventListener is firing, an easy strategy would be to overwrite in all cases. You can avoid unique id duplication error with following code:
#Insert(onConflict = OnConflictStrategy.REPLACE)
If replace is not suitable for you there are other options.
Classes implementing ChildEventListener interface can be used to receive events about changes in the child locations of a given database reference. You should use this interface only if you want to respond to child-level changes because this interface has separate methods for when a child is added, removed, changed or moved (onChildAdded(), onChildRemoved(), onChildChanged() and onChildMoved() respectively).
In what cases firebase ChildEventListener will fetch whole data and in what cases a single child?
For example, onChildAdded() method is called once when you start the app to get all the children from a particular location but is also called everytime a new child is added at that location.
You can use below code to remove duplicate of data
#Entity(tableName = "post",indices = #Index(value = {"id"},unique = true))
In your model class you can give your "id" as unique so that same data will not store multiple times. Remember one point don't provide your primary key as unique you have to provide a separate id for your Room Database. hope it will help you :)
I have parts of data the coming not by order. Let me explain :
User with id 1 sending data ("phone number") but in the same time user with id 78 sending another data ("home address").
All the incoming data have the same receiver.
When all set of the data per id is set the data would be sent to mysql database and deleted from temporarily storage.
Each user id need to fill 6 different information before the complete data is sent.
So the question is how to to store the temporary data parts using their personal id (without mixing them up) and only then I have all the parts I will proceed to the next task ?
Should I use arrayList or something different ?
Edit (answering the duplicate suggestion ): my question is a bit different and the answer there not helping me at all !
You could, as you say, store it in some structure in memory. I'd pick some method which isn't tied to an Activity, otherwise you need to track the activity lifecycle. E.g. you could have a HashMap to map entity name (i.e. name, address, etc) to its value (or in case you're storing multiple instances of a single entity, go with some kind of a Data class suggested in the other answer). Personally, I find this method most cumbersome/cluttered, because you need to either divorce your storage from your Activity or handle all activity changes in order not to lose state.
Second approach is using SharedPreferences. Make a separate prefs file for that purpose, obtain them (getSharedPreferences("partial_data_dl", MODE_PRIVATE)), store each column as one field in the prefs, and clear them before commiting them to the database. It's easier than storing everything in memory and shouldn't be noticeably slower.
Third is the obvious one: why don't you update the database record as the data comes in? First time insert a new record with only one column populated, every other issue an update query to add value for the new column. This is admittedly the heaviest solution and might not work for your use case, but I'd give it a try and test it out. As the saying goes, premature optimization is the root of all evil.
you can create object 'Data'
class Data {
private param1,param2,...,param6;
public Data(){
}
/* PARAMS getters */
/* PARAMS setters */
public void insert(){
//insert to database
}
}
then use HashMap to update the data :
HashMap<Integer, Data> map = new HashMap<Integer, Data>();
and every time you recive some data and id check the type of the data (which param in the Data object) and update it in the map
Data toUpdate = map.get(id);
toUpdate.setParam...
you can make a listener for each time you set some param to indicate that all the data had been set and ready to be inserted
I'm working with Firebase and have seen somewhere you can work the data so that they are in groups.
In my app I'm going to have genres and sub genres of documentaries ie:
(Genre)
History {
(sub genre)
world history {
(items)
pyramids
}
hidden history {
(items)
pyrmids
}
}
As you can see some of the items will be the same. Instead of rewriting the same item over and over again, is there a way to add them to a group so both could access the same item?
There is no problem with duplicating data, when it comes to Firebase. This is a quite common practice, which is named denormalization and for that, I recommend you see this video, Denormalization is normal with the Firebase Database.
When you are duplicating data, there is one thing that need to keep in mind. In the same way you are adding data, you need to maintain it. With other words, if you want to update/detele an item, you need to do it in every place that it exists.
In your case, you should consider augmenting your data structure to allow a reverse lookup like this:
pyrmids
|
--- "worldHistory": true
|
--- "hiddenHistory": true
I need help with adding a database in my app for android. What I want to do is save name, address, if certain checkboxes are checked and textbox data. so far this is what I have in my main activity xml and java (download links):
https://www.dropbox.com/s/vp4fg0f5kx0p5it/activity_main.xml?dl=1
https://www.dropbox.com/s/p2dkqtoe8h1dl85/MainActivity.java?dl=1
All the other questions and tutorials just don't seem able to be grasped by my mind. I want to save the "name", "address", "AirSealing", "airsealnotes", "DuctSealing", "BlowerDoor", "ductsealnotes", "blowdoornotes", "Light", "lightnotes", "othernotes", "HPD", "HPGJGNY", "Placeholder", and the save button to save data.
There's a lot of boilerplate code associated with creating and handling an SQLite database (even just one with a single table), so don't feel bad about not getting it right away.
Here's the tutorial I personally used to grasp SQLite databases in Android:
http://www.androidhive.info/2011/11/android-sqlite-database-tutorial/
Essentially, you'll want to follow the tutorial, only replacing their columns (a name and phone number) with the data you want to store. The custom helper class isn't mandatory - you can simply take the code they put in the DB handler class and write it in your main activity class instead - but it makes writing and retrieving data a lot easier.
Also, a word of warning: you won't be able to store every type of data in a database out of the box. For instance, if you want to store a boolean (which would be your checkboxes), you'll have to do it by storing, say, an integer or a string, instead of an actual boolean.