I'm working on a client's Point-of-Sales (POS) software. I was going to use django with MySQL but the client can't pay for a host, so I decided to write it in Java with Firebase. I'm having some trouble thinking in terms of Firebase coming from MySQL.
According to the Firebase documentation, to do relations like I would in SQL, it would have to look like:
inventory : {
CD001 : {
genre : {
"CLASSICAL" : TRUE
}
}
genre : {
CLASSICAL : {
Name : "CLASSICAL"
inventory : {
CD001 : TRUE
}
}
}
Whereas in SQL I would just put the genre primary key as a foreign key in inventory. Is there a better way to do this in Firebase? It seems for every product that has the genre CLASSICAL I would have to make two updateChildAsync(). Also, any changes (like removing a genre from inventory) I would also have to loop through two DatabaseReference.
If I were to use push to get the generated primary key it would be even worse because I would have to loop through each child just to get the genre name.
I know this may not be the optimal way to make a POS, but given the constraints of the project and that I like learning new stuff, I'm going to stick with it.
In NoSQL you will often have the same values in multiple places, to allow the specific use-cases of your app. This is normal, but may take some getting used to if you're coming from a background in relational databases.
To get to grips with Firebase's NoSQL model, I recommend reading NoSQL data modeling, watching Firebase for SQL developers (for Firebase Realtime Database) and Getting to know Clouf Firestore.
For a good example of how to model many-to-many relationships, see Many to Many relationship in Firebase. Your example looks more like a categorization problem, in which case I also recommend reading Firebase query if child of child contains a value.
Related
So I have a simple android app which is like this: It has a list of Car Manufacturers such as Honda, Ford, Mazda etc.
Then when one of the manufacturer is chosen, I need to display their car models. So if user selects Honda, I would display "Civic, CRV, Odyssey" etc.
And when user selects a specific model, I would display another list with more information.
So what kinda of data structure would be simple and easy to implement in my android app. I was thinking about using nested ArrayLists.
Also, can you please provide a link to an example on how to save the data internally using android studio and get the data when I need to.
Note: The list is dynamic, meaning that the user will have the option to add or delete any item from any of lists.
Thanks :)
This is practically what databases are for (of course I'm talking about SQLite here). I'm assuming you will have loads of data that are interconnected in a way.
I do not know how well you know database design, but you should have a table called Manufacturer which will hold the data about those (think about what data you want to save and therefore create proper columns of the table) and a table called CarModel which will have a foreign key which connects it to the Manufacturer (called manufacturer_id). These two tables need to have Many-To-One relationship -> Each manufacturer can have Many models, but each model can be produced by only One manufacturer.
This is just a basic idea which you can still improve to your own liking, but this should cover everything that you asked for very easily. For example, to get a list of Manufacturers, you'd simply Select * from Manufacturer. When you click a particular manufacturer and wanna display it's models, you'd just write a quite simple query Select * from CarModel where manufacturer_id = id (id being the parameter you pass to some function, the id of the manufacturer you want the models for. The models you select can have arbitrary amount of data like additional information about that model.
If you're not that good with raw queries, there are useful query builder which can make your life a whole lot easier.
I think that the most approaches to implement data in Android is based on SQLite. If you don't have enough experience with it i strongly recommend to investigate following tutorials with content provider:
Content-provider basics
Content-provider creating
You are right that it is very time consuming (even for experienced developers) just because this approach needs to specify entire database scheme using simple strings. But there are a few ORM-solutions you can use, such as ORMLite, but it still demanding task to investigate.
In addition i would reccomend Realm, that significantly simpler to implement, but, in my opinion, has many features you should consider (such as multithreading work)
Hope, it'll help.
I tried to make this as simple as possible with a short example.
We have two databases, one in MSSQLServer and other in Progress.
We have the user DTO as it follows that we shown in a UI table within a web application.
User
int, id
String, name
String, accountNumber
String, street
String, city
String, country
Now this DTO(Entity) is not stored only in one database, some information (fields) for the same user are stored in one database and some in the other database.
MSsql
Table user
int, id
String, name
String, accountNumber
Table userModel
int, id
String, street
String, city
String, country
As you can see the key is the only piece that link two tables in both databases, as I said before they are not in the same database and not using same database vendor.
We have a requirement for sorting the UI table for each column. Obviously we need to create user dto with the information coming from both databases.
Our proposal at this moment is if user want to apply sorting using street field, we run a query in the Progress database and obtain a page (using pagination) using this resultset and go directly to the MSSQLServer User table with those keys and run another query to extract the missing information and save it to our DTO and transfer it to the UI. With implies run a query in one database then other query based on the returned keys in the second database.
The order of the database could change depending in which column(field) the user wants to apply sorting.
Technically we will create a jparepository that acts as a facade and depending on the field make the process in the correct database.
My question is:
There is some kind of pattern that is commonly used in this scenarios, we are using spring, so probably spring have some out of the box features to support this requirement, will be great if this is possible using jparepositories (I have several doubts about it as we will use two different entitymanagers, one for each database).
Note: Move data from one database to another is not an option.
For this, you need to have separate DataSource/EntityManagerFactory/JpaRepository.
There is no out-of-the-box support for this architecture in the Spring framework, but you can easily hide the dual DataSource pair behind a Service layer. You can even configure JTA DataSources for ACID operations.
As you will always need to fetch data from both databases, why not populate local java User objects then sort these objects (using a comparator with the appropriate fields you want to sort on).
The advantage of sorting locally vs doing the sort in the database query is that you won't have to send requests to the database every time you change the sorting field.
So, to summarize:
1- Issue two sql queries for the two databases to get your users
2- Build your User objects using the retrieved values
3- Use Java comparators to sort the users on any field without having to issue new queries to the database.
My advice would be to find a way to link 2 databases together so that you can utilize database driver features without your code being affected.
Essentially if Progress database can be linked to SQL Server, you will be able to query both databases using a single SQL query with a join on id column and you will get a merged, sorted and paginated result set for your application to display.
I am not an expert in Progress database but it seems there is an ODBC driver for it so you might try to link it to SQL Server.
I am currently developing a Google AppEngine (GAE) application and I am struggling a bit with the GAE DataStore best practices. I would like to use the DataStore in the most efficient way. I am using the Objectify framework, but am flexible to use something else if there is a better alternative.
My application uses three objects/tables:
- Items (id, description)
- List (id, listId, listDescription
- SecurityProfile (id,listId, username, accessType)
I an relational world, my Items and SecurityProfiles tables would have an external key to link them to a list (ListId) and I would then use joins in my queries.
The typical Queries I need to make:
- Get all lists accessible to a particular user (need an index on "username" to filter by username and need to get the description from the List table)
- Get all items in list for a particular user (get the Items linked to the Lists retrieved in the query above)
I am struggling a bit to come up with a way to link the different objects in an efficient way (minimizing the DataStore queries and indexes).
I have seen in other posts that joins should be avoided and that I should de-normalize the model as much as possible.
So kind of creating one object only:
- Data (id, description, listId, listDescription, username, accessType)
I can see how that work from a read point of view, but if I update a listDescription, an accessType or add a new username, I could potentially have to update a massive amount of records. Is this really the way to go ?
I'm only familiar with the Python NDB API, but things are similar in Java.
In Python NDB, I would recommend to create a Model for each
User,
List,
List item
Then, you can reference them with repeated KeyProperties, e.g.
class SecurityProfiles(ndb.Model):
accessibleLists = ndb.KeyProperty(repeated=true)
class List(ndb.Model):
listItems = ndb.KeyProperty(repeated=true)
Like this, you can pull a user's profile from the DataStore, and with the keys stored in accessibleLists you can get the lists accessible to the user.
Alternatively, you could do it the other way around:
class List(ndb.Model):
usersWithAccess = ndb.KeyProperty(repeated=true)
and then you could immediately query for lists that are accessible to a given user.
I am creating a Google App Engine app that has a kind of User-Timeline in it (as in Facebook or Twitter), which consists of series of events sorted chronologically, using the Datastore.
However, I'm unable to come up with some efficient way.
In present solution, i define an entity as:
Entity:Activities:-
- Id
- Details...
- ...
- Timestamp
Is this a way so that i can index and get the activities in some sorted order, according to Timestamp ?
I am working with Java and using Objectify for data access.
The rule is : if you you want to query/order using a property, you need to index it.
I think it's pretty straight forward, what you're doing works.
The question is : what else do you want ? is performance a major need ? (probably, since your app seems user oriented).
If you only need to log user activities and retrieve it, I might suggest that you put your "Activity" entity under a parent entity "User" or "UserActivity".
UserActivity (key name = user id)
Activity (id, details, timestamp [indexed])
That way you will be able to run ancestor queries using your user's ID to retrieve all his posts.
The only drawback is that it will limit you to one write per second on this entity group (basically "all the user activity").
i am in the early stages of designing a VERY large system (its an enterprise level point of sale system). as some of you know the data models on these things can get very complicated. i want to run this thing on google app engine because i want to put more of my resources to developing the software rather than building and maintaining an infrastructure.
in that spirit of things, ive been doing a lot of reading on GAE and DataStore. im an old school relational database modeler and ive seen several different concepts of what a schemaless database is and i think ive figured out what datastore is but i want to make sure i have it right
so, if im right gae is a sorta table based system. so if i create a java entity
class user
public string firstname
public string lastname
and deploy it, the "table" user is automatically created and running. then in subsquent releases if i modify class user
class user
public string firstname
public string lastname
public date addDate
and deploy it, the "table" user is automatically updated with the new field.
now, in relating data, as i understand it, its very similar to some of the massively complex systems like SAP where the data is in fact very organized, but due to the volume its referential integrity is a function of the application, not the database engine. so i would have code that looks like this
class user
public long id
public string firstname
public string lastname
class phone
public string phonenumber
public user userentity
and to pull up the phone numbers for a user from scratch instead of
select phone from phone inner join user as phone.userentity = user where user.id = 5
(lay off i know the syntax is incorrect but you get the point)
i would do something like
select user from user where user.id = 5
then
select phone from phone where phone.userentity = user
and that would retrieve all the phone numbers for the user.
so, as i understand, its not so much a huge change in how to think about structuring data and organizing data, as its a big change on how to access it. i do joins manually with code instead of joins automatically with the database engine. beyond that its the same. am i correct or am i clueless.
There are really no tables at all. If you make some users with only a first and last name, and then later add addDate, then your original entities will still not have an addDate property. None of the user entities are connected at all, in any way. They are not in a table of Users.
You can access all of the objects you wrote to the database that have the name "User" because appengine keeps big, long lists (indexes) of all of the objects that have each name. So, any object you put in there that has the name (kind) "User" will get an entry in this list. Later, you can read that index to get the location of each of your objects, and use those locations (keys) to fetch the objects. They are not in a table, they're just floating around. Some of them have some properties in common, but this is a coincidence, and not a requirement.
If you want to fetch all of the User objects that have a certain name (Select * from User where firstname="Joe") then you have to maintain another big long index of keys. This index has the firstname property as well as the key of an entity on each row. Later you can scan the index for a certain firstname, get all the keys, and then go look up the actual entities you stored with those keys. All of THOSE entities will have the firstname property (because you wouldn't enter an entity without the firstname property on your firstname index), but they may not have any other fields in common, because they are not in a table that enforces any data structure at all.
These complications affect the way data is accessed pretty dramatically, and really affect things like transactions and complex queries. You're basically right that you don't have to change your thinking too much, but you should definitely understand how indexes and transactions work before planning your data structures. It is not always simple to efficiently tack on extra queries that you didn't think of before you got started, and it's pretty expensive to maintain these indexes, so the fewer you can get by with the better.
Great introduction to Google datastore is written by the creator of objectify framework: Fundamental Concepts of the Datastore