I have the following schema and I am not sure how to model it in Firestore.
I will be having clients and invoices. Clients "has" invoices. I need to be able to do these 2 queries:
Show invoices that a client has
Update all invoices in the system (change a boolean attribute from true to false).
What would be the right way to model this? The first query is satisfied by having a collection of clients with subcollection of their invoices. But the second one is satisfied by having a collection of all invoices?
Any experienced help is appreciated
Thank you
I have another recommendation which involves you to create two top level collections like this:
Firestore-root
|
--- users (collection)
| |
| --- userId (documents)
| |
| --- //user details
|
--- invoices (collection)
|
--- invoiceId (documents)
|
--- yourBooleanProperty: true
|
--- userId: true
As you can see, the simplest way to achieve this, is to have a single collection named invoices that can hold as documents all the invoices in your database. Because a single invoice can belong only to a single user, you can have the userId as a property. To get all the invoices that correspond to a specific user, I recommend you to use the following query:
FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
Query query = rootRef.collection("invoices").whereEqualTo(userId, true);
And if you want to change the boolean property of all invoices from true to false at once, simply use the following query:
Query query = rootRef.collection("invoices").whereEqualTo(yourBooleanProperty, true);
Remember that Firestore uses a document-oriented NoSQL model, similar to MongoDB and CouchDB, which leads to fundamentally different data structuring decisions.
You can think this structure in a relational way, and you can achieve the same results.
And you already did that here
The first query is satisfied by having a collection of clients with
subcollection of their invoices.
so in order to solve your last requirement i would make a collection with all the invoices and then share with the clients invoice the current invoice id, so you will have inside your first query a subcollection with the ids to refer to your all invoices, this way you can refer to them and make any changes you want just querying the first collection.
let me just illustrates how they can connect
Related
I am currently implementing a blacklist feature for my application. Therefore I want to use a event based datatable, so I can also track, when a item has been blocked and by whom.
To give you a bit of a context: This is how the table looks like
id|object_id |object_type|change_time |change_type|
--|----------|-----------|-------------------|-----------|
0|1234567890|ITEM |2019-04-29 15:12:42|BLACKLISTED|
1|654321 |MATERIAL |2019-04-29 15:14:19|BLACKLISTED|
2|654321 |MATERIAL |2019-04-29 15:14:58|CLEARED |
As I am using spring and spring-data-jpa it is quite easy to get the current state of a single Item when querying for the first result ordered by time.
#Repository
public interface ItemFilterRepository extends JpaRepository<ItemFilterDpo, Integer> {
ItemFilterDpo findFirstByObjectIdAndObjectTypeOrderByChangeTimeDesc(String objectId, ItemFilterObjectTypeDpo type);
}
However, I can't find a nice solution for showing all Items that are currently blocked.
So I had a look here in stack overflow and found an answer using subqueries in the sql (SQL Query for current state of all entities in event table).
select
object_id, object_type, change_time, change_type as last_modified
from
item_filter ife
where
ife.change_time = (
select max(ife2.change_time)
from item_filter ife2
where ife.object_id=ife2.object_id
)
That gives me the following result, which I can filter for BLACKLISTED afterwards:
object_id |object_type|change_time |last_modified|
----------|-----------|-------------------|-------------|
1234567890|ITEM |2019-04-29 15:12:42|BLACKLISTED |
654321 |MATERIAL |2019-04-29 15:14:58|CLEARED |
To use that with spring-data, my first approach would be to create a view and query from that.
I'd really like to know, whether there is a better approach using spring-data to query the current state of all objects in a event datatable.
If an other framework suites better for my problem, I am happy to know.
Edit:
Using distinct on feels a bit better, however this doesn't solves my problem with spring-data.
select distinct on (object_id, object_type)
object_id, object_type, change_time, change_type as last_modified
from
item_filter bl
order by
object_id, object_type, change_time DESC;
I want to define these two collections in Cloud Firestore. Some sample data has been shown below:
Playlists:
name: "playlist1"
songCount: //To be calculated on real time
Songs:
title: "song1"
playlistId: "playlist1"
title: "song2"
playlistId: "playlist1"
title: "song3"
playlistId: "playlist1"
Further, in my Android app, I want to show separate lists of all playlists and all songs. However, while displaying the list of all playlists, I want to show the name of each playlist alongwith the number of songs contained in each, based on the value in "songCount".
Please advise what sort of query I need to make in Cloud Firestore to achieve this objective. Is there a JOIN function available or would I have to put the Songs collection in a loop and count the ones having playlistId as "playlist1", and then repeat it for all the playlists that are to be presented in the list of playlists? I want to have a smarter solution but couldn't find one on the internet.
Any help would be a great help. Thanks for your time.
Is there a Join function available?
Unfortunately, there is no JOIN query in Firestore. Queries in Firestore are shallow: they only get items from the collection that the query is run against. There is no way to get documents from a top-level collection and other collections or sub-collections in a single query. Firestore doesn't support queries across different collections in one go. A single query may only use properties of documents in a single collection. So the most simple solution I can think of would be to use a database structure that looks similar to this:
Firestore-root
|
--- playlists (collection)
| |
| --- playListId (document) //The unique id of the play list
| |
| --- name: "playlist1"
|
--- songs (collection)
|
--- playListId (document) //The same as the above playListId
|
--- playListSongs
|
--- songId
|
--- title: "song1"
In order to display all playlists, just attach a listener to the playlists reference and get all playlist objects. If you want to get all songs that correspond to a particular playlist, just attach a listener to songs\playListId\playListSongs and get all song objects.
Regarding the count of all songs that correspond to a playlist, I recommend you see my answer from this post, where I have explained what you can achieve this. So according to the last part of my answer, your Firebase Realtime Database structure should look like this:
Firebase-Realtime-Database-root
|
--- playlists
|
--- playListIdOne: numberOfSongs
|
--- playListIdTwo: numberOfSongs
Edit:
I can't say I've understood it fully especially because the first answer involves Firestore and the second involves Firebase Realtime Database.
I gave you this solution because if you want to use Cloud Firestore to count and update elements every time a song is added or deleted, you'll be charged by Firestore for every write/delete operation. Firebase Realtime Database has another pricing plan, so you'll not be charged for that. See Firestore pricing Please read again till the end, my answer from this post.
Also, I couldn't really get the procedure for calculating the count of songs.
This is how you can get the number of songs from Firestore and write it to the Firebase Realtime Database:
rootRef.collection("songs").document(playListId).collection("playListSongs")
.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
#Override
public void onComplete(#NonNull Task<QuerySnapshot> task) {
if (task.isSuccessful()) {
Log.d("TAG", task.getResult().size() + "");
DatabaseReference rootRef = FirebaseDatabase.getInstance().getReference();
DatabaseReference playListIdRef = rootRef.child("playlists").child(playListId);
playListIdRef.setValue(task.getResult().size());
} else {
Log.d(TAG, "Error getting documents: ", task.getException());
}
}
});
And this is how you can read:
DatabaseReference rootRef = FirebaseDatabase.getInstance().getReference();
DatabaseReference playListIdRef = rootRef.child("playlists").child(playListId);
ValueEventListener valueEventListener = new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
long playListIdOne = dataSnapshot.getValue(String.Long);
Log.d(TAG, "playListIdOne: " + playListIdOne);
}
#Override
public void onCancelled(#NonNull DatabaseError databaseError) {
Log.d(TAG, databaseError.getMessage());
}
};
playListIdRef.addListenerForSingleValueEvent(valueEventListener);
Secondly, as per your suggested structure, how would I get a list of all songs across all playlists?
In this case, you should create another collection, named allSongs. The additional collection in your database should look like this:
Firestore-root
|
--- allSongs
|
--- songId
|
--- //song details
This practice is called denormalization and is a common practice when it comes to Firebase. For a better understanding, I recommend you see this video, Denormalization is normal with the Firebase Database. It is for Firebase Realtime Database but same rules apply to Cloud Firestore.
Also, when you are duplicating data, there is one thing that you need to keep in mind. In the same way you are adding data, you need to maintain it. In other words, if you want to update/delete an item, you need to do it in every place that it exists.
Keeping in mind that one same song can be found in more than one playlists.
It doesn't matter because each song has a different id.
In my application each user can have multiple authorization roles. Depending on his roles the user should be allowed to see certain excerpts of data. I want to provide this data from my relational database via a REST-API.
For example:
table "Role"
UserName | Role
---------------------------
Anne | ViewFreshFruits
Mike | ViewFreshFruits
Mike | ViewTinySoft
table "Company"
Name | Address | Role
--------------------------------------------
FreshFruits | 123 America | ViewFreshFruits
TinySoft | 543 Britain | ViewTinySoft
table "Contract"
ID | CompanyName | Dollar
---------------------------
147 | FreshFruits | 15549
148 | FreshFruits | 16321
149 | TinySoft | 2311
To implement the REST-Resource http://api:8080/Application/Contracts/getAll the data (without permission check) could simply be:
SELECT Contract.* FROM Contract
But Anne is only allowed to see 147 and 148. Mike can see 147, 148, 149. And Tomy must not get any results.
I started to implement the permission check like this:
SELECT Contract.* FROM Contract
INNER JOIN Company ON Contract.CompanyName = Company.Name
INNER JOIN Role ON Role.Role = Company.Role
WHERE User = #CurrentlyAuthenticatedUser
This kind of SQL gains complexity with the number of tables in my database. I'm looking for an easier approach: less complex and easier to maintain. Performance is not my primary concern.
How can I filter certain rows of data, depending on the user, as simple as possible?
I'm using a Microsoft SQL Server 2012, Java Tomcat 8 and Connection Pooling.
That seems the easiest way to create and maintain.
If you want to have a faster SQL query, that doesn't need joins, you could create a materialized view based on that query, but without the WHERE clause and with the User on the column selection.
SELECT User, Contract.* FROM Contract
INNER JOIN Company ON Contract.CompanyName = Company.Name
INNER JOIN Role ON Role.Role = Company.Role
That way you would have a virtual table that keeps that query saved and up to date for fast authentication data retrieval. To select you would only need to:
SELECT * FROM MaterializedView
WHERE User = #CurrentlyAuthenticatedUser
How about stored procedures where you pass the role. Or to simplify you select use a view to hide some of the details.
edit
Create a view on company and authorisation tables (or a stored procedure or CTE)
CREATE VIEW CompanySecurity AS SELECT Company.*, role.username FROM Company INNER JOIN Role ON Role.Role = Company.Role
If you want contract details join that view to CompanySecurity and filter on username
If you want to see Sales, join sales to CompanySecurity and filter on username
I would like to know how to create custom setups/teardown mostly to fix cyclyc refence issues where I can insert custom SQL commands with Spring Test Dbunit http://springtestdbunit.github.io/spring-test-dbunit/index.html.
Is there an annotation I can use or how can this be customized?
There isn't currently an annotation that you can use but you might be able to create a subclass of DbUnitTestExecutionListener and add custom logic in the beforeTestMethod. Alternatively you might get away with creating your own TestExecutionListener and just ordering it before DbUnitTestExecutionListener.
Another, potentially better solution would be to re-design your database to remove the cycle. You could probably drop the reference from company to company_config and add a unique index to company_id in the company_config table:
+------------+ 1 0..1 +--------------------------------+
| company |<---------| company_config |
+------------+ +--------------------------------+
| company_id | | config_id |
| ... | | company_id (fk, notnull, uniq) |
+------------+ +--------------------------------+
Rather than looking at company.config_id to get the config you would do select * from company_config where company_id = :id.
Dbunit needs the insert statements (xml lines) in order, because they are performed sequentially. There is no or magic parameter or annotation so dbunit can resolve your cyclyc refences or foreign keys automatically.
The most automate way I could achieve if you your data set contain many tables with foreign keys:
Populate your database with few records. In your example: Company, CompanyConfig and make it sure that the foreign keys are met.
Extract a sample of your database using dbunit Export tool.
This is an snippets you could use:
IDatabaseConnection connection = new DatabaseConnection(conn, schema);
configConnection((DatabaseConnection) connection);
// dependent tables database export: export table X and all tables that have a // PK which is a FK on X, in the right order for insertion
String[] depTableNames = TablesDependencyHelper.getAllDependentTables(connection, "company");
IDataSet depDataset = connection.createDataSet(depTableNames);
FlatXmlWriter datasetWriter = new FlatXmlWriter(new FileOutputStream("target/dependents.xml"));
datasetWriter.write(depDataset);
After running this code, you will have your dbunit data set in "dependents.xml", with all your cycle references fixed.
Here I pasted you the full code: also have a look on dbunit doc about how to export data.
I am trying to understand that how object initialization works for returned object by Session Get method.Please validate my understanding. When it executes, it checks for object with given identifier in the first level cache and then the Second level cache (If it is configured), If not found then fires the select query to retrieve the data from database.
My question is, Does it include associations in select query which are configured for lazy loading or null value is set for such associations in returned object?
If this is case then session.get does not do the complete initialization of the returned object which is contradictory to what is written on most of hibernate tutorials available on web.
Hibernate Session provide different methods to fetch data from database. Two of them are – get() and load().
get() returns the object by fetching it from database or from hibernate cache.
when we use get() to retrieve data that doesn’t exists, it returns null, because it try to load the data as soon as it’s called.
We should use get() when we want to make sure data exists in the database.
For Example :
In a Stock application , Stock and StockTransactions should have a “one-to-many” relationship, when you want to save a stock transaction, it’s common to declared something like below.
Stock stock = (Stock)session.get(Stock.class, new Integer(2));
StockTransaction stockTransactions = new StockTransaction();
//set stockTransactions detail
stockTransactions.setStock(stock);
session.save(stockTransactions);
Output :
Hibernate:
select ... from mkyong.stock stock0_
where stock0_.STOCK_ID=?
Hibernate:
insert into mkyong.stock_transaction (...)
values (?, ?, ?, ?, ?, ?)
In session.get(), Hibernate will hit the database to retrieve the Stock object and put it as a reference to StockTransaction.
To answer the question:
Does it include associations in select query which are configured for lazy loading or null value is set for such associations in returned object?
1) The session.get() will NOT initiate lazy stuff. NEVER. In fact that is the central thought of the design. Otherwise - we would be able to load whole DB in one SHOT (in one JAVA call to session.get())
2) And also there WILL NOT be null instead. Each reference or collection will be represented by proxy. This is the way how we can avoid to load compelte DB in one shot (all stuff initialized with one method get). Because each proxy is in fact a promise - once we will touch it... it will load the real data.
And so on. So get is very safe way how to recieve as few data as was configured....
Simply
When get() method is called, it will directly hit the database, fetch the result and return. If no matching fields are found, it will gladly return null.
Depending on the annotations on references, Lazy or Eager, data will be returned. if Lazy, proxy will be returned instead of null, if Eager, fully initialized object will be returned.
Better to monitor the queries at the backend, for good understanding.
1) Customer entity class that map the T_CUSTOMER DB table:
#Entity
#Table(name= “T_CUSTOMER”)
public class Customer {
#Id
#Column (name=“cust_id”)
private Long id;
#OneToMany(fetch=FetchType.EAGER)
#JoinColumn (name=“cid”)
private Set<Address> addresses;
…
…
…
}
2) Address entity class that map the T_ADDRESS DB table:
#Entity
#Table(name= “T_ADDRESS”)
public class Address {
// Fields and Properties
}
Consider this Customers table :
----------------------------------------------------------------------------
| Cust_id | Cust_firstname | Cust_lastname | Cust_email | Cust_mobile |
----------------------------------------------------------------------------
| 101 | XXXX | YYYYY |xxx#xyz.com | 8282263131 |
----------------------------------------------------------------------------
The Above customers table is having one record with cust_id as 101.
Now Consider this Address Table :
----------------------------------------------------------------------------
| id | street | suburb | city | zipcode | cid |
----------------------------------------------------------------------------
| 1 | streetX | AreaY | cityZ | 54726 | 101 |
----------------------------------------------------------------------------
| 2 | streetXA | AreaYB | cityZS | 60660 | 101 |
----------------------------------------------------------------------------
Now When you invoke :
Customer cust = (Customer)session.get(Customer.class, 101);
Then Hibernate will fire a SQL Query Something like :
1). In case of EAGER LOADING :
SELECT * FROM T_CUSTOMER cust JOIN T_ADDRESS add ON cust.cust_id=add.cid
i.e, It will load all the data related to the T_CUSTOMERS table and it's associated tables, which is T_ADDRESS table in this case.
2). I case of LAZY LOADING :
SELECT * FROM T_CUSTOMER WHERE cust_id=101;
So, it only fetches the data corresponding to the T_CUSTOMER table and uses Proxy for the T_ADDRESS table as said above by #Radim Köhler. It will fetch the data from the T_ADDRESS TABLE only when you'll call :
cust.getAddresses();