I would like to know how to create custom setups/teardown mostly to fix cyclyc refence issues where I can insert custom SQL commands with Spring Test Dbunit http://springtestdbunit.github.io/spring-test-dbunit/index.html.
Is there an annotation I can use or how can this be customized?
There isn't currently an annotation that you can use but you might be able to create a subclass of DbUnitTestExecutionListener and add custom logic in the beforeTestMethod. Alternatively you might get away with creating your own TestExecutionListener and just ordering it before DbUnitTestExecutionListener.
Another, potentially better solution would be to re-design your database to remove the cycle. You could probably drop the reference from company to company_config and add a unique index to company_id in the company_config table:
+------------+ 1 0..1 +--------------------------------+
| company |<---------| company_config |
+------------+ +--------------------------------+
| company_id | | config_id |
| ... | | company_id (fk, notnull, uniq) |
+------------+ +--------------------------------+
Rather than looking at company.config_id to get the config you would do select * from company_config where company_id = :id.
Dbunit needs the insert statements (xml lines) in order, because they are performed sequentially. There is no or magic parameter or annotation so dbunit can resolve your cyclyc refences or foreign keys automatically.
The most automate way I could achieve if you your data set contain many tables with foreign keys:
Populate your database with few records. In your example: Company, CompanyConfig and make it sure that the foreign keys are met.
Extract a sample of your database using dbunit Export tool.
This is an snippets you could use:
IDatabaseConnection connection = new DatabaseConnection(conn, schema);
configConnection((DatabaseConnection) connection);
// dependent tables database export: export table X and all tables that have a // PK which is a FK on X, in the right order for insertion
String[] depTableNames = TablesDependencyHelper.getAllDependentTables(connection, "company");
IDataSet depDataset = connection.createDataSet(depTableNames);
FlatXmlWriter datasetWriter = new FlatXmlWriter(new FileOutputStream("target/dependents.xml"));
datasetWriter.write(depDataset);
After running this code, you will have your dbunit data set in "dependents.xml", with all your cycle references fixed.
Here I pasted you the full code: also have a look on dbunit doc about how to export data.
Related
I have the following schema and I am not sure how to model it in Firestore.
I will be having clients and invoices. Clients "has" invoices. I need to be able to do these 2 queries:
Show invoices that a client has
Update all invoices in the system (change a boolean attribute from true to false).
What would be the right way to model this? The first query is satisfied by having a collection of clients with subcollection of their invoices. But the second one is satisfied by having a collection of all invoices?
Any experienced help is appreciated
Thank you
I have another recommendation which involves you to create two top level collections like this:
Firestore-root
|
--- users (collection)
| |
| --- userId (documents)
| |
| --- //user details
|
--- invoices (collection)
|
--- invoiceId (documents)
|
--- yourBooleanProperty: true
|
--- userId: true
As you can see, the simplest way to achieve this, is to have a single collection named invoices that can hold as documents all the invoices in your database. Because a single invoice can belong only to a single user, you can have the userId as a property. To get all the invoices that correspond to a specific user, I recommend you to use the following query:
FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
Query query = rootRef.collection("invoices").whereEqualTo(userId, true);
And if you want to change the boolean property of all invoices from true to false at once, simply use the following query:
Query query = rootRef.collection("invoices").whereEqualTo(yourBooleanProperty, true);
Remember that Firestore uses a document-oriented NoSQL model, similar to MongoDB and CouchDB, which leads to fundamentally different data structuring decisions.
You can think this structure in a relational way, and you can achieve the same results.
And you already did that here
The first query is satisfied by having a collection of clients with
subcollection of their invoices.
so in order to solve your last requirement i would make a collection with all the invoices and then share with the clients invoice the current invoice id, so you will have inside your first query a subcollection with the ids to refer to your all invoices, this way you can refer to them and make any changes you want just querying the first collection.
let me just illustrates how they can connect
Given the following SQL structure of MY_TABLE:
GROUP_LABEL | FILE | TOPIC
-----------------------------
group A | 1.pdf | topic A
group A | 1.pdf | topic B
group A | 2.pdf | topic A
group B | 2.pdf | topic B
My task is to get this stuff grouped by GROUP_LABEL, while forgetting about the different TOPICs of a file. So my expected result is
GROUP_LABEL | COUNT(*)
----------------------
group A | 2 -- two different files 1.pdf and 2.pdf here
group B | 1 -- only one file here
In pure SQL I would do it like
SELECT GROUP_LABEL, COUNT(*) FROM (
SELECT DISTINCT GROUP_LABEL, FILE FROM MY_TABLE
);
Is it possible to transform it into a JPA Criteria API query? I don't have any idea to get my inner query into the from construct of the Criteria query, in 9.3.1 of https://docs.jboss.org/hibernate/entitymanager/3.5/reference/en/html/querycriteria.html it seems like this is not possible.
But I just can't believe it ;-) Has anyone done this before? The inner query would be enriched with various, well-tested, filter Predicates which I would want to reuse.
I'm using spring-boot-starter-data : 1.5.6.RELEASE with mainly standard configuration.
Try this,
Query: select label, count(distinct file) from tableName group by label;
Criteria: criteria.setProjection(Projections.projectionList().add(Projections.groupProperty("label")).add(Projections.countDistinct("file")));
Firstly your sql query can be resumed to this :
Select distinct GLOBAL_LABEL ,count (distinct FILE) from MY_TABLE group by GLOBAL_LABEL
Secondly it's always good to not name your columns with primary names to avoid problems.
Finaly you can use this as your HQL query (with no magic) :
Select distinct ge.globalLabel,count (distinct ge.file) from GlobalEntity ge group by ge.globalLabel
Yes it is possible by using JPA javax.persistence.criteria API.
Take a look at this example in the official Documentation.
I'm currently trying to insert in batch many records (~2000) and Jooq's batchInsert is not doing what I want.
I'm transforming POJOs into UpdatableRecords and then I'm performing batchInsert which is executing insert for each record. So Jooq is doing ~2000 queries for each batch insert and it's killing database performance.
It's executing this code (jooq's batch insert):
for (int i = 0; i < records.length; i++) {
Configuration previous = ((AttachableInternal) records[i]).configuration();
try {
records[i].attach(local);
executeAction(i);
}
catch (QueryCollectorSignal e) {
Query query = e.getQuery();
String sql = e.getSQL();
// Aggregate executable queries by identical SQL
if (query.isExecutable()) {
List<Query> list = queries.get(sql);
if (list == null) {
list = new ArrayList<Query>();
queries.put(sql, list);
}
list.add(query);
}
}
finally {
records[i].attach(previous);
}
}
I could just do it like this (because Jooq is doing same thing internally):
records.forEach(UpdatableRecord::insert);
instead of:
jooq.batchInsert(records).execute();
How can I tell Jooq to create new records in batch mode? Should I transform records into bind queries and then call batchInsert? Any ideas? ;)
jOOQ's DSLContext.batchInsert() creates one JDBC batch statement per set of consecutive records with identical generated SQL strings (the Javadoc doesn't formally define this, unfortunately).
This can turn into a problem when your records look like this:
+------+--------+--------+
| COL1 | COL2 | COL3 |
+------+--------+--------+
| 1* | {null} | {null} |
| 2* | B* | {null} |
| 3* | {null} | C* |
| 4* | D* | D* |
+------+--------+--------+
.. because in that case, the generated SQL strings will look like this:
INSERT INTO t (col1) VALUES (?);
INSERT INTO t (col1, col2) VALUES (?, ?);
INSERT INTO t (col1, col3) VALUES (?, ?);
INSERT INTO t (col1, col2, col3) VALUES (?, ?, ?);
The reason for this default behaviour is the fact that this is the only way to guarantee ... DEFAULT behaviour. As in SQL DEFAULT. I gave a rationale of this behaviour here.
With this in mind, and as each consecutive SQL string is different, the inserts unfortunately aren't batched as a single batch as you intended.
Solution 1: Make sure all changed flags are true
One way to enforce all INSERT statements to be the same is to set all changed flags of each individula record to true:
for (Record r : records)
r.changed(true);
Now, all SQL strings will be the same.
Solution 2: Use the Loader API
Instead of batching, you could import the data (and specify batch sizes there). For details, see the manual's section about importing records:
https://www.jooq.org/doc/latest/manual/sql-execution/importing/importing-records
Solution 3: Use a batch statement instead
Your usage of batchInsert() is convenience that works when using TableRecords. But of course, you can generate an INSERT statement manually and batch the individual bind variables by using jOOQ's batch statement API:
https://www.jooq.org/doc/latest/manual/sql-execution/batch-execution
A note on performance
There are a couple of open issues regarding the DSLContext.batchInsert() and similar API. The client side algorithm that generates SQL strings for each individual record is inefficient and might be changed in the future, relying on changed() flags directly. Some relevant issues:
https://github.com/jOOQ/jOOQ/issues/4533
https://github.com/jOOQ/jOOQ/issues/6294
I am trying to understand that how object initialization works for returned object by Session Get method.Please validate my understanding. When it executes, it checks for object with given identifier in the first level cache and then the Second level cache (If it is configured), If not found then fires the select query to retrieve the data from database.
My question is, Does it include associations in select query which are configured for lazy loading or null value is set for such associations in returned object?
If this is case then session.get does not do the complete initialization of the returned object which is contradictory to what is written on most of hibernate tutorials available on web.
Hibernate Session provide different methods to fetch data from database. Two of them are – get() and load().
get() returns the object by fetching it from database or from hibernate cache.
when we use get() to retrieve data that doesn’t exists, it returns null, because it try to load the data as soon as it’s called.
We should use get() when we want to make sure data exists in the database.
For Example :
In a Stock application , Stock and StockTransactions should have a “one-to-many” relationship, when you want to save a stock transaction, it’s common to declared something like below.
Stock stock = (Stock)session.get(Stock.class, new Integer(2));
StockTransaction stockTransactions = new StockTransaction();
//set stockTransactions detail
stockTransactions.setStock(stock);
session.save(stockTransactions);
Output :
Hibernate:
select ... from mkyong.stock stock0_
where stock0_.STOCK_ID=?
Hibernate:
insert into mkyong.stock_transaction (...)
values (?, ?, ?, ?, ?, ?)
In session.get(), Hibernate will hit the database to retrieve the Stock object and put it as a reference to StockTransaction.
To answer the question:
Does it include associations in select query which are configured for lazy loading or null value is set for such associations in returned object?
1) The session.get() will NOT initiate lazy stuff. NEVER. In fact that is the central thought of the design. Otherwise - we would be able to load whole DB in one SHOT (in one JAVA call to session.get())
2) And also there WILL NOT be null instead. Each reference or collection will be represented by proxy. This is the way how we can avoid to load compelte DB in one shot (all stuff initialized with one method get). Because each proxy is in fact a promise - once we will touch it... it will load the real data.
And so on. So get is very safe way how to recieve as few data as was configured....
Simply
When get() method is called, it will directly hit the database, fetch the result and return. If no matching fields are found, it will gladly return null.
Depending on the annotations on references, Lazy or Eager, data will be returned. if Lazy, proxy will be returned instead of null, if Eager, fully initialized object will be returned.
Better to monitor the queries at the backend, for good understanding.
1) Customer entity class that map the T_CUSTOMER DB table:
#Entity
#Table(name= “T_CUSTOMER”)
public class Customer {
#Id
#Column (name=“cust_id”)
private Long id;
#OneToMany(fetch=FetchType.EAGER)
#JoinColumn (name=“cid”)
private Set<Address> addresses;
…
…
…
}
2) Address entity class that map the T_ADDRESS DB table:
#Entity
#Table(name= “T_ADDRESS”)
public class Address {
// Fields and Properties
}
Consider this Customers table :
----------------------------------------------------------------------------
| Cust_id | Cust_firstname | Cust_lastname | Cust_email | Cust_mobile |
----------------------------------------------------------------------------
| 101 | XXXX | YYYYY |xxx#xyz.com | 8282263131 |
----------------------------------------------------------------------------
The Above customers table is having one record with cust_id as 101.
Now Consider this Address Table :
----------------------------------------------------------------------------
| id | street | suburb | city | zipcode | cid |
----------------------------------------------------------------------------
| 1 | streetX | AreaY | cityZ | 54726 | 101 |
----------------------------------------------------------------------------
| 2 | streetXA | AreaYB | cityZS | 60660 | 101 |
----------------------------------------------------------------------------
Now When you invoke :
Customer cust = (Customer)session.get(Customer.class, 101);
Then Hibernate will fire a SQL Query Something like :
1). In case of EAGER LOADING :
SELECT * FROM T_CUSTOMER cust JOIN T_ADDRESS add ON cust.cust_id=add.cid
i.e, It will load all the data related to the T_CUSTOMERS table and it's associated tables, which is T_ADDRESS table in this case.
2). I case of LAZY LOADING :
SELECT * FROM T_CUSTOMER WHERE cust_id=101;
So, it only fetches the data corresponding to the T_CUSTOMER table and uses Proxy for the T_ADDRESS table as said above by #Radim Köhler. It will fetch the data from the T_ADDRESS TABLE only when you'll call :
cust.getAddresses();
Following is my Cassandra table structure.
Advertisement
AdvertisementId | Ad_Language | Ad_Caption | Others
----------------------------------------------------------------------
A01(UUID) | EN_US (text)| englishCaption (text) | Other Info(text)
A01(UUID) | FR_CA (text)| French Caption (text) | Other Info (text)
Primary key is (AdvertisementId, Ad_Language);
I am using java to integrate with Cassandra. There is a Java API call to fetch List<advertisements>
Is there a possiblity to fetch the rows like
Query : select * from ad_details orderBy advertisementId; (Unfortunately I cannot specify a col_name that will be used in WHERE or In clause)
I cannot have advertisement Id as cluster key as I need to maintain the UUID as partition key of the composite primary key in Cassandra.
The following query works: Select * from ad_details where advertisementId=xxx orderBy language ASC;
Can someone please help me in carrying out the orderBy advertisementId?
You can not order by a partition key when using the MurMur3partitioner or RandomPartitioner. If you are using an ordered partitioner the results will be in the order of the type specified for the partition key when creating the table.
You can't order by primary key unless you are using IN.
If you are not limiting your search with "where" you probably need to redesign your table as it is not an efficient design, when table gets big you can't query it in efficient way.