I want to transfer data between databases over http and thought to use Cachedrowset since its already serialized.
How is the best way to insert a populated CachedRowset object into another database?
Is it possible at all?
Related
We can read/write data to sql server using JDBC, but for certain scalability reasons (volume & frequency), we want to use the bulk copy functionality.
According to the documentation here, there is a SQLServerBulkCopy class. There are numerous examples, includind reading from other tables, and reading from files, but there is no example on how to insert an array of rows.
SQLServerBulkCopy has 3 writeToServer methods, which take Rowset, Resultset, and ISQLServerBulkData. Is converting our array/list into one of these classes the only way to do a bulk copy? Is there any other way to do it?
Would be glad of any pointers if you've come across this before.
Is converting our array/list into one of these classes the only way to do a bulk copy? Is there any other way to do it?
Yes. The JDBC driver will also use bulk copy API for batch insert operation.
OK, so let's say I have a list that contains the following types of objects:
Objects that are already stored in the database (have the same PK),
and are the same as in the database, not modified
Objects that are already stored in the database (have the same PK), and are modified in regards to the stored ones, so they need to be updated
Objects that don't yet exist in the database, and are about to be saved
Such list of objects is being sent as a JSON to the web-service, and the web-service now has to communicate to the database, and decide what objects to store, update or ignore.
My question is how to do this effectively?
One idea is to iterate the list, and for every object's PK make a query to the database, and check if the object in the database is non-existent, the same, or modified. And then choose the action based on that information.
What bothers me with that approach is a whole lot of queries to the database, just to save some objects. What if only 1 of 100 should really be saved? It is so ineffective.
Is there any better way to do that?
You can send the whole list to DB (MYSQL) and do upsert :
INSERT ... ON DUPLICATE KEY UPDATE
There are some JDBC Rowsets like CachedRowSet, WebRowSet, FilteredRowSet and JoinRowSet. Does any bode know where they are used?
Ok, may be CachedRowSet is good where I do not want open and connection, may be WebRowSet good when I need insert some XML data ("may be" but I do not sure). But what about others?
Obviously, It is better for performance to write join in SQL query instead of create 2 JoinRowSet, take all data from they and join fields in java. The same about FilteredRowSet - it is more efficient to add where clause to the SQL query instead of grub a lot of the data and filtered it by java.
But somebody "invented" CachedRowSet, WebRowSet, FilteredRowSet and JoinRowSet why? Does anybody has some good experience about they usage?
The CachedRowSet interface defines the basic capabilities available to all disconnected RowSet objects. The other three are extensions of the CachedRowSet interface, which provide more specialized capabilities. The following information shows how they are related:
A CachedRowSet object has all the capabilities of a JdbcRowSet object plus it can also do the following:
Obtain a connection to a data source and execute a query.
Read the data from the resulting ResultSet object and populate itself with
that data.
Manipulate data and make changes to data while it is
disconnected.
Reconnect to the data source to write changes back to
it.
Check for conflicts with the data source and resolve those
conflicts
A WebRowSet object has all the capabilities of a CachedRowSet object plus it can also do the following:
Write itself as an XML document
Read an XML document that describes a WebRowSet object
A JoinRowSet object has all the capabilities of a WebRowSet object (and therefore also those of a CachedRowSet object) plus it can also do the following:
Form the equivalent of a SQL JOIN without having to connect to a data source
A FilteredRowSet object likewise has all the capabilities of a WebRowSet object (and therefore also a CachedRowSet object) plus it can also do the following:
Apply filtering criteria so that only selected data is visible. This is equivalent to executing a query on a RowSet object without having to use a query language or connect to a data source.
RowSet interface, the rows are retrieved from a JDBC data source, but a rowset may be customized so that its data can also be from a spreadsheet, a flat file, or any other data source with a tabular format.
Disconnected (not connected to the data source except when reading data from it or writing data to it)
CachedRowSet
JoinRowSet
FilteredRowSet
WebRowSet
Cached Rowset - being disconnected and able to operate without a driver, is designed to work especially well with a thin client for passing data in a distributed application or for making a result set scrollable and updatable
WebRowSet - the ability to read and write a rowset in XML format.
FilteredRowSet- used to the filtered subset of data from a rowset.
JoinRowSet -used to combine data from two different RowSet objects. This can be especially valuable when related data is stored in different data sources
Documentation
Not sure but this is what I think about FilteredRowSet. One can get the data from database by making connection in one shot. For example data for city, state and country. Later one can further subset the data with out going back to database in Java. Such as all records related to city or state or country or combination of them.
In Java,
Whether DB driver request to DB for each row of resultset? Means if there are total 200 rows are fetched then would there be 200 requests to database? Or the result set is stored locally not in DB?
If i populate a resultset (based on some conditions in SQL statement) and then i do some updation/changes in data in database. Whether resultset will return updated data?
You should clarify exactly what you mean by "request." It varies by driver and configuration where the data is cached. However, no driver will do 200 separate queries.
For the second question, it depends on the isolation level.
Each JDBC driver may implement their own fetching strategy. The driver will fetch a number of rows prior to you iterating through them. Thus a large result set would not pre-fetch ALL rows, but scroll through in fetch-sized batches.
You can provide the JDBC driver a hint as to how many you'd like to get.
Just as with in-progress SQL statements, you should not expect dirty reads by other transactions unless you'd really like to and set the isolation levels as such.
Most commonly, the Connection establishes the session and network connection the the DB and the 200 records of which you speak will be streamed (& multiplexed) through the network connection, with proper flow control.
Another approach is to use RowSet instead of ResultSet whereby you can be notified via events of changes to those rows that are affected.
I know that if you close your Statement,you can not iterate through the ResultSet.
I am trying to develop a Java application which merges data from multiple data source basically RDBMS. The scenario is some thing like this.
I have creates a connection to two data sources, lets say a MSSQL database and other Oracle. Also on each connection a user can create a DataObject( a Java object) which contains a SQL query and a connection. The query is executed on the connection and result are displayed.
Now what I want is that my user can join and filter result obtained from multiple DataObject.
Currently I am looking on the following solution:
JDO/Hibernate - I will create a object from the ResultSet obtained from the query execution and will use the multiple objects with filter and joining condition.
Java RowSet - I will create a RowSet object over result sets and user JoinRowSet and FilteredRowSet to join multiple result set.
Please advice me on my choice. Also please can other solution be looked into.
I would suggest the former. To me its as simple as getting the list of entities, and add those in a single list, based on some filter.
Oracle comes with a generic ODBC gateway that allows you to link the oracle database with another database, so you can join tables from both databases etc. with SQL, as if both tables were on Oracle. See this link for details. By doing that, you don't have to replicate database features in your java program.