I have a table PERSON with more than 5 millions rows and I need to update field NICKNAME on each one of them based on the field NAME inside the same table.
ResultSet rs = statement.executeQuery("select NAME from PERSON");
while(rs.next())
{
// some parsing function like:
// Nickname = myparsingfunction(rs.getString("NAME"));
rs.updateString( "NICKNAME", Nickname );
rs.updateRow();
}
But I got this error:
not implemented by SQLite JDBC driver
I'm using sqlite-jdbc-3.8.11.2.jar downloaded at https://bitbucket.org/xerial/sqlite-jdbc/downloads.
I know I could use the following SQL query:
statement.executeUpdate("update PERSONS set NICKNAME = Nickname where ID = Id");
But that would take forever and I understand updating ResultSet would be faster. So what options do I have to update the table on the fastest way? Any other driver available? Should I move out of Java?
UPDATE
I was able to find a fast solution using below syntax. The block between CASE and END was a concatenated string that I built before executing the SQL query, so I could send all updates at once.
update PERSON
set NICKNAME= case ID
when 173567 then 'blabla'
when 173568 then 'bleble'
...
when 173569 then 'blublu'
end
where ID in (173567, 173568, 173569)
As you have encountered, the SQLite JDBC driver does not currently support the updateString operation. This can be seen in the source code for this driver.
I can think of three options:
As you stated in your question, you can select the name and ID of the person and then update the person by its ID. Those updates could be done in a batch (using PreparedStatement.addBatch()) to improve performance (tutorial).
Implement the method myparsingfunction in pure SQL so that the query could become UPDATE PERSONS SET NICKNAME = some_function(NAME).
Create an user-defined function (using org.sqlite.Function), implemented in Java, and call it inside the SQL. Example, taken from this answer:
Function.create(db.getConnection(), "getNickName", new Function() {
protected void xFunc() throws SQLException {
String name = value_text(0);
String nickName = ...; // implement myparsingfunction here
result(nickName);
}
});
and use it like this: UPDATE PERSONS SET NICKNAME = getNickName(NAME);
SQLite does not support stored procedures so that option is out of the table.
I'm not sure which of these options would provide the best performance (certainly using pure SQL would be faster but that may not be a viable solution). You should benchmark each solution to find the one that fits you.
Related
I am using SpringBoot connceted with Hibernate and Cassandra Database. I made couple of methods using ResultSet and everything works perfect till now. I create another method, create query and then ResultSet.
String queryString = query.toString().replace("?", dayList.toString());
ResultSet rS = dataSource.executeQuery(queryString);
It throws me:
com.datastax.driver.core.exceptions.InvalidQueryException: No keyspace has been specified. USE a keyspace, or explicitly specify keyspace.tablename
Query is correct. When I execute query in database it returns me proper data.
It is wierd because I use same implementation in previous method and it works.
Here is my query:
SELECT * FROM object_action_statistics WHERE day IN ('2018-04-29','2018-04-30') AND action_id=14 AND timestamp_from>=1525099500073 AND timestamp_from<1525120897000 ALLOW FILTERING
Correct query should be like this:
SELECT * FROM KEYSPACE_NAME.object_action_statistics WHERE day IN ('2018-04-29','2018-04-30') AND action_id=14 AND timestamp_from>=1525099500073 AND timestamp_from<1525120897000 ALLOW FILTERING
I guess you forgot to put keyspace name ahead of table name.
if i have multiple database with same Tables and Columns how can i retrieve Data from those Databases using single Query in Java.
Done this for single Database, i am newbie in java, please suggest.
public class MultipleDBTest{
public void dbConnect(String db_connect_string, String db_userid, String db_password){
try{
Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
Connection conn = DriverManager.getConnection(db_connect_string, db_userid, db_password);
System.out.println("connected");
Statement statement = conn.createStatement();
String queryString = "select <Col1>, <Col2> from <Table>";
ResultSet rs = statement.executeQuery(queryString);
while(rs.next()){
System.out.println(rs.getString(1) + " | " + rs.getString(2));
}
}
catch(Exception e){
e.printStackTrace();
}
}
public static void main(String[] args){
ConnectMSSQLServer connServer = new ConnectMSSQLServer();
connServer.dbConnect("jdbc:sqlserver://localhost;databaseName=<Database1>","<Username>","<Password>");
}
}
The easiest way to get data from multiple servers is linking them, querying the data from each table using the fully qualified table name, i.e. Server.Database.Schema.Table, and make the union of all.
Yo can only specify the desired server in the fully qualified name, Server, if you link the other servers to the server where you're making the query.
You'd end up with something like this
select * from Server1.Database1.dbo.Table
union
select * from Server2.Database2.dbo.Table
union
select * from Server3.Database2.dbo.Table
Please, see this article to understand what are linked servers and how you set them up: Linked Servers (Database Engine).
Let Sql Server do the work for you. Create a view in one of the databases that references the data from the tables in the other databases. That way your code need only access one object in one database, the view.
This is easiest if the databases are on the same server. If the databases are on separate servers you will need to link them.
If by "multiple databases" - you mean multiple schemas in the same database, then you can use the schema name and make the JOIN. Also, ensure that you have sufficient privileges to read both the schemas. SQL query would be of the form:
select S1T1.Col1, S1T1.Col2, S2T1.Col1, S2T1.Col2
from Schema1.T1 S1T1, Schema2.T1 S2T1
where S1T1.Col1=S2T1.Col1
And if you mean multiple databases on different database instances, then you may have to create links between the database instances. Refer to this SO post for more information:
Querying data by joining two tables in two database on different servers
If the information helps, don't forget to vote. Thanks! :)
Database.Schema.Table when referencing tables
and
Database.Schema.Table.Column when referencing columns
You can write joins between databases this way and deffinately pull data from more than one database.
USE [DatabaseA]
SELECT * FROM DatabaseA.dbo.DSNA_tblMaiin
INNER JOIN DatabaseB.dbo.DSNB_tblMaiin ON DatabaseA.dbo.DSNA_tblMaiin.Serialnumber = DatabaseB.dbo.DSNB_tblMaiin.Serialnumber
INNER JOIN DatabaseB.dbo.DSNC_tblMaiin ON DatabaseA.dbo.DSNA_tblMaiin.Serialnumber = DatabaseC.dbo.DSNC_tblMaiin.Serialnumber
What you are looking for is the federation layer. the layer will parse the SQL, Queries per DB will be created. Those independent queries will get fired on DB and result will be joined based on where clause. There are some Antlr based SQL Grammars available on the net, so you use them for parsing the SQL and generating DB specific SQLs.
I have an application developed based on MySQL that is connected through Hibernate. I used DAO utility code to query the database. Now I need optimize my database query by indexes. My question is, how can I query data through Hibernate DAO utility code and make sure indexes are used in MySQL database when queries are executed. Any hints or pointers to existing examples are appreciated!
Update: Just want to make the question more understandable a little bit. Following is the code I used to query the MySQL database through Hibernated DAO utility codes. I'm not directly using HQL here. Any suggestions for a best solution? If needed, I will rewrite the database query code and use HQL directly instead.
public static List<Measurements> getMeasurementsList(String physicalId, String startdate, String enddate) {
List<Measurements> listOfMeasurements = new ArrayList<Measurements>();
Timestamp queryStartDate = toTimestamp(startdate);
Timestamp queryEndDate = toTimestamp(enddate);
MeasurementsDAO measurementsDAO = new MeasurementsDAO();
PhysicalLocationDAO physicalLocationDAO = new PhysicalLocationDAO();
short id = Short.parseShort(physicalId);
List physicalLocationList = physicalLocationDAO.findByProperty("physicalId", id);
Iterator ite = physicalLocationList.iterator();
while(ite.hasNext()) {
PhysicalLocation physicalLocation = (PhysicalLocation)ite.next();
List measurementsList = measurementsDAO.findByProperty("physicalLocation", physicalLocation);
Iterator jte = measurementsList.iterator();
while(jte.hasNext()){
Measurements measurements = (Measurements)jte.next();
if(measurements.getMeasTstime().after(queryStartDate)
&& measurements.getMeasTstime().before(queryEndDate)) {
listOfMeasurements.add(measurements);
}
}
}
return listOfMeasurements;
}
Just like with SQL, you don't need to do anything special. Just execute your queries as usual, and the database will use the indices you've created to optimize them, if possible.
For example, let's say you have a HQL query that searches all the products that have a given name:
select p from Product where p.name = :name
This query will be translated by Hibernate to SQL:
select p.id, p.name, p.price, p.code from product p where p.name = ?
If you don't have any index set on product.name, the database will have to scan the whole table of products to find those that have the given name.
If you have an index set on product.name, the database will determine that, given the query, it's useful to use this index, and will thus know which rows have the given name thanks to the index. It willl thus be able to only read a small subset of the rows to return the queries data.
This is all transparent to you. You just need to know which queries are slow and frequent enough to justify the creation of an index to speed them up.
I am attempting to create a test database (based off of my production db) at runtime, but rather than have to maintain an exact duplicate test db i'd like to copy the entire data structure of my production db at runtime and then when I close the test database, drop the entire database.
I assume I will be using statements such as:
CREATE DATABASE test //to create the test db
CREATE TABLE test.sampleTable LIKE production.sampleTable //to create each table
And when I am finished with the test db, calling a close method will run something like:
DROP DATABASE test //delete the database and all its tables
But how do I go about automatically finding all the tables within the production database without having to manually write them out. The idea is that I can manipulate my production db without having to be concerned with maintaining the structure identically within the test db.
Would a stored procedure be necessary in this case? Some sample code on how to achieve something like this would be appreciated.
If the database driver you are using supports it, you can use DatabaseMetaData#getTables to get the list of tables for a schema. You can get access to DatabaseMetaData from Connection#getMetaData.
In your scripting language, you call "SHOW TABLES" on the database you want to copy. Reading that result set a row at a time, your program puts the name of the table into a variable (let's call it $tablename) and can generate the sql: "CREATE TABLE test.$tablename LIKE production.$tablename". Iterate through the result set and you're done.
(You won't get foreign key constraints that way, but maybe you don't need those. If you do, you can run "SHOW CREATE TABLE $tablename" and parse the results to pick out the constraints.)
I don't have a code snippet for java, but here is one for perl that you could treat as pseudo-code:
$ref = $dbh->selectall_arrayref("SHOW TABLES");
unless(defined ($ref)){
print "Nothing found\n";
} else {
foreach my $row_ref (#{$ref}){
push(#tables, $row_ref->[0]);
}
}
The foreach statement iterates over the result set in an array reference returned by the database interface library. The push statement puts the first element of the current row of the result set into an array variable #tables. You'd be using the database library appropriate for your language of choice.
I would use mysqldump : http://dev.mysql.com/doc/refman/5.1/en/mysqldump.html
It will produce a file containing all the sql commands needed to replicate the prod database
The solutions was as follows:
private static final String SQL_CREATE_TEST_DB = "CREATE DATABASE test";
private static final String SQL_PROD_TABLES = "SHOW TABLES IN production";
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
jdbcTemplate.execute(SQL_CREATE_TEST_DB);
SqlRowSet result = jdbcTemplate.queryForRowSet(SQL_PROD_TABLES);
while(result.next()) {
String tableName = result.getString(result.getMetaData().getColumnName(1)); //Retrieves table name from column 1
jdbcTemplate.execute("CREATE TABLE test2." + tableName + " LIKE production." + tableName); //Create new table in test2 based on production structure
}
This is using Spring to simplify the database connection etc, but the real magic is in the SQL statements. As mentioned by D Mac, this will not copy foreign key constraints, but that can be achieved by running another SQL statement and parsing the results.
I have been messing with Oracle DB queries that run from my JAVA app. I can successfully get them all to run in SQL Developer. But when I am trying to execute them from my JAVA app I usually get UpdatadbleResultSet Error/Exception on certain queries.
Also, sometimes I receive, ExhaustedResultset. As I mention at the bottom I will re work the question to break it down(When I get a chance). I keep editing and pretty soon it'll be a book.
Why is this? I cannot seem to pinpoint the problem.
Some queries run successfully such as:
SELECT table_name
FROM all_tables
SELECT column_name, data_length
FROM all_tab_columns
WHERE table_name = 'mytable'
But when I try and run something like
SELECT length(<myColumnName>)
FROM mytable
I get the updateableResultSetError
I am running my queries as methods called on button clicks (example below).
static void testQuery() {
String query = "SELECT blah from blah"
String length;
ResultSet rs = db.runQuery(query);
Length = rs.getString("length(myCol)")
System.out.println(length);
}
I have also tried while rs.next()
I can only think that for some reason I am unable to get into each table and I can only pull the "bigger" picture.
EDIT: Explained DB Connection
I am connecting using some other jarfiles that have been added to my project.
private static IDriver driver = null;
private static Database db = null;
I then pass in all my connection credentials in a separate method.
private void connectDB(){
driver = new OracleDriver();
db = new Database(driver)
driver.getPassword;
driver.getetc;
driver.getEtc;
}
EDIT:
When I getstacktrace all I am returning is.
Ljava.lang.StatckTraceElement;(assortment of random characters).
I may not be getting stack traces right so someone can fill me in. After all I am offering a bounty.
Also I will edit this question and break it down again when I have the time.
Your problem is that you're trying to update a query that can't be updated, hence the updateable result error. It seems that whoever is creating your database connection or executing your query is creating an updatable result set.
You can't use certain types of select in an updatable result set: you can't use aggregated functions (such as length, min, max), you can't use select * etc.)
For the full list see Result Set Limitations and Downgrade Rules
Try retrieving the value in your select statement via the columnIndex instead of the column name and see if that makes a difference.
Currently, its hard to tell what your db.runQuery() does since that code is not posted.
String query = "SELECT length(myCol) FROM myTable";
String length;
ResultSet rs = db.runQuery(query);
while (rs.next()) {
length = rs.getString(1);
System.out.println(length);
}
I've got an inkling what may be happening here (which would explain why some queries work, and some don't). Accoring to the jdbc ResultSet javadocs, when using the getString() method of the result set, the column label.
the label for the column specified with the SQL AS clause.
If the SQL AS clause was not specified, then the label is the name of the column
As "length(myCol)" is neither a label nor a column name, it may be that it fell over because of that (but without stacktrace it is difficult to say what your problem actually is).
Try
String query = "SELECT length(myCol) AS myCol_len FROM myTable"
ResultSet rs = db.runQuery(query);
String length = rs.getString("myCol_len");
Though are you sure, you didn't want:
int length = rs.getInt("myCol_len");
Alternatively (as written by Kal), you can use the column index to get the data from the result set, which oblivates the need for a SQL AS label:
String query = "SELECT length(myCol) FROM myTable"
ResultSet rs = db.runQuery(query);
String length = rs.getString(1);