So I have a MYSQL db in which boolean values are stored as binary(1). I was to investigate why certain queries were slow even though there was an index on the relevant columns. The issue was that when building the SELECT query, the system was using the setBoolean method of PreparedStatement which, as I understand it, converts the value to MYSQL TINYINT. The query found the correct rows, but never used the index since the index was on a binary column. However, if I instead used the setString method and converted the boolean to a string, namely '0' for false and '1' for true, MYSQL was able to use the index and find the wanted rows fast.
Basically, the first query is what I got when using setBoolean and the second when using setString:
SELECT someColumn FROM table WHERE binaryColumn = 1 //Does not use index
SELECT someColumn FROM table WHERE binaryColumn = '1'//Uses index
In Java the change was this:
PreparedStatement ps1 = ...
ps1.setBoolean(1, true);
...
PreparedStatement ps2 = ...
ps2.setString(1, "1");
...
My question is simply if there is a better way to do this? Everything works fine but for some reason I think the code "smells" but I cant really motivate why.
I prefer always the setBoolean, because of abstraction.
The real interesting point is when your DB uses the index.
The optimizier of the DB use a index only, if it makes sense. If you have 1000 entries and a booleanvalue only split it into 50/50 it make no sense for that index, especial when its not the PK - but if you use a additional limitation, to get only 10 rows, as result, a good optimizer should use the index you specified - maybe a "composed index" on 2 columns (booleanColumn1, StringColumn1)
MySQL uses TINYINT(1) for the SQL BOOL/BOOLEAN. So I would change the data type to BOOLEAN, in accordance to standard SQL.
By your relay, the issue should then be resolved. By the way BIT(1) would be another option.
Related
I post a similar question previously, but have opened another question to be more specific as the previous one gave me a solution but I have now encountered another problem.
We have an existing Oracle database that had boolean columns defined like so
CREATE TABLE MY_TABLE
(
SOME_TABLE_COLUMN NUMBER(1) DEFAULT 0 NOT NULL,
etc..
And with the corresponding java field defined like private boolean someTableColumn; I've come to understand this is because Oracle does not have a Boolean datatype, but under the hood does the conversion from boolean to integer when inserting data, and the reverse when retriving data.
This has caused an issue when I have been working on migrating our database from Oracle to Postgres. Thanks to answers on my previous question, I have migrated the column type from NUMBER(1) to BOOLEAN. This has solved the issue with inserting data. However, our codebase uses JDBCTemplate and we unfortunately have hundereds of hardcoded queries in the code that make queries like SELECT * FROM MY_TABLE WHERE TABLE_COLUMN=1.
When these run against the Postgres DB, I get the following error ERROR: operator does not exist: boolean = integer. We have a requirement to have backwards compatability with Oracle, so I can't simply update these queries to replace 1 and 0 with TRUE and FALSE respectively.
Is there a way I can configure Postgres so it can do a conversion behind the scenes to resolve this? I have looked at casts but I don't really understand the documentation and the examples given don't seem to match my use case. Any help is appreciated.
Can you try to use '0' and '1' instead of 0 and 1 in your requests ?
I used to work on apps compliant with both Oracle and Postgresql using this syntax. Apps were using JPA but can say with certainty that we were also using this syntax with nativeQuery = true.
Note: I would have posted this as a comment but I don't have the required reputation to do so, hence the post as an answer
Update:
Duh! Brain Freeze. On later thought there is a way to get this conversion in both directions. Process via a View.
Steps:
Rename your table.
Create a view having the same name as the old table. In this view
translate the boolean column to the appropriate integer.
Create an trigger function and an instead of trigger on the view for insert/update dml. In the trigger function translate the column value to boolean as appropriate.
See revised demo.
alter table testb rename to testb_tab;
create or replace view testb (id, name, is_ok)
as
select i,
, name
, is_ok::int
from testb_tab;
create or replace
function testb_act_dml()
returns trigger
language plpgsql
as $$
begin
if tg_op = 'INSERT' then
insert into testb_tab(name,is_ok)
values (new.name, new.is_ok::boolean) ;
else
update testb_tab
set name = new.name
, is_ok = new.is_ok::boolean
where id = old.id;
end if;
return new;
end;
$$;
create trigger testb_biuri -- before insert update row instead of
instead of insert or update on testb
for each row execute function testb_act_dml();
Finally, there is another option which probably has the lease work. Do not change the column description to Boolean. Either leave it as an integer or define it as a smallint. Either way a check constraint may come in useful. So something along the line of:
create table tests( id int generated always as identity primary key
, name text
, is_ok smallint
, constraint is_ok_ck
check ( is_ok in (0,1) or is_ok is null)
);
This is one of the issues I have with the concept of "database independence". It simply does not exist. Vendors implementation often simply vary too much. In this case Postures to the rescue, perhaps but also perhaps extreme: create you own create your own operators. Proceed with caution:
-- function to compare boolean = integer
create or replace
function"__bool=int"( b boolean, i int)
returns boolean
language sql
as $$
select (b=i::boolean);
$$;
-- create the Operator for boolean = integer
create operator = (
leftarg = boolean
, rightarg = int
, function = "__bool=int"
, commutator = =
);
The above will not allow your code: SELECT * FROM MY_TABLE WHERE TABLE_COLUMN=1 (see demo here).
However, this road may lead to unexpected twists, and lots of function/operator pairs. For example the above does not support SELECT * FROM MY_TABLE WHERE TABLE_COLUMN<>1. That requires another function/operator combination. Further I do not see a retrieval function that converts a boolean back to an integer. If you follow this path be sure to massively test your boolean-to-integer (integer-to-boolean) operations. It may just be better to just byte the bullet and updated those few queries (you did say hundreds not thousands) as #mlogario suggests.
I am trying to batch updates using PreparedStatement where the data you get in varies in what you have available.
Simple example:
You have a table with column x and y. Your inputdata is a representation of rows where only the modified column is present. So for row 1 you have x='foo' and for row 2 you have y='bar'. In order to batch these with a PreparedStatement you need one SQL statement: "update table where x=?,y=? where etc".
The solution that I am working towards is setting the column where you dont have any value to the value that is already present, but I'm not sure if this is possible. In raw SQL you could write "update table where x=x and y='foo' where etc", however I haven't found any ways to achieve this by setting the "?" parameter to be a reference to the column, it seems it's not possible.
Is it possible to handle this case at all with PreparedStatements? I apologize if my explanation is poor.
Assuming the values you want to set are all non-null, you could use a statement like
update sometable set column1 = coalesce(?, column1), colum2 = coalesce(?, column2)
Then when the value should not be updated, use either PreparedStatement.setNull with an appropriate java.sql.Types value or a PreparedStatement.setXXX of the appropriate type with a null as value.
As discussed in the comments, an alternative if you do need to update to null, is to use a custom function with a sentinel value (either for updating to null or for using the current value). Something like:
update sometable set column1 = conditional_update(?, column1), colum2 = conditional_update(?, column2)
Where conditional_update would be something like (using Firebird 3 PSQL syntax):
create function conditional_update(new_value varchar(500), original_value varchar(500))
returns varchar(500)
as
begin
if (new_value = '$$NO_UPDATE$$') then
return original_value;
else
return new_value;
end
Where using $$NO_UPDATE$$ is a sentinel value for not updating. A potential downside of this solution is the typing of columns as a string type. You always need to use string types (I'm not sure if there are databases that would supports dynamic parameters and return types in this situation).
Let me rephrase your problem and you please tell me if this is right:
You want to be able to have the update process
sometimes you only update for a specific x
sometimes you only update for a specific y
sometimes you update for specific x and y combined
Correct? If 'yes' then the next issue is implementation possibilities.
You suggested trying to put the column itself into the parameter. Even if that's possible I'd still recommend against it. The code will get confusing.
I suggest you create a java method that builds and returns the query string (or at least the 'where' clause) based on your available parameters (x, y, etc)
Your code for invoking the JDBC Prepared statement will invoke that method to get a query String.
You will still benefit from using prepared statements. The cost is that your database will be caching several different statements instead of one, but I imagine that is a minimal issue.
You can think of auxiliar "?" variable for each column which will manage if the column should be updated or not.
update table
set x = case when 0 = ? then x else ? end,
y = case when 0 = ? then y else ? end
where etc;
Passing 0 for each 0 = ? will not update the column whereas passing 1 will update it to the value you specified.
So I am querying a MySql database from my java application and I am trying to use a query,
Select count(*) from table where `NUMERIC`='1'
to count the rows from a database. When I run this query it works fine, and I get a 1 returned (I am using a test db with 12 records, Numeric has values 1-12 so this makes sense). However I wanted to try to break this and do some error handling. I changed my query to
Select count(*) from table where `Numeric`='1adjfa'
I expected this to return 0, however it still returns 1. In fact, as long as I have 1 at the beginning of the value it will work, if I change the value to just 'adjfa' than it returns 0. I have confirmed this through both my Java App and the MySQL workbench. Any ideas as to why this returns 1, even with the junk at the end of it?
Two different data types can not be compared. Instead one of the two needs to be cast/coerced to the same data type as the other.
In your case you're not doing the coercion, so the DB Engine is doing an implicit coercion.
Based on data-type-order-of-precedence, the database engine chooses the string to be coerced to a numeric.
The value '1adjfa' therefore becomes a 1, and then your comparison is being made.
This results is your query effectively being:
Select count(*) from table where `Numeric` = 1
You should either not be comparing numerics and strings, or do the coercion yourself, for example...
Select count(*) from table where CAST(`Numeric` AS VARCHAR(32)) = '1adjfa'
In terms of breaking the query, I'm hoping that in your application you're actually using parameterised queries. This will allow you to define the data-type of the parameter, and your application should throw the error if the wrong data-type is supplied.
Numeric has a number data type. To make the comparision to 1adjfa the DB engine tries to convert it also to a number which results in 1 and the rest gets cut off.
I feel like I'm missing something very obvious here, but it seems that the only way to go about doing this is to get the value, and then see if it returns a null (empty) value, which I would rather not do.
Is there an equivalent to List.contains(Object o) in SQL? Or perhaps the JDBC has something of that nature? If so, what is it?
I am using Microsoft Access 2013.
Unfortunately I don't have any useful code to show, but here is the gist of what I am trying to do. It isn't anything unique at all. I want to have a method (Java) that returns the values of a user that are stored in the database. If the user has not previously been added to the database, the user should be added, and the default values of the user should be set. Then those newly created values will be returned. If a player has already been added to the database (with the username as the primary key), I don't want to overwrite the data that is already there.
I would also advise against using MS Access for this purpose, but if you are familiar with MS Office applications, the familiar UI/UX structure might help you get your footing and require less time to learn other database environments. However, MS Access tends to be quite limited, and I would advise considering alternative options if available.
The only way to see if an SQL table contains a row with some condition on a column is to actually make an SQL query. I don't see why you wouldn't do that. Just make sure that you have an index on the column that you will be constraining the results on. Also for better speed use count to prevent from retrieving all the data from the rows.
SELECT count(*) FROM foos WHERE bar = 'baz'
Assuming you have an index on the bar column this query should be pretty fast and all you have to do is check whether it returns > 0. If it does then you have rows matching your criteria.
You can use "IF EXISTS" which returns a boolean value of 1 or 0.
select
if(
exists( select * from date1 where current_date()>now() ),
'today > now',
'today is not > now'
) as 'today > now ?' ;
+--------------------+
| today > now? |
+--------------------+
| today is not > now |
+--------------------+
1 row in set (0.00 sec)
Another Example:
SELECT IF(
EXISTS( SELECT col from tbl where id='n' ),
colX, colY
) AS 'result'
FROM TBL;
I'm also new to sql and I'm using Oracle.
In Oracle, suppose we have: TYPE: value.
We can use:
where value not in (select TYPE from table)
to make sure value not exist in the column TYPE of the table.
Don't know if it helps.
You can simply use Query with condition.
For example if you have to check records with particular coloumn, you can use where condition
select * from table where column1 = 'checkvalue'
You can use count property to check the no. of records existing with your specified conditon
select count(*) from table where column1 = 'checkvalue'
I have created the following method, which to my knowledge works perfectly. (Using the java.sql package)
public static containsUser(String username)
{
//connection is the Connection object used to connect to my Access database.
Statement statement = this.connection.createStatement();
//"Users" is the name of the table, "Username" is the primary key.
String sql = "SELECT * FROM Users WHERE Username = '" + username + "'";
Result result = statement.executeQuery(sql);
//There is no need for a loop because the primary key is unique.
return result.next();
}
It's an extremely simple and extremely basic method, but hopefully it might help someone in the future.
If there is anything wrong with it, please let me know. I don't want anyone learning from or using poorly written code.
IMPORTANT EDIT: It is now over half a decade after I wrote the above content (both question and answer), and I now advise against the solution I illustrated above.
While it does work, it prioritizes a "Java-mindset-friendly" approach to SQL. In short, it is typically a bad idea to migrate paradigms and mindsets of one language to another, as it is inevitable that you will eventually find yourself trying to fit a square peg into a round hole. The only way to make that work is to shave the corners off the square. The peg will then of course fit, but as you can imagine, starting with a circle peg in the first place would have been the better, cleaner, and less messy solution.
Instead, refer to the above upvoted answers for a more realistic, enterprise-friendly solution to this problem, especially as I imagine the people reading this are likely in a similar situation as I was when I originally wrote this.
i have a table containing 15+ million records in oracle. its sort of a log table which has a created_ts column of type "date" . i have a simple "non-unique" type index on created_ts column.
i have a simple range query :
select * from table1 where created_ts >= ? and created_ts <= ?;
when i run this query from SQLPlus or SQL Developer etc like this :
select * from table1
where created_ts >= TO_DATE( '2009-11-10 00:00:00', 'YYYY-MM-DD HH24:MI:SS')
and created_ts <= TO_DATE( '2009-11-10 23:59:59', 'YYYY-MM-DD HH24:MI:SS');
the query returns within 1-2 second max.
but when I run the exact same query in java over JDBC and set the corresponding "?" params using java.sql.Timestamp object . the query takes long time . Analyzing the oracle process it goes for full table scan and doesnt use the index.
the jdbc driver i am using is ojdbc5 11.1.0.7.0
Can any one please help .. how to create the index correctly so that it uses the index.
My problem was resolved when i used "oracle.sql.DATE" objects to set the bind variables instead of "java.sql.timestamp" . The query used the index and executed almost within 1-2 seconds.
Thanks to all who replied and helped.
But its problematic for me as this solution is DB dependent and my app receives DB connection and query as param and load and process data in a generic way. The DB connection can be of any RDBMS like oracle, mysql, etc.
This is classic behaviour for an implicit datatype conversion. Because the database is having to convert the datatype of the column it cannot use any index on that column.
In your case I suspect this is due to your use of java.sql.Timestamp. Would it be possible to use the equivalent type from the Oracle datatypes package, oracle.sql.Timestamp? Obviously that may have some knock-on effects but I think you should at least test it, to see whether that solves your problem.
The difference may because of bind variables vs. literal values. You are not comparing the same things.
Try this in SQL*Plus:-
explain plan for
select * from table1 where created_ts >= :1 and created_ts <= :2;
set markup html preformat on
set linesize 100
set pagesize 0
select plan_table_output
from table(dbms_xplan.display('plan_table',null,'serial'));
This will show you the plan Oracle will pick when using bind variables. In this scenario, Oracle has to make up a plan before you have provided values for your date range. It does not know if you are selecting only a small fraction of the data or all of it. If this has the same plan (full scan?) as your plan from java, at least you konw what is happening.
Then, you could consider:-
Enabling bind peeking (but only after testing this does not cause anything else to go bad)
Carefully binding literal values from java in a way that does not allow SQL injection
Putting a hint in the statement to indicate it should use the index you want it to.
You should try a hint of the form /*+ USE_INDEX(table_name, index_name) */
My guess is that the optimizer is choosing a full table scan because it sees that as the best option in absence of knowing the bind values.