I'm using database metadata to find some tables in a given schema in database like that:
DatabaseMetaData dbmd = connections.getMetaData();
ResultSet rs = dbmd.getTables(null,"schema_name","table_name_pattern","type");
It works, but my problem is that I only want to find tables that begin with t and three others tables for which I have the exact names:
books_table, froots, and colors.
How can I make a pattern that gives me only these three tables and the tables that begin with t?
I had a similar problem time ago, and the answer is no, you can't do it this way.
The JDBC metadata classes have only very limited functionality.
However, you can query the engine metadata directly. For example you can do:
select *
from pg_catalog.pg_tables
where schemaname = 'public'
and tablename like 't%' or tablename in ('books_table', 'froots', 'colors')
The parameter is interpreted as as wildcard value for a LIKE condition. So to find all tables starting with t use t%.
Quote from the JavaDocs
Some DatabaseMetaData methods take arguments that are String patterns. These arguments all have names such as fooPattern. Within a pattern String, "%" means match any substring of 0 or more characters, and "_" means match any one character.
Note that this is case-sensitive. If you created your tables without quotes (which is what you should be doing) they are stored in lowercase, so t% will work fine.
The last parameter should be a string array with all possible "object types". As you want tables, use {"TABLE"}. Alternatively if you don't care about the "type", you can also pass null
DatabaseMetaData dbmd = connections.getMetaData();
ResultSet rs = dbmd.getTables(null,"schema_name","t%",new String[]{"TABLE"});
You can't however specify multiple OR conditions. If you want to find if other tables exist where you can't use a wildcard, you will have to call getTables() once for each table name.
Another option is to simply get all tables from that schema, then discard those that you don't want while processing the ResultSet. That is most probably faster than using multiple calls to getTables()
Related
I am trying to pass a string value which is comma separated to birt report as parameter but failing
Java code
String userlist="\"a\",\"b\",\"c\"";
task.setParameterValue("userlist", userlist);
BeforeOpen has
params["userlist"].value.join("','");
SQL Query is
select * from users where name in (?)
I have already linked data set parameter to report parameter param_1
It's always giving me empty report even though DB table has 3 users. Any advise ?
Know your tools!
In this context: You have to understand SQL and the concept of bind variables, Javascript and BIRT.
Unfortunately every single piece of code you posted is wrong (or at least incomplete).
But you are on the right track: You can modify the SQL text in the beforeOpen event of the database. I'll sketch the idea here:
In your SQL, replace the ? with a placeholder like 'IN-LIST' (such that it is valid SQL).
You should still use the bind variable in the SQL (to avoid pitfalls caused by BIRT's caching mechanism), but in an effective no-op way, e.g. "where ? is not null".
In the beforeOpen event of the data set, you can modify the SQL text:
Get the original SQL text (var query = this.queryText; IIRC).
Split your report parameter into the individual search terms. How exactly to do this depends on your input format. In your example, you are using " around your search terms, which looks overly complicated, unless individual search terms may contain commas. You should now have a list of your search strings, e.g. ["a", "b", "c"].
Convert each term into a valid SQL string literal. Beware of SQL injection attacks, so carefully escape characters like single-quotes! You should now have a list of valid SQL string literals, e.g. ["'a"', "'b'", "'c'"].
Join your list of SQL string literals from 3) into a single string with ", ".
You should now have a string like 'a', 'b', 'c'.
In your query, replace your placeholder string with the string from 4).
Write the modified SQL text back to the DS object: this.queryText = query;
Probably there is also an example somewhere in the mists of the internet.
If you had invested five minutes more, you should have found an existing answer here on stack overflow: How to create a BIRT dataset that accepts multiple (CSV) values that it can be used inside "IN" clause in select statement, the only difference being that you are searching for a list of strings, while that question was about a list of numbers.
I have the following select-query creation:
final DSLContext create = DSL.using(..., SQLDialect.POSTGRES);
create
.select(DSL.field("identifier"), DSL.field("name"),
create.selectCount()
.from(DSL.table("person"))
.where(DSL.field("identifier").eq(DSL.field("personOuter.identifier")))
.asField("count"))
.from(DSL.table("person").as("personOuter"))
jOOQ generates the following query:
select
identifier,
name,
(select count(*)
from person
where identifier = personOuter.identifier) as "count"
from person as "personOuter"
The query should be:
select
identifier,
name,
(select count(*)
from person
where identifier = personOuter.identifier) as "count"
from person as personOuter
The latter query works perfectly in PostgreSQL. The table alias should not be surrounded by quotes.
Is this a bug?
(Note that the query is pretty dumb. I am playing around with jOOQ to evaluate.)
The following "hack" works:
create
.select(DSL.field("identifier"), DSL.field("name"),
create.selectCount()
.from(DSL.table("person"))
.where(DSL.field("identifier").eq(DSL.field("personOuter.identifier")))
.asField("count"))
.from("person as personOuter")
A note on using the code generator
I'm assuming you have a good reason to avoid using the code generator (e.g. you work on a dynamic schema), because working with generated code prevents having to worry about such details. Plus, you get access to many advanced features, like implicit joins, embeddable types, etc.
What's a string in the jOOQ API?
By default, jOOQ will wrap all your identifiers in quotes in order to be able to handle case-sensitivity correctly.
The confusing part is why this isn't done for DSL.field(String), but only for Field.as(String). The reason for this is that jOOQ re-uses the String type for both:
Plain SQL as in DSL.field(String), where the input String doesn't really represent an identifier, but an arbitrary SQL expression
Identifiers as in DSL.name(String), where the input String represents a name / identifier. There is also DSL.fieldByName(String) to create Field types composed of (schema) / table / column identifiers.
In order to remove the quotes from all generated identifiers, you can also change the Settings.renderNameStyle to RenderNameStyle.AS_IS.
More information about Settings can be found here. And also in this blog post about "What’s a “String” in the jOOQ API?"
I am having trouble puting those single quotes for ASCII/Timestamp columns and not puting for other types like Int, Decimal, Boolean etc.
The data comes from another db/table, which is a sql.
I have all the column data as string. I don't want to format each column data to check null values and then decide to put quote or not.
Is it possible to pass in insert data value without giving single quotes, using prepared statement or whatever.
If you don't want to write a loader that uses prepared statements (via the CQL driver...which is a good idea), I can think of one other way. To import without using single quotes, you should be able to accomplish this with the COPY FROM CQL3 command (setting the QUOTE parameter to an empty string). If you can dump your RDBMS data to a csv file, you should be able to insert those values into Cassandra like this:
COPY myColumnFamily (colname1,colname2,colname3)
FROM '/home/myUser/rdbmsdata.csv' WITH QUOTE='';
Check out the documentation on the COPY command for more information. Examples can be found here.
EDIT:
I also read the above question and assumed that you did not want a prepared statement-based answer. Since that's obviously not the case, I thought I'd also provide one here (using DataStax's Java CQL driver). Note that this answer is based on my column family and column names from my example above, and assumes that col1 is the (only) primary key.
PreparedStatement statement = session.prepare(
"UPDATE myKeyspace.myColumnFamily " +
"SET col2=?, col3=? " +
"WHERE col1=?");
BoundStatement boundStatement = statement.bind(
strCol2, strCol3, strCol1);
session.execute(boundStatement);
This solution does not require you to encapsulate your string data in single quotes, and has a few added benefits over your String.ReplaceAll:
Allows you to insert values containing single quotes.
Escapes your values, protecting you from CQL-Injection (the lesser-known relative of SQL-Injection).
In CQL, both UPDATE and INSERT add a record if it does not exist and update it if it does (effectively known as an "UPSERT"). Using an UPDATE over an INSERT supports counter columns (if your schema ends up using them).
Prepared statements are faster, because they allow Cassandra to only have to parse the query once, and then re-run that same query with different values.
For more information, check out DataStax's documentation on using prepared statements with the Java Driver.
Finally did it using String.format clubbed with replace
String.format("INSERT INTO xyz_zx(A,B,C,D) VALUES('%s','%s',%s,%s);",(Object[])Strings).replaceAll("'null'","null");
I have to execute below query through JDBC call
select primaryid from data where name in ("abc", adc", "anx");
Issue is inside in clause I have to pass 11000 strings. Can I use prepared statement here? Or any other solution any one can suggest. I dont want to execute the query for each record, as it is consuming time. I need to run this query in very less time.
I am reading the strings from an XML file using DOMParser. and I am using sql server db.
I'm just wondering why you would need to have a manual set of 11,000 items where you need to specify each item. It sounds like you need to bring the data into a staging table
(surely it's not been selected from the UI..?), then join to that to get your desired resultset.
Using an IN clause with 11k literal values is a really bad idea - off the top of my head, I know one major RDBMS (Oracle) that doesn't support more than 1k values in the IN list.
What you can do instead:
create some kind of (temporary) table T_NAMES to hold your names; if your RDBMS doesn't support "real" (session-specific) temporary tables, you'll have to add some kind of session ID
fill this table with the names you're looking for
modify your query to use the temporary table instead of the IN list: select primaryid from data where name in (select name from T_NAMES where session_id = ?session_id) or (probably even better) select primaryid from data join t_names on data.name = t_names.name and t_names.session_id = ?session_id (here, ?session_id denotes the bind variable used to pass your session id)
A prepared statement will need to know the number of arguments in advance - something along the lines of :
PreparedStatement stmt = conn.prepareStatement(
"select id, name from users where id in (?, ?, ?)");
stmt.setInt(1);
stmt.setInt(2);
stmt.setInt(3);
11,000 is a large number of parameters. It may be easiest to use a 'batch' approach as described here (in summary - looping over your parameters, using a prepared statement
each time)
Note - if your 11,000 strings are the result of an earlier database select, then the best approach is to write a stored procedure to do the whole calculation in the database (avoiding passing the 11,000 strings back and forth with your code)
You can merge all your parameter strings into one bitg string separating by ';' char
bigStrParameter=";abc;adc;anx;"
And use LOCATE to find substring.
select primaryid from data where LOCATE(concat(';',name,';'),?)>=0;
I am writing a web service that essentially allows users to submit queries to pre-existing tables in various SQL databases against advertised columns.
I have a PostgreSQL table defined like that:
CREATE TABLE stpg.test (
test integer,
"Test" integer,
"TEST" integer
);
insert into stpg.test values (1,2,3);
To determined the names of the available columns I run the following Java code:
ResultSet rs = dbmd.getColumns(null, "stpg", "test", null);
while (rs.next()) {
System.out.println(rs.getString("COLUMN_NAME"));
}
I get:
test
Test
TEST
If a user submits a query, referring to the columns as they were returned, like
select test, Test, TEST from stpg.test he will get 1 1 1 instead of expected 1 2 3.
Is this a bug?
I know that doing select test, "Test", "TEST" from stpg.testreturns results correctly. But my users would not know that to fetch the values of "capitalized" columns that were defined in quotes they need to use quotes in the query.
Is there a way I could could determine that a column name is case sensitive so that I could report its name in quotes? I need to do that generically against different databases, so JDBC api approach is preferable. I tried using ResultSetMetaData and invoking getColumnName and getColumnLabel but they return the names without the quotes. Calling isCaseSensitive always returns false.
Is there a way I could could determine that a column name is case sensitive so that I could report its name in quotes?
It looks like you are saying that a column name needs to be quoted if it contains any upper-case letters. In that case:
if (!name.equals(name.toLowercase())) {
// needs quoting.
}
But this is moot:
if you just quote all column names, or
if you treat user-supplied column names as case insensitive.
(On the latter point, having column names where case sensitivity matters is probably a bad design. Case sensitivity is certainly not something that you'd want your website users to have to worry about ...)
I tried using ResultSetMetaData and invoking getColumnName and getColumnLabel but they return the names without the quotes.
As they should! The quotes are not part of the column names! They are part of the (Postgres) SQL syntax for identifiers (in general). The name is the stuff within the quotes.
Calling isCaseSensitive always returns false.
To be honest, it is not entirely clear (from the javadoc) what the result of that method means. But it sounds like you might have found a bug in the JDBC driver you are using. (Or maybe you are just mistaken. The code for that implements that method in the current Postgres does consult the column type information ...)
I would suggest to always quote the column names. There is no real reason why you would remove the quotes. And, more importantly, the code to decide whether to quote or not is certainly going to span over 10-15ish lines with no added value. That's about 15 lines of code which can introduce new bugs, typos, conceptual errors.
Just quoting each column is straight-forward and always correct!
Also, regarding to your question if the result of select test, Test, TEST from stpg.test is a bug: It's not. It's the default behaviour of PostgreSQL. All column names (or, db-object names) are always lowered except if they are enclosed in quotes. This also leads us to isCaseSensitive. It is always false because it is not case-sensitive.
A more important note: If you let your users type in SQL queries, you will likely run into other weird problems. You will never know what kind of shenanigans your users type. Either by design or by accident ;)
If this is one of the first times you allow users to enter SQL queries, consider your plan of action carefully! Users type errors, mistakes (full-cartesian products on 5 tables with millions of rows? And only then apply filters?... fun times...), or might even try to play with your DB. If you decide on really doing this, buckle up! :) It all depends on the technical knowledge of you user-base.
Also, in Postgres I find it useful to keep everything lower-cased and user underscores to separate words. Like user_account instead of UserAccount.