Here's the code that gives me headaches:
public List<String> listColumnsForTable(String tableName) throws SQLException {
List<String> columns = new ArrayList<String>();
DatabaseMetaData metadata = _connection.getMetaData();
ResultSet resultSet = metadata.getColumns(null, null, tableName, null);
while (resultSet.next())
columns.add(resultSet.getString("COLUMN_NAME"));
return columns;
}
This code works fine with SQL Server (I haven't checked with MySQL, Oracle or others), but I need to run some integration tests on an in memory database. All the databases I tried (h2, hsqldb and derby) fail.
Here is the link on github.
If you want the full project (with tests for h2, hsqldb, derby and sql server) do the following:
git clone git://github.com/sensui/InMemoryColumns.git
cd InMemoryColumns
gradlew
All the dependencies will be automatically downloaded. If you want to check the library versions look in the build.gradle script.
Now import the project in your favorite IDE (eclipse or idea).
The tests are available in the DatabaseMetadataCheckerTests class (canListColumnsForTable and canCheckIfColumnsExistInTable).
Normally you shouldn't modify those. I have created 4 test classes that provide connection details for each in memory database and you need to run those (the DatabaseMetadataCheckerTests is abstract so you don't run that).
NOTE:
When/if you find a solution than the tests for that specific database will pass.
You can easily try other databases like Oracle or MySQL just by extending the DatabaseMetadataCheckerTests class and providing the connection details (check the other tests).
Issue solved
The table names and column names should be in UPPERCASE. Check this commit for details.
H2, HSQLDB (as well as Oracle and DB2) comply with the SQL standard and thus unquoted object names are folded to uppercase (SQL Server does not do that, it keeps whatever case you used plus it might be configured to be not case sensitive for string comparisons).
create table foo (id integer) will be stored as FOO with the column name ID in the system catalog.
So you will need to pass the table name in uppercase to the JDBC API calls.
A note on porting this to other DBMS:
Postgres does not comply with the standard here and folds everything to lowercase
For MySQL there is no definite answer on how it does this. It depends on various configuration settings (and the storage engine, and the filesystem) so you can never be sure how an unquoted table name will actually be stored in the system.
Related
We have an Oracle database with the following charset settings
SELECT parameter, value FROM nls_database_parameters WHERE parameter like 'NLS%CHARACTERSET'
NLS_NCHAR_CHARACTERSET: AL16UTF16
NLS_CHARACTERSET: WE8ISO8859P15
In this database we have a table with a CLOB field, which has a record that starts with the following string, stored obviously in ISO-8859-15: X²ARB (here correctly converted to unicode, in particular that 2-superscript is important and correct).
Then we have the following trivial piece of code to get the value out, which is supposed to automatically convert the charset to unicode via globalization support in Oracle:
private static final String STATEMENT = "SELECT data FROM datatable d WHERE d.id=2562456";
public static void main(String[] args) throws Exception {
Class.forName("oracle.jdbc.driver.OracleDriver");
try (Connection conn = DriverManager.getConnection(DB_URL);
ResultSet rs = conn.createStatement().executeQuery(STATEMENT))
{
if (rs.next()) {
System.out.println(rs.getString(1).substring(0, 5));
}
}
}
Running the code prints:
with ojdbc8.jar and orai18n.jar: X�ARB -- incorrect
with ojdbc7.jar and orai18n.jar: X�ARB -- incorrect
with ojdbc-6.jar: X²ARB -- correct
By using UNISTR and changing the statement to SELECT UNISTR(data) FROM datatable d WHERE d.id=2562456 I can bring ojdbc7.jar and ojdbc8.jar to return the correct value, but this would require an unknown number of changes to the code as this is probably not the only place where the problem occurs.
Is there anything I can do to the client or server configurations to make all queries return correctly encoded values without statement modifications?
It definitely looks like a bug in the JDBC thin driver (I assume you're using thin). It could be related to LOB prefetch where the CLOB's length, character set id and the first part of the LOB data is sent inband. This feature was introduced in 11.2. As a workaround, you can disable lob prefetch by setting the connection property
oracle.jdbc.defaultLobPrefetchSize
to "-1". Meanwhile I'll follow up on this bug to make sure that it gets fixed.
Please have a look at Database JDBC Developer's Guide - Globalization Support
The basic Java Archive (JAR) file ojdbc7.jar, contains all the
necessary classes to provide complete globalization support for:
CHAR or VARCHAR data members of object and collection for the character sets US7ASCII, WE8DEC, WE8ISO8859P1, WE8MSWIN1252, and UTF8.
To use any other character sets in CHAR or VARCHAR data members of
objects or collections, you must include orai18n.jar in the CLASSPATH
environment variable:
ORACLE_HOME/jlib/orai18n.jar
I am new to JOOQ and love learning and using it.
I am at the point where I want to do some testing, but instead of testing
on the 'real' database I want to use a 'copy' of that database.
When using jdbc all it really took was changing the database name in the
create statement and using that name when connecting to the database.
I quickly discovered that anything I tried to write to my test database
was ending up in the production database
In the JOOQ documentation it looked like I could solve the problem with some mapping.
I added a Settings class and got the DSLContext using the settings, as shown below
My base database name is 'kpi'.
Connection conn = JooqUtil.getConnection("user", "pw", kpitest, hostip, sb);
Settings settings = new Settings()
.withRenderMapping(new RenderMapping()
.withSchemata(
new MappedSchema().withInput(kpi)
.withOutput(kpitest)));
// Add the settings to the DSLContext
if (sb.length() == 0) {
dsl = DSL.using(conn, SQLDialect.MYSQL, settings);
}
The above resulted in seeing the testName database being used,
BUT access to the tables and fields was using the productionName.
Going back to the documentation it looks like there is a way to map the tables
also but it looks like a lot of work.
Further reading, I found some settings in the configuration file using schemata
My current JOOQ configuration xml file contains
<inputSchema>kpi</inputSchema>
It looks like I should change it to something like
<inputSchema></inputSchema> << maybe drop altogether
<schemata>
<schema>
<inputSchema>kpi</inputSchema>
<outputSchema>kpi</outputSchema>
</schema>
<schema>
<inputSchema>kpi</inputSchema>
<outputSchema>kpitest</outputSchema>
</schema>
</schemata>
In reading the manual I was not sure if DEV and PRODUCTION names had literal significance or not.
What would help is a (real world) example where there was a production and test database,
or one develpment database per developer, and how the tables and fields were accessed
if there was a change in syntax. A change in syntax would be difficult for doing unit type testing though.
Thanks for any quidance and links to examples
From within a java code - where I already have a connection to a database - I need to find the default schema of the connection.
I have the following code that gives me a list of all schemas of that connection.
rs = transactionManager.getDataSource().getConnection().getMetaData().getSchemas();
while (rs.next()) {
log.debug("The schema is {} and the catalogue is {} ", rs.getString(1), rs.getString(2));
}
However, I don't want the list of all the schemas. I need the default schema of this connection.
Please help.
Note1: I am using H2 and DB2 on Windows7 (dev box) and Linux Redhat (production box)
Note2: I finally concluded that it was not possible to use the Connections object in Java to find the default schema of both H2 and DB2 using the same code. I fixed the problem with a configuration file. However, if someone can share a solution, I could go back and refactor the code.
Please use connection.getMetaData().getURL() method which returns String like
jdbc:mysql://localhost:3306/?autoReconnect=true&useUnicode=true&characterEncoding=utf8
We can parse it easily and get the schema name. It works for all JDBC drivers.
I am using an APACHE DERBY database, and basing my database interactions on EntityManager, and I don't want to use JDBC class to build a query to change my tables' names (i just need to put a prefix to each new user to the application, but have the same structure of tables), such as:
//em stands for EntityManager object
Query tableNamesQuery= em.createNamedQuery("RENAME TABLE SCHEMA.EMP_ACT TO EMPLOYEE_ACT");
em.executeUpdate();
// ... rest of the function's work
// The command works from the database command prompt but i don't know how to use it in a program
//Or as i know you can't change system tables data, but here's the code
Query tableNamesQuery= em.createNamedQuery("UPDATE SYS.SYSTABLES SET TABLENAME='NEW_TABLE_NAME' WHERE TABLETYPE='T'");
em.executeUpdate();
// ... rest of the function's work
My questions are :
This syntax is correct?
Will it work?
Is there any other alternative?
Should I just use the SYS.SYSTABLES and find all the tables that has 'T' as tabletype and alter their name their, will it change the access name ?
I think you're looking for the RENAME TABLE statement: http://db.apache.org/derby/docs/10.10/ref/rrefsqljrenametablestatement.html
Don't just issue update statements against the system catalogs, you will corrupt your database.
I want to insert exchange.body to a database table for one of the condition of my route.
Is there any example/tutorial of camel-jdbc component to insert message body?
Can I import the SQL statement itself and pass exchange.body to it?
I looked at http://camel.apache.org/jdbc.html example, but could not understand it.
Here Spring example is confusing for me. I didn't get why is it setting the body as SQL query and again importing some query from the class path. (There is no insert query example mentioned here.)
If you want to insert using the same statement (changing the parameters only) - use SQL component.
If you want to insert using arbitrary SQL statement into the component - use JDBC component.
SQL component usage:
from("direct:start").to("sql:insert into table foo (c1, c1) values ('#','#')");
com.google.common.collect.Lists;
producerTemplate.sendBody("direct:start", Lists.newArrayList("value1","value2"));
JDBC component usage:
from("direct:start").to("jdbc:dataSource");
producerTemplate.sendBody("direct:start", "insert into table foo (c1, c1) values ('value1','value2')");
You probably need to do some restructure of your payload before inserting it anyway, so there should probably be no issue to do a transformation using whatever method in Camel to set the body to the appropriate INSERT statement.
The important thing is what kind of payload structure your incoming message have. In the basic case - it's a string - it should be fairly simple
// In a Java bean/processor before the JDBC endpoint.
// Update: make sure to sanitize the payload from SQL injections if it contains user inputs or external data not generated by trusted sources.
exchange.getIn().setBody("INSERT INTO MYTABLE VALUES('" + exchange.getIn().getBody(String.class) + "', 'fixedValue', 1.0, 42)");
In case your message contains complex data structures, this code will of course be more complex, but it's pretty much the same way regular application will generate SQL queries.
The classpath example you are refering to
<jdbc:embedded-database id="testdb" type="DERBY">
<jdbc:script location="classpath:sql/init.sql"/>
</jdbc:embedded-database>
Simply shows how to test the JDBC component by starting a Database server embedded (Apache Derby) and populate it with some initial data (the sql/init.sql file). This part is not really part of the core jdbc component, but simply in the documentation to get up and running a sample without needing to configure a DB server and setup the JDBC connection properties.
That said, you might want to use the SQL component for more complex scenarios.