I maintain a piece of software that runs as a Servlet and can make use of MySQL, Oracle or SQL Server as the DB backend - depending on what the Customer wants to use.
Everything works perfectly with MySQL and Oracle, and SQL Server works great too, except I cannot insert/update Unicode sequences into the database.
I can do a manual insert in SQL Server Management Studio of a unicode sequence like this
INSERT INTO mytable (msg) VALUES (N'Modern Standard Hindi (मानक हिन्दी), is a standardised and Sanskritised register of the Hindustani language.')
This data is output in my software correctly, so this verifies that the database and the web front end can both handle unicode no problem.
And here's my connection string
jdbc:jtds:sqlserver://<server ip>:1433/MyDb
As I said, Oracle and MySQL work perfectly using this setup. What's different about SQL Server?
Note: I also tried the official Microsoft-provided JDBC driver with exactly the same results.
If it makes a difference, I'm using JPA Repositories to do my DB interactions. The whole webapp is also set up as a SpringMVC application.
Edit: I also tried adding useUnicode=true;characterEncoding=UTF-8 to the end of my connection string with the same results
You may need to specify in your connection string that you're using Unicode, and also what encoding you're using. Maybe something like this?
jdbc:jtds:sqlserver://<server_ip>:1433;databaseName=MyDb;useUnicode=true;characterEncoding=UTF-8
So there was one thing I left off my original question.
I had the system set up with Spring Security and a CharacterEncodingFilter in the filter chain to force UTF-8 in all requests and responses, but I had it set up in the afterSpringSecurityChain section of my web security initializer.
I moved the filter into the beforeSpringSecurityChain method and boom - everything UTF-8 works perfectly with all DB vendors.
I ended up not adding any params to my connection string or anything either. It was literally just the filter that I changed.
Related
I have a Scala (Java) Play Web Application where I wrap database identifiers with ANSI double-quotes in queries e.g.
select *
from "account"
where "deleted" is null
order by "account_name"
This is necessary because I use among others H2 in-memory and Postgres databases for deploying the application in different scenarios e.g. CI server. Now we need to deploy it also in MySQL and by default wrapping identifiers in double-quotes is not supported. However, following directions from this post mysql double-quoted table names I see we can set this session parameter and then it should work.
How can I set this session parameter while opening the connection through the MySQL Java Connector? My database URL looks like this: jdbc:mysql://odysseus:3306/idxsrs-trading?param=xxx
Mysql session variable in JDBC string
Using the above to link in combination with the link you've posted, I'd try this.
jdbc:mysql://localhost:3306/db?sessionVariables=sql_mode=ANSI_QUOTES
I have been given a java web application for which I have the source code to. The application queries an Oracle database to return data back to the user in web page. I need to update a value for the returned data in the database, without knowing the table or column names. Is there a general way to determine what query the application is submitting to return the data for a particular page so I can find the right tables and columns?
I am new to java and web development so not sure where to start looking.
Thanks!
Well, there's always the old fashioned way of finding out. You can find the source code for the specific page you're looking at and identify the query that's being executed to retrieve the data. I'm assuming that's not what you're looking for, though.
Some other options include using JDBC (Enabling and Using JDBC Logging) logging feature or JProfiler (the JDBC probe shows you all SQL statements in the events view). Once you find the SQL statement, you can use standard text search features within your IDE to locate the specific code and make alterations.
Hope that helps!
If you can run a controlled test (e.g., you are the only person on that web application), you could turn on SQL tracing on the DB connection and then run your transaction several times. To do this
look at all the connections from that application using v$session -- you can control this by tweaking your connection pool setting (e.g., set min and max connection to 1). Assuming this is your test environment.
turn on 10046 trace (see https://oracle-base.com/articles/misc/sql-trace-10046-trcsess-and-tkprof -- there are many other examples).
The 10046 trace will show you what the application is doing -- SQL by SQL. You can even set the level to 12 to get the bind variable values (assuming you are using prepared statements).
Earlier I was trying to get batch inserts working in Hibernate. I tried everything: For the config I set batch_size(50), order_inserts(true), order_updates(true), use_second_level_cache(false), use_query_cache(false). For the session I used setCacheMode(CacheMode.IGNORE) and setFlushMode(FlushMode.MANUAL). Still the MySQL query log showed that each insert was coming in separately.
The ONLY thing that worked was setting rewriteBatchedStatements=true in the JBDC connection string. This worries me, as my application is supposed to support any JBDC database and I'm trying to avoid DB specific optimizations.
Is the only reason hibernate can't actually use batch statements because the MySQL driver doesn't support them by default? What about other drivers, do I have to add options to the connection string so they can support batched inserts? If you need specific db's, think SQL server, sqlite, Postgres, etc
One reason it could not be working is that hibernate disables batching if you use the Identity id generation strategy.
Also MySQL doesn't support JDBC batch prepared statements the same way as other databases without turning on the rewrite option.
I don't see that it is a problem to turn this flag on though, if your are setting up your application for a different database you will have to change the settings such as dialect, driver name, etc. anyway and as this is part of the JDBC connect String then you are isolated from the configuration.
Basically I think you are doing the right thing.
As batch insert (or bulk insert) is part of the SQL standard, ORM frameworks like Hibernate support and implement it. Please see Chapter 13. Batch Processing and Hibernate / MySQL Bulk insert problem .
Basically, you need to set the JDBC batch size via the variable named hibernate.jdbc.batch_size to a reasonable size. Also don't forget to end the batch transaction with flush() and clear().
I have a service method on an api that can be called to check the health of my database connection.
The method is pulling the query string from a properties file (depends on DB vendor, using Sybase and HSQL for now, more in future), and executing it. Then the method lets the caller know if it succeeded or failed.
In addition to this, I was using the Query.setHint("javax.persistence.query.timeout") to set a timeout on the query:
javax.persistence.EntityManager entityManager;
...
Query heartbeatQuery = entityManager.createNativeQuery(heartbeatQueryString);
heartbeatQuery.setHint("javax.persistence.query.timeout", heartbeatTimeout);
heartbeatQuery.getResultList();
My problem is the timeout property is working against my Sybase DB, but not against my HSQL DB. It sounds like it depends on the vendor, so I don't know for sure when it will work.
Is there a better way to generically test the DB connection & include some kind of timeout parameter?
Well sadly no. JPA's query hints are not mandatory, i.e. it's up to the implementator (EclipseLink, Hibernate, etc) to enforce them or not. Moreover, even if the implementator does chose to recognize a certain query hint, if that hint's functionality is not supported by the database then it won't work (here some implementators are nice and tell you if a certain hint won't work agains the current db while others fail silently). In the case of HSQLDB there's no way to set the query timeout. You can only set a timeout for the login (i.e. how long should it wait for a successful login before failing), but not for the queries duration.
Things are not so grim however. On the one hand, even if you'd solve this, you'd still stumble over other issues with HSQLDB, as it does not support a lot of other nice functionalities that most dbs have. You should only use HSQLDB for basic integration/unit testing. For more involved testing, you can use the integrated MySQL Java library. You can find it here:
http://dev.mysql.com/doc/refman/5.0/en/connector-mxj.html
This is simply a packaged fully working Mysql server, which has a Java api for star and stop, works on most major OSs (win,lin, os x, etc). This way you can have your integration tests start a real Mysql server, and try your code there, where such stuff as a query timeout hint will work fine.
We've got an application using spring for making calls to DB2 stored procedures.
The application was working fine with jdbc version 1.XX.XX (and DB2 V8). An recent upgrade to DB2 V9 moves us to using jdbc versiong 3.58.90 instead.
However, this seems to have broken the named parameters mapping in spring (version 2.5.5). Some previously working code had call string
call storedproc123(:id,:date)
now throws exceptions with
[jcc][t4][10427][12544][3.58.90] Error parsing FLOAT literal value starting at index 19. Error Detail:Unexpected character ':' found in FLOAT literal.
...
ERRORCODE=-4463, SQLSTATE=42601
...
Has anyone encountered something similar?
Thanks in advance!
---edited to add more info---
I've tried swapping the old jdbc back after the upgrade. The application works just fine with the old driver, however we'd like to upgrade to the newer version since another app on the same server needs this newer jdbc, and it's difficult to have different versions of the same jdbc deployed on our server (we uses JBoss).
http://redneckprogrammer.blogspot.com/2009/10/running-multiple-versions-of-oracle.html discussed how to deploy multiple versions of the same JDBC driver, however this seems to be too much of a hack and I'd like to avoid it if at all possible.
Found the cause, see IBM patch PK87567 .
...
All Connectivity: Driver code has been enhanced to support
a new API on com.ibm.db2.jcc.DB2ParameterMetaData:
getParameterMarkerNames, which returns a list of parameter
marker names used in the SQL Statement as a String .
This method returns null if property
enableNamedParameterMarkers is set to
DB2BaseDataSource.NOT_SET or DB2BaseDataSource.NO, or if
there are no named parameter markers in the SQL Statement.
The list returned contains unique parameter marker names.
If a named parameter marker appears more than once in the
SQL Statement, it will only appear once in the list
returned. (120191)
...
Fixed the problem by enabling the marker.