I have a spring boot application which is using a JPA query. The same query when executed directly on the live oracle DB tends to give results in some 20-40ms. On the other hand, when I try to hit using the application takes variable time ranging from 1-2 seconds to 50-60 seconds.
I want to understand the reason for this behavior as to why it is behaving unpredictably. We suspected it could be the limited number of threads in pool but later after isolating the application from external use now with only one user showed the same behavior.
The query should execute in a fast manner consistently.
I wanted to know the possible reasons behind this behavior.
It could really be anything e.g. unreliable network, contended database resources, JDBC driver miss-configuration or JVM GC pause. Try to establish where is the problem: is it Java client or is the database server that is taking the time when the problem occurs.
If you suspect that the problem is the database it would be best to trace the connection and SQL query on the database server side. This will give you the most information e.g. query execution plan. Each database has it's own tools e.g. Oracle docs have entire chapter on Performing Application Tracing.
One possible reason could be your entity relationships
try enabling hibernate statistics for more detail:
You can enable by following:
<persistence>
<persistence-unit name="my-persistence-unit">
...
<properties>
<property name="hibernate.generate_statistics" value="true" />
...
</properties>
</persistence-unit>
Related
I'm trying to check the database connection with jpa using the EntitityManager/Session class.
To check both cases (connected/not connected), I simply start/deactivate the service mysql before running the code.
Asking simple sql queries to the database is out of mind since it doesn't cover all cases.
I already tried:
Session session = entityManager.unwrap(Session.class)
session.isConnected();
But that always returns true...
When I disable the mySQL service I want session.isConnected() to return false, but it always returns true;
Is there any other way to check for the database connection(using any other classes maybe)?
Short answer: don't
Long answer: if you're developing an application where this is important, you will probably be using some sort of database connection pooling framework, like the
Tomcat JDBC pool
Apache Commons DBCP
HikariCP
C3P0 Connection pool
This will manage many important aspects for you, including connection lifecycle. In tomcat jdbc for example you can specify:
testOnBorrow="true"
validationQuery="select 1 from sysibm.dual"
This on an IBM DB2 database will test whether a connection went stale every time you use it, and also swap it out for a fresh one if the select fails. Done this way, you will never have to worry about this on the level of you application code.
I am using eclipse link as a JPA implementation and am connected to a DB running on "jdbc:XXX://localhost:35001/". Is there a way I can track all the sql calls? I am running this inside a java project in eclipse on my local machine.
Thanks
Several options you can try:
You could try using a proxy like P6spy as mentioned by Andreas, or alternative ones like log4jdbc etc. This can be useful in debugging when you are trying to trace calls from multiple clients since the proxy could intercept the calls from them all.
However for your case I would suggest using the built in logging facilities of EclipseLink. In eclipse link you can configure logging of the statements via entries in the persistence XML like shown below:
<property name="eclipselink.logging.level.sql" value="FINE"/>
<property name="eclipselink.logging.parameters" value="true"/>
I would suggest that after making the code changes to undeploy the application and stop and restart the application server before rebuilding and then deploying the application again. I have seen more than few instances where the logging does not start with you going through this entire cycle.
The last option would be a SQL trace. Depending on your database backend you might be able to run a profile or trace. SQL-Server would allow you to trace it. You can then view all SQL executed against the database. This is probably overkill in your scenario as it will log all activity unless configured incorrectly.
I have a Java Spring server, working with Hibernate and MySQL, and I have a view in my database, which I've mapped as an entity with hibernate.
The view is completely flat, no references to other entities. No fetches etc.
I can query the view using a simple select with no problems, both directly on the sql server using "Workbench", and using hql.
Locally I can also use "order by" clauses with no noticeable impact on the performance.
My problem is that When I move to a dev server, the "order by" clause causes the query to be extremely slow, which causes a timeout.
This happens only when the server makes the hql query.
I can still query the dev DB directly using "Workbench" with no performance problems.
The only differences between the two servers (that I can think of) is that the dev server is deployed on Google's appengine, and that the hibernate drivers are different.
On the local server I use com.mysql.jdbc.Driver, and on appengine com.mysql.jdbc.GoogleDriver.
My local DB is a MySQL database, and the dev one is on Google's Cloud SQL, which is also MySQL (pretty much).
On my Dev DB I have about 28000 rows in total in the view, and on my local DB I have about 21000, so that also shouldn't make the difference.
I can add code if you think it could help. I wasn't sure where to start.
Thank you very much in advance.
I would enable the 'show_sql=true' to see the generated queries, and also will use the 'GoogleDriver' with local DB to eliminate any issues with driver.
Would it be possible to connect to DEV DB from your local workspace? if so, try that option using 'com.mysql.jdbc.Driver' to eliminate any issues with DEV DB.
if possible connect to DEV_DB from Mysql Manager and run the query directly on DEV_DB.
28K is very less count to get problems with 'Order by' clause.
Edit
If you have not enabled the 'connector-j' before deploying to the appEngine, it will still try to use the 'mysql' driver, and will timeout eventually.
<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
...
<use-google-connector-j>true</use-google-connector-j>
</appengine-web-app>
More info.
You might want to look to see if an index on the ORDER BY field exists in the DEV server but does not on PROD. That can have a profound effect on both ordering and searching.
I have a service method on an api that can be called to check the health of my database connection.
The method is pulling the query string from a properties file (depends on DB vendor, using Sybase and HSQL for now, more in future), and executing it. Then the method lets the caller know if it succeeded or failed.
In addition to this, I was using the Query.setHint("javax.persistence.query.timeout") to set a timeout on the query:
javax.persistence.EntityManager entityManager;
...
Query heartbeatQuery = entityManager.createNativeQuery(heartbeatQueryString);
heartbeatQuery.setHint("javax.persistence.query.timeout", heartbeatTimeout);
heartbeatQuery.getResultList();
My problem is the timeout property is working against my Sybase DB, but not against my HSQL DB. It sounds like it depends on the vendor, so I don't know for sure when it will work.
Is there a better way to generically test the DB connection & include some kind of timeout parameter?
Well sadly no. JPA's query hints are not mandatory, i.e. it's up to the implementator (EclipseLink, Hibernate, etc) to enforce them or not. Moreover, even if the implementator does chose to recognize a certain query hint, if that hint's functionality is not supported by the database then it won't work (here some implementators are nice and tell you if a certain hint won't work agains the current db while others fail silently). In the case of HSQLDB there's no way to set the query timeout. You can only set a timeout for the login (i.e. how long should it wait for a successful login before failing), but not for the queries duration.
Things are not so grim however. On the one hand, even if you'd solve this, you'd still stumble over other issues with HSQLDB, as it does not support a lot of other nice functionalities that most dbs have. You should only use HSQLDB for basic integration/unit testing. For more involved testing, you can use the integrated MySQL Java library. You can find it here:
http://dev.mysql.com/doc/refman/5.0/en/connector-mxj.html
This is simply a packaged fully working Mysql server, which has a Java api for star and stop, works on most major OSs (win,lin, os x, etc). This way you can have your integration tests start a real Mysql server, and try your code there, where such stuff as a query timeout hint will work fine.
Is it possible to specify connection/query timeout for the Oracle database queries? Either on Oracle side or in Oracle's JDBC driver (10.2.0.4)? So, that Java client just got an error back after, let's say, 2 minutes instead of waiting until Oracle finishes executing the query?
If you are executing the query in the context of a transaction, the transaction timeout value of the JTA transaction monitor will be the determinant to query timeout. The configuration for this depends from one application server to another.
At an individual query level (in the absence of a JTA transaction monitor), the setQueryTimeout method can be used to set the timeout on the execution of a Statement/PreparedStatement/CallableStatement object.
Update
setQueryTimeout is not to be relied on, although it works (atleast from a J2SE client). It works via the JDBC driver performing a full round-trip to the Oracle database server. Then, it is upto the database to halt execution of the query. Don't rely on it for time critical applications.
Have a look at Oracle profiles. This allows you to specify several limits at the database level. One of them is a maximum CPU time per query.
If you have queries running for more than 2 minutes on a regular basis you might want to do some tuning of your queries first.
According to http://www.javamonamour.org/2012/09/oraclenetconnecttimeout.html
oracle.net.READ_TIMEOUT for jdbc versions < 10.1.0.5 oracle.jdbc.ReadTimeout for jdbc versions >=10.1.0.5
So if you are using a JDBC driver version 10.1.0.5 or higher, then oracle.jdbc.ReadTimeout is the correct property.
Setting oracle.jdbc.ReadTimeout helped to timeout the jdbc calls. I have used it in a production spring boot app by specifying datasource properties as below
spring.datasource.hikari.connection-timeout=1000
spring.datasource.hikari.dataSourceProperties=oracle.jdbc.ReadTimeout=2000
Note: Earlier this app was using tomcat connection pool and after setting the above properties for the same, timeouts were happening but the pool was not able to handle the closed connection objects efficiently. Therefore, I switched to hikariCP and got lot better results. I have made a video simulating the slow jdbc calls which compares the results with tomcat and hikari connection pools while using the ReadTimeout property.