I need to monitor an oracle table for changes and I'm trying to choose between Database Change Notifications and Advanced Queuing.
I don't understand certain points in Database JDBC Developer's Guide
and getDatabaseChangeRegistration javadoc
If I register a DB change registration (DCR) with the NTF_QOS_RELIABLE flag, I expect the notifications to persist while my jdbc application is down. However, I don't see a way to restore existing DCR after my app restarts: according to javadoc, getDatabaseChangeRegistration() is only for PLSQL listeners. And it seems that jdbc DCRs are destroyed when my app dies and I don't even have to unregister them.
After my program restarts I sometimes get notifications with a previous registration id. It is not necessary to call stmt.setDatabaseChangeRegistration() every time I start my app.
I never receive changes that happened while my app was down and this is the biggest problem. What does NTF_QOS_RELIABLE do then?
package org.foo;
import static oracle.jdbc.OracleConnection.*;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.util.Locale;
import java.util.Properties;
import oracle.jdbc.OracleConnection;
import oracle.jdbc.OracleStatement;
import oracle.jdbc.dcn.DatabaseChangeEvent;
import oracle.jdbc.dcn.DatabaseChangeListener;
import oracle.jdbc.dcn.DatabaseChangeRegistration;
// CHECKSTYLE.OFF: Name|Reg
public final class TestDbListener {
private TestDbListener() {}
public static void main(final String[] args) throws Exception {
Locale.setDefault(Locale.US);
Class.forName("oracle.jdbc.OracleDriver");
final OracleConnection conn =
(OracleConnection) DriverManager.getConnection(
"jdbc:oracle:thin:#192.168.56.150:1521:xe", "scott", "tiger");
final Properties props = new Properties();
props.setProperty(NTF_QOS_RELIABLE, "true");
final DatabaseChangeRegistration dcr = conn.registerDatabaseChangeNotification(props);
final DCNListener list = new DCNListener();
dcr.addListener(list);
if (true) {
// now you need to add whatever tables you want to monitor
final OracleStatement stmt = (OracleStatement) conn.createStatement();
// associate the statement with the registration:
stmt.setDatabaseChangeRegistration(dcr);
final String sql = "select * from a where 1=2";
final ResultSet rs = stmt.executeQuery(sql);
}
final String[] tableNames = dcr.getTables();
for (int i = 0; i < tableNames.length; i++) {
System.out.println(tableNames[i] + " has been registered.");
}
Thread.sleep(1000000);
// rs.close();
}
}
class DCNListener implements DatabaseChangeListener {
#Override
public void onDatabaseChangeNotification(final DatabaseChangeEvent event) {
System.out.println("onDatabaseChangeNotification: " + event);
}
}
SQL:
-- sysdba:
-- grant change notification to scott;
-- scott:
create table a ( a int );
insert into a values ( 1 );
commit;
NTF_QOS_RELIABLE can be used to control which type of queue is used on the server to handle the notifications. See this doc:
https://docs.oracle.com/cd/B28359_01/appdev.111/b28395/oci10new.htm#CHDFCCJE
About NTF_QOS_RELIABLE:
Surviving instances of Oracle RAC can be used to send and retrieve
continuous query notification messages, even after a node failure
because invalidations associated with this registration are queued
persistently into the database. If FALSE, then invalidations are
enqueued into a fast in-memory queue. Note that this option describes
the persistence of notifications and not the persistence of
registrations. Registrations are automatically persistent by default.
If you app dies, the assumption is that during restart you'll fetch the latest data from the table you want to monitor. So the notifications that were sent between the crash and the restart are not needed.
Note that upon restart it's required to call conn.registerDatabaseChangeNotification to restart the driver's listener.
Related
Here is my application's code to create the database, connect to it, and make a table in the database called Accounts.
package eportfolio.application;
import java.io.File;
import java.io.FileWriter;
import javax.swing.JOptionPane;
import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Scanner;
/**
*
* #author valeriomacpro
*/
public class HomePage extends javax.swing.JFrame {
public static String username;
public static String password;
public static int SelectedPost;
/**
* Creates new form HomePage
*/
public static boolean doesTableExists (String tableName, Connection conn)
throws SQLException {
DatabaseMetaData meta = conn.getMetaData();
ResultSet result = meta.getTables(null, null, tableName.toUpperCase(), null);
return result.next();
}
public HomePage() {
initComponents();
try
{
String databaseURL = "jdbc:derby:eportdatabase;create=true";
Connection con = DriverManager.getConnection(databaseURL);
Statement st = con.createStatement();
if (!doesTableExists("Accounts", con))
{
String sql = "CREATE TABLE Accounts (Username varchar(250), Password varchar(250)) ";
st.execute(sql);
System.out.println("Table Does Not Yet Exist!");
}
else if(doesTableExists("Accounts", con)) {
System.out.println("Table Already Exists!");
}
con.close();
} catch(SQLException e) {
do {
System.out.println("SQLState:" + e.getSQLState());
System.out.println("Error Code:" + e.getErrorCode());
System.out.println("Message:" + e.getMessage());
Throwable t = e.getCause();
while(t != null) {
System.out.println("Cause:" + t);
t = t.getCause();
}
e = e.getNextException();
} while (e != null);
}
}
Additionally, here is my code that interacts with the Accounts table.
try
{
String databaseURL = "jdbc:derby:eportdatabase;";
Connection con1 = DriverManager.getConnection(databaseURL);
Statement st = con1.createStatement();
String sql = " INSERT INTO Accounts VALUES ('"+txtNewUsername.getText()+"','"+txtNewPassword.getText()+"') ";
st.executeUpdate(sql);
JOptionPane.showMessageDialog(null, "Account Info Saved!");
txtNewUsername.setText("");
txtNewPassword.setText("");
txtNewConfirm.setText("");
}
When I run the application, the code works fine. However, if I open DBeaver and connect it to my database, then the following error message comes up. Does not come up if DBeaver is closed, even if it is connected to the database.
Message:Failed to start database 'eportdatabase' with class loader jdk.internal.loader.ClassLoaders$AppClassLoader#45ee12a7, see the next exception for details.
Cause:ERROR XJ040: Failed to start database 'eportdatabase' with class loader jdk.internal.loader.ClassLoaders$AppClassLoader#45ee12a7, see the next exception for details.
Cause:ERROR XSDB6: Another instance of Derby may have already booted the database /Users/(username)/NetBeansProjects/ePortfolio Application/eportdatabase.
SQLState:XSDB6
Error Code:45000
Message:Another instance of Derby may have already booted the database /Users/(username)/NetBeansProjects/ePortfolio Application/eportdatabase.
Cause:ERROR XSDB6: Another instance of Derby may have already booted the database /Users/(username)/NetBeansProjects/ePortfolio Application/eportdatabase.
Why is this? Am I connecting the Database to DBeaver incorrectly? Or am I coding the database incorrectly in Netbeans? It could be that my drivers and db derby version are old, but I have not been able to find help on that online either. Also important to know that the table does show up in DBeaver, but does not update. I have to delete the database folder in my application's folder every time I want to use the application with DBeaver open. Any help appreciated.
By using this line of code:
String databaseURL = "jdbc:derby:eportdatabase;";
you are using Derby in the "embedded" configuration. With Embedded Derby, only one Java application at a time can use the database. Other applications that try to use it concurrently are rejected with the message
Another instance of Derby may have already booted the database
as you saw when you tried it.
There are other configurations in which Derby can be deployed and run; specifically there is a Client-Server configuration in which multiple applications may all run as clients, and may connect to the same Derby server, allowing the applications to run concurrently.
To learn more about these aspects of Derby, start here: https://db.apache.org/derby/docs/10.15/getstart/cgsquck70629.html
I need some Listener for any change(update, insert, delete) of Oracle database table.
Problem: I get many detection by single update on my table.
I think its oracle cache etc.
Is it possible only real changes to detect?
my code:
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;
import oracle.jdbc.OracleConnection;
import oracle.jdbc.OracleDriver;
import oracle.jdbc.OracleStatement;
import oracle.jdbc.dcn.DatabaseChangeEvent;
import oracle.jdbc.dcn.DatabaseChangeListener;
import oracle.jdbc.dcn.DatabaseChangeRegistration;
public class OracleDCN {
static final String USERNAME = "scott";
static final String PASSWORD = "tiger";
static String URL = "jdbc:oracle:thin:#localhost:1521:stingdev";
public static void main(String[] args) {
OracleDCN oracleDCN = new OracleDCN();
try {
oracleDCN.run();
} catch (Exception ex) {
ex.printStackTrace();
}
}
private void run() throws Exception{
OracleConnection conn = connect();
Properties prop = new Properties();
prop.setProperty(OracleConnection.DCN_NOTIFY_ROWIDS, "true");
DatabaseChangeRegistration dcr = conn.registerDatabaseChangeNotification(prop);
try{
dcr.addListener(new DatabaseChangeListener() {
public void onDatabaseChangeNotification(DatabaseChangeEvent dce) {
System.out.println("Changed row id : "+dce.getTableChangeDescription()[0].getRowChangeDescription()[0].getRowid().stringValue());
}
});
Statement stmt = conn.createStatement();
((OracleStatement) stmt).setDatabaseChangeRegistration(dcr);
ResultSet rs = stmt.executeQuery("select * from EXAMPLE where ID=1");
while (rs.next()) {
}
rs.close();
stmt.close();
}catch(SQLException ex){
if (conn != null)
{
conn.unregisterDatabaseChangeNotification(dcr);
conn.close();
}
throw ex;
}
}
OracleConnection connect() throws SQLException {
OracleDriver dr = new OracleDriver();
Properties prop = new Properties();
prop.setProperty("user", OracleDCN.USERNAME);
prop.setProperty("password", OracleDCN.PASSWORD);
return (OracleConnection) dr.connect(OracleDCN.URL, prop);
}
}
Output:
Changed row id : AAAFSzAAAAAAAG8AAA
Changed row id : AAAFSzAAAAAAAG8AAA
Changed row id : AAAFSzAAAAAAAG8AAA
Changed row id : AAAFSzAAAAAAAG8AAA
Problem was I registered to database many times. The solution was check registered user for change events before register.
I checked that with this query:
select TABLE_NAME
from USER_CHANGE_NOTIFICATION_REGS
As already said you are responsible for releasing your DatabaseChangeNotification by unregistering them at your connection. In my case for some reason I am doing this when application server starts on tomcat. First I am releasing old registrations and then create new. I think I can reuse the old registration but anyway i am not.
I am doing something like this:
This query returns existing registrations. You must login as the same user that registered the notification.
SELECT REGID, CALLBACK FROM USER_CHANGE_NOTIFICATION_REGS
then code like this pseudo code unregisters the notification.
connection.unregisterDatabaseChangeNotification(REGID, CALLBACK) this is called for every row returned by the query.
Something else I found is that when updating a row using PL/SQL Developer using the included editor I receive multiple notifications for the same update. When using sql to update the row I receive one notification as expected. I don't know why is this happening but it is happening.
The point is that according to
https://docs.oracle.com/cd/E18283_01/appdev.112/e13995/oracle/jdbc/OracleConnection.html
you are responsible for releasing your DatabaseChangeNotification by unregister them at your Connection. Otherwise as described "The registration will continue to live after this connection is closed. You need to explicitly unregister it to destroy it in the server and release the resources in the driver."
Meaning if you test your sample code and you kill it, your registrations remains living on the server and you receive extra notifications after registering next DatabaseChangeNotification. Unfortunately I haven't found out yet hot to reconnect with living registration - I would need regId for that, but I have no clue yet how to obtain it from the OracleConnection.
From what I've read so far, I'm a bit confused as well.
Have you tried printing out the transactionId to see if it is, in fact, a unique event being generated?
System.out.println("DCE : regId="+dce.getRegristrationId()+"; transactionId="+dce.getTransactionId());
The listeners that I've been working with for JMS require an acknowledgment, but I don't see anything of that sort in the Oracle docs (http://docs.oracle.com/cd/B28359_01/java.111/b31224/dbmgmnt.htm).
I also found that it might disturb the oracle cache if the event treatment is unthreaded, but it looks you're already doing that too...
Good luck!
You will get an event for each commit that modifies one of the tables you're using in your query (in your code sample only one table called "EXAMPLE"). Think of it as "Table Change Notification" TCN. In other words you may get a lot of false positives because you're only interested in one row but you'll be notified if other rows are changed. It's then up to you to filter the events. This is why this feature should only be used for read mostly tables.
In 11gR2, Oracle improved this notification mechanism to allow a finer notification called "Query Change Notification". This time you will only be notified by changes that affect your query results. There is an option that needs to be turned on to enable QCN instead of TCN. Note that the server may not always be able to enable QCN. If the query is too complex it will fall back to TCN.
Please modify the event handler as follows:
public void onDatabaseChangeNotification(DatabaseChangeEvent dce) {
if (e.getRegId() == dcr.getRegId())
System.out.println("Changed row id : "+dce.getTableChangeDescription()[0].getRowChangeDescription()[0].getRowid().stringValue());
}
Correcting typo error:
Please modify the event handler as follows:
public void onDatabaseChangeNotification(DatabaseChangeEvent dce) {
if (dce.getRegId() == dcr.getRegId())
System.out.println("Changed row id : "+dce.getTableChangeDescription()[0].getRowChangeDescription()[0].getRowid().stringValue());
}
In the application we are working on, the user can connect to external RDBMSs by entering an arbitrary JDBC connection URL into a text field. One of our customers reported that our application server freezes (indefinitly) at 0% CPU when he accidentally tried to connect to a Microsoft SQL Server with a MySQL JDBC URL.
The following Java snippet illustrates the situation:
public static void main(String[] args){
// note: the application running on localhost:1433 is ACTUALLY
// an MS SQL Server instance!
String jdbcUrl = "jdbc:mysql://localhost:1433/my-db";
// enable JDBC Driver Manager logging
DriverManager.setLogWriter(new PrintWriter(System.err));
// set a timeout of 5 seconds for connecting (which is blissfully ignored!)
DriverManager.setLoginTimeout(5);
// open the connection (which should fail, but freezes instead)
try (Connection c = DriverManager.getConnection(jdbcUrl)){
System.out.println("This should never be reached due to wrong JDBC URL.");
}catch(Exception e){
System.out.println("This is expected (but never printed).");
}
System.out.println("This is never printed either.");
}
To run the snippet:
have a SQL Server instance running on localhost:1433 (content does not matter)
have the MariaDB JDBC driver version 2.2.5. (latest) on your classpath.
Questions:
1) Could this be a bug in the MariaDB JDBC driver? A google search revealed nothing in this regard.
2) How should I work around this issue? I don't want my server to freeze when the user accidentally inserts an invalid JDBC URL.
I tried several other JDBC drivers (MySQL, DB2, Oracle...) and they all handle this issue gracefully, only the MariaDB JDBC driver freezes the JVM.
Here's what I did to resolve the issue. The trick is to add a socketTimeout to the connection. To fix the program in the question, it is enough to modify the JDBC URL to be:
jdbc:mysql://localhost:1433/my-db?socketTimeout=2000
This answer to a related question was the hint I needed.
Answer 1: Yes, it is a bug. They missed to use the login timeout in the implementation of mariadb jdbc driver.
Answer 2: I worked around by using a task that that wraps the getConnection method. This task is stopped after a defined login time if it hasn't finished. Here is my implementation.
import java.sql.Connection;
import java.sql.DriverManager;
import java.util.Properties;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.FutureTask;
import java.util.concurrent.TimeUnit;
import org.mariadb.jdbc.util.DefaultOptions;
public class ConnectionTest {
private static final String CONNECTION_STRING = "jdbc:mariadb://localhost:3306/test";
private static final String USER = "root";
private static final String PW = "";
private static final int LOGIN_TIMEOUT_SEC = 2;
public static void main(String[] args) throws Exception {
var test = new ConnectionTest();
Connection connection = test.getConnection();
if(connection != null && connection.isValid(LOGIN_TIMEOUT_SEC)) {
System.out.println("Connected!");
}
}
private Connection getConnection() throws Exception {
ConnEstablishSync sync = new ConnEstablishSync();
Properties conProps = new Properties();
conProps.setProperty(DefaultOptions.USER.getOptionName(), USER);
conProps.setProperty(DefaultOptions.PASSWORD.getOptionName(), PW);
FutureTask<Connection> task = new FutureTask<>(() -> {
Connection c = DriverManager.getConnection(CONNECTION_STRING, conProps);
if(sync.canceled && c != null) {
c.close();
c = null;
}
return c;
});
Connection connection = null;
ExecutorService executor = Executors.newSingleThreadExecutor();
try {
executor.submit(task);
connection = task.get(LOGIN_TIMEOUT_SEC, TimeUnit.SECONDS);
} finally {
sync.canceled = true;
task.cancel(true);
executor.shutdown();
}
return connection;
}
private static class ConnEstablishSync {
private volatile boolean canceled = false;
}
}
I'm developing an student database web application using java servlets and h2 embedded database. I need to know how to find whether the typical student database exists or not. so that, if it is not there, it must be created or else the program should continue with the existing one.
tl;dr
Add IFEXISTS=TRUE to your connection URL string.
H2 creates database, if absent
You asked:
I need to know how to find whether the typical student database exists or not. so that, if it is not there, it must be created or else the program should continue with the existing one.
By default, H2 automatically creates the database if not already in existence. So that part of your Question does not make sense.
Specify a file location as part of establishing a DataSource object. Here we use org.h2.jdbcx.JdbcDataSource as our implementation of javax.sql.DataSource.
private javax.sql.DataSource establishDataSource() {
org.h2.jdbcx.JdbcDataSource ds = Objects.requireNonNull( new JdbcDataSource() ); // Implementation of `DataSource` bundled with H2. You may choose to use some other implementation.
ds.setURL( "jdbc:h2:/path/to/MyDatabase;" );
ds.setUser( "scott" );
ds.setPassword( "tiger" );
ds.setDescription( "An example database." );
return ds ;
}
Instantiate a DataSource object. Keep it around for use when you need access to the database.
DataSource dataSource = this.establishDataSource();
…
The DataSource object does not cause anything to happen. A DataSource object merely holds the pieces of information needed to locate and connect to a particular database.
Attempt to connect to database. This is when things start to happen.
try(
Connection conn = dataSource.getConnection() ;
)
{ … }
The first call to DataSource#getConnection causes H2 to:
Establish a new database file at the specified location in the file system, if not already existing.
Open a connection to the database.
But to answer the title of your Question, read on.
IFEXISTS
You can specify in your connection attempt that a connection should only be completed if the requested database already exists. This feature uses IFEXISTS=TRUE syntax.
String url = "jdbc:h2:/path/to/MyDatabase;IFEXISTS=TRUE";
This causes H2 to:
Look for existing database at given location.
If no existing database, throw exception.
If existing database found, open a connection.
Trap for the exception thrown when the database does not exist.
File
You check for the existence of the database file in the file system, assuming your database is persisted to storage (as opposed to in-memory database).
package com.dora.databasecheck;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import javax.sql.DataSource;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.jdbc.datasource.SimpleDriverDataSource;
import org.springframework.jdbc.datasource.init.DatabasePopulator;
import org.springframework.jdbc.datasource.init.DatabasePopulatorUtils;
import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator;
#Configuration
public class DataSourceInit {
boolean dbExists = false;
#Bean(name = "dataSource")
public DataSource getDataSource(){
DataSource dataSource = createDataSource();
DatabasePopulatorUtils.execute(createDatabasePopulator(), dataSource);
return dataSource;
}
private DatabasePopulator createDatabasePopulator() {
ResourceDatabasePopulator databasePopulator = new ResourceDatabasePopulator();
databasePopulator.setContinueOnError(true);
if(!dbExists){
databasePopulator.addScript(new ClassPathResource("db/sql/create-db.sql"));
databasePopulator.addScript(new ClassPathResource("db/sql/insert-data.sql"));
}
return databasePopulator;
}
private SimpleDriverDataSource createDataSource() {
SimpleDriverDataSource simpleDriverDataSource = new SimpleDriverDataSource();
simpleDriverDataSource.setDriverClass(org.h2.Driver.class);
simpleDriverDataSource.setUsername("sa");
simpleDriverDataSource.setPassword("");
Connection conn = null;
try{
conn = DriverManager.getConnection("jdbc:h2:~/doradb;IFEXISTS=TRUE","sa","");
this.dbExists =true;
}
catch(Exception e){
this.dbExists = false;
}
simpleDriverDataSource.setUrl("jdbc:h2:~/doradb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE");
return simpleDriverDataSource;
}
}
I found this method solving my problem, First the connection object checks for the existence of the database, if not it throws an exception where the database is created.
try
{
Connection c=DriverManager.getConnection("jdbc:h2:aaa;IFEXISTS=TRUE",username,password);
}
Catch(final Exception e)
{
connection c=DriverManager.getConnection("jdbc:h2:aaa","user","pass");
}
I am developing a simple CRUD application, using JDBC to establish connection and perform basic CRUD operations. In that process, created a DatabaseListener to create a connection object at startup and storing it in the context attribute for reuse.
Below is the code.
import javax.servlet.ServletContext;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import org.apache.log4j.Logger;
public class DatabaseInitListner implements ServletContextListener {
private static final Logger LOG = Logger.getLogger(DatabaseInitListner.class);
private DBUtil databaseUtil = null;
#Override
public void contextDestroyed(ServletContextEvent event) {
databaseUtil.closeConnection();
}
#Override
public void contextInitialized(ServletContextEvent contextinitEvent) {
ServletContext servletContext = contextinitEvent.getServletContext();
String database = servletContext.getInitParameter("db_name");
String url = servletContext.getInitParameter("db_url")
+ database;
String username = servletContext.getInitParameter("db_user");
String password = servletContext.getInitParameter("db_password");
String driverName = servletContext.getInitParameter("db_driver");
databaseUtil = new DBUtil(url, username, password,
driverName);
servletContext.setAttribute("databaseSingleConnectionObject",
databaseUtil.getConnection());
}
}
public class DBUtil {
private Connection connection = null;
private static final Logger LOG = Logger.getLogger(DatabaseUtil.class);
public DatabaseUtil(String url, String username, String password,
String driver) {
try {
Class.forName(driver);
this.connection = DriverManager.getConnection(url, username,
password);
LOG.debug("Connection Established... ");
} catch (ClassNotFoundException | SQLException e) {
LOG.error("Could not create connection... ", e);
}
}
public Connection getConnection() {
return connection;
}
public void closeConnection() {
if (connection != null) {
try {
connection.close();
} catch (SQLException e) {
LOG.error("Unable to close connection... ", e);
}
}
}
}
I am accessing the connection in servlets like this
Connection jdbcConnection = (Connection) getServletContext().getAttribute("databaseSingleConnectionObject");
I am not sure if this is right approach. What are the effects of single database connection?
When you use a single database connection like this you make your application slow and brittle.
Slow: because the connection implementation is synchronized, each user has to wait until the connection is free. If one user's query takes a while to come back that directly increases the time any other concurrent users spend waiting. If there were multiple connections available from a pool then the time spent by one user would not impact other users nearly as greatly (unless a query's results take all the JVM's memory or a big query bogs down the database server).
Brittle: The connection is a network connection, they tend to go down. Without a provision to create new connections any kind of timeout, network hiccup, or period of database non-availability (such as taking the database offline for maintenance) is going to require an application restart. Using a connection pool will mean your application will be able to survive these episodes and recover without outside intervention.
This will not be threadsafe, and if it were, performance would be really poor.
Look into using a Connection Pool, like DBCP or C3PO
You should let your application server manage database connection. Add a JNDI datasource in its configuration file and make a lookup from your application to get a connection when needed (for instance when you instantiate a class that must access your database).
You may configure the datasource to manage a connection pool so that each user session will get its own.
Depending on the AS you use run a search with keywords 'JNDI' and 'datasource' and you will get further details about the AS configuration and how to implement it in your application.