We Keep Coding

Why oracle data base change listener is triggering multiple events in case of single update? [duplicate]

I need some Listener for any change(update, insert, delete) of Oracle database table.
Problem: I get many detection by single update on my table.
I think its oracle cache etc.
Is it possible only real changes to detect?
my code:
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;
import oracle.jdbc.OracleConnection;
import oracle.jdbc.OracleDriver;
import oracle.jdbc.OracleStatement;
import oracle.jdbc.dcn.DatabaseChangeEvent;
import oracle.jdbc.dcn.DatabaseChangeListener;
import oracle.jdbc.dcn.DatabaseChangeRegistration;
public class OracleDCN {
static final String USERNAME = "scott";
static final String PASSWORD = "tiger";
static String URL = "jdbc:oracle:thin:#localhost:1521:stingdev";
public static void main(String[] args) {
OracleDCN oracleDCN = new OracleDCN();
try {
oracleDCN.run();
} catch (Exception ex) {
ex.printStackTrace();
}
}
private void run() throws Exception{
OracleConnection conn = connect();
Properties prop = new Properties();
prop.setProperty(OracleConnection.DCN_NOTIFY_ROWIDS, "true");
DatabaseChangeRegistration dcr = conn.registerDatabaseChangeNotification(prop);
try{
dcr.addListener(new DatabaseChangeListener() {
public void onDatabaseChangeNotification(DatabaseChangeEvent dce) {
System.out.println("Changed row id : "+dce.getTableChangeDescription()[0].getRowChangeDescription()[0].getRowid().stringValue());
}
});
Statement stmt = conn.createStatement();
((OracleStatement) stmt).setDatabaseChangeRegistration(dcr);
ResultSet rs = stmt.executeQuery("select * from EXAMPLE where ID=1");
while (rs.next()) {
}
rs.close();
stmt.close();
}catch(SQLException ex){
if (conn != null)
{
conn.unregisterDatabaseChangeNotification(dcr);
conn.close();
}
throw ex;
}
}
OracleConnection connect() throws SQLException {
OracleDriver dr = new OracleDriver();
Properties prop = new Properties();
prop.setProperty("user", OracleDCN.USERNAME);
prop.setProperty("password", OracleDCN.PASSWORD);
return (OracleConnection) dr.connect(OracleDCN.URL, prop);
}
}
Output:
Changed row id : AAAFSzAAAAAAAG8AAA
Changed row id : AAAFSzAAAAAAAG8AAA
Changed row id : AAAFSzAAAAAAAG8AAA
Changed row id : AAAFSzAAAAAAAG8AAA

Problem was I registered to database many times. The solution was check registered user for change events before register.
I checked that with this query:
select TABLE_NAME
from USER_CHANGE_NOTIFICATION_REGS

As already said you are responsible for releasing your DatabaseChangeNotification by unregistering them at your connection. In my case for some reason I am doing this when application server starts on tomcat. First I am releasing old registrations and then create new. I think I can reuse the old registration but anyway i am not.
I am doing something like this:
This query returns existing registrations. You must login as the same user that registered the notification.
SELECT REGID, CALLBACK FROM USER_CHANGE_NOTIFICATION_REGS
then code like this pseudo code unregisters the notification.
connection.unregisterDatabaseChangeNotification(REGID, CALLBACK) this is called for every row returned by the query.
Something else I found is that when updating a row using PL/SQL Developer using the included editor I receive multiple notifications for the same update. When using sql to update the row I receive one notification as expected. I don't know why is this happening but it is happening.

The point is that according to
https://docs.oracle.com/cd/E18283_01/appdev.112/e13995/oracle/jdbc/OracleConnection.html
you are responsible for releasing your DatabaseChangeNotification by unregister them at your Connection. Otherwise as described "The registration will continue to live after this connection is closed. You need to explicitly unregister it to destroy it in the server and release the resources in the driver."
Meaning if you test your sample code and you kill it, your registrations remains living on the server and you receive extra notifications after registering next DatabaseChangeNotification. Unfortunately I haven't found out yet hot to reconnect with living registration - I would need regId for that, but I have no clue yet how to obtain it from the OracleConnection.

From what I've read so far, I'm a bit confused as well.
Have you tried printing out the transactionId to see if it is, in fact, a unique event being generated?
System.out.println("DCE : regId="+dce.getRegristrationId()+"; transactionId="+dce.getTransactionId());
The listeners that I've been working with for JMS require an acknowledgment, but I don't see anything of that sort in the Oracle docs (http://docs.oracle.com/cd/B28359_01/java.111/b31224/dbmgmnt.htm).
I also found that it might disturb the oracle cache if the event treatment is unthreaded, but it looks you're already doing that too...
Good luck!

You will get an event for each commit that modifies one of the tables you're using in your query (in your code sample only one table called "EXAMPLE"). Think of it as "Table Change Notification" TCN. In other words you may get a lot of false positives because you're only interested in one row but you'll be notified if other rows are changed. It's then up to you to filter the events. This is why this feature should only be used for read mostly tables.
In 11gR2, Oracle improved this notification mechanism to allow a finer notification called "Query Change Notification". This time you will only be notified by changes that affect your query results. There is an option that needs to be turned on to enable QCN instead of TCN. Note that the server may not always be able to enable QCN. If the query is too complex it will fall back to TCN.

Please modify the event handler as follows:
public void onDatabaseChangeNotification(DatabaseChangeEvent dce) {
if (e.getRegId() == dcr.getRegId())
System.out.println("Changed row id : "+dce.getTableChangeDescription()[0].getRowChangeDescription()[0].getRowid().stringValue());
}

Correcting typo error:
Please modify the event handler as follows:
public void onDatabaseChangeNotification(DatabaseChangeEvent dce) {
if (dce.getRegId() == dcr.getRegId())
System.out.println("Changed row id : "+dce.getTableChangeDescription()[0].getRowChangeDescription()[0].getRowid().stringValue());
}

persistent Database Change Notifications

I need to monitor an oracle table for changes and I'm trying to choose between Database Change Notifications and Advanced Queuing.
I don't understand certain points in Database JDBC Developer's Guide
and getDatabaseChangeRegistration javadoc
If I register a DB change registration (DCR) with the NTF_QOS_RELIABLE flag, I expect the notifications to persist while my jdbc application is down. However, I don't see a way to restore existing DCR after my app restarts: according to javadoc, getDatabaseChangeRegistration() is only for PLSQL listeners. And it seems that jdbc DCRs are destroyed when my app dies and I don't even have to unregister them.
After my program restarts I sometimes get notifications with a previous registration id. It is not necessary to call stmt.setDatabaseChangeRegistration() every time I start my app.
I never receive changes that happened while my app was down and this is the biggest problem. What does NTF_QOS_RELIABLE do then?
package org.foo;
import static oracle.jdbc.OracleConnection.*;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.util.Locale;
import java.util.Properties;
import oracle.jdbc.OracleConnection;
import oracle.jdbc.OracleStatement;
import oracle.jdbc.dcn.DatabaseChangeEvent;
import oracle.jdbc.dcn.DatabaseChangeListener;
import oracle.jdbc.dcn.DatabaseChangeRegistration;
// CHECKSTYLE.OFF: Name|Reg
public final class TestDbListener {
private TestDbListener() {}
public static void main(final String[] args) throws Exception {
Locale.setDefault(Locale.US);
Class.forName("oracle.jdbc.OracleDriver");
final OracleConnection conn =
(OracleConnection) DriverManager.getConnection(
"jdbc:oracle:thin:#192.168.56.150:1521:xe", "scott", "tiger");
final Properties props = new Properties();
props.setProperty(NTF_QOS_RELIABLE, "true");
final DatabaseChangeRegistration dcr = conn.registerDatabaseChangeNotification(props);
final DCNListener list = new DCNListener();
dcr.addListener(list);
if (true) {
// now you need to add whatever tables you want to monitor
final OracleStatement stmt = (OracleStatement) conn.createStatement();
// associate the statement with the registration:
stmt.setDatabaseChangeRegistration(dcr);
final String sql = "select * from a where 1=2";
final ResultSet rs = stmt.executeQuery(sql);
}
final String[] tableNames = dcr.getTables();
for (int i = 0; i < tableNames.length; i++) {
System.out.println(tableNames[i] + " has been registered.");
}
Thread.sleep(1000000);
// rs.close();
}
}
class DCNListener implements DatabaseChangeListener {
#Override
public void onDatabaseChangeNotification(final DatabaseChangeEvent event) {
System.out.println("onDatabaseChangeNotification: " + event);
}
}
SQL:
-- sysdba:
-- grant change notification to scott;
-- scott:
create table a ( a int );
insert into a values ( 1 );
commit;

NTF_QOS_RELIABLE can be used to control which type of queue is used on the server to handle the notifications. See this doc:
https://docs.oracle.com/cd/B28359_01/appdev.111/b28395/oci10new.htm#CHDFCCJE
About NTF_QOS_RELIABLE:
Surviving instances of Oracle RAC can be used to send and retrieve
continuous query notification messages, even after a node failure
because invalidations associated with this registration are queued
persistently into the database. If FALSE, then invalidations are
enqueued into a fast in-memory queue. Note that this option describes
the persistence of notifications and not the persistence of
registrations. Registrations are automatically persistent by default.
If you app dies, the assumption is that during restart you'll fetch the latest data from the table you want to monitor. So the notifications that were sent between the crash and the restart are not needed.
Note that upon restart it's required to call conn.registerDatabaseChangeNotification to restart the driver's listener.

Cloudant AuthCookie may not be null

I am currently trying to add entries into Cloudant using this link:
https://github.com/cloudant/java-cloudant#installation-and-usage
Below is my code
package sample;
import java.util.List;
import com.cloudant.client.api.CloudantClient;
public class Cloudant {
public static void main (String[] args){
String password = System.getProperty("gffgasdas");
CloudantClient client = new CloudantClient("wiz.cloudant.com",password);
System.out.println("Connected to Cloudant");
System.out.println("Server Version: " + client.serverVersion());
List<String> databases = client.getAllDbs();
System.out.println("All my databases : ");
for ( String db : databases ) {
System.out.println(db);
}
}
}
I am getting this error:
Exception in thread "main" java.lang.IllegalArgumentException: AuthCookie may not be null.
at org.lightcouch.internal.CouchDbUtil.assertNotEmpty(Unknown Source)
at com.cloudant.client.api.CloudantClient.<init>(Unknown Source)
at sample.Cloudant.main(Cloudant.java:11)
I have all the necessary important imports. Any help would be appreciated thanks.

I'm not sure you're using the right constructor. It looks like you need to use the three-argument constructor CloudantClient(cloudantAccountName, username, password).
Your line:
CloudantClient client = new CloudantClient("wiz.cloudant.com",password);
Needs to be:
CloudantClient client = new CloudantClient("wiz", "wiz", password);
The two-argument version assumes you are passing a cookie rather than a password.

JMX: Read attribute from Server

We're using Adobe CQ (5.5) as CMS. Now, our CQ environment consists of one author server, where users can create content, and 2 publish servers which serve the content to the internet.
Now there's a replication agent which pushs content from the author server to both publish server. Unfortunately some articles block the queue of the replication agents, so no more new content is beeing published. This is not much of a problem, as it is easy to fix. The real problem is that we don't notice this blockage until users start to complain that no more changes are beeing published.
I searched around and found out that CQ provides a JMX API where monitoring applications could attach itself to it. I then tried to find some open source software which would allow me to configure alerts, so we can react faster, but I couldn't find something.
This is when I decided that I could try to write my own Java Application which just reads the attribute and sends a mail if the attribute should be true. I guess that was more complicated than I tought.
First off, I'm not a Java Developer, but since CQ is based on Java I tought I'd give it a try. I read some documentation about JMX and Java and was able to get a working connection to the CQ server. But this is almost everything I could realize.
I was able to find out that the class com.adobe.granite.replication has a type agent which stores an id for every replication agent (the id would be the name of the replication agent, for example id=replication-publish-1). Every replication-agent has different attributes, but the attribute relevant for me would be "QueueBlocked".
This is the code I've got so far (it's based on this example):
public static void main(String[] args) {
try {
JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://servername:9010/jmxrmi");
JMXConnector jmxc = JMXConnectorFactory.connect(url, null);
ClientListener listener = new ClientListener();
MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
// This outputs the domains, one of them is com.adobee.granite.replication, the one which I need to use
// This is why I'm sure that at least the connection works, I don't have any com.adobe.granite.replication class on my Eclipse installation, so the output has to come from the server
String domains[] = mbsc.getDomains();
for (int i = 0; i < domains.length; i++) {
echo("\tDomain[" + i + "] = " + domains[i]);
}
ObjectName replication = new ObjectName("com.adobe.granite.replication:type=Agent,id=replication-publish-1");
mbsc.getAttribute(replication, "QueueBlocked"); // This throws the error
} catch(Exception e) {
}
}
The error thrown is the following:
javax.management.InstanceNotFoundException: com.adobe.granite.replication:type=Agent,id=replication-publish-1
From what I understand I should be creating some kind of instance, but I don't really have an idea what instance and how to create it. I'd really appreciate any help I can get no matter if it's a documentation or code snippet :)

Solved it :)
This is the code I'm using:
import java.io.IOException;
import java.util.Iterator;
import java.util.Set;
import javax.management.Attribute;
import javax.management.MBeanServerConnection;
import javax.management.MBeanServerInvocationHandler;
import javax.management.ObjectName;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;
public class Client {
public static void main(String[] args) {
try {
JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://servername:9010/jmxrmi");
JMXConnector jmxc = JMXConnectorFactory.connect(url, null);
MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
ObjectName replication1 = new ObjectName("com.adobe.granite.replication:type=agent,id=\"replication-publish-1\"");
ObjectName replication2 = new ObjectName("com.adobe.granite.replication:type=agent,id=\"replication-publish-2\"");
String replication1Status = mbsc.getAttribute(replication1, "QueuePaused").toString();
String replication2Status = mbsc.getAttribute(replication2, "QueuePaused").toString();
} catch (Exception e) {
e.printStackTrace();
}
}
}

Easy way to start a standalone JNDI server (and register some resources)

For testing purposes, I'm looking for a simple way to start a standalone JNDI server, and bind my javax.sql.DataSource to "java:/comp/env/jdbc/mydatasource" programmatically.
The server should bind itself to some URL, for example: "java.naming.provider.url=jnp://localhost:1099" (doesn't have to be JNP), so that I can look up my datasource from another process. I don't care about which JNDI server implementation I'll have to use (but I don't want to start a full-blown JavaEE server).
This should be so easy, but to my surprise, I couldn't find any (working) tutorial.

The JDK contains a JNDI provider for the RMI registry. That means you can use the RMI registry as a JNDI server. So, just start rmiregistry, set java.naming.factory.initial to com.sun.jndi.rmi.registry.RegistryContextFactory, and you're away.
The RMI registry has a flat namespace, so you won't be able to bind to java:/comp/env/jdbc/mydatasource, but you will be able to bind to something so it will accept java:/comp/env/jdbc/mydatasource, but will treat it as a single-component name (thanks, #EJP).
I've written a small application to demonstrate how to do this: https://bitbucket.org/twic/jndiserver/src
I still have no idea how the JNP server is supposed to work.

I worked on the John´s code and now is working good.
In this version I'm using libs of JBoss5.1.0.GA, see jar list below:
jboss-5.1.0.GA\client\jbossall-client.jar
jboss-5.1.0.GA\server\minimal\lib\jnpserver.jar
jboss-5.1.0.GA\server\minimal\lib\log4j.jar
jboss-remote-naming-1.0.1.Final.jar (downloaded from http://search.maven.com)
This is the new code:
import java.net.InetAddress;
import java.util.Hashtable;
import java.util.concurrent.Callable;
import javax.naming.Context;
import javax.naming.InitialContext;
import org.jnp.server.Main;
import org.jnp.server.NamingBeanImpl;
public class StandaloneJNDIServer implements Callable<Object> {
public Object call() throws Exception {
setup();
return null;
}
#SuppressWarnings("unchecked")
private void setup() throws Exception {
//configure the initial factory
//**in John´s code we did not have this**
System.setProperty(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
//start the naming info bean
final NamingBeanImpl _naming = new NamingBeanImpl();
_naming.start();
//start the jnp serve
final Main _server = new Main();
_server.setNamingInfo(_naming);
_server.setPort(5400);
_server.setBindAddress(InetAddress.getLocalHost().getHostName());
_server.start();
//configure the environment for initial context
final Hashtable _properties = new Hashtable();
_properties.put(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
_properties.put(Context.PROVIDER_URL, "jnp://10.10.10.200:5400");
//bind a name
final Context _context = new InitialContext(_properties);
_context.bind("jdbc", "myJDBC");
}
public static void main(String...args){
try{
new StandaloneJNDIServer().call();
}catch(Exception _e){
_e.printStackTrace();
}
}
}
To have good logging, use this log4j properties:
log4j.rootLogger=TRACE, A1
log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
To consume the Standalone JNDI server, use this client class:
import java.util.Hashtable;
import javax.naming.Context;
import javax.naming.InitialContext;
/**
*
* #author fabiojm - Fábio José de Moraes
*
*/
public class Lookup {
public Lookup(){
}
#SuppressWarnings("unchecked")
public static void main(String[] args) {
final Hashtable _properties = new Hashtable();
_properties.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory");
_properties.put("java.naming.provider.url", "jnp://10.10.10.200:5400");
try{
final Context _context = new InitialContext(_properties);
System.out.println(_context);
System.out.println(_context.lookup("java:comp"));
System.out.println(_context.lookup("java:jdbc"));
}catch(Exception _e){
_e.printStackTrace();
}
}
}

Here's a code snippet adapted from JBoss remoting samples. The code that is
in the samples (version 2.5.4.SP2 ) no longer works. While the fix
is simple it took me more hours than I want to think about to figure it out.
Sigh. Anyway, maybe someone can benefit.
package org.jboss.remoting.samples.detection.jndi.custom;
import java.net.InetAddress;
import java.util.concurrent.Callable;
import org.jnp.server.Main;
import org.jnp.server.NamingBeanImpl;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class StandaloneJNDIServer implements Callable<Object> {
private static Logger logger = LoggerFactory.getLogger( StandaloneJNDIServer.class );
// Default locator values - command line args can override transport and port
private static String transport = "socket";
private static String host = "localhost";
private static int port = 5400;
private int detectorPort = 5400;
public StandaloneJNDIServer() {}
#Override
public Object call() throws Exception {
StandaloneJNDIServer.println("Starting JNDI server... to stop this server, kill it manually via Control-C");
//StandaloneJNDIServer server = new StandaloneJNDIServer();
try {
this.setupJNDIServer();
// wait forever, let the user kill us at any point (at which point, the client will detect we went down)
while(true) {
Thread.sleep(1000);
}
}
catch(Exception e) {
e.printStackTrace();
}
StandaloneJNDIServer.println("Stopping JBoss/Remoting server");
return null;
}
private void setupJNDIServer() throws Exception
{
// start JNDI server
String detectorHost = InetAddress.getLocalHost().getHostName();
Main JNDIServer = new Main();
// Next two lines add a naming implemention into
// the server object that handles requests. Without this you get a nice NPE.
NamingBeanImpl namingInfo = new NamingBeanImpl();
namingInfo.start();
JNDIServer.setNamingInfo( namingInfo );
JNDIServer.setPort( detectorPort );
JNDIServer.setBindAddress(detectorHost);
JNDIServer.start();
System.out.println("Started JNDI server on " + detectorHost + ":" + detectorPort );
}
/**
* Outputs a message to stdout.
*
* #param msg the message to output
*/
public static void println(String msg)
{
System.out.println(new java.util.Date() + ": [SERVER]: " + msg);
}
}

I know I'm late to the party, but I ended up hacking this together like so
InitialContext ctx = new InitialContext();
// check if we have a JNDI binding for "jdbc". If we do not, we are
// running locally (i.e. through JUnit, etc)
boolean isJndiBound = true;
try {
ctx.lookup("jdbc");
} catch(NameNotFoundException ex) {
isJndiBound = false;
}
if(!isJndiBound) {
// Create the "jdbc" sub-context (i.e. the directory)
ctx.createSubcontext("jdbc");
//parse the jetty-web.xml file
Map<String, DataSource> dataSourceProperties = JettyWebParser.parse();
//add the data sources to the sub-context
for(String key : dataSourceProperties.keySet()) {
DataSource ds = dataSourceProperties.get(key);
ctx.bind(key, ds);
}
}

Have you considered using Mocks? If I recall correctly you use Interfaces to interact with JNDI. I know I've mocked them out at least once before.
As a fallback, you could probably use Tomcat. It's not a full blown J2EE impl, it starts fast, and is fairly easy to configure JNDI resources for. DataSource setup is well documented. It's sub-optimal, but should work.

You imply you've found non-working tutorials; that may mean you've already seen these:
J2EE or J2SE? JNDI works with both
Standalone JNDI server using jnpserver.jar
I had a quick go, but couldn't get this working. A little more perseverance might do it, though.

For local, one process standalone jar purpouses I would use spring-test package:
SimpleNamingContextBuilder builder = new SimpleNamingContextBuilder();
SQLServerConnectionPoolDataSource myDS = new SQLServerConnectionPoolDataSource();
//setup...
builder.bind("java:comp/env/jdbc/myDS", myDS);
builder.activate();
startup log:
22:33:41.607 [main] INFO org.springframework.mock.jndi.SimpleNamingContextBuilder - Static JNDI binding: [java:comp/env/jdbc/myDS] = [SQLServerConnectionPoolDataSource:1]
22:33:41.615 [main] INFO org.springframework.mock.jndi.SimpleNamingContextBuilder - Activating simple JNDI environment

I have been looking for a similar simple starter solution recently. The "file system service provider from Sun Microsystems" has worked for me well. See https://docs.oracle.com/javase/jndi/tutorial/basics/prepare/initial.html.
The problem with the RMI registry is that you need a viewer - here you just need to look at file contents.
You may need fscontext-4.2.jar - I obtained it from http://www.java2s.com/Code/Jar/f/Downloadfscontext42jar.htm