I need to make a web application in java, that offers a dashboard based on the content of a db table.
It needs to be "autorefreshing", and always syncronized with the actual data in the db.
For the browser <-> servlet interaction I can use websockets or at least long polling to achieve the "freshness", but I'm stuck with the java <-> db communication.
I can have some polling, but I would really have some "notification" from the db itself.
Is there some way / some library to achieve?
For my case the db is oracle, but I'm interested also in solution for postgres.
To monitor db changes debezium connector is good. By using this u will get every change event of database in kafka topics.
For oracle look at this tutorial
For postgresql look at this tutorial
Related
We have a command and control system which persists historical data in a database. We'd like to make the system independent of the database. So if the database is there, great we will persist data there, if it is not, we will do some backup storage to files and memory until the database is back. The command and control functionality must be able to continue uninterrupted by the loss or restoration of the database; it should not even know the database exists. So the database and DAO functionality needs to be decoupled from the rest of the application.
We are using RESTful service calls, Spring framework, ActiveMQ, JDBCTemplate with SQL Server database. Currently following standard connection practices using Hikari datasource and JTDS driver. The problem is that if the database goes down or the database connection is lost we start to have data issues as too many service calls (mainly the getters) are still too dependent on the database existence for processing. This dependence is what we'd like to eliminate.
What are the best practices/technologies for totally decoupling the database from the application? We are considering using AMQ to broadcast data updates and have the DAO listen for those messages and then do the update to the database if it is available or flat files as a backup. Then for the getters, provide replies based on what is available either from the actual database or from the short-term backup.
My team has little experience with this and we want to know what others have done that works well.
I am building a java REST APIs based website whose function is to connect to any user entered database and get the schemas, tables, indexes etc and the user can pick whatever schemas / tables / indexes they want and send to another system.
So the site takes the database details, then shows the schemas - user selects the schemas they need- then the site brings back the corresponding tables etc. So in the backend I have separate calls for getting schema/tables/indexes.
I am using plain JDBC calls in the server to do this. Each time I am opening the connection, getting the metadata(schema/table/index), closing the connection. I think performance can be improved if I keep the database connection open between requests.
Since the database details are dynamic and each user is connecting to a different database,I cannot use the connection pool facility provided in the (play) framework. Is there a better way to do this? Thanks in advance!
I am using play framework 2.x with Angular JS.
You can use a singleton or static Map of JDBC DataSource and get the connexion from it. The datasource will manage the connexion pull.
What we're trying to do is what Meteor is doing with Mongo with LiveQuery, which is this:
Livequery can connect to the database, pretend to be a replication
slave, and consume the replication log. Most databases support some
form of replication so this is a widely applicable approach. This is
the strategy that Livequery prefers with MongoDB, since MongoDB does
not have triggers.
Source of that quote here
So is there a way with com.mongodb.*; in Java to create such replication slave so that it receives any notifications for each update that happens on the primary Mongo server?
Also, I don't see any replication log in the local database. Is there a way to turn them on?
If it's not possible to do it in Java, is it possible to create such solution in other languages (C++ or Node.js maybe)?
You need to start your database with the --replSet rsName option, and then run rs.initiate(). After that you will see a rs.oplog collection in the local database.
What you are describing is commonly referred to as "tailing the oplog", which is based on using a Tailable Cursor on a capped collection (the MongoDB oplog in this case). The mechanics are relatively simple, there are numerous oplog tailing examples out there written in Java, here are a few:
Event Streaming with MongoDB
TailableCursorExample
Wordnik mongo-admin-utils
IncrementalBackupUtil
I need to develop some services and expose an API to some third parties.
In those services I may need to fetch/insert/update/delete data with some complex calculations involved(not just simple CRUD). I am planning to use Spring and MyBatis.
But the real challenge is there will be multiple DB nodes with same data(some external setup will takes care of keeping them in sync). When I got a request for some data I need to randomly pick one DB node and query it and return the results. If the selected DB is unreachable or having some network issues or some unknown problem then I need to try to connect to some other DB node.
I am aware of Spring's AbstractRoutingDataSource. But where to inject the DB Connection Retry logic? Will Spring handle transactions properly if I switch the dataSource dynamically?
Or should I avoid Spring & MyBatis out-of-the-box integration and do Transaction management by myself using MyBatis?
What do you guys suggest?
I propose to you using of NoSQL database like MongoDB. It is easy clustering. You can configure for example use 10 servers and do replication of data 3 times.
Thats mean that if 2 of your 10 servers will fails - you still got data save.
NoSQL databases is different comparing to RDBS, but they can give hight performance for clustering.
Also, there is no transactions support for NoSQL - you have to do it manually in case of financial operations.
Actually you should thing in different way when developing with NoSQL.
Yes, it will work. Get AbstractRoutingDataSource and code your own one. The only thing you cannot do is to change the target database while a transaction is running.
So what you have to do is putting the db retry code in the getConnection. If during the transaction that connection becomes invalid you should let it fail.
I have a java application that is connected to a view on a remote Oracle db.
Does anyone know of a way in Java to monitor this table for changes? I.e. if there are inserts of updates etc I would need to react.
Look at Oracle Change Notification, a so interesting Oracle feature.
From the Oracle documentation: "Database Change Notification is a feature that enables client applications to register queries with the database and receive notifications in response to DML or DDL changes on the objects associated with the queries. The notifications are published by the database when the DML or DDL transaction commits."
You can place a INSERT/UPDATE/DELETE trigger on the table to perform some action when 'data' changes are made to the table. (as opposed to changes to the structure of the table)
I believe 10g also supports triggers on views.
but I'm not sure how you can notifiy the java process of this other then by polling.
sorry.
you could possibly create some solution where the java app has a 'listen' server and the database pushes it back a message. but that sounds hard to maintain.
Justin Cave in the comments suggests you could configure Oracle Streams to send out logical change records (LCRs) that a Java app could subscribe to via JMS. Or the trigger could write records to an Advanced Queue that Java could subscribe to via JMS.
you will still need to be wary of Oracle transations..
due to the way Oracle transactions work, the trigger will fire on the change, but it may also fire multiple times..
and in anycase the java app would not bee able to 'see' the changes until a commit had been performed.