In my application (app A) I have many trades. For each of these trades I need to fetch the clearing status from a second application (app B) database.
Currently, I have a Spring Boot Java Application with a Scheduled component that queries app B database every 10 minutes to check for all the cleared trades to then match with app A records and update accordingly if not already updated.
While I am querying app B database looking at only today's updates to minimize the dataset, it is expected to grow.
What I would like to do?
I would like to avoid retrieving the same set of records throughout the day every 10 minutes from app B database.
Is there a clean approach to solving this problem?
Instead of looking on app A for uncleared records and then going to app B database to query if trades cleared, I want to pick up the events in app B that are new and update accordingly in app A.
What would you suggest? Are there any tools I can use?
Ideally the owner of the application that has the App B database would expose an API for you to retrieve these trades rather than having you connect to their database directly or they would publish them to a message queue for you to consume from. These are standard patterns that are used in trading environments.
Generally it isn't a good idea to integrate directly with another applications database and polling is usually used as an integration of last resort.
Related
My scenario is the following: I have two applications, one allows the user to interact with the product catalogue (only GET API, totally passive) and the other one allows admins to create/modify/delete (crud API) products from catalogue.
In order to speed up the user application, I have been thinking about implement Spring Cache. The problem is that if an admin does any interaction with the database (Oracle19c), the app for the citizen does not detect anything.
How can I solve this problem?
In the past, I managed something like this with the Change Streams, by using Mongo or thanks to Spring Data Events, so that any operation to database could be perceived.
I would need to detect operation made on database to empty and reload my cache with always last updates and I do not if is possible.
Any advice?
The Architecture I am working on today consists of 2 instances of the same springboot app connected to a single datasource i.e PostgreSQL Database.
For all database queries I rely heavily on Spring Data JPA. I use the JpaRepository Interface to perform actions like findById , save etc.
The Spring Boot application mostly behaves like an Events ingestor, whose primary task is to take in requests and make updates in the database.
The Load balancer directs requests alternatively two each application server.
It is highly likely that 2 or more incoming concurrent requests need to access the same row/entity in the Database.
Today, even though we say repository.saveAndFlush(), we observe that the final save happened with a stale entity i.e some columns are not updated with the info from previous incoming requests.
Can someone point me in the right direction with the best design and spring data features to avoid such inconsistent states in the DB ?
In my java/spring application a database record is fetched at the server init and is stored as a static field. Currently we do a mbean refresh to refresh the database values across all instances. Is there any other way to programatically refresh the database value across all the instances of the server? I am reading about EntityManager refresh.Will that work across all instances?Any help would be greatly appreciated.
You could schedule a reload every 5 minutes for example.
Or you could send events and all instance react to that event.
Till now, Communication between databases and servers is one-sided i.e. app server requests for data from the database. This generally results in the problem, and as you mentioned, that all application servers cannot know about a database change if an application is being run in cluster mode.
The current solution includes refreshing the fields time-to-time (A poll based technique).
To make this a push based model, We can create wrapper APIs over databases and let those wrapper APIs pass on the change to all the application servers.
By this I mean, Do not directly update database values from one application server but instead, on an update request send this change request to another application which keeps track of your application servers and pushes an event (via API call or queues) for a refresh of the passed database table.
Luckily, if you are using some new database (like MongoDB), they provide this update push to app servers out of the box now.
So i have a problem. Currently my application connects with single database and supports multi user. So for different landscapes, we deploy different application all together.
I need a solution that my application remains the same (single WAR deployment) but is able to connect to different DB across different landscapes.
For example a user in UK is using same application but underlying DB is in UK and subsequently another user logs in from Bangladesh and he sees the data of DB schema for Bangladesh and so on.
Currently we are creating JDBC connections in a connection pool created out of java and taking the same through out the application. Also we load static datas in hashmaps during the start up of server. But the same would not be possible with multiple DB since one would overwrite the other static data.
I have been scratching here and there , If some one can point me in the right direction, would be grateful.
You have to understand that your application start up and a user's geography are not connected attributes. You simply need to switch / pick correct DB connection while doing CRUD operations for a user of a particular geography.
So in my opinion, your app's memory requirement is going to be bigger now ( than previous ) but rest of set up would be simple.
At app start up, You need to initialize DB Connection pools for all databases and load static data for all geographies and then use / pick connection & static data as per logged in user's geography.
There might be multiple ways to implement this switching / choosing logic and this very much dependent on what frameworks & libraries you are using.
We are using spring as back end process, hibernate as dao layer and maven as build tool for the project and data tables as the front end data display as a dashboard. Dashboard has almost 30 columns and 25 of them are editable by selected users who has admin rights.
Let say 5 users are viewing the Dashboard at the same time and one user change the data in some column then how we push updated data to other 4 users who are viewing same data live. In other words, how we push updated or changed data to all other live users if one live user changes something.
Have a look to Websocket or server side event.
You can also implement your own mechanism. Create an URL endpoint where javascript clients connect regulary to check for updates. The idea is to have a service exposing updates to clients each time a data is updated in the database.
With the release of Spring 4, Spring now supports WebSockets and actually make them easy to use. To get your hands dirty check out this tutorial.
An older solution that is fairly common is Comet.