I have an application solution which is made up of a web app written in Python (using Django framework) and a Java application which runs on the server.
The web application receives data and stores it into a database queue. The Java application is then to process the received data and also store the results in a database.
My question is how can the Java application be notified that there is new data in the database? Right now, it seems like I will have to regularly poll the database for new data. Is there any way around this?
PS. I have considered running the web app using Jython and using the Observer pattern but my host does not support Servlets.
Unless the database specifically supports it, polling is the only option I know of.
However, if your concern is load on the Java server, you could have another server that does nothing but polls for changes and then notifies your Java server when changes have occurred. I don't know if that is any better than doing a simple polling from the Java server (not knowing your specific problem space and hardware constraints).
Hope that helps.
Edit: after reading your statement again, it seems like you are already doing a messaging like framework (with the queue in the java application) so the database change could simply be another message that goes into the queue. If it needs priority, you could give the messages priority marks so that they get processed when they need to be processed.
Related
We have several Java standalone applications (in form of Jar files) running on multiple servers. These applications mainly read and stream data between systems. We are using Java 8 mainly in our development. I was put in charge recently. My main function is to manage and maintain these apps.
Currently, I check these apps manually by accessing these servers, check if the app is running, and sometimes run some database queries to see if the app started pulling data. My problem is that in many cases, some of these apps fail and shutdown due to data issue or edge cases without anyone noticing. We need some monitoring and application recovery in place.
We don't have docker infrastructure in place. We plan to implement docker in the future, but for now this is not an option.
After research, the following are options I thought of or solutions I tried:
Have the apps create a socket client which sends a heartbeat to a monitoring app (which needs to be developed). I am keeping this as my last option.
I tried to use Eclipse Vertx to wrap the apps into Verticles. Then create a web view that can show me status and other info. After several tries, the apps fail to parse the data correctly (might be due to my lack of understanding to Vertx library).
Have a third party solution that does this, but I have no idea what solutions are out there. I am open for suggestions.
My requirements are:
Proper monitoring of the apps running and their status.
In case of failure, the app should start again while notifying the admin/developer.
I am willing to develop a solution or implement a third party one. I need you guidance on this.
Thank you.
You could use spring-boot-actuator (see health). It comes with a built-in endpoint that has some health checks(depending on your spring-boot project), but you can create your own as well.
Then, doing a http request to http://{host}:{port}/{context}/actuator/health (replace with yours), you could see those health checks status and also use the response status code to monitor your application.
Have you heard of Java Service Wrappers? Not a full management functionality, however it would monitor for JVM crashes and out of memory conditions and restart your application for sure. Alerting should also be possible.
There is a small comparison table here: https://yajsw.sourceforge.io/#mozTocId284533
So some basic monitoring and management is included already. If you need more, I suggest using JMX (https://www.oracle.com/java/technologies/javase/javamanagement.html) or Prometheus (https://prometheus.io/ and https://github.com/prometheus/client_java)
I am using a 3rd party (address validation) WebService in our application. The license to the WS includes only certain number of calls, exceeding which we pay more. I am trying to keep track of the usage within our application, so we can warn the users, before they exceed. Sort of like a hit counter for Web Services.
I am currently using a Static variable in the (controller) class to track it. This works, but only until the server gets restarted, at which time it resets to 0 again.
Is there a way to keep the counter running across restarts? I saw some suggestions about serializing static variable. Is this the right approach? Or should I read/write to a file/DB table every time I make the request (sounds costly).
MY webservice client will be running in an old Sybase EAServer (built around Apache Tomcat). So, I can only use Java 1.4.
Thanks for any comments or suggestions.
I would go: every WS call, update a counter on database.
If you have many users accessing the WS Client and all of them writing to simple text file, you are going to have trouble on concurrent access to the file.
If you try to use some in-memory architecture, and your app. crashes you are going to lose the count information.
So use a database.
I have a requirement where a Groovy Application is supposed to send event notifications to another Java Web Application Which will than display that data on
web interface.
I don't want to use Queues like ActiveMQ or RabbitMQ because this will introduce an extra layer and will be used if no other solution exists.
An idea have been shared with me that I should expose a web-service from my Java application, which will be consumed by the first application, and the data
sent to the web-service will be then received in second application and somehow displayed on it's web interface.
I am not sure how this will work i.e how the data which is received in web-service of second application will be displayed on its web interface.
Kindly help me to figure out the right solution for this task.
Your problem actually is "how to send notifications from server to browser/mobile client issued by another application".
If you have very strict requirements for latency, then I would suggest to use https://github.com/OpenHFT/Chronicle-Queue
It was created by HFT guys to process 6 millions of messages per second in a single thread.
To display events on user's screen please consider using mechanisms like WebSockets, Server Sent Events, Push Notifications, Long polling, whatsoever depending on your requirements ( like browser support ).
Actually in most cases it doesn't matter what transport are you using. Unless you have super strict non-functional requirements like sub microsecond latency you're free to choose any mechanism, e.g. HTTP, JMS.
Try not to over engineer and design your software based on your actual requirements - not on stackoverflow answers.
Cheers!
I would suggest you create an XML representation of the data you wish to transfer to the java web app. On the java web app if using simple servlets, create a new servlet to which you can post this xml. The servlet could then persist this to a database. This can then be retrieved when a user logs in to the web app at some point.
Let me know if you need any more help. I could only answer only so much based on the question. Some more light on the framework the java web app is using and the data you wish to transfer, might make it easier to add more info.
Here's what i need.. I have a UI where a user has the capability to upload a file and extract a report based on the inputted(uploaded) data. Since there is a huge data to be extracted, once the user uploads the data i would like to come out of the servlet control so that user doesn't have to wait in the same page and that the control to be passed on to a java stand alone program there by making it possible for the user to work on something else. So once the control goes on to the java standalone,it would invoke back-end sps and build an extract out of it and place it in a file path on the server.
The user how-over has a capability from UI to check if the extract is ready for them to download.
So the question here is, what is the best practice or possibility in achieving the same? Please let me know your valuable comments.
Thanks!
If you're running in a Java EE environment I would suggest having the servlet dispatch the task to a JMS queue and use a message driven bean to do the (async) processing.
As others suggest, it would be fairly trivial to have the upload servlet redirect the user to some ajax-enabled page that polls the backend for job completion.
If you're not in an EE environment, you could create a standalone (thread pooled) application to consume from the queue and provide signalling eg. through the database (I assume the result goes in a DB anyway). The Spring framework provides very capable and extensive facilities for binding it all together.
But really, there are several free/open source EE containers available, from light weight up to enterprise, so there's no need to build the necessary stuff yourself.
Cheers,
Its very easy.
Have one thread in your servlet class.
Run the thread (Thread will extract the data etc).
After running the thread redirect user to a page where you have auto-refresh or something to show how much extraction is done.(You mentioned that you have a way to find it)
If you can't use message driven beans, you could have your servlet upload the data to a location on the filesystem and record a row in a DB table to say there's a job to be processed.
Then you have your standalone program polling for jobs, processing the data and updating the DB row on completion (including reasons for failure etc.).
Finally, you can poll the status of the job from the UI using an ajax request.
Allows the user to build up a queue of data jobs to be processed while they're doing something else.
My initial approach to this was to use a stored procedure in database of Application A, triggered on an insert to gather additional data and call a web-service, hosted by Application B to do the necessary mapping and persistence there. Application A and Application B may not be on the same machine. The initial requirement is to support a SQL Server database on Application A's side. A CLR stored procedure came to mind. However, it was felt that calling out to a web-service would have both severe performance implications in the SQL Server engine and also require elevation of permissions for the procedure that DBA's do not like to give.
I am now thinking in terms of the procedure creating some form of reference table on database A and a polling application consuming this data and cleaning up once processed at Application B. However, I cannot but think that there is a better way of doing this beside polling for the data.
Application A is Windows only. Application B could be Windows, UNIX, or LINUX, so Java would be the obvious choice on this side.
You've given us a list of approaches you've considered but you haven't really outlined exactly what it is you are trying to accomplish, what your goals are, etc., other than the one-sentence title of this question. Can you clarify exactly what your requirements are?
The standard answer for asynchronous messaging between applications is to use JMS. Application A places messages onto a queue whenever events that should be audited occur within it, and Application B is written to consume messages from the queue at a certain rate (if you want "realtime", then you can poll the queue very often). Application B can then do whatever it needs to with these messages - write them to a database, sent them to another web service etc.
This way, the actions in Application A - what you want to audit - and the behavior of Application B - how you want to audit the messages - are completely de-coupled from one another. This allows you to change things on either side - audit new types of events, change the payload of the message, output the messages to somewhere else, etc. - without changing the other side.
It also allows you to scale both applications independently of the other - you can add more instances of A without affecting B, A can produce messages at a much higher rate than B consumes them, and A does not wait for B to finish consuming a message before it is able to respond to the user's actions.