I want to make some performance tests on my Camel app (I'm using Spring Boot for development) and build a report based on the time that each transaction took to finish, but I'm not quite sure what will be the best approach (first time doing something like this).
This app is layered in three parts: each part will publish a message to a JMS queue, so the next part can pick it up. So, I know that I can get the elapsed time for each Exchange like this:
List<MessageHistory> historyList = exchange.getProperty(Exchange.MESSAGE_HISTORY, List.class);
for (MessageHistory history : historyList) {
history.getElapsed();
}
But, as I will have three different parts inside the app I will get three different Exchanges. Each message coming inside the app will have an ID inside the payload so maybe I can group each transaction using that (unless there is a more Camel-oriented way to do it that I'm not aware of).
I can write something custom to make the report I suppose, but I'm wondering if maybe there are some frameworks that I can use.
Any recommendations are appreciated
Thanks!
I suppose you want to capture the total time it took for the message to traverse in your application. If so, you can use camel interceptors.
You can capture the starting time using the FromInterceptor, save the time in a property, and get the end time using the sendToEndPoint interceptor. Calculate the time it took for the message to travel
Related
This might seem like an easy solution got on the internet, but believe me, I have seen through a lot of examples and couldn't figure out which approach to choose.
Requirement :
I have a subscriber at the application service(spring boot/Java) end, subscribed to blockchain events( corda ). I want to push this event to UI (ReactJS) whenever there is a change in state.
I could subscribe to the blockchain events successfully but stuck with multiple in-complete or tangled ideas of pushing it to the UI and how UI would receive my events ( kindly don't suggest paid services, APIs, Libraries etc ).
I have come across and tried out all approach, since I'm newly working on events I need some ray of light as to how to approach towards a complete solution.
Publisher-subscriber pattern
Observable pattern
Sse emitter
Flux & Mono
Firebase ( a clear NO )
+Boggler :
events handling between service and UI , should it be via API/endpoint calls or can it be emitted just in air( i'm not clear) and based on event name can we subscribe to it in UI ?
should i have two APIs dedicated for this ? one trigger subscribe and other actually executes emitter ?
If the endpoint is always being heard doesn't it needs dedicated resource ?
I basically need a CLEAR approach to handle this.
Code can be provided based on demand
I see you mention you are able to capture events in Spring Boot. So you are left with sending the event information to the front-end. I could think of three ways to do this.
Websockets: Might be an over-kill, as I suppose you won't need bi-directional communication.
SEE: Perhaps a better choice than WebSockets.
Or simply Polling: Not a bad choice either, if you are not looking for realtime notifications.
Yes Long Polling.
The solution seems to be pretty simple. Make the connection once and let them wait for as long as possible. So that in the meanwhile if any new data comes to the server, the server can directly give the response back. This way we can definitely reduce the number of requests and response cycles involved.
You will find multiple implementation examples of How Long Polling is done as part of Spring Boot project on internet.
Let's say I have an API that gives me the values of stock for the last month. The data is sampled every hour.
Now I want to make a web app that would visualize this data on a line chart. I don't need all the hourly samples, so my question is how should I make this work?
My idea is that there would be a backend app (i.e. in Java Spring) that would GET the data from the API and calculate the average for each day (using a stream, maybe parallel stream?) and then put that in a new collection and pass it on to the front end to put in a chart.
Start thinking from the UI, what do you need there, how often do you need it and how fast ?
Then get the data from the backend, if there is too much data at once and the API cannot do otherwise, either:
get data and reduce to what the UI needs (backend), use once and throw away
OR get data and reduce to what the UI needs (backend), keep in cache for a while
OR pre-process the data so that when the UI needs it, it will be ready
For the return format, consider something lightweight, like some simple named json array {"dayAverages": [0.34, 1253.432, ...]}, "month" : 2, "year": 2018}, then in the UI adapt to the needs of your lib (that is debatable).
Also observe how users use the UI, then you may get some ideas on how to optimize the experience (preload next month ...)
If you do this for learning purposes, consider doing it async + lambdas = bonus :)
As to your question "...how should I make this work?" --
This is extremely broad. There are many, many ways to do this. Some of these ways depend heavily on your architecture, how much traffic is expected to your app, what request-load the API can handle, etc. Here are a few general things to consider:
Any sort of MVC architecture (or similar) would be a good fit for your Web app.
You mention needing a "backend app" of some type. Not sure what you mean here, but the averaging features can be built directly into your Web app framework without needing a separate back-end app.
If you're going to calculate averages for display in the Web app, you will need to maintain state somewhere. Assuming the API doesn't give this to you, you'll need a database of some type, or at least some type of memory caching storage engine to facilitate this. How you do this will depend on your architecture and the traffic/load on your app (e.g. will you have multiple, load-balanced servers).
Hope that helps. We could give more if you ask some specific questions.
I'm developing an application with play framework in java, in order to evaluate the performance of my application, I want to record the time on processing each http request. Based on the official document provided, I can only find one place in Global.java, where I can override the onRequest method to record the start time of a request.
Is there any plugin or hooks in this framework so I can add my code like this way?
long start = System.getsystem.currenttimemillis(); processRequest(request); long end = System.getsystem.currenttimemillis();
You can achieve this by using Filters. There is a working example which is exactly what you are looking for.
https://www.playframework.com/documentation/2.3.x/ScalaHttpFilters
recently I dove into the world of JMX, trying to instrument our applications, and expose some operations through a custom JMXClient. The work of figuring out how to instrument the classes without having to change much about our existing code is already done. I accomplished this using a DynamicMBean implementation. Specifically, I created a set of annotations, which we decorate our classes with. Then, when objects are created (or initialized if they are used as static classes), we register them with our MBeanServer through a static class, that builds a dynamicMBean for the class and registers it. This has worked out beautifully when we just use JConsole or VisualVM. We can execute operations and view the state of fields all like we should be able to. My question is more geared toward creating a semi-realtime JMXClient like JConsole.
The biggest problem I'm facing here is how to make the JMXClient report the state of fields in as close to realtime as I can reasonably get, without having to modify the instrumented libraries to push notifications (eg. in a setter method of some class, set the field, then fire off a JMX notification). We want the classes to be all but entirely unaware they are being instrumented. If you check out JConsole while inspecting an attribute, there is a refresh button at the bottom of the the screen that refreshes the attribute values. The value it displays to you is the value retrieved when that attribute was loaded into the view, and wont ever change without using the refresh button. I want this to happen on its own.
I have written a small UI which shows some data about connection states, and a few field on some instrumented classes. In order to make those values reflect the current state, I have a Thread which spins in the background. Every second or so the thread attempts to get the current values of the fields I'm interested in, then the UI gets updated as a result. I don't really like this solution very much, as its tricky to write the logic that updates the underlying models. And even trickier to update the UI in a way that doesn't cause strange bugs (using Swing).
I could also write an additional section of the JMXAgent in our application side, with a single thread that runs through the list of DynamicMBeans that have been registered, determines if the values of their attributes have change, then pushes a notification(s). This would move the notification logic out of the instrumented libraries, but still puts more load on the applications :(.
I'm just wondering if any of you have been in this position with JMX, or something else, and can guide me in the right direction for a design methodology for the JMXClient or really any other advice that could make this solution more elegant than the one I have.
Any suggestions you guys have would be appreciated.
If you don't want to change the entities then something is going to have to poll them. Either your JMXAgent or the JMX client is going to have to request the beans every so often. There is no way for you to get around this performance hit although since you are calling a bunch of gets, I don't think it's going to be very expensive. Certainly your JMXAgent would be better than the JMX client polling all of the time. But if the client is polling all of the beans anyway then the cost may be exactly the same.
You would not need to do the polling if the objects could call the agent to say that they have been changed or if they supported some sort of isDirty() method.
In our systems, we have a metrics system that the various components used. Each of the classes incremented their own metric and it was the metrics that were wired into a persister. You could request the metric values using JMX or persist them to disk or the wire. By using a Metric type, then there was separation between the entity that was doing the counting and the entities that needed access to all of the metric values.
By going to a registered Metric object type model, your GUI could then query the MetricRegistrar for all of the metrics and display them via JMX, HTML, or whatever. So your entities would just do metric.increment() or metric.set(...) and the GUI would query the metric whenever it needed the value.
Hope something here helps.
Being efficient here means staying inside the mbean server that contains the beans you're looking at. What you want is a way to convert the mbeans that don't know how to issue notifications into mbeans that do.
For watching numeric and string attributes, you can use the standard mbeans in the monitor package. Instantiate those in the mbean server that contains the beans you actually want to watch, and then set the properties appropriately. You can do this without adding code to the target because the monitor package is standard in the JVM. The monitor beans will watch the objects you select for changes and will emit change notifications only when actual changes are observed. Use setGranularityPeriod to tell the monitor beans how often to look at the target.
Once the monitor beans are in place, just register for the MonitorNotifications that will be created upon change.
not a solution per se but you can simplify your polling-event translator JMXAgent implementation using spring integration. It has something called JMX Attribute Polling Channel which seems to fulfill your need. example here
I am building an application in Java (with a jQuery frontend) that needs to talk to a third party application. it needs to update the interface every two seconds at the most.
Would it be a good idea to use comets? If so, how do they fit into the picture?
What other means/technologies can I use to make the application better?
The application will poll stock prices from a third party app, write it to a database and then push it to the front end every second, for the polling, I have a timer that runs every second to call the third party app for data, I then have to display it to the front end using JSP or something,
well at this point im not sure if I should use a servlet to write this out to the front end, what would you recommend? how should I go about it?
is there any new technology that I can use instead of servlets?
I am also using Berkeley db to store the data, do you think its a good option? what would be the drawbacks, if any for using berkeley..
im absolutely clueless so any advice will be much appreciated.
Thanks!
edit : I am planning to do this so that a deskop app constantly polls from the thrid part and writes to the database and a web app only reads and displays from the database, this will reduce the load on the web app and all it has to do is read from db.
Take a look at using a web application framework instead of Servlets - unless it's a really basic project with one screen. There are lots in the Java world unfortunately and it can be a bit of a minefield. Stick with maybe SpringMVC or Struts 2, the worst part is setting these up, but take a look at a sample application plus a tutorial or two and work from there.
http://www.springsource.org/about
http://struts.apache.org/2.x/index.html
Another option to look at is using a template framework such as Appfuse to get yourself up and running without having to integrate a lot of the framework together, see:
http://appfuse.org/display/APF/AppFuse+QuickStart
It provides you with a template to setup SpringMVC with MySQL as a database plus Spring as an POJO framework. It may be a quick way to get started and up and building a prototype.
Judging by your latency requirement of 2 seconds it would be wise to look at some sort of AJAX framework - JQuery or Prototype/Scriptaculous are both good places to start.
http://jquery.com/
http://www.prototypejs.org/
In terms of other technoloqies to make things better you will want to consider a build system, Ant/Maven are fine with Maven the slightly more complex of the two.
http://ant.apache.org/
http://maven.apache.org/download.html
Also, consider JUnit for testing the application. You might want to consider Selenium for functional testing of the front end.
http://www.junit.org
http://seleniumhq.org/
Is this really a stock trading application? Or just a stock price display application? I am asking because from your description it sounds like the latter.
How critical is it that data is polled every second? Specifically would it matter if some polls are a second or two late?
If you are building a stock trading application (where the timing is absolutely critical), or if you cannot afford to be delayed on your polling, I'd recommend you have a look at one of the Java Real Time solutions:
Sun Java Real-Time System (http://java.sun.com/javase/technologies/realtime/index.jsp)
WebSphere Real Time (http://www-01.ibm.com/software/webservers/realtime/)
Oracle JRockit Real Time (http://download.oracle.com/docs/cd/E13150_01/jrockit_jvm/jrockit/docs30/index.html)
Other than that, my only advice is that you stick to good OO design practices. For instance, use a DAO to write to your database, this way, if you find that Berkeley DB isn't quite for you, you can switch to a relational database system with relative ease. It also makes it easy for you to move on to some database partitioning solutions (e.g., Hibernate Shards) if you decide you need it.
While I may have my own technology preferences (for instance, I'd choose Spring MVC for the front end as others have mentioned, I'd try and use Hibernate for persistance), I really cannot claim that these would be better than other technologies out there. Go with something you are familiar with, if it fits the bill.
I think you should focus on your architectural design before picking technologies with a focus on scalability and extendability. Once an architectural design is in place you can look to see what's available and what you need to build, all of which should be pretty obvious.
While not directly comparable look at how Google, eBay and YouTube deal with the scalability problems they face. While a trading system won't have the issues these guys have with sheer numbers of users, you'll get similar problems with data volumes and being able to process price ticks in a timely fashion.
The LSE has getting on for 3000 names, multiply this by the 10 or so popular exchanges round the world and you've got a lot of data being updated continuously over the period each market is open. To give you an idea of what involved in capturing data from a single exchange take a look at http://kx.com/.
From a database perspective you've going to need something industrial strength that allows clustering and has reliable replication - for me this means Oracle. You also want to look at a Time-series Database Design, which in my experience is the best way to build this sort of system.
The same scaling and reliability requirements will apply to your app servers, with JBoss being the logical choice there, although I'd also consider the OSGi Spring Server (http://www.springsource.com/products/dmserver) as its lightweight nature could make it faster.
You'll also want Apache servers for load balancing and to serve static content - a quick Google will turn up stacks of information on that so I won't repeat it here.
Also forget polling, it doesn't scale. Look at using messaging and consumer processes for the cross-process communication, events and worker threads for the in-process communication. Both techniques achieve a natural load balancing effect that can be tuned by increasing the number of consumer processes or worker threads as need be.
Also a static front-end isn't going to cut the mustard, IMHO. Take a look at what's out in the market already - CNC Markets, IG Index, etc all have pretty impressive real-time trading apps.
As an aside, assuming this is a commercial project and not meaning to put a downer on the whole thing, companies like CNC Markets, IG Index, etc make their money from trading fees, the software being a means to an end, which you get access to for free simply by having an account. The other target for the trading software is commercial institutions such as the banks, investment managers, etc. I'd want a pretty watertight plan for how I was going to break into either of these markets before expending too much time and effort.
PostgreSQL is probably the right database. It's a little more enterprisy than MySQL. As for the front-end, there's lots of stuff that can go "on top" of servlets, SpringMVC, Tapestry, and so on and so forth. The actual servlet implementation will be hidden from you.
Many will suggest, and it's probably not a bad suggestion to use Spring to configure the application and to do any dependency injection.
If you're looking for something a little more lightweight, you might consider grails. It's quick to develop with and becoming mature.
Really though, it's kind of hard to recommend things without knowing what kind of "production" environment this would be. Are we talking lots of transactions? (sure, it's a stock trading program, but is it a simulation with a small number of users etc...) It's fun to suggest things, but if you're serious, I'm not sure I would start a major project like this. There are lots of ways to do this, and lots of ways to do this wrong.
Your intention is to build a web UI which shows realtime data eg: time, market data etc...
One of the technologies I have personally used is Web Firm Framework, an opensource framework under Apache License 2.0. It is a java server side framework to build web UI. For each and every tag & attribute there is a corresponding java class. We are just building the UI with Java code instead of pure HTML and JavaScript. The advantage is whatever changes we are making in the server tag & attribute objects will be reflected to the browser page without any explicit trigger from the client. In your case we can simply use ScheduledExecutorService to make data changes in the UI.
Eg:
AtomicReference<BigDecimal> oneUSDToOneGBPRef = new AtomicReference<>(new BigDecimal("0.77"));
SharedTagContent<BigDecimal> amountInBaseCurrencyUSD = new SharedTagContent<>(BigDecimal.ZERO);
Div usdToGBPDataDiv = new Div(null).give(dv -> {
//the second argument is formatter
new Span(dv).subscribeTo(amountInBaseCurrencyUSD, content -> {
BigDecimal amountInUSD = content.getContent();
if (amountInUSD != null) {
return new SharedTagContent.Content<>(amountInUSD.toPlainString(), false);
}
return new SharedTagContent.Content<>("-", false);
});
new Span(dv).give(spn -> {
new NoTag(spn, " USD to GBP: ");
});
new Span(dv).subscribeTo(amountInBaseCurrencyUSD, content -> {
BigDecimal amountInUSD = content.getContent();
if (amountInUSD != null) {
BigDecimal oneUSDToOneGBP = oneUSDToOneGBPRef.get();
BigDecimal usdToGBP = amountInUSD.multiply(oneUSDToOneGBP);
return new SharedTagContent.Content<>(usdToGBP.toPlainString(), false);
}
return new SharedTagContent.Content<>("-", false);
});
});
amountInBaseCurrencyUSD.setContent(BigDecimal.ONE);
//just to test
// will print <div><span>1</span><span> USD to GBP: </span><span>0.77</span></div>
System.out.println(usdToGBPDataDiv.toHtmlString());
ScheduledExecutorService scheduledExecutorService =
Executors.newScheduledThreadPool(1);
Runnable task = () -> {
//dynamically get USD to GBP exchange value
oneUSDToOneGBPRef.set(new BigDecimal("0.77"));
//to update latest converted value
amountInBaseCurrencyUSD.setContent(amountInBaseCurrencyUSD.getContent());
};
ScheduledFuture scheduledFuture = scheduledExecutorService.schedule(task, 1, TimeUnit.SECONDS);
//to cancel the realtime update
//scheduledFuture.cancel(false);
For displaying time in real-time you can use SharedTagContent<Date> and ContentFormatter<Date> to show time in specific timezone. You can watch this video for better understanding. You can also download sample projects from this github repository.