We Keep Coding

Computing Monitoring metrics of Spring-Boot Micro-Service in Java

Problem statement: Expose Latency, Error-Rate, Throughput, Retry-Count (For persisting into SQL DB failures and Kafka publishing failures), and Unsuccessful-Retries of the Spring Boot application as ManagedBeans on JConsole. There are no REST endpoints in this application. It is purely asynchronously event-driven via Kafka (Use of Spring StreamListener).
I cannot use Prometheus due to the licensing issue on the Production level.
My Approach: Created a MetricsBean class (implementing an interface) essentially containing the POJO for the above metrics:
import org.springframework.context.annotation.Lazy;
import org.springframework.jmx.export.annotation.ManagedAttribute;
import org.springframework.jmx.export.annotation.ManagedOperation;
import org.springframework.jmx.export.annotation.ManagedResource;
#Lazy(false)
#ManagedResource
public class ConfigMBean implements IConfigMBean {
private Double latency;
private Double errorRate;
private Double throughput;
private Long retryCount;
private Long unsuccessfulRetries;
public GatewayConfigMBean(Double latency, Double errorRate, Double throughput, Long retryCount,
Long unsuccessfulRetries) {
super();
this.latency = latency;
this.errorRate = errorRate;
this.throughput = throughput;
this.retryCount = retryCount;
this.unsuccessfulRetries = unsuccessfulRetries;
}
public GatewayConfigMBean() {
}
#ManagedAttribute
#Override
public Double getLatency() {
return latency;
}
#ManagedAttribute
#Override
public Double getErrorRate() {
return errorRate;
}
#ManagedAttribute
#Override
public Double getThroughput() {
return throughput;
}
#ManagedAttribute
#Override
public Long getRetryCount() {
return retryCount;
}
#ManagedAttribute
#Override
public Long getUnsuccessfulRetries() {
return unsuccessfulRetries;
}
#ManagedOperation
#Override
public String showMetrics() {
StringBuilder builder = new StringBuilder();
builder.append("GatewayConfigMBean [errorRate=").append(errorRate).append(", latency=").append(latency)
.append(", retryCount=").append(retryCount).append(", throughput=").append(throughput)
.append(", unsuccessfulRetries=").append(unsuccessfulRetries).append("]");
return builder.toString();
}
}
Then I exposed this Bean in the Config class:
#Bean("My_Microservice_Name")
public ConfigMBean configMBean () {
return new ConfigMBean ();
}
I was able to see My_Microservice_Name on the JConsole MBeans tab, but as expected all values were null.
Question: How to use Spring Micrometer to have correct values assigned to these metrics each? Also if I need to use some kind of AOP to get these metrics, how I need to incorporate that in my code?
I was searching on the internet and found a few hits:
import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.Metrics;
private Counter counter = Metrics.counter("ErrorCount");
counter.increment();
private Timer timer = Metrics.timer("Latency");
// Assume startTime was declared earlier
timer.record(System.nanoTime() - startTime, TimeUnit.NANOSECONDS);
But not sure how to get the other Metrics and set these values to ConfigMBean class?
Assume all dependencies are available for use in Spring.
Need help with the above queries, please mention if any other information is also required.

The best way to tackle this was to add the following dependency:
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-jmx</artifactId>
</dependency>
Using this dependency, we can have
A. private static final Counter counterErrorCount = Metrics.counter("ErrorRate");
B. private static final Counter counterThroughput = Metrics.counter("Throughput");
C. private static final Timer timerLatency = Metrics.timer("Latency");
D. private static final Counter counterRetryCount = Metrics.counter("RetryCount");
E. private static final Counter counterUnsuccessfulRetryCount = Metrics.counter("UnsuccessfulRetryCount");
Import from the following package wherever necessary: io.micrometer.core.instrument.*
For the latency part, have some logic like this:
long startTime = System.nanoTime();
try {
// Some code already here
} catch (){
// Some code already here
} finally {
timerLatency.record(System.nanoTime() - startTime, TimeUnit.NANOSECONDS);
}
And for the Counters, we need to identify where to put the incremental code, i.e. counterErrorCount.increment(); and others.
Finally, we can equate these Metrics with the above POJO class, and Metrics shall be exposed to JConsole.
Open JConsole (for the respective PID) -> MBeans tab -> metrics check-box -> Desired metrics -> Attributes.

Sending bulk mails periodically

I have a requirement where i have to send 100 mails/minute/user and it is time specific.(i.e All the mails i.e 100*1000 = 10000 mails/min should go on same time)
Currently there are almost 1000 users.
As well as for each email i am saving it first then sending it.
What things i need to implement for better performance and achieve goal optimally.
[Note: All the emails are sent via different accounts, so limit wont increase]
Any suggestion will be very helpful.
I am currently using Spring Boot for the project.

You can use Windows Service for this, if project run on windows server.

//Remember to set #EnableScheduling
//in the class containing your main method
#SpringBootApplication
#EnableScheduling
#EnableAsync
public class Application {
public static void main(String[] args) throws Exception {
SpringApplication.run(Application.class);
}
}
#Component
public class ScheduledTasks {
List<Email> listOfEmails;
int nextBatch = 50;
int curBatch = 0;
//This method will run every 15 second.
#Scheduled(fixedDelay = 15000)
public void yourMethodName() {
//This will process all of your objects all at once using treads
for(int i = curBatch; i < maxBatchSize(); i++){
listOfEmails.get(i).process();
}
nextBatch+=50;
curBatch+=50;
}
private int maxBatchSize(){
if(nextBatch < listOfEmails.size()){
return nextBatch;
} else {
return listOfEmails.size();
}
}
}
public class YourObject {
Integer someTest = 0;
#Async
public void process(Email e) {
e.send();
}
}

Spring Boot Progress report

I would like to be able to report a certain method's progress in Spring Boot. I used a seperate class which I store the current status in and return as the current view:
It looks like this:
public class SearchTableReloadState {
//STATIC STORAGE
public static long TABLE_ROW_COUNT = 0;
public static long CURRENT_OFFSET = 0;
public static long CURRENT_LIMIT = 0;
public static long DEFAULT_LIMIT = 20000;
public static void reset() {
TABLE_ROW_COUNT = 0;
CURRENT_OFFSET = 0;
CURRENT_LIMIT = DEFAULT_LIMIT;
}
public static void setDefaultLimit(long defaultLimit) {
DEFAULT_LIMIT = defaultLimit;
}
// VIEWMODEL
public long tableRowCount = 0;
public long currentOffset = 0;
public long currentLimit = 0;
public static SearchTableReloadState getState() {
SearchTableReloadState reloadState = new SearchTableReloadState();
reloadState.tableRowCount = TABLE_ROW_COUNT;
reloadState.currentOffset = CURRENT_OFFSET;
reloadState.currentLimit = CURRENT_LIMIT;
return reloadState;
}
}
And the methods:
#RequestMapping(value = {"/manage/searchtable/reload/state"}, method = RequestMethod.GET)
public #ResponseBody SearchTableReloadState searchTableReloadState() {
return SearchTableReloadState.getState();
}
#ResponseStatus(HttpStatus.OK)
#RequestMapping(value = {"/manage/searchtable/reload"}, method = RequestMethod.GET)
public void searchTableReload() throws ResourceAlreadyExistsException, ParameterMissingIdException {
SearchTableReloadState.reset();
SearchTableReloadState.TABLE_ROW_COUNT = productDataReferenceDao.countJobReferences();
productDataReferenceDao.truncateSearchTable();
while (SearchTableReloadState.CURRENT_OFFSET < SearchTableReloadState.TABLE_ROW_COUNT) {
... long running task
....
SearchTableReloadState.CURRENT_OFFSET += SearchTableReloadState.CURRENT_LIMIT;
}
}
The method with the /state would report the current state, so I could call these with Ajax on a site. Problem is, If I start the long running one, the state report request won't complete until the long running did not complete. I thought Spring uses separate threads for each request. Do I need to implement threading in Spring?
If I use the #Async annotation for the long running process, it works like I expected, but I still don't understand, why could two separate HTTP requests for a different method block each other!

If I use the #Async annotation on the method that is supposed to take a long time, the HTTP Request calling it will get a response immediately and it will run in the background and I can call the state method as I expected. Even though it is working, I still don't know why it won't work without the asynchronous execution.
If you want to use the #Async annotation, you have to put the #EnableAsync annotation on the class you used the #SpringBootApplication and/or #EnableAutoConfiguration.
I hope someone can provide a better answer later.

How to create some sort of event framework in java?

I don't have a GUI (my classes are part of a Minecraft Mod). I wanted to be able to mimic C# event framework: A class declares events and lets others subscribe to them.
My first approach was to create a class called EventArgs and then do something like this:
public class EventArgs
{
public boolean handled;
}
#FunctionalInterface
public interface IEventHandler<TEvtArgs extends EventArgs>
{
public void handle(Object source, TEvtArgs args);
}
public class Event<TEvtArgs extends EventArgs>
{
private final Object owner;
private final LinkedList<IEventHandler<TEvtArgs>> handlers = new LinkedList<>();
public Event(Object owner)
{
this.owner = owner;
}
public void subscribe(IEventHandler<TEvtArgs> handler)
{
handlers.add(handler);
}
public void unsubscribe(IEventHandler<TEvtArgs> handler)
{
while(handlers.remove(handler));
}
public void raise(TEvtArgs args)
{
for(IEventHandler<TEvtArgs> handler : handlers)
{
handler.handle(owner, args);
if(args.handled)
break;
}
}
}
Then a class would do something like this:
public class PropertyChangedEvtArgs extends EventArgs
{
public final Object oldValue;
public final Object newValue;
public PropertyChangedEvtArgs(final Object oldValue, final Object newValue)
{
this.oldValue = oldValue;
this.newValue = newValue;
}
}
public class SomeEventPublisher
{
private int property = 0;
private final Random rnd = new Random();
public final Event<PropertyChangedEvtArgs> PropertyChanged = new Event<>(this);
public void raiseEventOrNot(int value)
{
if(rnd.nextBoolean())//just to represent the fact that the event is not always raised
{
int old = property;
property = value;
PropertyChanged.raise(new PropertyChangedEvtArgs("old(" + old + ")", "new(" + value + ")"));
}
}
}
public class SomeSubscriber
{
private final SomeEventPublisher eventPublisher = new SomeEventPublisher();
public SomeSubscriber()
{
eventPublisher.PropertyChanged.subscribe(this::handlePropertyAChanges);
}
private void handlePropertyAChanges(Object source, PropertyChangedEvtArgs args)
{
System.out.println("old:" + args.oldValue);
System.out.println("new:" + args.newValue + "\n");
}
public void someMethod(int i)
{
eventPublisher.raiseEventOrNot(i);
}
}
public class Main
{
private static final SomeSubscriber subscriber = new SomeSubscriber();
public static void main(String[] args)
{
for(int i = 0; i < 10; ++i)
{
subscriber.someMethod(i);
}
}
}
The biggest problem with this naïve approach is that it breaks proper encapsullation by exposing raise as public. I can't see a way around it, and maybe my whole pattern is wrong. I would like some ideas.
There's also a related problem: I would like the events to be raised immediately after the method raising them returns. Is there a way to synchronize this using threads or some other construct? The caller code, of course, can't be involved in the task of synchronization. It has to be completely transparent to it.

The best thing to do here is to avoid implementing your own event framework in the first place, and instead rely on some existing library. Out of the box Java provides EventListener, and at a minimum you can follow the patterns documented there. Even for non-GUI applications most of this advice applies.
Going beyond the JDK Guava provides several possible options, depending on your exact use case.
The most likely candidate is EventBus, which:
allows publish-subscribe-style communication between components without requiring the components to explicitly register with one another (and thus be aware of each other).
Or ListenableFuture (and ListeningExecutorService) which:
allows you to register callbacks to be executed once [a task submitted to an Executor] is complete, or if the computation is already complete, immediately. This simple addition makes it possible to efficiently support many operations that the basic Future interface cannot support.
Or the Service API which:
represents an object with an operational state, with methods to start and stop. For example, webservers, RPC servers, and timers can implement the Service interface. Managing the state of services like these, which require proper startup and shutdown management, can be nontrivial, especially if multiple threads or scheduling is involved.
This API similarly lets you register listeners to respond to state changes in your services.
Even if none of these options directly work for your use case, take a look at Guava's source code for examples of event-driven behavior and listeners you can try to emulate.

Expose current progress of an #Asynchronous function to use in View

In my JEE6-App (running on Glassfish 3.0.1) I have an EmailEJB which has to send lots of mails. The mails are sent asynchronously, so its annotated with the new EJB3.1 #Asynchronous, letting it be run in a separate Thread. Now i want the user to be informed about the current status of the method: How many mails have already been sent?
Sending the mails asynchronously works fine, but i can't figure out how to let the progress be accessible from outside. Seems like my approach to do that is quite wrong, but somehow it has to be possible (maybe another approach). This is how my EmailEJB currently looks like (its kind of pseudo code, but explains what i want):
#Stateful
public class EmailEJB {
#Asynchronous
public Future<Integer> sendMails() {
for (int i=0; i<mails.size; i++) {
sendMail(mails[i])
// i want to return the progress without returning ;)
return new AsyncResult<Integer>(i)
}
}
}
//Just for the completeness... from outside, i'm accessing the progress like this:
Future<Integer> progress = emailEJB.sendEmails();
Integer currentvalue = progress.get();
How can i return the current progress inside my asynchronous function, without cancelling it with a return? How can i show the user the progress of a loop inside a function? Do i need another asynchronous method? Any hints?

Nobody? Ok so this is my solution. Im not sure if this is a big fat workaround or just a way to get this done.
Since an #Asynchronous method cannot access the Session context, and therefore also no Session Beans (at least i dont know how, i always got ConcurrentModificationErrors or similar ones) i created a Singleton ProgressEJB, which contains a HashMap:
#Singleton #LocalBean #Startup
public class ProgressEJB {
private HashMap<String, Integer> progressMap = new HashMap<String, Integer>
// getters and setters
}
This hashmap should map the SessionId (a String) to an Integer value (the progress 0->100). So a user session is associated with a progress.
In my EmailEJB, i'm injecting this ProgressEJB, and in my #Asynchronous method, i'm increasing the value everytime an email has been sent:
#Stateful #LocalBean
public class EmailEJB {
#Inject
private ProgressEJB progress;
// Mail-Settings
...
#Asynchronous
public void sendEmails(user:User, message:Message, sessionId:String) {
progress.progressMap.put(sessionId, 0);
for (int i=0; i<mails.size; i++) {
sendMail(mails[i])
progress.getProgressMap().put(sessionId, (i / mails.size) * 100)
}
progress.getProgressMap().remove(sessionId);
}
The sessionId comes from my Managed (Weld) Bean, when calling the function:
#SessionScoped
#Named
public class EmailManager {
#Inject
private ProgressEJB progress;
#Inject
private FacesContext facesContext;
private String sessionId;
#PostConstruct
private void setSessionId() {
this.sessionId = ((HttpSession)facesContext.getExternalContext().getSession(false)).getId();
}
public Integer getProgress() {
if (progress.getProgressMap().get(sessionId) == null)
return 100;
else
return progress.getProgressMap().get(sessionId);
}
}
Now i can access progress from EmailManager from my JSF view with Ajax Polling, telling the user how many mails already have been sent. Just tested it with 2 users, seems to work.

I also see only a #Singleton solution here.
But this imply the need of Housekeeping in ProgressEJB. E.g. some effort is needed to prune old session from Hashmap.

Another solution is described in
Is there any way to know the progress of a EJB Asynchronous process?
This solution does not need a Stateful Bean.
#Stateless
public class EmailEJB {
// Mail-Settings
...
#Asynchronous
public void sendEmails(User user, Message message, WorkContext context) {
progress.progressMap.put(sessionId, 0);
for (int i=0; i<mails.size; i++) {
sendMail(mails[i])
context.setProgress((i / mails.size) * 100)
}
context.setRunning(false);
}
}
The Context-Object, which holds the progress.
public class WorkContext {
//volatile is important!
private volatile Integer progress = 0;
private volatile boolean running = false;
// getters & setters
}
The usage is very easy.
#SessionScoped
#Named
public class EmailManager {
#Inject
private EmailEJB emailEJB;
private WorkContext workContext;
public void doStuff() {
workContext = new WorkContext();
emailEJB.sendEmails(user, message, workContext)
}
public Integer getProgress() {
return workContext.getProgress();
}
....
}