Currently running SpringBoot applications in a containerised environment (ECS) and I've observed scenarios in which the container gets terminated during start-up and while it's still holding the Liquibase changelock.
This leads to issues in all containers that are spun afterwards and ends up requiring manual intervention.
Is it possible to ensure that if the process receives a SIGTERM, it will gracefully handle termination and release the lock?
I've already ensured that the container is receiving the signals by enabling via InitProcessEnabled (in the CloudFormation template) and use of "exec java ..." as a java agent we use does gracefully shutdown on this circumstances.
Heyo,
As mentioned in the GitHub issue I have a workaround. A solution is yet to be implemented.
You can manually register a shutdown hook before running spring boot.. That hook should assure that the Termination is postponed until liquibase is done.
package dang;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
#EnableJpaRepositories
#SpringBootApplication
public class DangApplication {
public static void main(String[] args) throws InterruptedException {
Thread thread = new GracefulShutdownHook();
Runtime.getRuntime().addShutdownHook(thread);
new SpringApplicationBuilder(DangApplication.class)
.registerShutdownHook(true)
.logStartupInfo(true)
.build()
.run();
Runtime.getRuntime().removeShutdownHook(thread);
}
}
And the hook:
package dang;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import java.util.Map;
#Slf4j
public class GracefulShutdownHook extends Thread {
#SneakyThrows
#Override
public void run() {
super.run();
log.info("Shutdown Signal received.. Searching for Liquibase instances!");
boolean liquibaseIsRunning = true;
while (liquibaseIsRunning) {
Map<Thread,StackTraceElement[]> stackTraces = Thread.getAllStackTraces();
for(Map.Entry<Thread, StackTraceElement[]> entry : stackTraces.entrySet()) {
StackTraceElement[] stackTraceElements = entry.getValue();
for (StackTraceElement stackTraceElement : stackTraceElements) {
if (stackTraceElement.getClassName().contains("liquibase") && stackTraceElement.getMethodName().contains("update")) {
try {
log.warn("Liquibase is currently updating");
entry.getKey().join();
liquibaseIsRunning = false;
} catch (InterruptedException e) {
log.error("Shutdown Hook was interrupted.. Fatal databaselock may be imminent", e);
if (Thread.interrupted()) {
throw e;
}
}
}
}
}
}
}
}
EDIT
After implementing my workaround a contributor of liquibase shared a different solution (It's actually the same solution just through Spring functionality) which is much better than what I did:
package dang;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
#EnableJpaRepositories
#SpringBootApplication
public class DangApplication {
public static void main(String[] args) throws InterruptedException {
new SpringApplicationBuilder(DangApplication.class)
.initializers(ConfigurableApplicationContext::registerShutdownHook) // Registers application hook before liquibase executes.
.logStartupInfo(true)
.build()
.run();
}
}
Related
My Spring Boot application is not a web server, but it's a server using custom protocol (using Camel in this case).
But Spring Boot immediately stops (gracefully) after started. How do I prevent this?
I'd like the app to stop if Ctrl+C or programmatically.
#CompileStatic
#Configuration
class CamelConfig {
#Bean
CamelContextFactoryBean camelContext() {
final camelContextFactory = new CamelContextFactoryBean()
camelContextFactory.id = 'camelContext'
camelContextFactory
}
}
I found the solution, using org.springframework.boot.CommandLineRunner + Thread.currentThread().join(), e.g.:
(note: code below is in Groovy, not Java)
package id.ac.itb.lumen.social
import org.slf4j.LoggerFactory
import org.springframework.boot.CommandLineRunner
import org.springframework.boot.SpringApplication
import org.springframework.boot.autoconfigure.SpringBootApplication
#SpringBootApplication
class LumenSocialApplication implements CommandLineRunner {
private static final log = LoggerFactory.getLogger(LumenSocialApplication.class)
static void main(String[] args) {
SpringApplication.run LumenSocialApplication, args
}
#Override
void run(String... args) throws Exception {
log.info('Joining thread, you can press Ctrl+C to shutdown application')
Thread.currentThread().join()
}
}
As of Apache Camel 2.17 there is a cleaner answer. To quote http://camel.apache.org/spring-boot.html:
To keep the main thread blocked so that Camel stays up, either include the spring-boot-starter-web dependency, or add camel.springboot.main-run-controller=true to your application.properties or application.yml file.
You will want the following dependency too:
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
<version>2.17.0</version>
</dependency>
Clearly replace <version>2.17.0</version> or use the camel BOM to import dependency-management information for consistency.
An example implementation using a CountDownLatch:
#Bean
public CountDownLatch closeLatch() {
return new CountDownLatch(1);
}
public static void main(String... args) throws InterruptedException {
ApplicationContext ctx = SpringApplication.run(MyApp.class, args);
final CountDownLatch closeLatch = ctx.getBean(CountDownLatch.class);
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
closeLatch.countDown();
}
});
closeLatch.await();
}
Now to stop your application, you can look up the process ID and issue a kill command from the console:
kill <PID>
Spring Boot leaves the task of running the application to the protocol around which the application is implemented. See, for example, this guide:
Also required are some housekeeping objects like a CountDownLatch to keep the main thread alive...
So the way of running a Camel service, for example, would to be to run Camel as a standalone application from your main Spring Boot application class.
This is now made even simpler.
Just add camel.springboot.main-run-controller=true to your application.properties
All threads are completed, the program will close automatically.
So, register an empty task with #Scheduled will create a loop thread to prevent shutdown.
file application.yml
spring:
main:
web-application-type: none
file DemoApplication.java
#SpringBootApplication
#EnableScheduling
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
}
file KeepAlive.java
#Component
public class KeepAlive {
private static final Logger log = LoggerFactory.getLogger(ScheduledTasks.class);
private static final SimpleDateFormat dateFormat = new SimpleDateFormat("HH:mm:ss");
#Scheduled(fixedRate = 1 * 1000 * 60) // 1 minute
public void reportCurrentTime() {
log.info("Keepalive at time {}", dateFormat.format(new Date()));
}
}
My project is NON WEB Spirng Boot.
My elegant solution is create a daemon thread by CommandLineRunner.
Then, Application do not shutdown immediately.
#Bean
public CommandLineRunner deQueue() {
return args -> {
Thread daemonThread;
consumer.connect(3);
daemonThread = new Thread(() -> {
try {
consumer.work();
} catch (InterruptedException e) {
logger.info("daemon thread is interrupted", e);
}
});
daemonThread.setDaemon(true);
daemonThread.start();
};
}
To keep the java process alive when not deploying a web application set the webEnvironment property to false like so:
SpringApplication sa = new SpringApplication();
sa.setWebEnvironment(false); //important
ApplicationContext ctx = sa.run(ApplicationMain.class, args);
for springboot app to run continously it has to be run in a container, otherwise it is just like any java app all threads are done it finishes,
you can add
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
and it will turn it into webapp, if not you are responsible keeping it alive in your implementation
I am working on a project that is basically a lot of processes that run periodically. Each process is a different class that extends an abstract class RunnableProcess we created, which contains a private attribute Map<String, String> result and the abstract method run with the signature below:
public abstract void run(Map processContext) throws IOException;
To improve modularization on the project, I'm starting to use Aspect Oriented Programming (AOP) to intercept the run calls from every RunnableProcess. I am still learning AOP, and I have the following code until now:
import static org.slf4j.LoggerFactory.getLogger;
import org.slf4j.Logger;
import process.RunnableProcess;
import java.util.Map;
public aspect ProcessRunInterceptor {
private Logger logger;
pointcut runProcess() : call(void RunnableProcess.run(Map));
after(): runProcess() {
logger = getLogger(thisJoinPoint.getClass());
logger.info("process run successfully");
}
}
It is working, but I want to log more information than just "process run successfully". I have this information inside the intercepted class, in the result attribute mentioned above. Is it possible to access it in the advice without changing the implementation of RunnableProcess?
I can (prefer not to, but if it would be the only choice...) change the attribute from private to protected, but I wouldn't change it to public. I also would not like to create a get method for it.
Building upon my answer to your other question, I will explain to you what you can do. A few hints:
Instead of before() and after() you could just use around().
You should not use a private member for the process instance in a singleton aspect because if you have asynchronous processes in multiple threads, the member could be overwritten. Thus, your approach is not thread-safe and you should use a local variable instead.
You should not print "process run successfully" in an after() advice because after() also runs after an exception. So you cannot safely assume that the process ran successfully, only that it ran at all. You should rather write "finished process" or similar. BTW, if you want to differentiate between successful processes and such ending with an exception, you might want to look into pointcut types after() returning and after() throwing().
It does not make sense to use an abstract base class and not define the member result there directly. Why add it as a private member in each subclass if you can have it as a protected member in the parent? We are still doing OOP here (beside AOP, of course), right? The advantage is that you can access the member directly from the aspect using the base class like you already do in your pointcut.
Here is an MCVE for you:
Process classes:
package de.scrum_master.app;
import java.io.IOException;
import java.util.Map;
public abstract class RunnableProcess {
protected String result = "foo";
public abstract void run(Map processContext) throws IOException;
}
package de.scrum_master.app;
import java.io.IOException;
import java.util.Map;
public class FirstRunnableProcess extends RunnableProcess {
#Override
public void run(Map processContext) throws IOException {
System.out.println("I am #1");
result = "first";
}
}
package de.scrum_master.app;
import java.io.IOException;
import java.util.Map;
public class SecondRunnableProcess extends RunnableProcess {
#Override
public void run(Map processContext) throws IOException {
System.out.println("I am #2");
result = "second";
}
}
Driver application:
package de.scrum_master.app;
import java.io.IOException;
public class Application {
public static void main(String[] args) throws IOException {
new FirstRunnableProcess().run(null);
new SecondRunnableProcess().run(null);
}
}
Aspect:
Here you just bind the target() object to a parameter in the pointcut and use it in both advices.
package de.scrum_master.aspect;
import static org.slf4j.LoggerFactory.getLogger;
import org.slf4j.Logger;
import de.scrum_master.app.RunnableProcess;
import java.util.Map;
public privileged aspect ProcessRunInterceptorProtocol {
pointcut runProcess(RunnableProcess process) :
call(void RunnableProcess.run(Map)) && target(process);
before(RunnableProcess process): runProcess(process) {
Logger logger = getLogger(process.getClass());
logger.info("logger = " + logger);
logger.info("running process = " + thisJoinPoint);
}
after(RunnableProcess process): runProcess(process) {
Logger logger = getLogger(process.getClass());
logger.info("finished process = " + thisJoinPoint);
logger.info("result = " + process.result);
}
}
Console log with JDK logging (some noise removed):
INFORMATION: logger = org.slf4j.impl.JDK14LoggerAdapter(de.scrum_master.app.FirstRunnableProcess)
INFORMATION: running process = call(void de.scrum_master.app.FirstRunnableProcess.run(Map))
I am #1
INFORMATION: finished process = call(void de.scrum_master.app.FirstRunnableProcess.run(Map))
INFORMATION: result = first
INFORMATION: logger = org.slf4j.impl.JDK14LoggerAdapter(de.scrum_master.app.SecondRunnableProcess)
INFORMATION: running process = call(void de.scrum_master.app.SecondRunnableProcess.run(Map))
I am #2
INFORMATION: finished process = call(void de.scrum_master.app.SecondRunnableProcess.run(Map))
INFORMATION: result = second
I am trying to hook into the creation of the context using a custom application listener like this
#Component
public class ContextStartedListener implements ApplicationListener<ContextStartedEvent> {
#Override
public void onApplicationEvent(ContextStartedEvent event) {
System.out.println("Context started"); // this never happens
}
}
But the onApplicationEvent method never fires. If I use a different event such as ContextRefreshedEvent then it works just fine, but I need to hook into before it is created. Any advice? Thanks!
[Edit]
Editing answer adding more info because of the downvote.
The reason why you are not getting a callback by the listener is because you are not explicitly calling the LifeCycle start() method (JavaDoc).
This cascades down to your ApplicationContext normally via the AbstractApplicationContext on in Spring Boot case via the ConfigurableApplicationContext.
Example of working code below demonstrating how your callback would work (just explicitly call the start() method)
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationListener;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.event.ContextStartedEvent;
import org.springframework.stereotype.Component;
#SpringBootApplication
public class DemoApplication {
public static void main(String[] args) {
ConfigurableApplicationContext applicationContext = SpringApplication.run(DemoApplication.class, args);
applicationContext.start();
}
#Component
class ContextStartedListener implements ApplicationListener<ContextStartedEvent> {
#Override
public void onApplicationEvent(ContextStartedEvent event) {
System.out.println("Context started");
}
}
}
The reason why I suggested below the ContextRefreshedEvent callback instead is because behind the scenes the refresh() code is getting invoked.
If you drill down the SpringApplication#run() method you'll eventually see it.
Again here's a working example of how this would work using the ContextRefreshedEvent:
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationListener;
import org.springframework.context.event.ContextRefreshedEvent;
import org.springframework.stereotype.Component;
#SpringBootApplication
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
#Component
class ContextStartedListener implements ApplicationListener<ContextRefreshedEvent> {
#Override
public void onApplicationEvent(ContextRefreshedEvent event) {
System.out.println("Context refreshed");
}
}
}
[Before Edit]
Change the Generic type to ContextRefreshedEvent instead and then it should work.
For more details read this article from the Spring Blog. Just to quote the part about the ContextRefreshedEvent:
[..]This allows MyListener to be notified when the context has refreshed
and one can use that to run arbitrary code when the application
context has fully started.[..]
I created a spring boot app which has a scheduler which runs every X seconds
Main Class :
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableScheduling;
#SpringBootApplication
#EnableScheduling
public class FiobenchmarkApplication {
public static void main(String[] args) {
SpringApplication.run(FiobenchmarkApplication.class, args);
}
}
My Spring Scheduler class is as below
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
#Component
public class Scheduler {
#Scheduled(fixedRate=1000)
public void getFioResult(){
System.out.println("hello class");
}
}
After gradle clean build I get a jar named:fiobench.jar
The application runs perfectly and calls the scheduler every X seconds when I run on my local machine(Which is ubuntu)
When the jar is transfered to another machine (Centos 7 Minimal and both ubutnu and centos have same Java versions ) .
The scheduler is called only once at the start and is never called after it calls one time .
Has any one faced any such issue with spring scheduler.
Also When I try to replicate the same using normal java multi threading it doesnt work on centos minimal while it works on my ubuntu machine
Below is my pure java implementation of thread
public class Scheduler extends Thread implements Runnable{
//#Scheduled(fixedRate=1000)
public void getFioResult(){
}
#Override
public void run() {
// TODO Auto-generated method stub
while(true){
try {
getFioResult();
Thread.sleep(TimeUnit.SECONDS.toMillis(10));
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
And my main class is
public class FiobenchmarkApplication {
public static void main(String[] args) {
Scheduler s=new Scheduler();
s.start();
}
}
How can I register/add a custom shutdown routine that shall fire when my Spring Boot application shuts down?
Scenario: I deploy my Spring Boot application to a Jetty servlet container (i.e., no embedded Jetty). My application uses Logback for logging, and I want to change logging levels during runtime using Logback's MBean JMX configurator. Its documentation states that to avoid memory leaks, on shutdown a specific LoggerContext shutdown method has to be called.
What are good ways to listen on Spring Boot shutdown events?
I have tried:
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext cac = SpringApplication.run(Example.class, args);
cac.addApplicationListener(new ApplicationListener<ContextClosedEvent>() {
#Override
public void onApplicationEvent(ContextClosedEvent event) {
logger.info("Do something");
}
});
}
but this registered listener does not get called when the application shuts down.
https://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#features.spring-application.application-exit
Each SpringApplication will register a shutdown hook with the JVM to ensure that the ApplicationContext is closed gracefully on exit. All the standard Spring lifecycle callbacks (such as the DisposableBean interface, or the #PreDestroy annotation) can be used.
In addition, beans may implement the org.springframework.boot.ExitCodeGenerator interface if they wish to return a specific exit code when the application ends.
have you tried this as mentioned by #cfrick ?
#SpringBootApplication
#Slf4j
public class SpringBootShutdownHookApplication {
public static void main(String[] args) {
SpringApplication.run(SpringBootShutdownHookApplication.class, args);
}
#PreDestroy
public void onExit() {
log.info("###STOPing###");
try {
Thread.sleep(5 * 1000);
} catch (InterruptedException e) {
log.error("", e);;
}
log.info("###STOP FROM THE LIFECYCLE###");
}
}
Your listener is registered too late (that line will never be reached until the context has already closed). It should suffice to make it a #Bean.
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.servlet.ServletListenerRegistrationBean;
import org.springframework.boot.web.support.SpringBootServletInitializer;
import org.springframework.context.annotation.Bean;
#SpringBootApplication
#EnableAutoConfiguration
public class Application extends SpringBootServletInitializer {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#NotNull
#Bean
ServletListenerRegistrationBean<ServletContextListener> myServletListener() {
ServletListenerRegistrationBean<ServletContextListener> srb =
new ServletListenerRegistrationBean<>();
srb.setListener(new ExampleServletContextListener());
return srb;
}
}
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
public class ExampleServletContextListener implements ServletContextListener {
#Override
public void contextInitialized(
ServletContextEvent sce) {
// Context Initialised
}
#Override
public void contextDestroyed(
ServletContextEvent sce) {
// Here - what you want to do that context shutdown
}
}
I have a similar use case, where I have to hold the server's shutdown process for some minutes, I have used the same approach mentioned in the question, the only change is instead of adding the listener after booting the service, I have added the listener (ContextClosedEvent) before running the application
#SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication application = new SpringApplication(Application.class);
application.addListeners((ApplicationListener<ContextClosedEvent>) event -> {
log.info("Shutdown process initiated...");
try {
Thread.sleep(TimeUnit.MINUTES.toMillis(5));
} catch (InterruptedException e) {
log.error("Exception is thrown during the ContextClosedEvent", e);
}
log.info("Graceful Shutdown is processed successfully");
});
application.run(args);
}
}