My Spring-Boot Application is running on Kubernetes.
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.12</version>
</parent>
According to Spring-Boot reference, livenessProbe and readinessProbe would be enabled automatically.
https://docs.spring.io/spring-boot/docs/2.5.14/reference/htmlsingle/#actuator.endpoints.kubernetes-probes
These health groups are only enabled automatically if the application is running in a Kubernetes environment.
They will also be exposed as separate HTTP Probes using Health Groups: /actuator/health/liveness and /actuator/health/readiness.
But the endpoints (/actuator/health/liveness and /actuator/health/readiness) in My Spring-Boot Application returns 404, what's wrong here?
The component of livenessState and readinessState is enabled, but the health group is not.
I debug the code and find that the method org.springframework.boot.actuate.autoconfigure.health.HealthEndpointConfiguration.HealthEndpointGroupsBeanPostProcessor#postProcessAfterInitialization doesn't work. Because there's no bean instanceof HealthEndpointGroups, Why?
/**
* {#link BeanPostProcessor} to invoke {#link HealthEndpointGroupsPostProcessor}
* beans.
*/
static class HealthEndpointGroupsBeanPostProcessor implements BeanPostProcessor {
private final ObjectProvider<HealthEndpointGroupsPostProcessor> postProcessors;
HealthEndpointGroupsBeanPostProcessor(ObjectProvider<HealthEndpointGroupsPostProcessor> postProcessors) {
this.postProcessors = postProcessors;
}
#Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
// there's no bean instanceof HealthEndpointGroups here, Why?
if (bean instanceof HealthEndpointGroups) {
return applyPostProcessors((HealthEndpointGroups) bean);
}
return bean;
}
private Object applyPostProcessors(HealthEndpointGroups bean) {
for (HealthEndpointGroupsPostProcessor postProcessor : this.postProcessors.orderedStream()
.toArray(HealthEndpointGroupsPostProcessor[]::new)) {
bean = postProcessor.postProcessHealthEndpointGroups(bean);
}
return bean;
}
}
I try to run the application locally with adding KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_HOST environment variables, to enable probes. But it didn't work as well.
/**
* {#link SpringBootCondition} to enable or disable probes.
* <p>
* Probes are enabled if the dedicated configuration property is enabled or if the
* Kubernetes cloud environment is detected/enforced.
*/
static class ProbesCondition extends SpringBootCondition {
private static final String ENABLED_PROPERTY = "management.endpoint.health.probes.enabled";
private static final String DEPRECATED_ENABLED_PROPERTY = "management.health.probes.enabled";
#Override
public ConditionOutcome getMatchOutcome(ConditionContext context, AnnotatedTypeMetadata metadata) {
Environment environment = context.getEnvironment();
ConditionMessage.Builder message = ConditionMessage.forCondition("Probes availability");
ConditionOutcome outcome = onProperty(environment, message, ENABLED_PROPERTY);
if (outcome != null) {
return outcome;
}
outcome = onProperty(environment, message, DEPRECATED_ENABLED_PROPERTY);
if (outcome != null) {
return outcome;
}
if (CloudPlatform.getActive(environment) == CloudPlatform.KUBERNETES) {
return ConditionOutcome.match(message.because("running on Kubernetes"));
}
return ConditionOutcome.noMatch(message.because("not running on a supported cloud platform"));
}
private ConditionOutcome onProperty(Environment environment, ConditionMessage.Builder message,
String propertyName) {
String enabled = environment.getProperty(propertyName);
if (enabled != null) {
boolean match = !"false".equalsIgnoreCase(enabled);
return new ConditionOutcome(match, message.because("'" + propertyName + "' set to '" + enabled + "'"));
}
return null;
}
}
I run application on Kubernetes, but it doesn't work for me. I find that removing dependency of Spring Cloud Sleuth, it would run normally. Is there any conflict between SpringBoot and Spring Cloud Sleuth?
Removing the follow dependency
<!--traceId -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-sleuth-zipkin</artifactId>
</dependency>
yaml:
management:
server:
port: 9235
endpoints:
web:
exposure:
include: ${MANAGEMENT_ENDPOINTS_WEB_EXPOSURE_INCLUDE:*}
k8s:
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: {{ .Values.deployment.managementPort }}
failureThreshold: 3
initialDelaySeconds: 150
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: {{ .Values.deployment.managementPort }}
failureThreshold: 3
initialDelaySeconds: 180
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
AFAIK Health groups are automatically enabled only if the application met below 2 conditions:
1)Runs in a Kubernetes deployment environment
Spring Boot auto-detects Kubernetes deployment environments by checking the environment for "*_SERVICE_HOST" and "*_SERVICE_PORT" variables. You can override this detection with the spring.main.cloud-platform configuration property. Spring Boot helps you to manage the state of your application and export it with HTTP Kubernetes Probes using Actuator.
2)The management.health.probes.enabled property is set to true
If the application meets either of the above conditions, then auto-configuration registers.
Refer to Baeldung, Liveness and Readiness Probes in Spring Boot, Auto-Configurations by Mona Mohamadinia, for more information.
Also refer to github link, Kubernetes readiness probe endpoint returning 404 #22562 for more information.
Related
I'm try reproduce the follow code Deploying a Camel Route in AWS Lambda : A Camel Quarkus example in my own code Quarkus Camel AWS Lambda, but the ProducerTemplate returns NullPointerExcetion, as can see in this link BUG_CAMEL_QUARKUS_LAMBDA
#Named("languageScoreLambda")
public class LanguageScoreLambda implements RequestHandler<Language, LanguageScoreDto> {
#Inject
ProducerTemplate template;
#Override
public LanguageScoreDto handleRequest(Language input, Context context) {
System.out.println("#Template isNull ===> " + (null == template)); // true
return new LanguageScoreDto("5", input.getLanguage());
}
}
I Found the issue, as I've been using Terraform to provide the AWS Lambda function the handler MUST BE io.quarkus.amazon.lambda.runtime.QuarkusStreamHandler::handleRequest
bellow the raw code snippted
resource "aws_lambda_function" "hello_lambda" {
function_name = var.AWS_LAMBDA_FUNCTION_NAME
filename = "${path.module}/function.zip"
role = aws_iam_role.hello_lambda_role.arn
depends_on = [aws_cloudwatch_log_group.hello_lambda_logging]
runtime = "java11"
handler = io.quarkus.amazon.lambda.runtime.QuarkusStreamHandler::handleRequest
timeout = 10
memory_size = 256
}
I have a simple Eureka Server, Config Server, Zuul Gateway, and a test service(named aService below) registed in eureka.
Besides, a implemention of FallbackProvider is registed and timeoutInMilliseconds for default command is 10000.
I send a request to aService, in which will sleep 15 seconds and print tick per second.After 10 seconds a HystrixTimeoutException occured and my custom fallbackResponse accessed, but the tick still go on until 15 seconds end.
My question is, abviously, why is the request not interrupted?Could someone please explain what hystrix and zuul do after HystrixTimeout?
Dependency version:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-commons-dependencies</artifactId>
<version>Edgware.SR2</version>
</dependency>
<dependency>
<groupId>com.netflix.zuul</groupId>
<artifactId>zuul-core</artifactId>
<version>1.3.0</version>
</dependency>
Some of my hystrix configurations:
zuul.servletPath=/
hystrix.command.default.execution.isolation.strategy=THREAD
hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds=10000
hystrix.command.aService.execution.isolation.strategy=THREAD
ribbon.ReadTimeout=60000
ribbon.ConnectTimeout=3000
Some of my FallbackProvider:
#Component
public class ServerFallback implements FallbackProvider {
#Override
public String getRoute() {
return "*";
}
#Override
public ClientHttpResponse fallbackResponse() {
// some logs
return simpleClientHttpResponse();
}
#Override
public ClientHttpResponse fallbackResponse(Throwable cause) {
// some logs
return simpleClientHttpResponse();
}
}
``
When using zuul with ribbon(default), the executionIsolationStrategy in HytrixCommandProperties will be overrided by AbstractRibbonCommand,which is SEMAPHORE by default.In this isolation strategy, request will not interrupted immediatelly.See ZuulProxy fails with “RibbonCommand timed-out and no fallback available” when it should do failover
I am working in an microservices architecture that works as follows
I have two service web applications (REST services) that register themselves correctly in an eureka server, then I have a client application that fetches the eureka registry and using ribbon as a client side load balancer, determines to which service application go (at the moment, a simple Round Robin is being used).
My problem is that when I stop one of the service applications (they currently run in docker containers btw), eureka does not remove them from their registry (seems to take a few minutes) so ribbon still thinks that there are 2 available services, making around 50% of the calls fail.
Unfortunately I am not using Spring Cloud (reasons out of my control). So my config for eureka is as follows.
For the service applications:
eureka.registration.enabled=true
eureka.name=skeleton-service
eureka.vipAddress=skeleton-service
eureka.statusPageUrlPath=/health/ping
eureka.healthCheckUrlPath=/health/check
eureka.port.enabled=8042
eureka.port=8042
eureka.appinfo.replicate.interval=5
## configuration related to reaching the eureka servers
eureka.preferSameZone=true
eureka.shouldUseDns=false
eureka.serviceUrl.default=http://eureka-container:8080/eureka/v2/
eureka.decoderName=JacksonJson
For the client application (eureka + ribbon)
###Eureka Client configuration for Sample Eureka Client
eureka.registration.enabled=false
eureka.name=skeleton-web
eureka.vipAddress=skeleton-web
eureka.statusPageUrlPath=/health/ping
eureka.healthCheckUrlPath=/health/check
eureka.port.enabled=8043
eureka.port=8043
## configuration related to reaching the eureka servers
eureka.preferSameZone=true
eureka.shouldUseDns=false
eureka.serviceUrl.default=http://eureka-container:8080/eureka/v2/
eureka.decoderName=JacksonJson
eureka.renewalThresholdUpdateIntervalMs=3000
#####################
# RIBBON STUFF HERE #
#####################
sample-client.ribbon.NIWSServerListClassName=com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList
# expressed in milliseconds
sample-client.ribbon.ServerListRefreshInterval=3000
# movieservice is the virtual address that the target server(s) uses to register with Eureka server
sample-client.ribbon.DeploymentContextBasedVipAddresses=skeleton-service
I had faced similar issue in my development ,
There are multiple things I tried and worked for me .
1)Instead of using eureka registry ,use only underlying ribbon and modify it's health check mechanism as per our need ,for doing this provide own implementation for IPing
public class PingUrl implements com.netflix.loadbalancer.IPing {
public boolean isAlive(Server server) {
String urlStr = "";
if (isSecure) {
urlStr = "https://";
} else {
urlStr = "http://";
}
urlStr += server.getId();
urlStr += getPingAppendString();
boolean isAlive = false;
try {
ResponseEntity response = getRestTemplate().getForEntity(urlStr, String.class);
isAlive = (response.getStatusCode().value()==200);
} catch (Exception e) {
;
}
return isAlive;
}
}
override the load balancing behaviour
#SpringBootApplication
#EnableZuulProxy
#RibbonClients(defaultConfiguration = LoadBalancer.class)
#ComponentScan(basePackages = {"com.test"})
public class APIGateway {
public static void main(String[] args) throws Exception {
SpringApplication.run(APIGateway .class, args);
}
}
public class LoadBalancer{
#Autowired
IClientConfig ribbonClientConfig;
#Bean
public IPing ribbonPing() {
return new PingUrl(getRoute() + "/ping");
}
}
private String getRoute() {
return RequestContext.getCurrentContext().getRequest().getServletPath();
}
providing availability filtering rule
public class AvailabilityBasedServerSelectionRule extends AvailabilityFilteringRule {
#Override
public Server choose(Object key) {
Server chosenServer = super.choose(key);
int count = 1;
List<Server> reachableServers = this.getLoadBalancer().getReachableServers();
List<Server> allServers = this.getLoadBalancer().getAllServers();
if(reachableServers.size() > 0) {
while(!reachableServers.contains(chosenServer) && count++ < allServers.size()) {
chosenServer = reachableServers.get(0);
}
}
return chosenServer;
}
}
2)You can specify the time interval for refreshing the list
ribbon.eureka.ServerListRefreshInterval={time in ms}
I am developing a Spring Boot [web] REST-style application with a ServletInitializer (since it needs to be deployed to an existing Tomcat server). It has a #RestController with a method that, when invoked, needs to write to a Redis pub-sub channel. I have the Redis server running on localhost (default port, no password). The relevant part of the POM file has the required starter dependency:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
When I deploy the WAR and hit the endpoint http://localhost:8080/springBootApp/health, I get this response:
{
"status": "DOWN",
"diskSpace": {
"status": "UP",
"total": 999324516352,
"free": 691261681664,
"threshold": 10485760
},
"redis": {
"status": "DOWN",
"error": "org.springframework.data.redis.RedisConnectionFailureException: java.net.SocketTimeoutException: Read timed out; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out"
}
}
I added the following to my Spring Boot application class:
#Bean
JedisConnectionFactory jedisConnectionFactory() {
return new JedisConnectionFactory();
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<String, Object> template = new RedisTemplate<String, Object>();
template.setConnectionFactory(jedisConnectionFactory());
return template;
}
I also tried adding the following to my #RestController before executing some test Redis code, but I get the same error as above in the stack trace:
#Autowired
private RedisTemplate<String, String> redisTemplate;
Edit (2017-05-09)
My understanding is that Spring Boot Redis starter assumes the default values of spring.redis.host=localhost and spring.redis.port=6379, I still added the two to application.properties, but that did not fill the gap.
Update (2017-05-10)
I added an answer to this thread.
I done a simple example with redis and spring boot
First I installed redis on docker:
$ docker run --name some-redis -d redis redis-server --appendonly yes
Then I Used this code for receiver :
import java.util.concurrent.CountDownLatch;
public class Receiver {
private static final Logger LOGGER = LoggerFactory.getLogger(Receiver.class);
private CountDownLatch latch;
#Autowired
public Receiver(CountDownLatch latch) {
this.latch = latch;
}
public void receiveMessage(String message) {
LOGGER.info("Received <" + message + ">");
latch.countDown();
}
}
And this is my spring boot app and my listener:
#SpringBootApplication
// after add security library then it is need to use security configuration.
#ComponentScan("omid.spring.example.springexample.security")
public class RunSpring {
private static final Logger LOGGER = LoggerFactory.getLogger(RunSpring.class);
public static void main(String[] args) throws InterruptedException {
ConfigurableApplicationContext contex = SpringApplication.run(RunSpring.class, args);
}
#Autowired
private ApplicationContext context;
#RestController
public class SimpleController{
#RequestMapping("/test")
public String getHelloWorld(){
StringRedisTemplate template = context.getBean(StringRedisTemplate.class);
CountDownLatch latch = context.getBean(CountDownLatch.class);
LOGGER.info("Sending message...");
Thread t = new Thread(new Runnable() {
#Override
public void run() {
for (int i = 0 ; i < 100 ; i++) {
template.convertAndSend("chat", i + " => Hello from Redis!");
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
});
t.start();
try {
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
return "hello world 1";
}
}
///////////////////////////////////////////////////////////////
#Bean
RedisMessageListenerContainer container(RedisConnectionFactory connectionFactory,
MessageListenerAdapter listenerAdapter) {
RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.addMessageListener(listenerAdapter, new PatternTopic("chat"));
return container;
}
#Bean
MessageListenerAdapter listenerAdapter(Receiver receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
#Bean
Receiver receiver(CountDownLatch latch) {
return new Receiver(latch);
}
#Bean
CountDownLatch latch() {
return new CountDownLatch(1);
}
#Bean
StringRedisTemplate template(RedisConnectionFactory connectionFactory) {
return new StringRedisTemplate(connectionFactory);
}
}
The important point is the redis IP. if you installed it on docker like me then
you should set ip address in application.properties like this:
spring.redis.host=172.17.0.4
I put all my spring examples on github here
In addition I used redis stat to monitor redis. it is simple monitoring.
Spring data redis properties are updated, e.g. spring.redis.host is now spring.data.redis.host.
You need to configure your redis server information using the application.properties:
# REDIS (RedisProperties)
spring.redis.cluster.nodes= # Comma-separated list of "host:port"
spring.redis.database=0 # Database index
spring.redis.url= # Connection URL,
spring.redis.host=localhost # Redis server host.
spring.redis.password= # Login password of the redis server.
spring.redis.ssl=false # Enable SSL support.
spring.redis.port=6379 # Redis server port.
Spring data docs: https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html#REDIS
This was a proxy related problem, where even access to localhost was somehow being curtailed. Once I disabled the proxy settings, Redis health was UP! So the problem is solved. I did not have to add any property to application.properties and neither did I have to explicitly configure anything in the Spring Boot application class, because Spring Boot and the Redis Starter auto-configures based on Redis defaults (as applicable in my development environment). I just added the following to the pom.xml:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
and the following to the #RestController annotated class, and Spring Boot auto-wired as needed (awesome!).
#Autowired
private RedisTemplate<String, String> redisTemplate;
To publish a simple message to a channel, this single line of code was sufficient for validating the setup:
this.redisTemplate.convertAndSend(channelName, "hello world");
I appreciate all the comments, which were helpful in backing up my checks.
I'm creating two springboot server & client applications communicating using JMS, and everything is working fine with the release 5.12.1 for activemq, but as soon as I update to the 5.12.3 version, I'm getting the following error :
org.springframework.jms.support.converter.MessageConversionException: Could not convert JMS message; nested exception is javax.jms.JMSException: Failed to build body from content. Serializable class not available to broker. Reason: java.lang.ClassNotFoundException: Forbidden class MyClass! This class is not trusted to be serialized as ObjectMessage payload. Please take a look at http://activemq.apache.org/objectmessage.html for more information on how to configure trusted classes.
I went on the URL that is provided and I figured out that my issue is related to the new security implemented in the 5.12.2 release of ActiveMQ, and I understand that I could fix it by defining the trusted packages, but I have no idea on where to put such a configuration in my SpringBoot project.
The only reference I'm making to the JMS queue in my client and my server is setting up it's URI in application.properties and enabling JMS on my "main" class with #EnableJms, and here's my configuration on the separate broker :
#Configuration
#ConfigurationProperties(prefix = "activemq")
public class BrokerConfiguration {
/**
* Defaults to TCP 10000
*/
private String connectorURI = "tcp://0.0.0.0:10000";
private String kahaDBDataDir = "../../data/activemq";
public String getConnectorURI() {
return connectorURI;
}
public void setConnectorURI(String connectorURI) {
this.connectorURI = connectorURI;
}
public String getKahaDBDataDir() {
return kahaDBDataDir;
}
public void setKahaDBDataDir(String kahaDBDataDir) {
this.kahaDBDataDir = kahaDBDataDir;
}
#Bean(initMethod = "start", destroyMethod = "stop")
public BrokerService broker() throws Exception {
KahaDBPersistenceAdapter persistenceAdapter = new KahaDBPersistenceAdapter();
persistenceAdapter.setDirectory(new File(kahaDBDataDir));
final BrokerService broker = new BrokerService();
broker.addConnector(getConnectorURI());
broker.setPersistent(true);
broker.setPersistenceAdapter(persistenceAdapter);
broker.setShutdownHooks(Collections.<Runnable> singletonList(new SpringContextHook()));
broker.setUseJmx(false);
final ManagementContext managementContext = new ManagementContext();
managementContext.setCreateConnector(true);
broker.setManagementContext(managementContext);
return broker;
}
}
So I'd like to know where I'm supposed to specify the trusted packages.
Thanks :)
You can just set one of the below spring boot properties in application.properties to set trusted packages.
spring.activemq.packages.trust-all=true
or
spring.activemq.packages.trusted=<package1>,<package2>,<package3>
Add the following bean:
#Bean
public ActiveMQConnectionFactory activeMQConnectionFactory() {
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory("your broker URL");
factory.setTrustedPackages(Arrays.asList("com.my.package"));
return factory;
}
The ability to do this via a configuration property has been added for the next release:
https://github.com/spring-projects/spring-boot/issues/5631
Method: public void setTrustedPackages(List<String> trustedPackages)
Description: add all packages which is used in send and receive Message object.
Code : connectionFactory.setTrustedPackages(Arrays.asList("org.api","java.util"))
Implementated Code:
private static final String DEFAULT_BROKER_URL = "tcp://localhost:61616";
private static final String RESPONSE_QUEUE = "api-response";
#Bean
public ActiveMQConnectionFactory connectionFactory(){
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(DEFAULT_BROKER_URL);
connectionFactory.setTrustedPackages(Arrays.asList("org.api","java.util"));
return connectionFactory;
}
#Bean
public JmsTemplate jmsTemplate(){
JmsTemplate template = new JmsTemplate();
template.setConnectionFactory(connectionFactory());
template.setDefaultDestinationName(RESPONSE_QUEUE);
return template;
}
If any one still looking for an answer, below snippet worked for me
#Bean
public ActiveMQConnectionFactory connectionFactory() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(BROKER_URL);
connectionFactory.setPassword(BROKER_USERNAME);
connectionFactory.setUserName(BROKER_PASSWORD);
connectionFactory.setTrustAllPackages(true); // all packages are considered as trusted
//connectionFactory.setTrustedPackages(Arrays.asList("com.my.package")); // selected packages
return connectionFactory;
}
I am setting Java_opts something like below and passing to java command and its working for me.
JAVA_OPTS=-Xmx256M -Xms16M -Dorg.apache.activemq.SERIALIZABLE_PACKAGES=*
java $JAVA_OPTS -Dapp.config.location=/data/config -jar <your_jar>.jar --spring.config.location=file:/data/config/<your config file path>.yml
Yes I found it's config in the new version
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.4.0.RELEASE</version>
</parent>
spring:
profiles:
active: #profileActive#
cache:
ehcache:
config: ehcache.xml
activemq:
packages:
trusted: com.stylrplus.api.model