I am playing around with Spring Cloud Data Flow. I have successfully deployed SCDF on Kubernetes using the related documentation. When registering the 1.5.x based starter apps, everything is working as expected, no further configuration of the starter apps during the deployment of a stream definition is needed.
When using the 2.x based starter apps, there are some changes introduced by the switch to Spring Boot 2.0 that need to be accommodated for, e.g. the actuator endpoints changed. For reference, here are the properties that I provide during the deployment of the stream:
app.*.management.endpoints.web.exposure.include=health,info,binders
deployer.*.cpu=2
deployer.*.memory=4096
deployer.http.count=2
deployer.*.kubernetes.livenessProbePath=/actuator/health
deployer.*.kubernetes.readinessProbePath=/actuator/info
However, the readiness probe fails since the health and the info endpoint now seem to be protected by default. Therefore, the pods end up in crashloops since from the Kubernetes perspective they get never ready.
I worked around the situation by following the guide on patching the starter apps that my stream definition relies on (e.g. throughput sink) like this:
#SpringBootApplication
#Import({org.springframework.cloud.stream.app.throughput.sink.ThroughputSinkConfiguration.class})
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
#Configuration
protected static class ThroughputSinkSecurityConfiguration extends WebSecurityConfigurerAdapter {
#Override
protected void configure(HttpSecurity http) throws Exception {
http
.authorizeRequests()
.requestMatchers(EndpointRequest.to("health", "info")).permitAll();
}
}
}
Is there a way to specify this kind of security configuration via flags or properties? Shouldn't such a WebSecurityConfigurerAdapter be there by default to make the health and info endpoints accessible for Kubernetes?
Artem's response is very relevant. I wanted to also share a few other approaches specific to security and OOTB apps.
In 1.6 SNAPSHOTs, we have recently added support via spring-cloud/spring-cloud-deployer-kubernetes#236 to plug basic-auth realm to interact with secured actuator endpoints. They are applicable to both liveness and readiness probes. Here's the commit/docs for your reference.
If you don't really want security at all, though not recommended, you can explicitly disable the Security configuration.
dataflow:>stream create foo -- definition "http | throughput"
dataflow:>stream deploy foo --properties app.*.spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration"
(i.e., all the Apps in the foo stream definition will start with SecurityAutoConfiguration excluded)
I would suggest to look into the situation from the other angle and provide credential from the Kubernetes to get access to your secured Microservice.
The problem of the current status-quo that all the resources has to be protected.
You can generate your own static password and store it in the application.properties do not reconfigure Kubernetes for each application restart: https://docs.spring.io/spring-boot/docs/2.0.3.RELEASE/reference/htmlsingle/#boot-features-security
Related
I have multiple microservices which communicates with each other through REST calls.
I have used spring boot and spring rest and have configured the URLS of the rest end points in application.properties file.
Now the problems is if the URL for one end point changes then I to have to manually modify all the property files of the services which are calling that particular end point which has got changed.
Is there a workaround for this so that the URLS can be somehow placed in a centralized location so that any modification does not impacts the other services which are using it.
You can use spring-cloud to achieve this. Usual way used in spring-cloud is by configuring the required properties in a git repo. And then those properties can be accessed by any micro-service you want with minimal configurations. You can refer projects in this repo
limits-services acts as a client that needs certain properties those are configured in spring-cloud-config-server. Hope this helps.
In case with microservices you can use Spring Cloud Config (Spring Cloud Config, Spring Cloud Config Server). It's very usefull and you can update your configuration at runtime.
Spring Cloud Config provides server and client-side support for externalized configuration in a distributed system. With the Config Server you have a central place to manage external properties for applications across all environments. The concepts on both client and server map identically to the Spring Environment and PropertySource abstractions, so they fit very well with Spring applications, but can be used with any application running in any language. As an application moves through the deployment pipeline from dev to test and into production you can manage the configuration between those environments and be certain that applications have everything they need to run when they migrate.
As others have mentioned you can use Spring Cloud Config Server to remotly load your application configuration. All you need is git repository containing your configuration.
Spring cloud configuration supporst Git, database as your store for configuration.
Idea is to create an spring-boot app that can provide configuration to other applications.
#SpringBootApplication
#EnableConfigServer
public class ConfigServer {
public static void main(String[] args) {
SpringApplication.run(ConfigServer.class, args);
}
}
You can configurae port and provide your git repository using key spring.cloud.config.server
server.port: 8888
spring.cloud.config.server.git.uri: file://${user.home}/config-repo
At client side, if you have spring-config in your classpath, application will try to connect to an application runnign at port 8888 to retrieve configuration.
More information can be found here.
may put configuration inside a database.
after that need have one centralize cache service that used by other services, can be .jar service,
then the values can be load inside a cache class in this service,
then in the front end side need have update button for updating the cache after modify the URL value in the database, so then all impacted services can use new value.
and also to be easier may have stand alone UI for update those configuration rather than updating database directly.
You can use Microconfig.IO to manage your service configuration and it's placeholders functionality to reference configuration values of certain services from others. So in your case you configure your deploy url in your server and put placeholders on it in your clients. This allows you to edit value only in one place and then everyone who depend on it will get it automatically.
I have a Spring Boot app as follows:
#SpringBootApplication
#EnableResourceServer
#EnableDiscoveryClient
public class App
{
public static void main(String[] args) {
SpringApplication.run(App.class, args);
}
}
And i have configured oauth2 properties in application.properties.
security.oauth2.resource.user-info-uri=http://localhost:9000/user
However, I need to change this URI depending on the request I send to my Spring Boot App.
For example:
If I receive a request to my Spring Boot App to URL htpp://test.localhost:8000 I would want the user-info-url to be http://test.localhost:9000/user.
But if I receive a request to http://localhost:8000 I want the user-info-url to be http://localhost:9000/user.
My intention is to manage different environments depending on the subdomain.
Is there a way to dynamically change this URI in code instead of hardcoding it as a property?
Spring Cloud Config
By leveraging a Spring Cloud Config server, you can externalize your configuration and push updates to application properties. A simple implementation is backed by a Git repository where you'll store your application.properties file. You commit your changes and then make a POST request the /refresh endpoint that will trigger a reload of your configuration.
There are more advanced scenarios as well but hopefully that's enough to get you started.
Here's a simple Getting Started guide.
I am looking for a way to configure during runtime (say, via a properties file, or deployment descriptor) which services in my Jersey-based application are available, i.e. so that services can be enabled or disabled by an administrator at the container level.
Our app presently exposes 15 different endpoints. It has 1 Application annotated with #ApplicationPath with 3 classes annotated with #Path, and within those 3 classes are 15 different methods annotated with the usual #Path/#/#Produces.
My app has a few different techniques for reading its runtime configuration, e.g. settings to connect to database resources, watching a properties file for changes, etc. What I want to do is add some configuration values so that administrators can enable/disable the 3 classes or any of the individual endpoints within those classes at the Jersey level. Could someone suggest the best way to do this?
Taking that a step further, we want to control this configuration in runtime, so if the configuration changes, we can update the jersey configuration to enable/disable the changed services without having to restart our container (which is Tomcat in this case).
Any advice appreciate! Thanks!
I don't know if this is possible natively with Jersey, but one thing I can think of is to just use a Jersey prematching filter to determine if the endpoints are disabled. You can use a service that can also act as a listener and update the disabled endpoints accordingly. If the endpoint is disabled, then just return a 404.
#Provider
#PreMatching
public class DisabledEndpointsFilter implements ContainerRequestFilter {
#Inject
private ConfigurationService configuration;
#Override
public void filter(ContainerRequestContext request) throws IOException {
final List<String> disabledEndpoints = this.configuration.getDisabledEndpoints();
final String path = stripLeadingSlash(request.getUriInfo().getPath());
for (String endpoint: disabledEndpoints) {
endpoint = stripLeadingSlash(endpoint);
if (path.startsWith(endpoint)) {
request.abortWith(Response.status(404).build());
return;
}
}
}
}
I was playing around with this and put together a working POC. You can checkout the Github Repo.
I would recommend use apache camel for this. With camel you can control the http requests combined with a embedded jetty server as proxy.
http://camel.apache.org/jetty.html
Consider you have a Spring application that gets its configuration from a config server. If it cannot connect to the config server, the application will continue to start but as all configurations are missing, it will eventually fail with a potentially misleading error.
Is it possible to configure Spring, so it immediately aborts when it cannot connect to its config server during startup?
Set spring.cloud.config.failFast to true in your bootstrap.yml or bootstrap.properties file. Also, you can add -Dspring.cloud.config.failFast=true to the JVM arguments.
From the documentation
Config Client Fail Fast
In some cases, it may be desirable to fail startup of a service if it cannot connect to the Config Server. If this is the desired behavior, set the bootstrap configuration property spring.cloud.config.failFast=true and the client will halt with an Exception.
You can achieve that by using spring cloud Spring Cloud Config Serverand Spring Cloud Config Client components.
1. Spring Cloud Config Server
The Server provides an HTTP, resource-based API for external configuration (name-value pairs, or equivalent YAML content). The server is easily embeddable in a Spring Boot application using the #EnableConfigServer annotation. So this app is a config server:
//ConfigServer.java
#SpringBootApplication
#EnableConfigServer
public class ConfigServer {
public static void main(String[] args) {
SpringApplication.run(ConfigServer.class, args);
}
}
2. Spring Cloud Config Client
A Spring Boot application can take immediate advantage of the Spring Config Server (or other external property sources provided by the application developer), and it will also pick up some additional useful features related to Environment change events.
Then in the config client side you can configure fail fast by setting the bootstrap configuration property spring.cloud.config.failFast=true
Spring cloud documentation Config Client Fail Fast
I'm trying to make a basic project using spring cloud with the netflix addons such as Hystrix, Eureka and Ribbon to learn how this works. The project I'm trying to make is a simple message server that will keep messages. And a message-client that will just ask the server for a message, and I want to use the auto discovery client for this, or the RestTemplate discovery. But I can't get either to work.
I have the following structure:
message-client (eureka client)
message-server (eureka client)
configuration-service (config server)
discovery-service (eureka server)
What I currently do is I start up the configuration-service, and expose the application.yml details to all of these "apps/clients" when they are connecting by the following structure:
config-service\src\main\resources\config\appname.yml
app\src\main\resources\bootstrap.yml (contains the appname and url to cloud config)
This is working just fine, and my apps start up on the port they receive from the config server, as well as they all connect to my eureka server, and all of them are visible there. As well as the Hystrix failover is also working, not that it is related to this but it tells me that it can't be completely wrong then.
But here comes my confusion...
When using the #Autowired annotation in my service class (#Service annotated) inside my client module, I get a discoveryClient object, but I am unable to find any other services using that one.
Message Client - boot class:
#EnableAutoConfiguration
#EnableHystrix
#EnableEurekaClient
#ComponentScan("cloud.rest.resources, spring.cloud.client")
public class ClientBoot {
public static void main(String[] args) {
SpringApplication.run(ClientBoot.class, args);
}
}
Message Client - REST Resource:
#RestController
public class MessageResource {
#Autowired
private MessageClient messageClient;
#RequestMapping(value = "/message/{client}", method = RequestMethod.GET)
public Message getMessage(#PathVariable String client) {
return messageClient.getMessage(client);
}
}
Message Client - MessageClient:
#Service
public class RestMessageClient implements MessageClient {
#Autowired
private DiscoveryClient discoveryClient;
#Autowired
private RestTemplate restTemplate;
#Override
public Message getMessage(String client) {
return restTemplate.getForObject(String.format("http://message-server/message/%s", client), Message.class);
}
}
My message server boot class that is holding the messages has the same annotations as my client one.
And as I said, my service class are unable to find anything..
Which leads me to all of my questions:
What is required to actually use ribbon load balancer?
Do I have to use ribbon to be able to use the "auto discovery", I thought not but now I'm just confused.
From what I've understood, when using EnableEurekaClient I should not need to use the EnableDiscoveryClient as well?
Can I change the yml files on my config-server for the clients in runtime and just have to reboot the client?
How much configuration is really meant to be shared by the config-server, because currently all of my clients just contain a super basic bootstrap.yml file.
Does anyone have a good link to where I can read more about all the properties that is being set in my yml files? Both a documentation of what the properties that exists actually do as well as some documentation on how I can use them in combination with spring cloud?
Do I need specific properties to enable my apps/clients to find other apps/clients?
Edited information
Thank you for your quick and excellent reply, I've gone through this over and over today and I finally got my application working..
The problem (I can't understand why and was hoping you could help me understand that) is that my discovery service contains yml files for each of my other clients where I specify things like port and eureka information.. What I specified here as well was:
eureka:
client:
serviceUrl:
defaultZone: http://localhost:8761/eureka
So, when I set this value it seems to override something that makes my service discovery not working.. Even tho I can see all my applications in the eureka server, they were unable to find each other when I had this value set.
I set this value by having a message-server.yml file in my configuration service that is sent out to my message-server application after bootstrap..
So then I have two new questions.
How do I override this eureka server property?
Why does my discovery client stop working when I set this value, what is it that it actually does?
What is required to actually use ribbon load balancer?
The ribbon-loadbalancer must be on the classpath (e.g. via "spring-cloud-starter-ribbon"). Then you can inject one as a LoadBalancerClient or you can inject a RestTemplate (it will be load-balancer aware if you have a LoadBalancerClient).
Do I have to use ribbon to be able to use the "auto discovery", I thought not but now I'm just confused.
What is "auto discovery"? You don't need to use Ribbon to use the DiscoveryClient (Ribbon is a load balancer, not a service registry).
From what I've understood, when using EnableEurekaClient I should not need to use the EnableDiscoveryClient as well?
Correct. #EnableEurekaClient is annotated with #EnableDiscoveryClient so it is only there to express a preference.
Can I change the yml files on my config-server for the clients in runtime and just have to reboot the client?
Yes. Or you can use the /refresh or /restart endpoints (a full reboot is probably best in production, at least periodically).
How much configuration is really meant to be shared by the config-server, because currently all of my clients just contain a super basic bootstrap.yml file.
As much as you want. How long is a piece of string? If I were you I would try and keep the central config to a minimum (only the things that change between environments, or at runtime).
Does anyone have a good link to where I can read more about all the properties that is being set in my yml files? Both a documentation of what the properties that exists actually do as well as some documentation on how I can use them in combination with spring cloud?
Spring Boot and Spring Cloud have autogenerated metadata for externalized properties. The new generation of IDEs understands them (so get STS 3.6.4 or IDEA 14.1), and they are listed in the user guide (for Spring Boot at least) under http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#common-application-properties
Do I need specific properties to enable my apps/clients to find other apps/clients?
You need to be able to locate your service registry (Eureka in this case). If you are using Eureka and your clients have registered then that is enough.