I have spring boot 2 application with REST API clients. There is a API to download large byte array (around 85MB), so I willing to compress it using GZip encoding. Added following properties to application.properties file
server.compression.enabled=true
server.compression.min-response-size=1024
server.compression.mime-types=application/octet-stream
Default compression reduce file size but it increase processing time rapidly. I saw there are compression levels in GZip encoding from 0-9.
How I set compression level in application.properties file.
Resolve my issue by changing embedded server to jetty.
First exclude embedded tomcat and add jetty dependency in pom.xml
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
</dependency>
Then add Custom JettyServletWebServerFactory as follows
#Configuration
public class GZipConfig {
#Bean
public JettyServletWebServerFactory jettyServletWebServerFactory() {
JettyServletWebServerFactory factory = new JettyServletWebServerFactory();
factory.addServerCustomizers((Server server) -> {
GzipHandler gzipHandler = new GzipHandler();
gzipHandler.setInflateBufferSize(1);
gzipHandler.setHandler(server.getHandler());
gzipHandler.setIncludedMethods("GET", "POST", "DELETE", "PUT");
gzipHandler.setCompressionLevel(Deflater.BEST_SPEED);
HandlerCollection handlerCollection = new HandlerCollection(gzipHandler);
server.setHandler(handlerCollection);
});
return factory;
}
}
Here we can add compression level.
Thanks
Related
I'm facing issue with Swagger Integration in Spring Boot. Have a look at the code and error snippet.
------------------POM--------------------
<properties>
<java.version>1.8</java.version>
<swagger.version>2.9.2</swagger.version>
</properties>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
<version>${swagger.version}</version>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger-ui</artifactId>
<version>${swagger.version}</version>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-bean-validators</artifactId>
<version>${swagger.version}</version>
</dependency>
-----------------App class--------------
#SpringBootApplication
#EnableSwagger2
public class ProducerApplication {
public static void main(String[] args) {
SpringApplication.run(ServletPocProducerApplication.class, args);
}
#Bean
public Docket api() {
return new Docket(DocumentationType.SWAGGER_2)
.select()
.apis(RequestHandlerSelectors.any())
.paths(PathSelectors.any())
.build();
}
}
Stack Trace
org.springframework.context.ApplicationContextException: Failed to start bean
'documentationPluginsBootstrapper'; nested exception is
java.lang.NullPointerException: Cannot invoke
"org.springframework.web.servlet.mvc.condition.PatternsRequestCondition.toString()"
because the return value of
"springfox.documentation.spi.service.contexts.Orderings.patternsCondition(springfox.docume
ntation.RequestHandler)" is null
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:181) ~[spring-context-5.3.13.jar:5.3.13]
How do I fix this??
I solved it by adding "spring.mvc.pathmatch.matching-strategy=ant-path-matcher" in application.properties.
For a long time I have tried to solve this problem and solution for this is:
a) adding this to application.properties:
spring.mvc.pathmatch.matching-strategy=ant-path-matcher
b) adding this to application.yaml(or application.yml):
spring:
mvc:
pathmatch:
matching-strategy: ant_path_matcher
I know this does not solve your problem directly, but consider moving to springdoc. Springfox is so buggy at this point that is a pain to use. I've moved to springdoc 2 years ago because of its Spring WebFlux support and I am very happy about it. Additionally, it also supports Kotlin Coroutines, which I am not sure Springfox does.
If you decide to migrate, springdoc even has a migration guide.
For the integration between spring-boot and swagger-ui, add the library to the list of your project dependencies (No additional configuration is needed):
<dependency>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-ui</artifactId>
<version>1.5.12</version>
</dependency>
The Swagger UI page will then be available at
http://server:port/context-path/swagger-ui.html and the OpenAPI
description will be available at the following url for json format:
http://server:port/context-path/v3/api-docs
server: The server name or IP
port: The server port
context-path: The context path of the application
Adding this "spring.mvc.pathmatch.matching-strategy=ant-path-matcher" to your application.properties file solves the problem.
It's what i used and i saved me alot of trouble.
My suggestion is when you are using spring-boot then it is better to use spring boot dependency for swagger. So, spring-boot will take care of your default settings.
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-boot-starter</artifactId>
<version>...</version>
</dependency>
I am trying to forward/add the Actuator Camel metrics from /actuator/camelroutes (route metrics like number of exchanges/transactions) to the Prometheus Actuator endpoint. Is there a way for me to configure Camel to add those metrics to the PrometheusMeterRegistry?
I have tried adding:
camel.component.metrics.metric-registry=io.micrometer.prometheus.PrometheusMeterRegistry
in my application.properties according to the documentation here: https://camel.apache.org/components/latest/metrics-component.html
But still nothing relating to Apache Camel is displayed in actuator/prometheus
Here are the dependencies I am using with Spring Boot 2.1.9 and Apache Camel 2.24.2:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-metrics-starter</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
Got the Camel Routes metrics working in the /actuator/prometheus endpoint.
Use the camel-micrometer-starter dependency as stated by #claus-ibsen 's comment.
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-metrics-starter</artifactId>
</dependency>
Set the following in your properties file:
camel.component.metrics.metric-registry=prometheusMeterRegistry
Then add set the Camel Context to use the MicrometerRouterPolicyFactory and MicrometerMessageHistoryFactory. Code seen below is places in a Configuration class:
#Configuration
public class AppConfig {
#Bean
public CamelContextConfiguration camelContextConfiguration() {
return new CamelContextConfiguration() {
#Override
public void beforeApplicationStart(CamelContext camelContext) {
camelContext.addRoutePolicyFactory(new MicrometerRoutePolicyFactory());
camelContext.setMessageHistoryFactory(new MicrometerMessageHistoryFactory());
}
#Override
public void afterApplicationStart(CamelContext camelContext) {
}
};
}
}
You need to trigger an exchange in a route for the metrics to appear in /actuator/prometheus.
Here are the metrics made available to Prometheus:
CamelMessageHistory_seconds_count
CamelMessageHistory_seconds_max
CamelRoutePolicy_seconds_max
CamelRoutePolicy_seconds_count
CamelRoutePolicy_seconds_sum
You can use the JMX Exporter jar for Prometheus to get the more detailed metrics from the JMX of Camel. I wanted to avoid this approach as it would mean that for each Camel Spring Boot App I have would use 2 ports; 1 for the JMX Metrics and 1 for the Actuator Metrics.
There is a camel-micrometer-starter dependency you should use instead that integrates with micrometer. And then you can use the micrometer route policy from that dependency to let it monitor all your routes. See the docs at: https://camel.apache.org/components/2.x/micrometer-component.html
I can see the metrics by keeping these dependencies intact
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-management</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-metrics</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-micrometer-starter</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
Why dont I see the actual process and bean names rather than like process3, bean1 etc ??
CamelMessageHistory_seconds_sum{camelContext="AppName",nodeId="process3",routeId="AppNameRoute",serviceName="MicrometerMessageHistoryService",} 0.041466
CamelMessageHistory_seconds_count{camelContext="AppName",nodeId="bean1",routeId="AppNameRoute",serviceName="MicrometerMessageHistoryService",} 100.0
CamelMessageHistory_seconds_sum{camelContext="AppName",nodeId="bean1",routeId="AppNameRoute",serviceName="MicrometerMessageHistoryService",} 4.8417576
I have created one microservice using Java8 and SpringBoot using Maven.
Lets call it as MicroServiceA
It has controller which returns ResponseEntity object as below:
#RestController
#RequestMapping("/api")
public class MicroserviceAController {
#GetMapping(value = "/all")
public ResponseEntity<ServiceAResponseWrapper<List<ServiceADto>>> getAll() {
ServiceAResponseWrapper<List<ServiceADto>> wrapper =
new ServiceAResponseWrapper<List<ServiceADto>>(ServiceAResponseStatus.SUCCESS,findAll());
return new ResponseEntity<ServiceAResponseWrapper<List<ServiceADto>>>(wrapper,HttpStatus.OK);
}
public static List<ServiceADto> findAll() {
//returns list of ServiceADto objects
}
}
When I start this service and verify it in any browser: http://localhost:8073/api/all/ , I get JSON response displayed.
Now if I want to introduce my service to EUREKA service registry then I will need to do following changes.
Create EUREKA server microservice. I start it - http://localhost:8761/
Make changes to MicroserviceA as follows -
Go to pom.xml and add dependency
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>
go to application.yml and add this:
eureka:
client:
registerWithEureka: true
fetch-registry: true
service-url:
defaultZone: http://localhost:8761/eureka/
As soon as I start my service then I start seeing it on server http://localhost:8761/
Now I again go to browser and try to check my microservice http://localhost:8073/api/all/ What I see is XML and not JSON.
I even tried to fix it by modifying my Microservice Controller by adding
annotation to my method:
#Produces( { MediaType.APPLICATION_JSON} )
But with that also I see XML and not JSON.
Am I missing something or its normal behavior with EUREKA ? If yes, how do I fix it?
If you are using older version of spring cloud starter, you might need to exclude Jackson dataformat XML dependency
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-eureka</artifactId>
<exclusions>
<exclusion>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
</exclusion>
</exclusions>
</dependency>
There are two potential solutions for this:
First: exclude the jackson-dataformat-xml dependency from all spring-cloud-starter-* artifacts if your application has nothing to do with XML conversions. One exclusion example from spring-cloud-starter-netflix-eureka-client is below. For my case, I had to exclude Jackson XML dependency from spring-cloud-starter-netflix-ribbon, spring-cloud-starter-openfeign and spring-cloud-starter-netflix-eureka-client
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-eureka</artifactId>
<exclusions>
<exclusion>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
</exclusion>
</exclusions>
</dependency>
Second: If you want to support both XML and JSON responses. You can pass Accept: header with the request to your API.
For getting JSON response:
curl -X GET \
http://localhost:8073/api/all/ \
-H 'Accept: application/json'
For getting XML response:
curl -X GET \
http://localhost:8073/api/all/ \
-H 'Accept: application/xml'
Hi I myself never used Eureka but from a quick search there is a ready to use API that converts the XML to json as Eureka uses XML and not json because json can’t hold attributes.
Link to the site explaining how to do this ->
https://automationrhapsody.com/json-format-register-service-eureka/amp/
Hope this helps you out
I am trying to submit distCP job from a spring boot application on a REST API call.
version of spring: 1.5.13.RELEASE
hadoop version: 2.7.3
below is the code I am using to instantiate the DistCP:
List<Path> srcPathList = new ArrayList<Path>();
srcPathList.add(new Path("hdfs://<cluster>/tmp/<user>/source"));
Path targetPath = new Path("hdfs://<cluster>/tmp/<user>/destination");
DistCpOptions distCpOptions = new DistCpOptions(srcPathList,targetPath);
DistCp distCp = new DistCp(configuration,distCpOptions);
Job job = distCp.execute();
The job is submitted successfully to the cluster, however the job fails due to ClassNotFoundException on the cluster. Below is the exception:
INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED;
cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
java.lang.RuntimeException: java.lang.ClassNotFoundException:
Class org.apache.hadoop.tools.mapred.CopyOutputFormat not found
Why does this happen? Any pointers around this would be very helpful!! Thanks!
I found the reason via viewing the job.jar on the NodeManager machine. The structure of job.jar is：
BOOT-INF/class/xxx
this is unreasonable.
I tried to replace the jar package with war，it works!
<packaging>war</packaging>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<!--exclude inner tomcat-->
<exclusions>
<exclusion>
<artifactId>spring-boot-starter-tomcat</artifactId>
<groupId>org.springframework.boot</groupId>
</exclusion>
</exclusions>
</dependency>
<!-- include tomcat-->
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-servlet-api</artifactId>
<version>7.0.47</version>
<scope>provided</scope>
</dependency>
...
and then add start class:
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.support.SpringBootServletInitializer;
public class SpringBootStartApplication extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder builder) {
//
return builder.sources(xxxPortalApplication.class);
}
}
i face a problem when i using spring cloud gateway
is if any dependency call spring-boot-starter-tomcat directly or recursively
it will not work because it will start the embedded tomcat server not the netty server that spring cloud gateway use
i started to solve this problem by excluding this dependency
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
the spring cloud gateway worked successfully
but sometimes i want to use spring-cloud-starter-oauth2 to use #EnableOAuth2Sso
i start to use
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-oauth2</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
at that time i face the big issue that throw exception saying
Caused by: java.lang.IllegalStateException: Failed to introspect annotated methods on class org.springframework.security.oauth2.config.annotation.web.configuration.OAuth2ClientConfiguration
......
Caused by: java.lang.NoClassDefFoundError: javax/servlet/Filter
As you've seen, the Spring cloud gateway uses the reactive model and is based on netty rather than tomcat. The reactive change is a major shift and currently isn't supported by Spring Security but work is in progress on it and you can track it at https://github.com/spring-cloud/spring-cloud-gateway/issues/179
Use the following dependencies (I copied from my build.gradle)
dependencies {
implementation 'org.springframework.cloud:spring-cloud-starter-gateway'
implementation 'org.springframework.cloud:spring-cloud-starter-security'
implementation 'org.springframework.boot:spring-boot-starter-oauth2-client'
}
Code your gateway app minimally as follows
#SpringBootApplication
public class App {
#Bean
public ForwardedHeaderTransformer forwardedHeaderTransformer() {
return new ForwardedHeaderTransformer();
}
public static void main(String[] args) {
SpringApplication.run(App.class, args);
}
}
Configure in application.yml
spring:
security:
oauth2:
client:
registration:
google:
client-id: XXX
client-secret: YYY
I am actively building my stack that uses OAuth2 with Docker Swarm Discovery https://github.com/trajano/spring-cloud-demo.git so you can see how it would work in action.
spring boot 2.1 with spring security 5 have resolve this problem
see this example