Elasticsearch Rest Client Still Giving IOException : Too Many Open Files - java

This is a follow up to the solution which was provided to me on this previous post:
How to Properly Close Raw RestClient When Using Elastic Search 5.5.0 for Optimal Performance?
This same exact error message came back!
2017-09-29 18:50:22.497 ERROR 11099 --- [8080-Acceptor-0] org.apache.tomcat.util.net.NioEndpoint : Socket accept failed
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) ~[na:1.8.0_141]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) ~[na:1.8.0_141]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) ~[na:1.8.0_141]
at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:453) ~[tomcat-embed-core-8.5.15.jar!/:8.5.15]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_141]
2017-09-29 18:50:23.885 INFO 11099 --- [Thread-3] ationConfigEmbeddedWebApplicationContext : Closing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext#5387f9e0: startup date [Wed Sep 27 03:14:35 UTC 2017]; root of context hierarchy
2017-09-29 18:50:23.890 INFO 11099 --- [Thread-3] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 2147483647
2017-09-29 18:50:23.891 WARN 11099 --- [Thread-3] o.s.c.support.DefaultLifecycleProcessor : Failed to stop bean 'documentationPluginsBootstrapper'
... 7 common frames omitted
2017-09-29 18:50:53.891 WARN 11099 --- [Thread-3] o.s.c.support.DefaultLifecycleProcessor : Failed to shut down 1 bean with phase value 2147483647 within timeout of 30000: [documentationPluginsBootstrapper]
2017-09-29 18:50:53.891 INFO 11099 --- [Thread-3] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown
2017-09-29 18:50:53.894 INFO 11099 --- [Thread-3] com.app.controller.SearchController : Closing the ES REST client
I tried using the solution from the previous post.
ElasticsearchConfig:
#Configuration
public class ElasticsearchConfig {
#Value("${elasticsearch.host}")
private String host;
#Value("${elasticsearch.port}")
private int port;
#Bean
public RestClient restClient() {
return RestClient.builder(new HttpHost(host, port))
.setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
#Override
public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) {
return requestConfigBuilder.setConnectTimeout(5000).setSocketTimeout(60000);
}
}).setMaxRetryTimeoutMillis(60000).build();
}
SearchController:
#RestController
#RequestMapping("/api/v1")
public class SearchController {
#Autowired
private RestClient restClient;
#RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json" )
public ResponseEntity<Object> getSearchQueryResults(#RequestParam(value = "criteria") String criteria) throws IOException {
// Setup HTTP Headers
HttpHeaders headers = new HttpHeaders();
headers.add("Content-Type", "application/json");
// Setup query and send and return ResponseEntity...
Response response = this.restClient.performRequest(...);
}
#PreDestroy
public void cleanup() {
try {
logger.info("Closing the ES REST client");
this.restClient.close();
}
catch (IOException ioe) {
logger.error("Problem occurred when closing the ES REST client", ioe);
}
}
}
pom.xml:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.4.RELEASE</version>
</parent>
<dependencies>
<!-- Spring -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!-- Elasticsearch -->
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>5.5.0</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>5.5.0</version>
</dependency>
<!-- Apache Commons -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.6</version>
</dependency>
<!-- Log4j -->
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
</dependencies>
This makes me think that the RestClient was never explicitly closing the connection, in the first place...
And this is surprising since my Elasticsearch Spring Boot based Microservice is load balanced on two different AWS EC-2 Servers.
That exception occurreded like 2000 times reported by the log file and only in the end did the preDestroy() close the client. See the INFO from the #PreDestroy() cleanup method being logged at the end of the StackTrace.
Do I need to explicitly put a finally clause inside the SearchController and close the RestClient connection explicitly?
It's really critical that this IOException doesn't happen again because this Search Microservice is dependent on a lot of different mobile clients (iOS & Android).
Need this to be fault tolerant and scalable... Or, at the very least, not to break.
The only reason this is in the bottom of the log file:
2017-09-29 18:50:53.894 INFO 11099 --- [Thread-3] com.app.controller.SearchController : Closing the ES REST client
Is because I did this:
kill -3 jvm_pid
Should I keep the #PreDestory cleanup() method but change the contents of my SearchController.getSearchResults() method to reflect something like this:
#RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json" )
public ResponseEntity<Object> getSearchQueryResults(#RequestParam(value = "criteria") String criteria) throws IOException {
// Setup HTTP Headers
HttpHeaders headers = new HttpHeaders();
headers.add("Content-Type", "application/json");
// Setup query and send and return ResponseEntity...
Response response = null;
try {
// Submit Query and Obtain Response
response = this.restClient.performRequest("POST", endPoint, Collections.singletonMap("pretty", "true"), entity);
}
catch(IOException ioe) {
logger.error("Exception when performing POST request " + ioe);
}
finally {
this.restClient.close();
}
// return response as EsResponse();
}
This way the RestClient connection is always closing...
Would appreciate if someone can help me with this.

From my point of view, there are few thing that you are doing wrong but I will go directly to the solution.
I'm not going to write the full solution (in fact, I didn't execute or test anything), but the important is to understand it. Also, it is better if you move all related with data access to another layer. Anyway, this is only an example so the design is not perfect.
Step 1: Import the right library.
Practically the same as your example.
I updated the example to use the last client library recommended in version 5.6.2
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.acervera</groupId>
<artifactId>elastic-example</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>elastic-example</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<es.version>5.6.2</es.version>
</properties>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.4.RELEASE</version>
</parent>
<dependencies>
<!-- Spring -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!-- Elasticsearch -->
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>${es.version}</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>${es.version}</version>
</dependency>
<!-- Log4j -->
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
</dependencies>
</project>
Step 2: Create and shutdown the client in the bean factory.
In the bean factory, create and destroy it. You can reuse the same client.
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import java.io.IOException;
#Configuration
public class ElasticsearchConfig {
// Here all init stuff with #Value(....)
RestClient lowLevelRestClient;
RestHighLevelClient client;
#PostConstruct
public void init() {
lowLevelRestClient = RestClient.builder(new HttpHost("host", 9200, "http")).build();
client = new RestHighLevelClient(lowLevelRestClient);
}
#PreDestroy
public void destroy() throws IOException {
lowLevelRestClient.close();
}
#Bean
public RestHighLevelClient getClient() {
return client;
}
}
Step 3: Execute the query using the Java Transport Client.
Use the Java Transport Client to execute the query.
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import java.io.IOException;
#RestController
#RequestMapping("/api/v1")
public class SearchController {
#Autowired
private RestHighLevelClient client;
#RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json" )
public ResponseEntity<Tweet> getSearchQueryResults(#RequestParam(value = "criteria") String criteria) throws IOException {
// This is only one example. Of course, this logic make non sense and you are going to put it in a DAO
// layer with more logical stuff
SearchRequest searchRequest = new SearchRequest();
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchAllQuery());
SearchResponse searchResponse = client.search(searchRequest);
if(searchResponse.getHits().totalHits > 0) {
SearchHit searchHit = searchResponse.getHits().iterator().next();
// Deserialize to Java. The best option is to use response.getSource() and Jackson
// This is other option.
Tweet tweet = new Tweet();
tweet.setId(searchHit.getField("id").getValue().toString());
tweet.setTittle(searchHit.getField("tittle").getValue().toString());
return ResponseEntity.ok(tweet);
} else {
return ResponseEntity.notFound().build();
}
}
}
Also, use a bean to build the response.
public class Tweet {
private String id;
private String tittle;
public String getTittle() {
return tittle;
}
public void setTittle(String tittle) {
this.tittle = tittle;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
// Here rest of bean stuff (equal, hash, etc) or Lombok
}
Step: 4
Enjoy Elasticsearch!
Notes: Java REST Client [5.6] ยป Java High Level REST Client
PS. It is necessary to refactor the example. It is only to understand the way.

Are you sure that you don't initiate a new (thread pool) connection to the elasticsearch server on every HTTP request? I.e., in line
Response response = this.restClient.performRequest(...);
Double-check the logs on the elasticsearch server after a single HTTP request. You should try implementing a Singleton pattern without the #Autowired annotation and see if the problem persists.

From your stacktrace, it appears that embedded tomcat(your application container) is not longer able to accept new connection due to too many open files error. From your code, elasticsearch rest client does not seems problematic.
Since you are re-using the single instance of RestClient while servicing your search request, there may not be more more than 30 (org.elasticsearch.client.RestClientBuilder.DEFAULT_MAX_CONN_TOTAL) open connections with ES cluster. So it is unlikely that RestClient it is causing the issue.
Other potential root cause may be your service's consumer are keeping connection open for longer time with your (tomcat) server or they are not closing connection properly.
Do I need to explicitly put a finally clause inside the
SearchController and close the RestClient connection explicitly?
No. You shouldn't. Rest client should be closed while bringing down your service(in a #PreDestroy method as you are already doing correctly).

Related

SpringBoot connection to Redis : Does not appear to be caching data

I am using SpringBoot to connect to Redis. I have Web dependency on SpringBoot and my intention is to write Product information to a runtime datastructure i.e., a Map. I want to test the Cache annotations that Spring provides to understand the usage.
Here is the POM.xml file
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.0</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>io.fireacademy</groupId>
<artifactId>redisconnectivity</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>redisconnectivity</name>
<description>Demo project for Spring Boot & Redis connectivity</description>
<properties>
<java.version>11</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
</dependency>
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
Here is the Spring Application class
package io.fireacademy.redisconnectivity;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cache.annotation.EnableCaching;
#SpringBootApplication
#EnableCaching
public class RedisconnectivityApplication {
public static void main(String[] args) {
SpringApplication.run(RedisconnectivityApplication.class, args);
}
}
Here is my main controller class
package io.fireacademy.redisconnectivity.controllers;
import io.fireacademy.redisconnectivity.model.Product;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cache.annotation.CacheEvict;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
#RestController
#RequestMapping("/products")
public class WebAppController {
private static final Logger logger = LoggerFactory.getLogger(WebAppController.class);
// This will serve as the database
private Map<String, Product> m_productDatabase = new HashMap<String, Product>();
#Cacheable(value="my-product-cache", key="#productId")
private Product getProductFromCacheOrDB(String productId)
{
logger.info("Loading the Product " + productId + " from the cache!.");
return m_productDatabase.get(productId);
}
#CacheEvict(value="my-product-cache", key="#productId")
private Product deleteFromCache(String productId)
{
logger.info("Remove the Product " + productId + " from the cache!.");
return m_productDatabase.get(productId);
}
#GetMapping(path="/")
public ResponseEntity<List<Product>> getProducts()
{
Collection<Product> allProducts = m_productDatabase.values();
List<Product> allProductsAsList = allProducts.stream().collect(Collectors.toList());
return new ResponseEntity<List<Product>>(allProductsAsList, HttpStatus.OK);
}
#GetMapping(path="/{productId}")
public ResponseEntity<Product> getProducts(#PathVariable String productId)
{
// Either from the Cache or from the DB.
Product product = getProductFromCacheOrDB(productId);
return new ResponseEntity<Product>(product, HttpStatus.OK);
}
#PostMapping(consumes = {MediaType.APPLICATION_XML_VALUE,MediaType.APPLICATION_JSON_VALUE},
produces = {MediaType.APPLICATION_XML_VALUE,MediaType.APPLICATION_JSON_VALUE})
public ResponseEntity<Product> createProduct(#RequestBody Product product)
{
m_productDatabase.put(product.getId(), product);
return new ResponseEntity<Product>(product, HttpStatus.CREATED);
}
#PutMapping(path="/{productId}",
consumes = {MediaType.APPLICATION_XML_VALUE,MediaType.APPLICATION_JSON_VALUE},
produces = {MediaType.APPLICATION_XML_VALUE,MediaType.APPLICATION_JSON_VALUE})
public ResponseEntity<Product> updateProduct(#PathVariable String productId, #RequestBody Product product)
{
m_productDatabase.put(productId, product);
return new ResponseEntity<Product>(product, HttpStatus.OK);
}
#DeleteMapping(path="/{productId}")
public ResponseEntity<Product> deleteProduct(#PathVariable String productId)
{
Product deletedProduct = getProductFromCacheOrDB(productId);
deleteFromCache(productId);
return new ResponseEntity<Product>(deletedProduct, HttpStatus.OK);
}
}
Here is my RedisConfig class
package io.fireacademy.redisconnectivity.configurations;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisStandaloneConfiguration;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.repository.configuration.EnableRedisRepositories;
import org.springframework.data.redis.serializer.JdkSerializationRedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
#Configuration
#EnableRedisRepositories
public class RedisConfig {
#Bean
public JedisConnectionFactory connectionFactory() {
RedisStandaloneConfiguration configuration = new RedisStandaloneConfiguration();
configuration.setHostName("localhost");
configuration.setPort(6379);
return new JedisConnectionFactory(configuration);
}
#Bean
public RedisTemplate<String, Object> template() {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(connectionFactory());
template.setKeySerializer(new StringRedisSerializer());
template.setHashKeySerializer(new StringRedisSerializer());
template.setHashKeySerializer(new JdkSerializationRedisSerializer());
template.setValueSerializer(new JdkSerializationRedisSerializer());
template.setEnableTransactionSupport(true);
template.afterPropertiesSet();
return template;
}
}
My application.properties looks as below:
# Redis Config
spring.cache.type=redis
spring.redis.host=localhost
spring.redis.port=6379
And I am running Redis as a docker container using this:
docker run -d -p 6379:6379 --name my-redis redis
When I inspect the docker container's logs, I see nothing happening.
D:\Development\springboot\learn_redis>docker logs -f c376f1be9be35281b900c2943fbf8ea37e1563157efb57d46ca1c74fc880bc5c
1:C 04 Jun 2022 17:49:51.854 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 04 Jun 2022 17:49:51.854 # Redis version=7.0.0, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 04 Jun 2022 17:49:51.854 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 04 Jun 2022 17:49:51.854 * monotonic clock: POSIX clock_gettime
1:M 04 Jun 2022 17:49:51.858 * Running mode=standalone, port=6379.
1:M 04 Jun 2022 17:49:51.858 # Server initialized
1:M 04 Jun 2022 17:49:51.858 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 04 Jun 2022 17:49:51.858 * The AOF directory appendonlydir doesn't exist
1:M 04 Jun 2022 17:49:51.858 * Ready to accept connections
My understanding was that the Cache annotations would make the data be stored onto the Redis cache. For instance, the getProductFromCacheOrDB, should store the Product object onto the cache with the input productId as the key because of the #Cacheable annotation and a subsequent call to get this specific Product using the GET productId should not invoke the method again. But this is not happening.
Please show some pointers on what I could have missed...
Thanks,
Pavan.
Edits:
I do not see anything on Redis getting created. I enabled monitoring on Redis via the redis-cli, but see nothing.
I tried removing the RedisConfig class and see no change.
Just created a simple working example at https://github.com/bsbodden/basic-caching-spring-data-redis
What you need:
POM dependencies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-cache</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>
In the app class or configurer #EnableCaching:
#SpringBootApplication
#EnableCaching
public class DemoApplication {
In the app class RedisCacheManager bean:
#Bean
public RedisCacheManager cacheManager(RedisConnectionFactory connectionFactory) {
RedisCacheConfiguration config = RedisCacheConfiguration.defaultCacheConfig() //
.prefixCacheNameWith(this.getClass().getPackageName() + ".") //
.entryTtl(Duration.ofHours(1)) //
.disableCachingNullValues();
return RedisCacheManager.builder(connectionFactory) //
.cacheDefaults(config) //
.build();
}
#Cacheable in your controller:
#GetMapping("/{id}")
#Cacheable("some-cache")
public SomeModel get(#PathVariable("id") String id) {
return repo.findById(id).orElse(SomeModel.of("nope"));
}

Unable to resolve View using Thymeleaf

I am new to Thymeleaf.I am creating a spring-boot app using thymeleaf in view part.
Please find below my pom.xml file[only the dependencies part]:
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.10.RELEASE</version>
</parent>
<properties>
<java.version>1.8</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-jasper</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-thymeleaf</artifactId>
</dependency>
</dependencies>
I have created an application.properties file in following location:src/main/resources
server.port = 8086
spring.mvc.static-path-pattern=/resources/**
spring.thymeleaf.prefix=/WEB-INF/views/
spring.thymeleaf.suffix=.html
Following is my Boot Application Main class:
#SpringBootApplication
#ComponentScan(basePackages="<common package structure>")
public class AdvMain extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(AdvMain.class);
}
public static void main(String[] args) {
SpringApplication.run(AdvMain.class, args);
}
}
I have a controller class as follows:
#Controller
public class HomeController {
#RequestMapping(path = "/info", method = { RequestMethod.GET, RequestMethod.POST })
public String info(Model model) {
model.addAttribute("message", "Thymeleaf");
return "info";
}
#RequestMapping(path = "/home", method = RequestMethod.GET)
public String index() {
return "index";
}
}
When I invoke the following url:
http://localhost:8086/AddViewer/home
I am getting page not found error.
Please find below my tomcat console log:
Mapped "{[/home],methods=[GET]}" onto public java.lang.String <package_Structure>.HomeController.index()
Mapped "{[/info],methods=[GET || POST]}" onto public java.lang.String <package_Structure>.HomeController.info(org.springframework.ui.Model)
Mapped URL path [/resources/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
Tomcat started on port(s): 8086 (http)
FrameworkServlet 'dispatcherServlet': initialization completed in 29 ms
No mapping found for HTTP request with URI [/AddViewer/home] in DispatcherServlet with name 'dispatcherServlet'
Can anyone provide any suitable solution to this???
This is not a problem of thymeleaf.
In your console message, you have two mappings, /home and /info.
Mapped "{[/home],methods=[GET]}" onto public java.lang.String <package_Structure>.HomeController.index()
Mapped "{[/info],methods=[GET || POST]}" onto public java.lang.String <package_Structure>.HomeController.info(org.springframework.ui.Model)
But you made a request /AddViewer/home. So, it does not matches with request mappings.
Request like this.
http://localhost:8086/home
if you using thymeleaf put your view files in src/main/resources/template and no need to add these two line -
spring.thymeleaf.prefix=/WEB-INF/views/
spring.thymeleaf.suffix=.html

How test #WebServlet with Jersey Test Framework? (Throws Http 404 not found)

When I run my Maven tests with the Jersey Test Framework for a class with #Path, it works fine. But when i try to test a #WebServlet class, it does not and says it's a 404 error.
how do I test a web servlet with Jersey Test Framework? (Or, if this isn't possible, how else can i test this?)
Web Servlet:
package com.testservlet;
#WebServlet("/test")
public class TestServlet extends HttpServlet
{
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException
{
resp.getWriter().println( "The Result" );
}
}
Test class:
import com.testservlet.TestServlet;
import org.glassfish.jersey.server.ResourceConfig;
import org.glassfish.jersey.test.JerseyTest;
import org.junit.Assert;
import org.junit.Test;
import javax.ws.rs.core.Application;
public class TestEndpoint extends JerseyTest
{
#Override
protected Application configure()
{
return new ResourceConfig( JPAServlet.class );
}
#Test
public void baseGetTest()
{
String response = target( "/test" ).request().get( String.class );
Assert.assertTrue( "92096".equals( response ) );
}
}
Maven:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>groupId</groupId>
<artifactId>TestWeb</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>war</packaging>
<!-- Tell Maven what language version to use -->
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<!-- Enables the annotations, etc needed -->
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-api</artifactId>
<version>7.0</version>
<exclusions>
<exclusion>
<groupId>javax.exterprise</groupId>
<artifactId>cdi-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- Our jersey libs -->
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-servlet-core</artifactId>
<version>2.25.1</version>
</dependency>
<!-- CDI to JAX-RS Binding -->
<!-- https://mvnrepository.com/artifact/org.glassfish.jersey.containers.glassfish/jersey-gf-cdi-ban-custom-hk2-binding -->
<dependency>
<groupId>org.glassfish.jersey.containers.glassfish</groupId>
<artifactId>jersey-gf-cdi</artifactId>
<version>2.14</version>
</dependency>
<!-- TESTING WEB -->
<dependency>
<groupId>org.glassfish.jersey.test-framework.providers</groupId>
<artifactId>jersey-test-framework-provider-jdk-http</artifactId>
<version>2.25.1</version>
</dependency>
</dependencies>
</project>
First you will need to use a container that supports servlet deployment, like the grizzly container
<dependency>
<groupId>org.glassfish.jersey.test-framework.providers</groupId>
<artifactId>jersey-test-framework-provider-grizzly2</artifactId>
<version>${jersey2.version}</version>
</dependency>
Then you need to configure the test class to deploy as a servlet container.
#Override
public TestContainerFactory getTestContainerFactory() {
return new GrizzlyWebTestContainerFactory();
}
#Override
public DeploymentContext configureDeployment() {
return ServletDeploymentContext.forServlet(TestServlet.class)
.build();
}
Here, you will not need to override configure method like you're currently doing. However, the above configuration doesn't configure a Jersey application. It only configures your test servlet. If you want to run Jersey in a servlet environment (which the default JerseyTest does not do), then you would configure Jersey like
#Override
public DeploymentContext configureDeployment() {
final ResouceConfig config = ResourceConfig();
return ServletDeploymentContext.builder(config)
.build();
}
There a lot of other things you can configure with the ServletDeploymentContext; just look at the docs.
UPDATE
Instead of trying to start up a server. You might be better off writing a unit test (rather than an integration test), and just mock dependencies using a mocking framework like Mockito. There's an example below. I added some comment as to what each line does. For more info on Mockito, please refer to the linked docs.
public class WebServletTest {
#Test
public void testServletGet() throws Exception {
// create some mocks
SomeDao dao = mock(SomeDao.class);
HttpServletRequest request = mock(HttpServletRequest.class);
HttpServletResponse response = mock(HttpServletResponse.class);
PrintWriter writer = mock(PrintWriter.class);
// do some stubbing
when(response.getWriter()).thenReturn(writer);
when(dao.getData(anyString())).thenReturn("Hello World");
// used to capture argument to writer.println
ArgumentCaptor<String> captor = ArgumentCaptor.forClass(String.class);
// create the servlet and call the method under test
TestServlet servlet = new TestServlet(dao);
servlet.doGet(request, response);
// capture the argument to writer.println
verify(writer).println(captor.capture());
String arg = captor.getValue();
// make assetions
assertThat(arg).isEqualTo("Hello World");
}
private interface SomeDao {
String getData(String param);
}
#WebServlet("/test")
public static class TestServlet extends HttpServlet {
private final SomeDao dao;
#Inject
public TestServlet(SomeDao dao) {
this.dao = dao;
}
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
resp.getWriter().println(this.dao.getData(""));
}
}
}

Spring RestTemplate & AsyncRestTemplate with Netty4 hangs forever

Very simple setup:
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>demo-rest-client</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>demo-rest-client</name>
<description>Demo project for Spring Boot</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.4.1.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.5.Final</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-buffer</artifactId>
<version>4.1.5.Final</version>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>3.4.1</version>
</dependency>
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-core</artifactId>
<version>9.3.1</version>
</dependency>
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-hystrix</artifactId>
<version>9.3.1</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
And a test case to demonstrate different usages of AsyncRestTemplate:
SampleTests.java
package com.example;
import com.netflix.hystrix.HystrixCommand;
import com.netflix.hystrix.HystrixCommandProperties;
import feign.RequestLine;
import feign.hystrix.HystrixFeign;
import feign.hystrix.SetterFactory;
import org.junit.Test;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.Netty4ClientHttpRequestFactory;
import org.springframework.http.client.OkHttp3ClientHttpRequestFactory;
import org.springframework.util.concurrent.ListenableFuture;
import org.springframework.web.client.AsyncRestTemplate;
import org.springframework.web.client.RestTemplate;
public class SampleTests {
private static final String URL = "https://api.github.com/users/octocat";
private static final int DEFAULT_SLEEP_MILLIS = 20;
private static final int DEFAULT_TIMEOUT = 10000;
#Test(timeout = DEFAULT_TIMEOUT)
public void syncRestNetty() throws Exception {
RestTemplate restTemplate = new RestTemplate(new Netty4ClientHttpRequestFactory());
ResponseEntity<String> response = restTemplate.getForEntity(URL, String.class);
System.out.println("response = " + response);
}
#Test(timeout = DEFAULT_TIMEOUT)
public void asyncRestNetty() throws Exception {
AsyncRestTemplate restTemplate = new AsyncRestTemplate(new Netty4ClientHttpRequestFactory());
ListenableFuture<ResponseEntity<String>> listenableFuture = restTemplate.getForEntity(URL, String.class);
listenableFuture.addCallback(result -> System.out.println("result = " + result), Throwable::printStackTrace);
while (!listenableFuture.isDone()) {
Thread.sleep(DEFAULT_SLEEP_MILLIS);
}
System.out.println("the end");
}
#Test
public void asyncRestOkHttp() throws Exception {
AsyncRestTemplate restTemplate = new AsyncRestTemplate(new OkHttp3ClientHttpRequestFactory());
ListenableFuture<ResponseEntity<String>> listenableFuture = restTemplate.getForEntity(URL, String.class);
listenableFuture.addCallback(result -> System.out.println("result = " + result), Throwable::printStackTrace);
while (!listenableFuture.isDone()) {
Thread.sleep(DEFAULT_SLEEP_MILLIS);
}
System.out.println("the end");
}
#Test
public void asyncRestHystrixFeign() throws Exception {
GitHub gitHub = HystrixFeign.builder()
.setterFactory((target, method) -> new SetterFactory.Default().create(target, method).andCommandPropertiesDefaults(HystrixCommandProperties.defaultSetter().withExecutionTimeoutInMilliseconds(10000)))
.target(GitHub.class, "https://api.github.com");
HystrixCommand<String> command = gitHub.octocatAsync();
command.toObservable().subscribe(result -> System.out.println("result = " + result), Throwable::printStackTrace);
while (!command.isExecutionComplete()) {
Thread.sleep(DEFAULT_SLEEP_MILLIS);
}
System.out.println("command.getExecutionTimeInMilliseconds() = " + command.getExecutionTimeInMilliseconds());
System.out.println("the end");
}
interface GitHub {
#RequestLine("GET /users/octocat")
HystrixCommand<String> octocatAsync();
}
}
When trying to run the tests which use Netty they just hang forever. (To see this please remove the JUnit timeout constraint). But if I run the exact same code with other clients everything works as expected.
I have tried different versions of Spring Boot and Netty but did not succeed. And from the logs everything looks ok.
What am I missing here?
EDIT:
Opened a ticket https://jira.spring.io/browse/SPR-14744 as suggested on Spring Gitter
EDIT-2:
Answer from Brian Clozel helped me find the issue which is related to Netty not realizing the server sent an empty response (a particular case with Github API and plain http) so I am marking it as accepted.
Can you try to configure your request factory with a Netty Sslcontext?
Netty4ClientHttpRequestFactory nettyFactory = new Netty4ClientHttpRequestFactory();
nettyFactory.setSslContext(SslContextBuilder.forClient().build());
AsyncRestTemplate restTemplate = new AsyncRestTemplate(nettyFactory);
Without that context, the client is trying to send plaintext requests to the https endpoint; in that case, you're probably getting an HTTP 400 response.
In your example code, the throwable should be an instance of HttpClientErrorException, and you could get that information by logging the response status or its body with exception.getResponseBodyAsString().

Enabling WebSockets using spring

I am trying to configure web sockets with spring using a Wildfly 10 server. As per this tutorial, I have the following files:
This is the web socket class:
package com.myapp.spring.web.controller;
import javax.websocket.OnClose;
import javax.websocket.OnError;
import javax.websocket.OnMessage;
import javax.websocket.OnOpen;
import javax.websocket.server.ServerEndpoint;
import org.springframework.web.socket.server.standard.SpringConfigurator;
#ServerEndpoint(value="/serverendpoint", configurator = SpringConfigurator.class)
/**
* This class creates web sockets, opens, and maintains connection with the client
*/
public class serverendpoint {
#OnOpen
public void handleOpen () {
System.out.println("JAVA: Client is now connected...");
}
#OnMessage
public String handleMessage (String message) {
if (message.equals("ping"))
return "pong";
else if (message.equals("close")) {
handleClose();
return null;
}
System.out.println("JAVA: Received from client: "+ message);
if (message.contains("//")) {
MyClass mc = new MyClass(message);
return mc.someMethod();
} else {
System.out.println("Message From Web Socket Not Understood");
return null;
}
}
#OnClose
public void handleClose() {
System.out.println("JAVA: Client is now disconnected...");
}
#OnError
public void handleError (Throwable t) {
t.printStackTrace();
}
}
This is the web socket config file:
package com.myapp.spring.security.config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.socket.server.standard.ServerEndpointExporter;
import com.myapp.spring.web.controller.serverendpoint;
#Configuration
public class EndpointConfig {
#Bean
public serverendpoint serverendpoint() {
return new serverendpoint();
}
#Bean
public ServerEndpointExporter endpointExporter() {
return new ServerEndpointExporter();
}
}
This is my pom.xml:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>4.3.2.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc</artifactId>
<version>4.3.2.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
<version>1.4.0.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-websocket</artifactId>
<version>1.4.0.RELEASE</version>
</dependency>
According to the tutorial, this is all I have to do. But I get the following errors:
Failed to start service jboss.undertow.deployment.default-server.default-host./ROOT: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./ROOT: java.lang.RuntimeException: java.lang.ClassCastException: org.apache.tomcat.websocket.server.WsServerContainer cannot be cast to io.undertow.websockets.jsr.ServerWebSocketContainer
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:85)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at org.jboss.threads.JBossThread.run(JBossThread.java:320)
Caused by: java.lang.RuntimeException: java.lang.ClassCastException: org.apache.tomcat.websocket.server.WsServerContainer cannot be cast to io.undertow.websockets.jsr.ServerWebSocketContainer
at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:231)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:100)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)
... 6 more
Caused by: java.lang.ClassCastException: org.apache.tomcat.websocket.server.WsServerContainer cannot be cast to io.undertow.websockets.jsr.ServerWebSocketContainer
at io.undertow.websockets.jsr.Bootstrap$WebSocketListener.contextInitialized(Bootstrap.java:104)
at io.undertow.servlet.core.ApplicationListeners.contextInitialized(ApplicationListeners.java:187)
at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:198)
... 8 more
What is the fix to this problem? In addition, are there any other config files I need to add in order for my web socket to be mapped correctly at the endpoint /serverendpoint as I did in my serverendpoint() class (I am asking this because I am a bit unsure if I only need one config file or not. It doesn't seem right. I looked around and others have included other files with, for instance, the #EnableWebSocket, but the tutorial says that I only need these two files.)?
Thank you so much!
Please go through https://github.com/spring-projects/spring-boot/issues/6166 and see if this solves your issue.
There is a similar issue reported in SO at Spring Boot Websockets in Wildfly. Hope this helps.

Categories