High latency with Spring multipart file - java

I'm currently working on a Spring-boot project in which I have to send files (one at a time) through a REST Web Service to a server, which sends it to a Cassandra cluster.
This part works, but my problem appeared when I began to do stress tests (with Gatling) on my project.
Here it is :
- When I tested my work without sending files, I had latency ~10-15 ms ;
- When I tested my work sending a file in the body of the request, I had latency ~10-15 ms too (maybe a little higher by 5 or 10 ms, I don't remember);
- BUT, and my problem shows up here, when I tested the Web Service by sending files with an input, and getting it with MultipartFile, the latency was very variable, from 10 ms for the fastest requests, to 12000ms (!!) for the longest.
I tried to set the files in memory limit to 1Go (it was originaly ~10ko), but it didn't change anything.
I don't get any error while executing the tests, it's just very long...
Plus, this project is being made to modernize the existing webservice, which runs as a JAX-WS, with WSDL and all. The team working on the old project (we are switching from a DBMS to an other) has response times of ~10 in mean !
Here's my conf (this is for a POC, the machines are cheap) :
- Web server : Wildfly 9-0-1.FINAL out of the box, on CentOS Server 7.1;
- Stress tests machine : Gatling 2.1.7, on the same CentOS version as Wildfly, but on an other machine;
- An 8 nodes Cassandra cluster.
Here is my pom (without company's secrets | concerning the spring-boot dependency, it's not my project's parent, it already has one):
<packaging>war</packaging>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>1.3.0.RELEASE</version>
<exclusions>
<exclusion>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-websocket</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<version>1.3.0.RELEASE</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.3.1</version>
</dependency>
</dependencies>
<properties>
<java.version>1.8</java.version>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.6</version>
<configuration>
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>spring-releases</id>
<url>https://repo.spring.io/libs-release</url>
</repository>
<repository>
<id>oss-jfrog-artifactory</id>
<name>oss-jfrog-artifactory-releases</name>
<url>http://oss.jfrog.org/artifactory/oss-release-local</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>spring-releases</id>
<url>https://repo.spring.io/libs-release</>
</pluginRepository>
</pluginRepositories>
<dependencyManagement>
<dependencies>
<dependency>
<!-- Import dependency management from Spring Boot -->
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>1.3.0.RELEASE</version>
<type>pom</type>
</dependency>
</dependencies>
</dependencyManagement>
StartApp.java:
package foo.bar;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.web.MultipartAutoConfiguration;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.context.web.SpringBootServletInitializer;
import org.springframework.context.annotation.Bean;
import org.springframework.web.multipart.MultipartResolver;
import org.springframework.web.multipart.commons.CommonsMultipartResolver;
#SpringBootApplication
#EnableAutoConfiguration(exclude = MultipartAutoConfiguration.class)
public class StartApp extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(
SpringApplicationBuilder application) {
return application.sources(StartApp.class);
}
public static void main(String[] args) {
SpringApplication.run(StartApp.class, args);
}
#Bean(name = "multipartResolver")
public MultipartResolver multipartResolver() {
CommonsMultipartResolver multipartResolver = new CommonsMultipartResolver();
multipartResolver.setMaxInMemorySize(1024 * 1024 * 1024); // 1Go
return multipartResolver;
}
}
WS.java (only the relevant function) :
import static org.springframework.web.bind.annotation.RequestMethod.POST;
import java.io.IOException;
import java.io.InputStream;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.multipart.MultipartFile;
import service.DaoException;
import service.ErrCode;
import service.IService;
...
#Inject IService service;
...
#RequestMapping(path = "/insert/{foo}/{bar}", method = POST)
public ResponseEntity<Void> insert(#PathVariable String foo,
#PathVariable String bar,
#RequestParam MultipartFile document) {
HttpStatus httpCode = HttpStatus.CREATED;
try (InputStream is = document.getInputStream()) {
service.insert(foo, bar, is);
} catch (DaoException e) {
logger.error("Problème lors de l'insertion du document : "
+ e.getMessage());
httpCode = handleError(e.getCode());
} catch (IOException e1) {
logger.error("Problème lors du traitement du document envoyé : "
+ e1.getMessage());
httpCode = handleError(ErrCode.PB_TECHNIQUE);
}
return ResponseEntity.status(httpCode).build();
}
Simulation.scala :
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._
import java.nio.file.Files
import java.nio.file.Paths
import java.nio.file.Path
import java.util.Base64
import java.io.File
import java.lang.String
class RestSimulation1 extends Simulation {
val DEBIT = 30 //Clients per second
val DUREE = 300 //Duration
val httpConf = http
.baseURL("http://myserver:8080") // Here is the root for all relative urls
.acceptHeader("text/html,application/xhtml+xml,application/json,application/xml;q=0.9,*/*;q=0.8") // Here are the common headers
.doNotTrackHeader("1")
.acceptLanguageHeader("en-US,en;q=0.5")
.acceptEncodingHeader("gzip, deflate")
.userAgentHeader("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0")
//Feeders creation
val feeder = csv("9k_rest.csv").queue
object Insert {
val insert = feed(feeder)
.exec(http("insertion")
.post("/webservice/ws/insert/${fooo}/${bar}")
.formUpload("document", "rest_insert_${format}.txt")
.check(status.is(201)))
}
val scn = scenario("").exec(Insert.insert)
setUp(scn.inject( constantUsersPerSec(DEBIT) during(DUREE seconds))).protocols(httpConf)
}
What do I do wrong ?
Thank you for your help :)

Related

AWS Lambda gives class not found exception for jdbc and org.json JARS

I'm trying to run an AWS Lambda function with the help of eclipse and Maven using AWS tool kit. When I run the project as a Maven test, it successfully runs. But when I export the project as a .jar and upload the jar to AWS Lambda, it gives me the following error trace:
Input: ToRupeeDatabase error: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
Done inserting the data in RDS
The code says it runs successfully and the data has been inserted in the RDS, but it is being caught as an exception and the code is crashing.
Code:
package com.amazonaws.torupee.demo;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import org.json.JSONObject;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.util.Arrays;
import java.util.HashSet;
import java.util.logging.Level;
import java.util.logging.Logger;
public class RemoteDataCRON implements RequestHandler<Object, String> {
#Override
public String handleRequest(Object input, Context context) {
context.getLogger().log("Input: " + input);
try {
// Exception in the following line when uploaded on AWS Lambda!
Class.forName("com.mysql.jdbc.Driver");
String endPoint = "pathtodatabase?useSSL=false";
String username = "foor";
String password = "bar";
PreparedStatement stmt = null;
try {
Connection dbConnection = DriverManager.getConnection(endPoint, username, password);
} catch (SQLException ex) {
Logger.getLogger(RemoteDataCRON.class.getName()).log(Level.SEVERE, null, ex);
}
// Some processing here and org.json exception here!
} catch (Exception ex) {
System.err.println("Database error: " + ex);
}
System.out.println("Done inserting the data in RDS");
// TODO: implement your handler
return "Hello from Lambda!";
}
}
.pom file:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.amazonaws.torupee</groupId>
<artifactId>demo</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<encoding>UTF-8</encoding>
<forceJavacCompilerUse>true</forceJavacCompilerUse>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-bom</artifactId>
<version>1.11.256</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-events</artifactId>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-core</artifactId>
<version>1.1.0</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.39</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20180130</version>
</dependency>
</dependencies>
</project>
Here is a list of solutions that I tried:
I tried adding external JARS for mysql and org.json but that gave me the same errors
I tried zipping the project instead of exporting it as a JAR but that gave me ClassNotFoundException on the handler itself
I then added the dependencies in the pom file itself as I thought that external JARS were not bundled when making a .jar file
I then changed the driver path from "com.mysql.jdbc.Driver" to "com.mysql.cj.jdbc.Driver" but that still gave me the exception on the console
Am I doing something wrong or building the Jar incorrectly, is it why Lambda gives me the error when I'm on the management console?
So I solved this by completely ditching the maven approach and did it using a simple java class. You can follow the solution for creating a simple AWS Lambda function in java step by step with this video tutorial I created.
https://www.youtube.com/watch?v=-Pm22SJibpQ&t=40s

Pact File Not Generated Inspite of test being Passed

I am trying to generate Pact file.The test is passing when is "Run As-Junit Test" in Eclipse. However, unable to really understand why the Pact Contract File is not generated. Can you please help? Below is my test code:
package pact;
import au.com.dius.pact.consumer.*;
import au.com.dius.pact.consumer.dsl.DslPart;
import au.com.dius.pact.consumer.dsl.PactDslJsonBody;
import au.com.dius.pact.model.PactFragment;
import org.junit.Assert;
import au.com.dius.pact.consumer.dsl.PactDslWithProvider;
import au.com.dius.pact.consumer.dsl.PactDslWithState;
import org.junit.Rule;
import org.junit.Test;
import utils.Configuration;
import java.io.IOException;
import static org.junit.Assert.assertEquals;
public class GetHelloWorldTest
{
#Rule
public PactProviderRule rule = new PactProviderRule("PP Provider", "localhost", 9000, this);
private String helloWorldResults;
#Pact(provider = Configuration.DUMMY_PROVIDER,consumer = Configuration.DUMMY_CONSUMER)
public PactFragment createFragment(PactDslWithProvider builder)//TODO
{
return builder
.uponReceiving("get hello world response")
.path("/hello-world")
.method("GET")
.willRespondWith()
.status(200)
.body("{\"id\":2,\"content\":\"Hello, Stranger!\"}")
.toFragment();
}
#Test
#PactVerification(value = "PP provider")
public void shouldGetHelloWorld() throws IOException
{
DummyConsumer restClient = new DummyConsumer(Configuration.SERVICE_URL);
Assert.assertEquals("{\"id\":32,\"content\":\"Hello, Stranger!\"}",restClient.getHelloWorld());
}
}
My POM File is as below :
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>consumer</groupId>
<artifactId>consumer</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-plugin-api</artifactId>
<version>3.3.9</version>
</dependency>
<dependency>
<groupId>org.apache.maven.reporting</groupId>
<artifactId>maven-reporting-impl</artifactId>
<version>2.2</version>
</dependency>
<dependency>
<groupId>commons-cli</groupId>
<artifactId>commons-cli</artifactId>
<version>1.4</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>fluent-hc</artifactId>
<version>4.5.2</version>
</dependency>
<dependency>
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
<version>3.0.24</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8</version>
</dependency>
<dependency>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-consumer-junit_2.10</artifactId>
<version>2.4.18</version>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>2.4.7</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.0.13</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18</version>
<configuration>
<systemPropertyVariables>
<pact.rootDir>PPPPP/pact</pact.rootDir>
<buildDirectory>${project.build.directory}</buildDirectory>
</systemPropertyVariables>
</configuration>
</plugin>
</plugins>
</build>
</project>
I had the same problem that my Pact file has not been generated in the folder that I specified in the maven-surfire-plugin.
However I realized that the file has been generated, but in the folder "target > pacts". To generate the file I just needed to execute the maven goal "mvn test".
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemPropertyVariables>
<pact.rootDir>src/test/resources/pacts</pact.rootDir>
<buildDirectory>${project.build.directory}</buildDirectory>
</systemPropertyVariables>
I have set the pact.rootDir like this but it always generates the pact contract in the target/pact folder so now I am taking it from there.
It would seem that you're missing your state specification in the createFragment function.
To fix it in your example, change your function to this:
#Pact(state = "PP provider", provider = Configuration.DUMMY_PROVIDER,consumer = Configuration.DUMMY_CONSUMER)
public PactFragment createFragment(PactDslWithProvider builder)
From here, the verification process will know to check for that state because of the name, verify it, then write the Pact file out under the pacts directory, which is the default.
Here's an example that works for me:
package pact;
import au.com.dius.pact.consumer.*;
import au.com.dius.pact.model.PactFragment;
import org.junit.Rule;
import org.junit.Test;
import utils.Configuration;
import java.io.IOException;
import static org.junit.Assert.assertEquals;
public class GetHelloWorldTest
{
#Rule
public PactRule rule = new PactRule(Configuration.MOCK_HOST, Configuration.MOCK_HOST_PORT, this);
private DslPart helloWorldResults;
#Pact(state = "HELLO WORLD", provider = Configuration.DUMMY_PROVIDER, consumer = Configuration.DUMMY_CONSUMER)
public PactFragment createFragment(ConsumerPactBuilder.PactDslWithProvider.PactDslWithState builder)
{
helloWorldResults = new PactDslJsonBody()
.id()
.stringType("content")
.asBody();
return builder
.uponReceiving("get hello world response")
.path("/hello-world")
.method("GET")
.willRespondWith()
.status(200)
.headers(Configuration.getHeaders())
.body(helloWorldResults)
.toFragment();
}
#Test
#PactVerification("HELLO WORLD")
public void shouldGetHelloWorld() throws IOException
{
DummyConsumer restClient = new DummyConsumer(Configuration.SERVICE_URL);
assertEquals(helloWorldResults.toString(), restClient.getHelloWorld());
}
}
Pact JVM appears to ignore the buildDirectory property
Try the following configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18</version>
<configuration>
<systemPropertyVariables>
<pact.rootDir>${project.build.directory}/pacts</pact.rootDir>
<buildDirectory>${project.build.directory}</buildDirectory>
</systemPropertyVariables>
</configuration>
</plugin>

Spring boot mvc async execution

I'm trying to execute simplest async REST controller using java.util.concurrent.Callable:
#RequestMapping("/AsyncRequest")
public Callable asyncRequest() {
return () -> {
Thread.sleep(3000);
return "reply";
};
}
Then I run http://localhost/AsyncRequest two times in browser.
The first answer I get in 3 seconds, but the second just after 6 seconds.
It seems like that the requests processed not asynchronously.
Why this happen?
Spring Boot behavior really depends on version and configuration.
The following configuration and code works for me:
Starter
package com.stackoverflow.spring.boot.async;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.context.annotation.Import;
#Import(WebConfig.class)
#SpringBootApplication
public class WebMain {
private final SpringApplication application;
public WebMain() {
final SpringApplicationBuilder applicationBuilder = new SpringApplicationBuilder(WebMain.class);
application = applicationBuilder.build();
}
public SpringApplication getApplication() { return application; }
public static void main(final String[] args) {
new WebMain().getApplication().run(args);
}
}
Configuration
package com.stackoverflow.spring.boot.async;
import org.springframework.boot.context.embedded.ConfigurableEmbeddedServletContainer;
import org.springframework.boot.context.embedded.EmbeddedServletContainerCustomizer;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;
#Configuration
public class WebConfig extends WebMvcConfigurerAdapter implements EmbeddedServletContainerCustomizer
{
public void customize(final ConfigurableEmbeddedServletContainer container) {}
}
Controller
I've changed sleep time here to 6 second and added some payload into response.
package com.stackoverflow.spring.boot.async;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;
import java.util.Date;
import java.util.concurrent.Callable;
#Controller
public class AsyncController {
#RequestMapping(path = "/AsyncRequest", method = RequestMethod.GET)
#ResponseBody
public Callable<String> asyncRequest() {
return () -> {
final long currentThread = Thread.currentThread().getId();
final Date requestProcessingStarted = new Date();
Thread.sleep(6000L);
final Date requestProcessingFinished = new Date();
return String.format(
"request: [threadId: %s, started: %s - finished: %s]"
, currentThread, requestProcessingStarted, requestProcessingFinished);
};
}
}
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.stackoverflow.spring.boot</groupId>
<artifactId>async</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<properties>
<java.version>1.8</java.version>
<spring-boot.version>1.4.1.RELEASE</spring-boot.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot</artifactId>
<version>${spring-boot.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>${spring-boot.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<version>${spring-boot.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<inherited>true</inherited>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>${spring-boot.version}</version>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
<configuration>
<classifier>exec</classifier>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Testing
java -version
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)
--
mvn spring-boot:run
...
--
# start following command from different consoles:
curl http://localhost:8080/AsyncRequest
Results
request: [threadId: 33, started: 15:50:28 - finished: 15:50:34]
request: [threadId: 35, started: 15:50:29 - finished: 15:50:35]
request: [threadId: 37, started: 15:50:30 - finished: 15:50:36]

Get client ip in Jersey 2.22.2

I am trying to access the clients IP that are calling my rest server but I only get null as a response. The web-server is running and I can access it from the web browser.
I have tried with
#Context HttpServletRequest
And also with
#Context ContainerRequest request
request.getRequestHeader("HTTP_FORWARDED")
//HTTP_X_FORWARDED
//HTTP_CLIENT_IP
But neither succeeds, the response is null or blank.
Setup
Jersey v: 2.22.2
Grizzly v: 2.3.22
Java v: 8
Rest.java
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Request;
import javax.ws.rs.core.UriInfo;
#Path("/rest")
public class Rest {
#GET
#Path("/test/")
#Produces(MediaType.APPLICATION_JSON)
public TestAddress test(#Context HttpServletRequest re){
System.out.println(re.getRemoteAddr());
TestAddress adr = new TestAddress();
adr.setAge(32);
adr.setName("Fidde");
adr.setSurename("Lass");
//System.out.println(uriInfo.getBaseUri());
return adr;
}
main.java
import org.glassfish.grizzly.http.server.HttpServer;
import org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpServerFactory;
import org.glassfish.jersey.server.ResourceConfig;
import java.io.IOException;
import java.net.URI;
public class Main {
// Base URI the Grizzly HTTP server will listen on
public static final String BASE_URI = "http://localhost:8080/myapp/";
/**
* Starts Grizzly HTTP server exposing JAX-RS resources defined in this application.
* #return Grizzly HTTP server.
*/
public static HttpServer startServer() {
// create a resource config that scans for JAX-RS resources and providers
// in com.example package
final ResourceConfig rc = new ResourceConfig().packages("com.example");
// create and start a new instance of grizzly http server
// exposing the Jersey application at BASE_URI
return GrizzlyHttpServerFactory.createHttpServer(URI.create(BASE_URI), rc);
}
/**
* Main method.
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
final HttpServer server = startServer();
System.out.println(String.format("Jersey app started with WADL available at "
+ "%sapplication.wadl\nHit enter to stop it...", BASE_URI));
System.in.read();
server.stop();
}
}
Pom.xml
http://maven.apache.org/xsd/maven-4.0.0.xsd">
4.0.0
Test
Bootstrap
jar
0.0.1-SNAPSHOT
<build>
<sourceDirectory>src/main/java</sourceDirectory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<mainClass>backend.Main</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.glassfish.jersey</groupId>
<artifactId>jersey-bom</artifactId>
<version>${jersey.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-json-jackson</artifactId>
<version>2.22.2</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-grizzly2-http</artifactId>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.containers</groupId>
<artifactId>jersey-container-servlet-core</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.2.3</version>
<scope>compile</scope>
</dependency>
</dependencies>
<properties>
<jersey.version>2.22.2</jersey.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
HttpServletRequest provides a getRemoteAddr() method that should returns the remote IP address.
Note that proxying or NATing may modify the IP address.
EDIT :
The solution is to inject a grizzly request :
#GET
#Path("/test/")
#Produces(MediaType.APPLICATION_JSON)
public TestAddress test(#Context org.glassfish.grizzly.http.server.Request re) {
System.out.println(re.getRemoteAddr());
...
}
This is working for me, but it is totally grizzly dependant.

Accessing compiled resources in a Pax Exam contained test

I've been trying to run a JMH benchmark test in an OSGI container created with Pax Exam (similarly to how it's described in this SO answer, but with Pax Exam thrown into the mix as well). But am having some problems getting the JMH generated resource files created during the compilation to load.
After the tests have been compiled, the following can be found in the target/test-classes directory:
$ ls -l target/test-classes/META-INF
BenchmarkList CompilerHints
In a test I use some code (that I cannot change) that (effectively) looks for the file like so:
getClass().getClassLoader().getResources("META-INF/BenchmarkList");
Running this fails (returns 0 results) and I get the following error:
java.lang.RuntimeException: ERROR: Unable to find the resource: /META-INF/BenchmarkList
at org.openjdk.jmh.runner.AbstractResourceReader.getReaders(AbstractResourceReader.java:96)
at org.openjdk.jmh.runner.BenchmarkList.find(BenchmarkList.java:104)
at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:228)
at org.openjdk.jmh.runner.Runner.run(Runner.java:178)
at com.company.project.performance.MyBenchmarkTest.launchBenchmark(MyBenchmarkTest.java:145)
I've tried creating a bundle containing the file, like so:
streamBundle(bundle()
.add("META-INF/BenchmarkList", new FileInputStream("target/test-classes/META-INF/BenchmarkList"))
.build()),
Which does create a JAR with the following contents:
META-INF/MANIFEST.MF
META-INF/
META-INF/BenchmarkList
But the problem persists. How could I make the resource file available for JMH?
An MCVE:
pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.company.project</groupId>
<artifactId>performance</artifactId>
<version>0.0.1</version>
<packaging>jar</packaging>
<name>MCVE for JMH+PaxExam issue</name>
<prerequisites>
<maven>3.0</maven>
</prerequisites>
<properties>
<apache-servicemix.version>5.4.0</apache-servicemix.version>
<junit.version>4.11</junit.version>
<jmh.version>1.10.1</jmh.version>
<pax-exam.version>4.4.0</pax-exam.version>
<tinybundles.version>2.1.0</tinybundles.version>
<maven-sunfire-report-plugin.version>2.18.1</maven-sunfire-report-plugin.version>
<maven-depends-plugin.version>1.2</maven-depends-plugin.version>
<maven-compiler-plugin.version>3.3</maven-compiler-plugin.version>
<javac.target>1.7</javac.target>
</properties>
<dependencies>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>${jmh.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>${jmh.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.ops4j.pax.exam</groupId>
<artifactId>pax-exam</artifactId>
<version>${pax-exam.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.ops4j.pax.exam</groupId>
<artifactId>pax-exam-container-karaf</artifactId>
<version>${pax-exam.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.ops4j.pax.exam</groupId>
<artifactId>pax-exam-junit4</artifactId>
<version>${pax-exam.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.servicemix</groupId>
<artifactId>apache-servicemix</artifactId>
<version>${apache-servicemix.version}</version>
<scope>test</scope>
<type>zip</type>
</dependency>
<dependency>
<groupId>org.ops4j.pax.tinybundles</groupId>
<artifactId>tinybundles</artifactId>
<version>${tinybundles.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.servicemix.tooling</groupId>
<artifactId>depends-maven-plugin</artifactId>
<version>${maven-depends-plugin.version}</version>
<executions>
<execution>
<id>generate-depends-file</id>
<goals>
<goal>generate-depends-file</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-sunfire-report-plugin.version}</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>${maven-sunfire-report-plugin.version}</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
<configuration>
<source>${javac.target}</source>
<target>${javac.target}</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
src/test/java/com/company/project/performance/MyBenchmarkTest.java:
package com.company.project.performance;
import static org.ops4j.pax.exam.CoreOptions.junitBundles;
import static org.ops4j.pax.exam.CoreOptions.maven;
import static org.ops4j.pax.exam.CoreOptions.options;
import static org.ops4j.pax.exam.CoreOptions.streamBundle;
import static org.ops4j.pax.exam.CoreOptions.wrappedBundle;
import static org.ops4j.pax.exam.karaf.options.KarafDistributionOption.karafDistributionConfiguration;
import static org.ops4j.pax.exam.karaf.options.KarafDistributionOption.keepRuntimeFolder;
import static org.ops4j.pax.tinybundles.core.TinyBundles.bundle;
import static org.ops4j.pax.tinybundles.core.TinyBundles.withBnd;
import java.io.File;
import java.io.FileInputStream;
import java.util.concurrent.TimeUnit;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import org.ops4j.pax.exam.Configuration;
import org.ops4j.pax.exam.MavenUtils;
import org.ops4j.pax.exam.Option;
import org.ops4j.pax.exam.junit.PaxExam;
import org.ops4j.pax.exam.options.MavenArtifactUrlReference;
#RunWith(PaxExam.class)
public class MyBenchmarkTest
{
public static final String BENCHMARK_LIST = "META-INF/BenchmarkList";
#Benchmark
public void measureThroughput() throws InterruptedException
{
TimeUnit.MILLISECONDS.sleep(100);
}
#Configuration
public Option[] config() throws Exception
{
String karafVersion = MavenUtils.getArtifactVersion("org.apache.karaf", "apache-karaf");
MavenArtifactUrlReference servicemixUrl = maven()
.groupId("org.apache.servicemix")
.artifactId("apache-servicemix")
.versionAsInProject()
.type("zip");
return options(karafDistributionConfiguration()
.frameworkUrl(servicemixUrl)
.useDeployFolder(false)
.karafVersion(karafVersion)
.unpackDirectory(new File("target/exam")),
keepRuntimeFolder(),
junitBundles(),
wrappedBundle(maven("org.openjdk.jmh", "jmh-core")),
streamBundle(bundle()
.add(BENCHMARK_LIST, new FileInputStream("target/test-classes/" + BENCHMARK_LIST))
.build(withBnd())));
}
#Test
public void launchBenchmark() throws Exception
{
Options opts = new OptionsBuilder()
.include("com.company.project.performance.*")
.warmupIterations(1)
.forks(1)
.build();
new Runner(opts).run();
}
}
The issue is clearly an import export issue.
As your test which is using your meta data, isn't aware of those extra data.
The bundle you generated with the streamBundle, needs to add an extra header information actually exporting those extra data.
streamBundle(bundle()
.add(BENCHMARK_LIST, new FileInputStream("target/test-classes/" + BENCHMARK_LIST))
.set(Constants.EXPORT_PACKAGE, BENCHMARK_LIST)
and in your test you'll need to make sure you are actually importing it.
#ProbeBuilder
public TestProbeBuilder probeConfiguration(TestProbeBuilder probe) {
//make sure the needed imports are there.
probe.setHeader(Constants.IMPORT_PACKAGE, "*,"+BENCHMARK_LIST);
return probe;
}
on the other hand it might just be a lot better to actually try to add those extra data into the test-bundle (your test class is generated into a bundle on the fly)
Therefore the following should be added to the configuration
.metaInfResource(BENCHMARK_LIST)

Categories