Spring WS request GZIP compression - java

I'm building a Spring Boot application that behaves as a client to another web service. Using WebServiceTemplate to send SOAP messages and I have a case where a request is big enough that the target service requires it to be gzip compressed. As I understand handling compressed responses is done by default on the client's side, but not requests as that is not the standard. I'm using Java 8, Spring Boot 2.1 and Spring WS 3.0.3
Setting mime headers does not do the trick for me as that does not get the payload compressed, neither does setting server.compression.enabled (along with the various mime-types) in the application properties and I know it's not a faulty service on the other end because it does work with SoapUI.
So my question is - how can I enable gzip compression for outgoing requests?

A solution that worked for us was making a Http interceptor that does the compression and giving the WebServiceTemplate a new HttpComponentMessageSender with that interceptor. Here's what the interceptor looks like:
import org.apache.http.HttpEntity;
import org.apache.http.HttpEntityEnclosingRequest;
import org.apache.http.HttpRequest;
import org.apache.http.HttpRequestInterceptor;
import org.apache.http.client.entity.GzipCompressingEntity;
import org.apache.http.protocol.HTTP;
import org.apache.http.protocol.HttpContext;
import java.net.URI;
import java.net.URISyntaxException;
public class GzipHttpRequestInterceptor implements HttpRequestInterceptor {
private final String targetHost;
public GzipHttpRequestInterceptor(String targetUrl) throws URISyntaxException {
this.targetHost = getDomainName(targetUrl);
}
private String getDomainName(String url) throws URISyntaxException {
URI uri = new URI(url);
String domain = uri.getHost() + ":" + uri.getPort();
return domain.startsWith("www.") ? domain.substring(4) : domain;
}
#Override
public void process(HttpRequest httpRequest, HttpContext httpContext) {
final HttpEntityEnclosingRequest entityRequest = (HttpEntityEnclosingRequest) httpRequest;
final HttpEntity entity = entityRequest.getEntity();
if (entity != null) {
final GzipCompressingEntity zippedEntity = new GzipCompressingEntity(entity);
entityRequest.setEntity(zippedEntity);
httpRequest.removeHeaders(HTTP.CONTENT_ENCODING);
httpRequest.addHeader(zippedEntity.getContentEncoding());
httpRequest.removeHeaders(HTTP.CONTENT_LEN);
httpRequest.removeHeaders("Accept");
httpRequest.removeHeaders(HTTP.TRANSFER_ENCODING);
httpRequest.addHeader(HTTP.TRANSFER_ENCODING, HTTP.CHUNK_CODING);
httpRequest.addHeader(HTTP.TARGET_HOST, targetHost);
}
}
}
In our web configuration we assemble the org.apache.http.protocol.HttpProcessor and org.springframework.ws.transport.http.HttpComponentsMessageSender beans:
#Bean
public HttpProcessor httpRequestCompressionProcessor(String url) throws URISyntaxException {
return HttpProcessorBuilder.create()
.add(new GzipHttpRequestInterceptor(url))
.build();
}
#Bean
public HttpComponentsMessageSender messageGzipSender(String url) throws URISyntaxException {
return new HttpComponentsMessageSender(HttpClients.custom()
.addInterceptorFirst(new HttpComponentsMessageSender.RemoveSoapHeadersInterceptor())
.setHttpProcessor(httpRequestCompressionProcessor(url))
.build());
}
And then assign that message sender to our WebServiceTemplate using setMessageSender(messageGzipSender(url)
I guess I wouldn't mind comments on this code, in case it can be improved and still eager to hear if there is a simpler way.

Related

Generating ETag in GET response Header Java spring To solve 412 http error

I am using some external API to GET and POST some ressources, locally my code works fine with the call of different endPoints (GET, POST...) and even with Postman, but when i try to run my code in another platerform (where the ressources are), i get the 412 HTTP error due to a POST call : after looking on the internet, i found out that i should generate an ETagd of the entity (that i went to modify) and add it into the header of my POST endPoint.
For that, i used ShallowEtagHeaderFilter and the #Bean annotation(above the filter method) and the #SpringBootApplication annotation above my class, here is my code :
package main.Runners;
import io.testproject.java.annotations.v2.Parameter;
import okhttp3.*;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.configurationprocessor.json.JSONArray;
import org.springframework.boot.configurationprocessor.json.JSONObject;
import org.springframework.context.annotation.Bean;
import org.springframework.web.filter.ShallowEtagHeaderFilter;
import javax.servlet.Filter;
#SpringBootApplication
public class ActionRunner {
#Parameter(description = "the project ID")
public static String projectId = "xxx";
#Parameter(description = "the test ID")
public static String testId = "yyy";
public static void main(String[] args) throws Exception {
try {
OkHttpClient client = new OkHttpClient().newBuilder()
.build();
Request request = new Request.Builder()
.url("https://api.testproject.io/v2/projects/"+projectId+"/tests/"+testId)
.method("GET", null)
.addHeader("Authorization", "nzmo4DI08ykizYgcp9-5cCTArlxq7k7zt9MYhGmTcRk1")
.build();
Response response = client.newCall(request).execute();
System.out.println("================ this is our response headers ::: \n"+ response.headers());
} catch(Exception e) {
System.out.println(e);
}
}
#Bean
public ShallowEtagHeaderFilter shallowEtagHeaderFilter(){
return new ShallowEtagHeaderFilter();
}
}
I really need Your help since i cant generate any ETag parameter on my GET response header(after checking reponse.headers() ).
Thanks in advance!

How to capture HTTP request and mock its response in Java?

Say that Java application makes requests to http://www.google.com/... and there's no way to configure the inherited library (making such requests internally), so I can not stub or replace this URL.
Please, share some best practices to create a mock like
whenCalling("http://www.google.com/some/path").withMethod("GET").thenExpectResponse("HELLO")
so a request made by any HTTP client to this URL would be redirected to the mock and replaced with this response "HELLO" in the context of current JVM process.
I tried to find a solution using WireMock, Mockito or Hoverfly, but it seems that they do something different from that. Probably I just failed to use them properly.
Could you show a simple set up from the main method like:
create mock
start mock simulation
make a request to the URL by an arbitrary HTTP client (not entangled with the mocking library)
receive mocked response
stop mock simulation
make the same request as on step 3
receive real response from URL
Here's how to achieve what you want with the API Simulator.
The example demonstrates two different ways to configure Embedded API Simulator as HTTP proxy for the Spring's RestTemplate client. Check with the documentation of the (quote from the question) "inherited library" - often times Java-based clients rely on system properties described here or may offer some way to configure HTTP proxy with code.
package others;
import static com.apisimulator.embedded.SuchThat.*;
import static com.apisimulator.embedded.http.HttpApiSimulation.*;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.Proxy.Type;
import java.net.URI;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.ClassRule;
import org.junit.Test;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;
import com.apisimulator.embedded.http.JUnitHttpApiSimulation;
public class EmbeddedSimulatorAsProxyTest
{
// Configure an API simulation. This starts an instance of
// Embedded API Simulator on localhost, default port 6090.
// The instance is automatically stopped when the test ends.
#ClassRule
public static final JUnitHttpApiSimulation apiSimulation = JUnitHttpApiSimulation
.as(httpApiSimulation("my-sim"));
#BeforeClass
public static void beforeClass()
{
// Configure simlets for the API simulation
// #formatter:off
apiSimulation.add(simlet("http-proxy")
.when(httpRequest("CONNECT"))
.then(httpResponse(200))
);
apiSimulation.add(simlet("test-google")
.when(httpRequest()
.whereMethod("GET")
.whereUriPath(isEqualTo("/some/path"))
.whereHeader("Host", contains("google.com"))
)
.then(httpResponse()
.withStatus(200)
.withHeader("Content-Type", "application/text")
.withBody("HELLO")
)
);
// #formatter:on
}
#Test
public void test_using_system_properties() throws Exception
{
try
{
// Set these system properties just for this test
System.setProperty("http.proxyHost", "localhost");
System.setProperty("http.proxyPort", "6090");
RestTemplate restTemplate = new RestTemplate();
URI uri = new URI("http://www.google.com/some/path");
ResponseEntity<String> response = restTemplate.getForEntity(uri, String.class);
Assert.assertEquals(200, response.getStatusCode().value());
Assert.assertEquals("HELLO", response.getBody());
}
finally
{
System.clearProperty("http.proxyHost");
System.clearProperty("http.proxyPort");
}
}
#Test
public void test_using_java_net_proxy() throws Exception
{
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
// A way to configure API Simulator as HTTP proxy if the HTTP client supports it
Proxy proxy = new Proxy(Type.HTTP, new InetSocketAddress("localhost", 6090));
requestFactory.setProxy(proxy);
RestTemplate restTemplate = new RestTemplate();
restTemplate.setRequestFactory(requestFactory);
URI uri = new URI("http://www.google.com/some/path");
ResponseEntity<String> response = restTemplate.getForEntity(uri, String.class);
Assert.assertEquals(200, response.getStatusCode().value());
Assert.assertEquals("HELLO", response.getBody());
}
#Test
public void test_direct_call() throws Exception
{
RestTemplate restTemplate = new RestTemplate();
URI uri = new URI("http://www.google.com");
ResponseEntity<String> response = restTemplate.getForEntity(uri, String.class);
Assert.assertEquals(200, response.getStatusCode().value());
Assert.assertTrue(response.getBody().startsWith("<!doctype html>"));
}
}
When using maven, add the following to project's pom.xml to include the Embedded API Simulator as a dependency:
<dependency>
<groupId>com.apisimulator</groupId>
<artifactId>apisimulator-http-embedded</artifactId>
<version>1.6</version>
</dependency>
... and this to point to the repository:
<repositories>
<repository>
<id>apisimulator-github-repo</id>
<url>https://github.com/apimastery/APISimulator/raw/maven-repository</url>
</repository>
</repositories>

Spring Boot RESTful Web Service to post json content to https(secure) url

I am searching for working code sample/ snippet of spring boot to post json content to HTTPS restful web service(developed in python). Below is sample code of standalone program which does the same, But I want to do it in spring boot Restful app. I found many examples in google and stack overflows example example but these are for get request and not suites for what I am looking for. Someone please share full working example for "https post request using spring boot service".
Thanks in advance.
import java.io.PrintStream;
import java.net.URL;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import javax.net.ssl.HttpsURLConnection;
public class App{
public static void main(String []args) throws NoSuchAlgorithmException, KeyManagementException{
String https_url = "https://192.168.4.5:55543/books";
String content = "{\"data\":{\"a\"}}";
System.setProperty("javax.net.useSystemProxies", "true");
System.setProperty("javax.net.ssl.keyStore", "C:/cert/client.jks");
System.setProperty("javax.net.ssl.keyStorePassword", "testdev");
System.setProperty("javax.net.ssl.trustStore", "C:/cert/server.jks");
System.setProperty("javax.net.ssl.trustStorePassword", "testdev");
System.setProperty("jdk.tls.client.protocals", "TLSv1.2");
System.setProperty("https.protocals", "TLSv1.2");
HttpsURLConnection.setDefaultHostnameVerifier((hostname, session) -> { return true;});
URL url = new URL(https_url);
HttpsURLConnection https_con = (HttpsURLConnection) url.openConnection();
https_con.setRequestProperty("Content-Type", "application/json");
https_con.setRequestMethod("POST");
https_con.setDoOutput(true);
PrintStream ps = new PrintStream(https_con.getOutputStream());
ps.println(content);
ps.close();
https_con.connect();
https_con.getResponseCode();
https_con.disconnect();
}
}
Ok, so here is the forked Github repo.
Following are the changes I made:
secure-server -> Added post endpoint with simply String payload:
#RestController
public class HomeRestController {
//...
#PostMapping(value = "/", consumes = MediaType.APPLICATION_JSON_UTF8_VALUE)
public String consumeData(#RequestBody String data, Principal principal){
return String.format("Hello %s! You sent: %s", principal.getName(), data);
}
}
secure-client -> added call to that post method:
#RestController
public class HomeRestController {
// . . .
#GetMapping("/post")
public String post() throws RestClientException {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
String data = "Test payload";
HttpEntity<String> request = new HttpEntity<>(data, headers);
return restTemplate.postForEntity("https://localhost:8443", request , String.class ).getBody();
}
}
So, when you make call to client endpoint as:
http://localhost:8086/post
You will get response:
Hello codependent-client! You sent: Test payload

Returning header to STOMP client frame object after handshake

Currently, when successfully connecting to an endpoint, I am passing a custom token (called access-token) back to the client via a response header. This header is being set correctly and I can verify the header by analyzing the HTTP response.
However, when trying to get the header from the frame object the header is not set (see the JavaScript below):
stompClient.connect(headers,
function(frame) {
console.log('=========================================');
console.log(frame.headers['access-token']);
console.log(frame);
console.log('=========================================');
stompClient.subscribe('/topic/test', function(stuff){
console.debug(stuff);
});
},
function(error) {
//error code
}
);
I am setting the response header as follows on the server:
public class HttpSessionHandshakeInterceptorImpl extends HttpSessionHandshakeInterceptor {
#Override
public boolean beforeHandshake(ServerHttpRequest request,
ServerHttpResponse response, WebSocketHandler wsHandler,
Map<String, Object> attributes) throws Exception {
response.getHeaders().set("access-token", token);
return super.beforeHandshake(request, response, wsHandler, attributes);
I stripped out some code and can confirm that the interceptor is being called. I take it that this is not the correct way to pass a header value back to the client when the connect function is called? I can't seem to find any documentation on how to accomplish this. Thanks.
ServerHttpResponse response - this is the response to a HTTP request, which you can inspect in more detail (by logging or in debug mode). Setting its headers will affect the HTTP response, not the STOMP stream which flows over the WebSocket connection. The STOMP headers of the CONNECTED are set by the message broker.
I do not know it for sure but I doubt that Spring is capable of inserting extra headers in the STOMP frames. (please correct me if I am wrong)
You need to implement ChannelInterceptor and register it with output bound channel
import java.net.InetAddress;
import java.net.UnknownHostException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Configuration;
import org.springframework.lang.Nullable;
import org.springframework.messaging.Message;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.simp.stomp.StompCommand;
import org.springframework.messaging.simp.stomp.StompHeaderAccessor;
import org.springframework.messaging.support.ChannelInterceptor;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.util.MultiValueMap;
#Configuration
public class WebSocketServiceCustomHeaderInterceptor implements ChannelInterceptor {
private static final Logger logger = LoggerFactory.getLogger(WebSocketServiceCustomHeaderInterceptor.class);
#Override
#Nullable
public Message<?> preSend(Message<?> message, MessageChannel channel) {
final StompHeaderAccessor headerAccessor = StompHeaderAccessor.wrap(message);
final StompCommand command = headerAccessor.getCommand();
if (command != null && command == StompCommand.CONNECTED) {
final StompHeaderAccessor accessor = StompHeaderAccessor.create(command);
accessor.setSessionId(headerAccessor.getSessionId());
#SuppressWarnings("unchecked")
final MultiValueMap<String, String> nativeHeaders = (MultiValueMap<String, String>) headerAccessor
.getHeader(StompHeaderAccessor.NATIVE_HEADERS);
accessor.addNativeHeaders(nativeHeaders);
// add custom headers
try {
accessor.addNativeHeader("HOSTNAME", InetAddress.getLocalHost().getHostName());
} catch (UnknownHostException e) {
logger.error("Error getting host name ", e);
}
final Message<?> newMessage = MessageBuilder.createMessage(new byte[0], accessor.getMessageHeaders());
return newMessage;
}
return message;
}
}
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketBrokerConfig implements WebSocketMessageBrokerConfigurer {
#Bean
public WebSocketServiceCustomHeaderInterceptor webSocketServiceCustomHeaderInterceptor() {
return new WebSocketServiceCustomHeaderInterceptor();
}
#Override
public void configureClientOutboundChannel(ChannelRegistration registration) {
registration.interceptors(webSocketServiceCustomHeaderInterceptor());
registration.taskExecutor().corePoolSize(outboundPoolCoreSize).maxPoolSize(outboundPoolMaxSize);
}
}
I'm afraid this is not possible without a major hack. The issue is, the headers are completely rebuilt AFTER the ChannelInterceptors are called. See method StompSubProtocolHandler.convertConnectAcktoStompConnected
The method StompSubProtocolHandler.handleMessageToClient will also add a few headers depending on the type of message, for example by calling afterStompSessionConnected which sets the header "user-name" if the authentication is done.
I've tried adding headers as well but I'm about to give up... My guess is that the people who wrote the STOMP implementation in Spring wanted to strictly respect the specifications

How to set HTTP header in RESTEasy client framework?

RESTEasy (a JAX-RS implementation) has a nice client framework, eg:
RegisterBuiltin.register(ResteasyProviderFactory.getInstance());
SimpleClient client = ProxyFactory.create(SimpleClient.class, "http://localhost:8081");
client.putBasic("hello world");
How do you set HTTP headers?
Clarification:
The solution proposed by jkeeler is a good approach, but I want to set HTTP headers on ProxyFactory level and I don't want to pass headers to the client object. Any ideas?
In your client proxy interface, use the #HeaderParam annotation:
public interface SimpleClient
{
#PUT
#Path("basic")
#Consumes("text/plain")
public void putBasic(#HeaderParam("Greeting") String greeting);
}
The call in your example above would add an HTTP header that looks like this:
Greeting: hello world
With RestEasy 3.x I use ClientRequestFilters. In the below example there is a continuous integration (CI) server listening for requests running in the background. The test and the CI server use the same database and entity classes.
Assume that a tenant named 'test-tenant' does in fact exist, and there is a user 'root' that belongs to that tenant, and the user has the password specified below.
private static final String BASE_URI = "http://localhost:" + PORT;
#Test(groups = "functionalTests")
public void testGetTenant() throws Exception {
Client client = ClientBuilder.newClient();
ResteasyWebTarget target = (ResteasyWebTarget)client.target(BASE_URI);
client.register(new AddAuthHeadersRequestFilter("root", "DefaultPasswordsAre:-("));
TenantResource resource = target.proxy(TenantResource.class);
RestTenant restTenant = resource.getTenant(tenant.id().value().toString());
assertThat(restTenant.getName(), is("test-tenant"));
assertThat(restTenant.isActive(), is(true));
}
And the AddAuthHeadersRequestFilter class:
public static class AddAuthHeadersRequestFilter implements ClientRequestFilter {
private final String username;
private final String password;
public AddAuthHeadersRequestFilter(String username, String password) {
this.username = username;
this.password = password;
}
#Override
public void filter(ClientRequestContext requestContext) throws IOException {
String token = username + ":" + password;
String base64Token = Base64.encodeBase64String(token.getBytes(StandardCharsets.UTF_8));
requestContext.getHeaders().add("Authorization", "Basic " + base64Token);
}
}
The import statements (assuming you just paste the test and the static class into a single TestNg test-class file):
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import org.jboss.resteasy.client.jaxrs.ResteasyWebTarget;
import org.testng.annotations.Test;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import javax.ws.rs.client.ClientRequestContext;
import javax.ws.rs.client.ClientRequestFilter;
import org.apache.commons.codec.binary.Base64;
Even easier:
ResteasyClient client = new ResteasyClientBuilder().build();
ResteasyWebTarget target = client.target("https://test.com");
Response response = target.request().header("Authorization", "Basic test123")
.acceptEncoding("gzip, deflate")
.post(Entity.entity(some_xml, "application/x-www-form-urlencoded"));
I have found a solution:
import org.apache.commons.httpclient.HttpClient;
import org.jboss.resteasy.client.ClientRequest;
import org.jboss.resteasy.client.ClientResponse;
import org.jboss.resteasy.client.ProxyFactory;
import org.jboss.resteasy.client.core.executors.ApacheHttpClientExecutor;
import org.jboss.resteasy.plugins.providers.RegisterBuiltin;
import org.jboss.resteasy.spi.ResteasyProviderFactory;
RegisterBuiltin.register(ResteasyProviderFactory.getInstance());
HttpClient httpClient = new HttpClient();
ApacheHttpClientExecutor executor = new ApacheHttpClientExecutor(httpClient) {
#Override
public ClientResponse execute(ClientRequest request) throws Exception {
request.header("X-My-Header", "value");
return super.execute(request);
}
};
SimpleClient client = ProxyFactory.create(SimpleClient.class, "http://localhost:8081", executor);
client.putBasic("hello world");

Categories