I'm trying to write unit tests for my program and use mock data. I'm a little confused on how to intercept an HTTP Get request to a URL.
My program calls a URL to our API and it is returned a simple XML file. I would like the test to instead of getting the XML file from the API online to receive a predetermined XML file from me so that I can compare the output to the expected output and determine if everything is working correctly.
I was pointed to Mockito and have been seeing many different examples such as this SO post, How to use mockito for testing a REST service? but it's not becoming clear to me how to set it all up and how to mock the data (i.e., return my own xml file whenever the call to the URL is made).
The only thing I can think of is having another program made that's running locally on Tomcat and in my test pass a special URL that calls the locally running program on Tomcat and then return the xml file that I want to test with. But that just seems like overkill and I don't think that would be acceptable. Could someone please point me in the right direction.
private static InputStream getContent(String uri) {
HttpURLConnection connection = null;
try {
URL url = new URL(uri);
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept", "application/xml");
return connection.getInputStream();
} catch (MalformedURLException e) {
LOGGER.error("internal error", e);
} catch (IOException e) {
LOGGER.error("internal error", e);
} finally {
if (connection != null) {
connection.disconnect();
}
}
return null;
}
I am using Spring Boot and other parts of the Spring Framework if that helps.
Part of the problem is that you're not breaking things down into interfaces. You need to wrap getContent into an interface and provide a concrete class implementing the interface. This concrete class will then
need to be passed into any class that uses the original getContent. (This is essentially dependency inversion.) Your code will end up looking something like this.
public interface IUrlStreamSource {
InputStream getContent(String uri)
}
public class SimpleUrlStreamSource implements IUrlStreamSource {
protected final Logger LOGGER;
public SimpleUrlStreamSource(Logger LOGGER) {
this.LOGGER = LOGGER;
}
// pulled out to allow test classes to provide
// a version that returns mock objects
protected URL stringToUrl(String uri) throws MalformedURLException {
return new URL(uri);
}
public InputStream getContent(String uri) {
HttpURLConnection connection = null;
try {
Url url = stringToUrl(uri);
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept", "application/xml");
return connection.getInputStream();
} catch (MalformedURLException e) {
LOGGER.error("internal error", e);
} catch (IOException e) {
LOGGER.error("internal error", e);
} finally {
if (connection != null) {
connection.disconnect();
}
}
return null;
}
}
Now code that was using the static getContent should go through a IUrlStreamSource instances getContent(). You then provide to the object that you want to test a mocked IUrlStreamSource rather than a SimpleUrlStreamSource.
If you want to test SimpleUrlStreamSource (but there's not much to test), then you can create a derived class that provides an implementation of stringToUrl that returns a mock (or throws an exception).
The other answers in here advise you to refactor your code to using a sort of provider which you can replace during your tests - which is the better approach.
If that isn't a possibility for whatever reason you can install a custom URLStreamHandlerFactory that intercepts the URLs you want to "mock" and falls back to the standard implementation for URLs that shouldn't be intercepted.
Note that this is irreversible, so you can't remove the InterceptingUrlStreamHandlerFactory once it's installed - the only way to get rid of it is to restart the JVM. You could implement a flag in it to disable it and return null for all lookups - which would produce the same results.
URLInterceptionDemo.java:
public class URLInterceptionDemo {
private static final String INTERCEPT_HOST = "dummy-host.com";
public static void main(String[] args) throws IOException {
// Install our own stream handler factory
URL.setURLStreamHandlerFactory(new InterceptingUrlStreamHandlerFactory());
// Fetch an intercepted URL
printUrlContents(new URL("http://dummy-host.com/message.txt"));
// Fetch another URL that shouldn't be intercepted
printUrlContents(new URL("http://httpbin.org/user-agent"));
}
private static void printUrlContents(URL url) throws IOException {
try(InputStream stream = url.openStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(stream))) {
String line;
while((line = reader.readLine()) != null) {
System.out.println(line);
}
}
}
private static class InterceptingUrlStreamHandlerFactory implements URLStreamHandlerFactory {
#Override
public URLStreamHandler createURLStreamHandler(final String protocol) {
if("http".equalsIgnoreCase(protocol)) {
// Intercept HTTP requests
return new InterceptingHttpUrlStreamHandler();
}
return null;
}
}
private static class InterceptingHttpUrlStreamHandler extends URLStreamHandler {
#Override
protected URLConnection openConnection(final URL u) throws IOException {
if(INTERCEPT_HOST.equals(u.getHost())) {
// This URL should be intercepted, return the file from the classpath
return URLInterceptionDemo.class.getResource(u.getHost() + "/" + u.getPath()).openConnection();
}
// Fall back to the default handler, by passing the default handler here we won't end up
// in the factory again - which would trigger infinite recursion
return new URL(null, u.toString(), new sun.net.www.protocol.http.Handler()).openConnection();
}
}
}
dummy-host.com/message.txt:
Hello World!
When run, this app will output:
Hello World!
{
"user-agent": "Java/1.8.0_45"
}
It's pretty easy to change the criteria of how you decide which URLs to intercept and what you return instead.
The answer depends on what you are testing.
If you need to test the processing of the InputStream
If getContent() is called by some code that processes the data returned by the InputStream, and you want to test how the processing code handles specific sets of input, then you need to create a seam to enable testing. I would simply move getContent() into a new class, and inject that class into the class that does the processing:
public interface ContentSource {
InputStream getContent(String uri);
}
You could create a HttpContentSource that uses URL.openConnection() (or, better yet, the Apache HttpClientcode).
Then you would inject the ContentSource into the processor:
public class Processor {
private final ContentSource contentSource;
#Inject
public Processor(ContentSource contentSource) {
this.contentSource = contentSource;
}
...
}
The code in Processor could be tested with a mock ContentSource.
If you need to test the fetching of the content
If you want to make sure that getContent() works, you could create a test that starts a lightweight in-memory HTTP server that serves the expected content, and have getContent() talk to that server. That does seem overkill.
If you need to test a large subset of the system with fake data
If you want to make sure things work end to end, write an end to end system test. Since you indicated you use Spring, you can use Spring to wire together parts of the system (or to wire the entire system, but with different properties). You have two choices
Have the system test start a local HTTP server, and when you have your test create your system, configure it to talk to that server. See the answers to this question for ways to start the HTTP server.
Configure spring to use a fake implementation of ContentSource. This gets you slightly less confidence that everything works end-to-end, but it will be faster and less flaky.
When I try to use a non-standard HTTP Method like PATCH with URLConnection:
HttpURLConnection conn = (HttpURLConnection) new URL("http://example.com").openConnection();
conn.setRequestMethod("PATCH");
I get an exception:
java.net.ProtocolException: Invalid HTTP method: PATCH
at java.net.HttpURLConnection.setRequestMethod(HttpURLConnection.java:440)
Using a higher level API like Jersey generates the same error. Is there a workaround to issue a PATCH HTTP request?
There are a lot of good answers, so here is mine (not work in jdk12):
import java.io.IOException;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Arrays;
import java.util.LinkedHashSet;
import java.util.Set;
public class SupportPatch {
public static void main(String... args) throws IOException {
allowMethods("PATCH");
HttpURLConnection conn = (HttpURLConnection) new URL("http://example.com").openConnection();
conn.setRequestMethod("PATCH");
}
private static void allowMethods(String... methods) {
try {
Field methodsField = HttpURLConnection.class.getDeclaredField("methods");
Field modifiersField = Field.class.getDeclaredField("modifiers");
modifiersField.setAccessible(true);
modifiersField.setInt(methodsField, methodsField.getModifiers() & ~Modifier.FINAL);
methodsField.setAccessible(true);
String[] oldMethods = (String[]) methodsField.get(null);
Set<String> methodsSet = new LinkedHashSet<>(Arrays.asList(oldMethods));
methodsSet.addAll(Arrays.asList(methods));
String[] newMethods = methodsSet.toArray(new String[0]);
methodsField.set(null/*static field*/, newMethods);
} catch (NoSuchFieldException | IllegalAccessException e) {
throw new IllegalStateException(e);
}
}
}
It also uses reflection, but instead of hacking into every connection object we're hacking HttpURLConnection#methods static field which is used in the checks internally.
Yes there is workaround for this. Use
X-HTTP-Method-Override
. This header can be used in a POST request to “fake” other HTTP methods. Simply set the value of the X-HTTP-Method-Override header to the HTTP method you would like to actually perform.
So use following code.
conn.setRequestProperty("X-HTTP-Method-Override", "PATCH");
conn.setRequestMethod("POST");
There is a Won't Fix bug in OpenJDK for this: https://bugs.openjdk.java.net/browse/JDK-7016595
However, with Apache Http-Components Client 4.2+ this is possible. It has a custom networking implementation, thus using all standard HTTP methods like PATCH is possible. It even has a HttpPatch class supporting the patch method.
CloseableHttpClient httpClient = HttpClients.createDefault();
HttpPatch httpPatch = new HttpPatch(new URI("http://example.com"));
CloseableHttpResponse response = httpClient.execute(httpPatch);
Maven Coordinates:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.2+</version>
</dependency>
If the project is on Spring/Gradle; the following solution will workout.
For the build.gradle, add the following dependency;
compile('org.apache.httpcomponents:httpclient:4.5.2')
And define the following bean in your #SpringBootApplication class inside the
com.company.project;
#Bean
public RestTemplate restTemplate() {
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setReadTimeout(600000);
requestFactory.setConnectTimeout(600000);
return new RestTemplate(requestFactory);
}
This solutions worked for me.
In java 11+ you can use the HttpRequest class to do what you want:
import java.net.http.HttpRequest;
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(uri))
.method("PATCH", HttpRequest.BodyPublishers.ofString(message))
.header("Content-Type", "text/xml")
.build();
Reflection as described in this post and a related post does not work if you are using a HttpsURLConnection on Oracle's JRE, becausesun.net.www.protocol.https.HttpsURLConnectionImpl is using the method field from the java.net.HttpURLConnection of its DelegateHttpsURLConnection!
So a complete working solution is:
private void setRequestMethod(final HttpURLConnection c, final String value) {
try {
final Object target;
if (c instanceof HttpsURLConnectionImpl) {
final Field delegate = HttpsURLConnectionImpl.class.getDeclaredField("delegate");
delegate.setAccessible(true);
target = delegate.get(c);
} else {
target = c;
}
final Field f = HttpURLConnection.class.getDeclaredField("method");
f.setAccessible(true);
f.set(target, value);
} catch (IllegalAccessException | NoSuchFieldException ex) {
throw new AssertionError(ex);
}
}
Using the answer:
HttpURLConnection Invalid HTTP method: PATCH
I'm created a sample request and work like a charm:
public void request(String requestURL, String authorization, JsonObject json) {
try {
URL url = new URL(requestURL);
httpConn = (HttpURLConnection) url.openConnection();
httpConn.setRequestMethod("POST");
httpConn.setRequestProperty("X-HTTP-Method-Override", "PATCH");
httpConn.setRequestProperty("Content-Type", "application/json");
httpConn.setRequestProperty("Authorization", authorization);
httpConn.setRequestProperty("charset", "utf-8");
DataOutputStream wr = new DataOutputStream(httpConn.getOutputStream());
wr.writeBytes(json.toString());
wr.flush();
wr.close();
httpConn.connect();
String response = finish();
if (response != null && !response.equals("")) {
created = true;
}
}
catch (Exception e) {
e.printStackTrace();
}
}
public String finish() throws IOException {
String response = "";
int status = httpConn.getResponseCode();
if (status == HttpURLConnection.HTTP_OK || status == HttpURLConnection.HTTP_CREATED) {
BufferedReader reader = new BufferedReader(new InputStreamReader(
httpConn.getInputStream()));
String line = null;
while ((line = reader.readLine()) != null) {
response += line;
}
reader.close();
httpConn.disconnect();
} else {
throw new IOException("Server returned non-OK status: " + status);
}
return response;
}
I hope it help you.
I had the same exception and wrote sockets solution (in Groovy) but I translate in the answer form to Java for you:
String doInvalidHttpMethod(String method, String resource){
Socket s = new Socket(InetAddress.getByName("google.com"), 80);
PrintWriter pw = new PrintWriter(s.getOutputStream());
pw.println(method +" "+resource+" HTTP/1.1");
pw.println("User-Agent: my own");
pw.println("Host: google.com:80");
pw.println("Content-Type: */*");
pw.println("Accept: */*");
pw.println("");
pw.flush();
BufferedReader br = new BufferedReader(new InputStreamReader(s.getInputStream()));
String t = null;
String response = "";
while((t = br.readLine()) != null){
response += t;
}
br.close();
return response;
}
I think it works in Java. You have to change the server and port number remember change the Host header too and maybe you have to catch some exception.
For anyone using Spring restTemplate looking for a detailed answer.
You will face the problem if you are using SimpleClientHttpRequestFactory as your restTemplate's ClientHttpRequestFactory.
From java.net.HttpURLConnection:
/* valid HTTP methods */
private static final String[] methods = {
"GET", "POST", "HEAD", "OPTIONS", "PUT", "DELETE", "TRACE"
};
As PATCH is not a supported operation, this line of code from the same class will execute:
throw new ProtocolException("Invalid HTTP method: " + method);
I ended up using the same as what #hirosht suggested in his answer.
Another dirty hack solution is reflexion:
private void setVerb(HttpURLConnection cn, String verb) throws IOException {
switch (verb) {
case "GET":
case "POST":
case "HEAD":
case "OPTIONS":
case "PUT":
case "DELETE":
case "TRACE":
cn.setRequestMethod(verb);
break;
default:
// set a dummy POST verb
cn.setRequestMethod("POST");
try {
// Change protected field called "method" of public class HttpURLConnection
setProtectedFieldValue(HttpURLConnection.class, "method", cn, verb);
} catch (Exception ex) {
throw new IOException(ex);
}
break;
}
}
public static <T> void setProtectedFieldValue(Class<T> clazz, String fieldName, T object, Object newValue) throws Exception {
Field field = clazz.getDeclaredField(fieldName);
field.setAccessible(true);
field.set(object, newValue);
}
You can find a detailed solution that can work even if you don't have direct access to the HttpUrlConnection (like when working with Jersey Client here: PATCH request using Jersey Client
If your server is using ASP.NET Core, you can simply add the following code to specify the HTTP method using the header X-HTTP-Method-Override, as described in the accepted answer.
app.Use((context, next) => {
var headers = context.Request.Headers["X-HTTP-Method-Override"];
if(headers.Count == 1) {
context.Request.Method = headers.First();
}
return next();
});
Simply add this code in Startup.Configure before your call to app.UseMvc().
In emulator of API 16 I received an exception: java.net.ProtocolException: Unknown method 'PATCH'; must be one of [OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE].
While an accepted answer works, I want to add one detail. In new APIs PATCH works well, so in conjunction with https://github.com/OneDrive/onedrive-sdk-android/issues/16 you should write:
if (method.equals("PATCH") && Build.VERSION.SDK_INT <= Build.VERSION_CODES.KITKAT) {
httpConnection.setRequestProperty("X-HTTP-Method-Override", "PATCH");
httpConnection.setRequestMethod("POST");
} else {
httpConnection.setRequestMethod(method);
}
I changed JELLY_BEAN_MR2 to KITKAT after testing in API 16, 19, 21.
I got mine with Jersey client.
The workaround was:
Client client = ClientBuilder.newClient();
client.property(HttpUrlConnectorProvider.SET_METHOD_WORKAROUND, true);
We have faced the same problem with slightly different behavior. We were using apache cxf library for making the rest calls.
For us, PATCH was working fine till we were talking to our fake services which were working over http.
The moment we integrated with actual systems (which were over https) we started facing the same issue with following stack trace.
java.net.ProtocolException: Invalid HTTP method: PATCH at java.net.HttpURLConnection.setRequestMethod(HttpURLConnection.java:428) ~[na:1.7.0_51] at sun.net.www.protocol.https.HttpsURLConnectionImpl.setRequestMethod(HttpsURLConnectionImpl.java:374) ~[na:1.7.0_51] at org.apache.cxf.transport.http.URLConnectionHTTPConduit.setupConnection(URLConnectionHTTPConduit.java:149) ~[cxf-rt-transports-http-3.1.14.jar:3.1.14]
Issue was happening in this line of code
connection.setRequestMethod(httpRequestMethod); in URLConnectionHTTPConduit class of cxf library
Now the real reason for the failure is that
java.net.HttpURLConnection contains a methods variable which looks like below
/* valid HTTP methods */
private static final String[] methods = {
"GET", "POST", "HEAD", "OPTIONS", "PUT", "DELETE", "TRACE"
};
And we can see that there is no PATCH method defined hence the error made sense.
We tried lots of different thing and looked over stack overflow. The only reasonable answer was to use reflection to modify the methods variable to inject another value "PATCH". But somehow we were not convinced to use that as the solution was kind of hack and is too much work and might have impact as we had common library to make all connection and performing these REST calls.
But then we realized that cxf library itself is handling the exception and there is code written in the catch block to add the missing method using reflection.
try {
connection.setRequestMethod(httpRequestMethod);
} catch (java.net.ProtocolException ex) {
Object o = message.getContextualProperty(HTTPURL_CONNECTION_METHOD_REFLECTION);
boolean b = DEFAULT_USE_REFLECTION;
if (o != null) {
b = MessageUtils.isTrue(o);
}
if (b) {
try {
java.lang.reflect.Field f = ReflectionUtil.getDeclaredField(HttpURLConnection.class, "method");
if (connection instanceof HttpsURLConnection) {
try {
java.lang.reflect.Field f2 = ReflectionUtil.getDeclaredField(connection.getClass(),
"delegate");
Object c = ReflectionUtil.setAccessible(f2).get(connection);
if (c instanceof HttpURLConnection) {
ReflectionUtil.setAccessible(f).set(c, httpRequestMethod);
}
f2 = ReflectionUtil.getDeclaredField(c.getClass(), "httpsURLConnection");
HttpsURLConnection c2 = (HttpsURLConnection)ReflectionUtil.setAccessible(f2)
.get(c);
ReflectionUtil.setAccessible(f).set(c2, httpRequestMethod);
} catch (Throwable t) {
//ignore
logStackTrace(t);
}
}
ReflectionUtil.setAccessible(f).set(connection, httpRequestMethod);
message.put(HTTPURL_CONNECTION_METHOD_REFLECTION, true);
} catch (Throwable t) {
logStackTrace(t);
throw ex;
}
}
Now this gave us some hopes, so we spent some time in reading the code and found that if we provide a property for URLConnectionHTTPConduit.HTTPURL_CONNECTION_METHOD_REFLECTION then we can make cxf to execute the exception handler and our work is done as by default the variable will be assigned to false due to below code
DEFAULT_USE_REFLECTION =
Boolean.valueOf(SystemPropertyAction.getProperty(HTTPURL_CONNECTION_METHOD_REFLECTION, "false"));
So here is what we had to do to make this work
WebClient.getConfig(client).getRequestContext().put("use.httpurlconnection.method.reflection", true);
or
WebClient.getConfig(client).getRequestContext().put(HTTPURL_CONNECTION_METHOD_REFLECTION, true);
Where WebClient is from cxf library itself.
Hope this answer helps some one.
**CloseableHttpClient http = HttpClientBuilder.create().build();
HttpPatch updateRequest = new HttpPatch("URL");
updateRequest.setEntity(new StringEntity("inputjsonString", ContentType.APPLICATION_JSON));
updateRequest.setHeader("Bearer", "auth");
HttpResponse response = http.execute(updateRequest);
JSONObject result = new JSONObject(IOUtils.toString(response.getEntity().getContent()));**
maven plugin
> <dependency>
> <groupId>org.apache.httpcomponents</groupId>
> <artifactId>httpclient</artifactId>
> <version>4.3.4</version>
> <!-- Exclude Commons Logging in favor of SLF4j -->
> <exclusions>
> <exclusion>
> <groupId>commons-logging</groupId>
> <artifactId>commons-logging</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
use this really it would helps you
I have a cloud storage at strato namely hidrive. It uses the webdav protocol. Note that it's based on HTTP. The client application they provide is poor and buggy so I tried various other tools for synchronization but none just worked the way I need it.
I'm therefore trying to implement it in Java using the Sardine project. Is there any code for hard-copying a local source folder to an external cloud folder? I haven't found anything in that direction.
The following code is supposed to upload the file...
Sardine sardine = SardineFactory.begin("username", "password");
InputStream fis = new FileInputStream(new File("some/file/test.txt"));
sardine.put("https://webdav.hidrive.strato.com/users/username/Backup", fis);
... but throws an exception instead:
Exception in thread "main" com.github.sardine.impl.SardineException: Unexpected response (301 Moved Permanently)
at com.github.sardine.impl.handler.ValidatingResponseHandler.validateResponse(ValidatingResponseHandler.java:48)
at com.github.sardine.impl.handler.VoidResponseHandler.handleResponse(VoidResponseHandler.java:34)
at com.github.sardine.impl.handler.VoidResponseHandler.handleResponse(VoidResponseHandler.java:1)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:218)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:160)
at com.github.sardine.impl.SardineImpl.execute(SardineImpl.java:828)
at com.github.sardine.impl.SardineImpl.put(SardineImpl.java:755)
at com.github.sardine.impl.SardineImpl.put(SardineImpl.java:738)
at com.github.sardine.impl.SardineImpl.put(SardineImpl.java:726)
at com.github.sardine.impl.SardineImpl.put(SardineImpl.java:696)
at com.github.sardine.impl.SardineImpl.put(SardineImpl.java:689)
at com.github.sardine.impl.SardineImpl.put(SardineImpl.java:682)
at com.github.sardine.impl.SardineImpl.put(SardineImpl.java:676)
Printing out the folders in that directory works so the connection/ authentication did succeed:
List<DavResource> resources = sardine.list("https://webdav.hidrive.strato.com/users/username/Backup");
for (DavResource res : resources)
{
System.out.println(res);
}
Please either help me fix my code or link me to some file synchronization library that works for my purpose.
Sardine uses (internally) HttpClient. There is similar question here where you can find an answer Httpclient 4, error 302. How to redirect?.
Try converting the InputStream obj into byte array before you call put(). Something like the below,
byte[] fisByte = IOUtils.toByteArray(fis);
sardine.put("https://webdav.hidrive.strato.com/users/username/Backup", fisByte);
It worked for me. Let me know.
I had to extend the "org.apache.http.impl.client.LaxRedirectStrategy" and also the getRedirect() Method of org.apache.http.impl.client.DefaultRedirectStrategy with a treatment of the needed methods: PUT, MKOL, etc. . By default only GET is redirected.
It looks like this:
private static final String[] REDIRECT_METHODS = new String[] { HttpGet.METHOD_NAME, HttpPost.METHOD_NAME, HttpHead.METHOD_NAME, HttpPut.METHOD_NAME, HttpDelete.METHOD_NAME, HttpMkCol.METHOD_NAME };
isRedirectable-Method
for (final String m : REDIRECT_METHODS) {
if (m.equalsIgnoreCase(method)) {
System.out.println("isRedirectable true");
return true;
}
}
return method.equalsIgnoreCase(HttpPropFind.METHOD_NAME);
getRedirect-Method:
final URI uri = getLocationURI(request, response, context);
final String method = request.getRequestLine().getMethod();
if (method.equalsIgnoreCase(HttpHead.METHOD_NAME)) {
return new HttpHead(uri);
} else if (method.equalsIgnoreCase(HttpGet.METHOD_NAME)) {
return new HttpGet(uri);
} else if (method.equalsIgnoreCase(HttpPut.METHOD_NAME)) {
HttpPut httpPut = new HttpPut(uri);
httpPut.setEntity(((HttpEntityEnclosingRequest) request).getEntity());
return httpPut;
} else if (method.equalsIgnoreCase("MKCOL")) {
return new HttpMkCol(uri);
} else if (method.equalsIgnoreCase("DELETE")) {
return new HttpDelete(uri);
} else {
final int status = response.getStatusLine().getStatusCode();
if (status == HttpStatus.SC_TEMPORARY_REDIRECT) {
return RequestBuilder.copy(request).setUri(uri).build();
} else {
return new HttpGet(uri);
}
}
That worked for me.
I am using the Kohsuke GitHub-API to connect to the GitHub from my Java (server-side) application and I wanted to use the OkHttp's ability to cache responses from the GitHub. This worked perfectly when I wrote a test for it, but it doesn't work in the application itself and I don't have a clue why that is. I have managed to trace the problem back to the creation of the URLConnection object that is created with its useCache variable set to false, but I cannot figure out why. Does it maybe have something to do with the server configuration or something like that?
I would appreciate any ideas or even a nudge in any direction, because frankly I don't have any ideas left... Thanks
Provider:
public class GitHubProvider implements Provider<GitHub> {
#Override
public GitHub get() {
GitHub gitHub = null;
HttpResponseCache cache = null;
OkHttpClient okHttpClient = new OkHttpClient();
File cacheDir = new File(System.getProperty("java.io.tmpdir"), UUID.randomUUID().toString());
try {
cache = new HttpResponseCache(cacheDir, 10L * 1024 * 1024);
} catch (IOException e) {
// NOTHING
}
okHttpClient.setResponseCache(cache);
try {
gitHub = GitHub.connectUsingPassword("user", "password");
} catch (Exception e) {
// NOTHING
}
gitHub.setConnector(new OkHttpConnector(okHttpClient));
return gitHub;
}
}
Test (works):
#RunWith(JukitoRunner.class)
public class SoftwareComponentServiceTest {
public static class Module extends TestModule {
#Override
protected void configureTest() {
bind(GitHub.class).toProvider(GitHubProvider.class);
}
}
#Inject
GitHub gitHub;
#Test
public void testInjectedGitHubResponseCache() {
try {
GHUser ghUser = gitHub.getUser("user");
GHRepository repository = ghUser.getRepository("repository");
int limit = gitHub.getRateLimit().remaining;
repository.getFileContent("README.md");
assertEquals(limit - 1, gitHub.getRateLimit().remaining);
repository.getFileContent("README.md");
assertEquals(limit - 1, gitHub.getRateLimit().remaining);
} catch (IOException e) {
e.printStackTrace();
}
}
}
Service that is used in the application (doesn't work):
#Singleton
#RequiresAuthentication
public class SoftwareComponentService {
#Inject
GitHub gitHub;
public List<SoftwareComponent> findAll() {
List<SoftwareComponent> softwareComponentList = new ArrayList<SoftwareComponent>();
try {
GHUser ghUser = gitHub.getUser("user");
List<GHRepository> repositories = ghUser.listRepositories().asList();
for (int i = 0; i < repositories.size(); i++) {
GHRepository repository = repositories.get(i);
if (!repository.getName().startsWith("sc_")) {
continue;
}
softwareComponentList.add(new SoftwareComponent(repository.getName(), repository.getDescription()));
}
} catch (IOException e) {
// NOTHING
}
return softwareComponentList;
}
}
The reason
The URLConnection object is created with its useCache variable set to false because its defaultUseCaches variable is also set to false by the Tomcat server at the time of initialization. Tomcat does this through its JreMemoryLeakPreventionListener class because reading resources from JAR files using java.net.URLConnections can sometimes result in the JAR file being locked (urlCacheProtection variable). The workaround they implemented to solve this problem was to disable URLConnection caching by default (!?!?).
The solution
The workaround to this workaround is to create a dummy URLConnection and use its setDefaultUseCaches() method to change the default value of every subsequently created URLConnection (as suggested by Jesse Wilson).
URL url = new URL("jar:file://dummy.jar!/");
URLConnection uConn = url.openConnection();
uConn.setDefaultUseCaches(true);
Big thanks to Jesse Wilson for pointing me in the right direction!
There's an insane method called URLConnection.setDefaultUseCaches() that could be doing it globally. That's an instance method that works like a static method: it sets the property for everyone.