I'm trying to write unit tests for my program and use mock data. I'm a little confused on how to intercept an HTTP Get request to a URL.
My program calls a URL to our API and it is returned a simple XML file. I would like the test to instead of getting the XML file from the API online to receive a predetermined XML file from me so that I can compare the output to the expected output and determine if everything is working correctly.
I was pointed to Mockito and have been seeing many different examples such as this SO post, How to use mockito for testing a REST service? but it's not becoming clear to me how to set it all up and how to mock the data (i.e., return my own xml file whenever the call to the URL is made).
The only thing I can think of is having another program made that's running locally on Tomcat and in my test pass a special URL that calls the locally running program on Tomcat and then return the xml file that I want to test with. But that just seems like overkill and I don't think that would be acceptable. Could someone please point me in the right direction.
private static InputStream getContent(String uri) {
HttpURLConnection connection = null;
try {
URL url = new URL(uri);
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept", "application/xml");
return connection.getInputStream();
} catch (MalformedURLException e) {
LOGGER.error("internal error", e);
} catch (IOException e) {
LOGGER.error("internal error", e);
} finally {
if (connection != null) {
connection.disconnect();
}
}
return null;
}
I am using Spring Boot and other parts of the Spring Framework if that helps.
Part of the problem is that you're not breaking things down into interfaces. You need to wrap getContent into an interface and provide a concrete class implementing the interface. This concrete class will then
need to be passed into any class that uses the original getContent. (This is essentially dependency inversion.) Your code will end up looking something like this.
public interface IUrlStreamSource {
InputStream getContent(String uri)
}
public class SimpleUrlStreamSource implements IUrlStreamSource {
protected final Logger LOGGER;
public SimpleUrlStreamSource(Logger LOGGER) {
this.LOGGER = LOGGER;
}
// pulled out to allow test classes to provide
// a version that returns mock objects
protected URL stringToUrl(String uri) throws MalformedURLException {
return new URL(uri);
}
public InputStream getContent(String uri) {
HttpURLConnection connection = null;
try {
Url url = stringToUrl(uri);
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Accept", "application/xml");
return connection.getInputStream();
} catch (MalformedURLException e) {
LOGGER.error("internal error", e);
} catch (IOException e) {
LOGGER.error("internal error", e);
} finally {
if (connection != null) {
connection.disconnect();
}
}
return null;
}
}
Now code that was using the static getContent should go through a IUrlStreamSource instances getContent(). You then provide to the object that you want to test a mocked IUrlStreamSource rather than a SimpleUrlStreamSource.
If you want to test SimpleUrlStreamSource (but there's not much to test), then you can create a derived class that provides an implementation of stringToUrl that returns a mock (or throws an exception).
The other answers in here advise you to refactor your code to using a sort of provider which you can replace during your tests - which is the better approach.
If that isn't a possibility for whatever reason you can install a custom URLStreamHandlerFactory that intercepts the URLs you want to "mock" and falls back to the standard implementation for URLs that shouldn't be intercepted.
Note that this is irreversible, so you can't remove the InterceptingUrlStreamHandlerFactory once it's installed - the only way to get rid of it is to restart the JVM. You could implement a flag in it to disable it and return null for all lookups - which would produce the same results.
URLInterceptionDemo.java:
public class URLInterceptionDemo {
private static final String INTERCEPT_HOST = "dummy-host.com";
public static void main(String[] args) throws IOException {
// Install our own stream handler factory
URL.setURLStreamHandlerFactory(new InterceptingUrlStreamHandlerFactory());
// Fetch an intercepted URL
printUrlContents(new URL("http://dummy-host.com/message.txt"));
// Fetch another URL that shouldn't be intercepted
printUrlContents(new URL("http://httpbin.org/user-agent"));
}
private static void printUrlContents(URL url) throws IOException {
try(InputStream stream = url.openStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(stream))) {
String line;
while((line = reader.readLine()) != null) {
System.out.println(line);
}
}
}
private static class InterceptingUrlStreamHandlerFactory implements URLStreamHandlerFactory {
#Override
public URLStreamHandler createURLStreamHandler(final String protocol) {
if("http".equalsIgnoreCase(protocol)) {
// Intercept HTTP requests
return new InterceptingHttpUrlStreamHandler();
}
return null;
}
}
private static class InterceptingHttpUrlStreamHandler extends URLStreamHandler {
#Override
protected URLConnection openConnection(final URL u) throws IOException {
if(INTERCEPT_HOST.equals(u.getHost())) {
// This URL should be intercepted, return the file from the classpath
return URLInterceptionDemo.class.getResource(u.getHost() + "/" + u.getPath()).openConnection();
}
// Fall back to the default handler, by passing the default handler here we won't end up
// in the factory again - which would trigger infinite recursion
return new URL(null, u.toString(), new sun.net.www.protocol.http.Handler()).openConnection();
}
}
}
dummy-host.com/message.txt:
Hello World!
When run, this app will output:
Hello World!
{
"user-agent": "Java/1.8.0_45"
}
It's pretty easy to change the criteria of how you decide which URLs to intercept and what you return instead.
The answer depends on what you are testing.
If you need to test the processing of the InputStream
If getContent() is called by some code that processes the data returned by the InputStream, and you want to test how the processing code handles specific sets of input, then you need to create a seam to enable testing. I would simply move getContent() into a new class, and inject that class into the class that does the processing:
public interface ContentSource {
InputStream getContent(String uri);
}
You could create a HttpContentSource that uses URL.openConnection() (or, better yet, the Apache HttpClientcode).
Then you would inject the ContentSource into the processor:
public class Processor {
private final ContentSource contentSource;
#Inject
public Processor(ContentSource contentSource) {
this.contentSource = contentSource;
}
...
}
The code in Processor could be tested with a mock ContentSource.
If you need to test the fetching of the content
If you want to make sure that getContent() works, you could create a test that starts a lightweight in-memory HTTP server that serves the expected content, and have getContent() talk to that server. That does seem overkill.
If you need to test a large subset of the system with fake data
If you want to make sure things work end to end, write an end to end system test. Since you indicated you use Spring, you can use Spring to wire together parts of the system (or to wire the entire system, but with different properties). You have two choices
Have the system test start a local HTTP server, and when you have your test create your system, configure it to talk to that server. See the answers to this question for ways to start the HTTP server.
Configure spring to use a fake implementation of ContentSource. This gets you slightly less confidence that everything works end-to-end, but it will be faster and less flaky.
Related
I have the following code:
public byte[] myFunction(String s1, String s2, String s3) {
try {
URL myUrl = new URL("https://myUrl");
HttpsURLConnection connection = (HttpsURLConnection) myUrl.openConnection();
connection.connect();
int responseCode = connection.getResponseCode();
String lastPacket = connection.getHeaderField("LastPacket");
byte fileContent[] = IOUtils.toByteArray((InputStream) connection.getInputStream());
return fileContent;
}
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
I want to unit test the above code.
So I've written JUnit using MockitoJUnitRunner as:
#RunWith(MockitoJUnitRunner.class)
public class EvmlSeviceImplTest {
#Mock
private URL url;
#Mock
private HttpsURLConnection httpsURLConnection;
#InjectMocks
private MyClass myClass;
#Before
public void preSetup() {
try {
Mockito.doReturn(httpsURLConnection).when(url).openConnection();
Mockito.doReturn(200).when(httpsURLConnection).getResponseCode();
Mockito.doReturn("-1").when(httpsURLConnection).getHeaderField("LastPacket");
} catch (Exception e) {
e.printStackTrace();
}
}
#Test
public void testMyFunction() {
myClass.myFunction("my", "dummy", "data");
}
}
But the problem here is that URL is final class and Mockito cannot spy final classes. Since we're using MockitoJUnitRunner, how can this be done?
I'm getting the exception:
org.mockito.exceptions.base.MockitoException: Cannot mock/spy class
java.net.URL
Let's look closer at your function and what you try to test:
URL myUrl = new URL("https://myUrl"); // (1)
HttpsURLConnection connection = (HttpsURLConnection) myUrl.openConnection(); // (2)
connection.connect(); // (3)
int responseCode = connection.getResponseCode(); // (4)
String lastPacket = connection.getHeaderField("LastPacket");
byte fileContent[] = IOUtils.toByteArray((InputStream) connection.getInputStream()); // (5)
return fileContent;
So in line 1, 2 and 3 you create a connection object to some remote resource and connect to it. You use SDK's classes for that.
Then you check the response code (4) and read from the stream (5). Again, you are using SDK's standard library for that.
Question is: do you really want to unit test code that we are safe to assume is working?
What I would do is extract lines from 1 to 4 to a separate class with a method that returns a stream that you can read. And then, in your test mock that stream according to your needs. But don't unit test such low-level code.
Looks to me like your preSetup is wrong as well.
Even though you mock HttpsURLConnection (httpsURLConnection) and URL (url), this isn't used in the test.
You can't just use "url" (the mocked URL) in your test or setup, since in the method you want to test (myFunction), a new URL (myUrl) is created. This "myUrl" is not the same as "url" you mocked.
I would pass an URLConnection as an argument to that method (or create a second function). You then have full control over it and can mock it.
// Edit:
Mocking a URL in Java
I am using the Kohsuke GitHub-API to connect to the GitHub from my Java (server-side) application and I wanted to use the OkHttp's ability to cache responses from the GitHub. This worked perfectly when I wrote a test for it, but it doesn't work in the application itself and I don't have a clue why that is. I have managed to trace the problem back to the creation of the URLConnection object that is created with its useCache variable set to false, but I cannot figure out why. Does it maybe have something to do with the server configuration or something like that?
I would appreciate any ideas or even a nudge in any direction, because frankly I don't have any ideas left... Thanks
Provider:
public class GitHubProvider implements Provider<GitHub> {
#Override
public GitHub get() {
GitHub gitHub = null;
HttpResponseCache cache = null;
OkHttpClient okHttpClient = new OkHttpClient();
File cacheDir = new File(System.getProperty("java.io.tmpdir"), UUID.randomUUID().toString());
try {
cache = new HttpResponseCache(cacheDir, 10L * 1024 * 1024);
} catch (IOException e) {
// NOTHING
}
okHttpClient.setResponseCache(cache);
try {
gitHub = GitHub.connectUsingPassword("user", "password");
} catch (Exception e) {
// NOTHING
}
gitHub.setConnector(new OkHttpConnector(okHttpClient));
return gitHub;
}
}
Test (works):
#RunWith(JukitoRunner.class)
public class SoftwareComponentServiceTest {
public static class Module extends TestModule {
#Override
protected void configureTest() {
bind(GitHub.class).toProvider(GitHubProvider.class);
}
}
#Inject
GitHub gitHub;
#Test
public void testInjectedGitHubResponseCache() {
try {
GHUser ghUser = gitHub.getUser("user");
GHRepository repository = ghUser.getRepository("repository");
int limit = gitHub.getRateLimit().remaining;
repository.getFileContent("README.md");
assertEquals(limit - 1, gitHub.getRateLimit().remaining);
repository.getFileContent("README.md");
assertEquals(limit - 1, gitHub.getRateLimit().remaining);
} catch (IOException e) {
e.printStackTrace();
}
}
}
Service that is used in the application (doesn't work):
#Singleton
#RequiresAuthentication
public class SoftwareComponentService {
#Inject
GitHub gitHub;
public List<SoftwareComponent> findAll() {
List<SoftwareComponent> softwareComponentList = new ArrayList<SoftwareComponent>();
try {
GHUser ghUser = gitHub.getUser("user");
List<GHRepository> repositories = ghUser.listRepositories().asList();
for (int i = 0; i < repositories.size(); i++) {
GHRepository repository = repositories.get(i);
if (!repository.getName().startsWith("sc_")) {
continue;
}
softwareComponentList.add(new SoftwareComponent(repository.getName(), repository.getDescription()));
}
} catch (IOException e) {
// NOTHING
}
return softwareComponentList;
}
}
The reason
The URLConnection object is created with its useCache variable set to false because its defaultUseCaches variable is also set to false by the Tomcat server at the time of initialization. Tomcat does this through its JreMemoryLeakPreventionListener class because reading resources from JAR files using java.net.URLConnections can sometimes result in the JAR file being locked (urlCacheProtection variable). The workaround they implemented to solve this problem was to disable URLConnection caching by default (!?!?).
The solution
The workaround to this workaround is to create a dummy URLConnection and use its setDefaultUseCaches() method to change the default value of every subsequently created URLConnection (as suggested by Jesse Wilson).
URL url = new URL("jar:file://dummy.jar!/");
URLConnection uConn = url.openConnection();
uConn.setDefaultUseCaches(true);
Big thanks to Jesse Wilson for pointing me in the right direction!
There's an insane method called URLConnection.setDefaultUseCaches() that could be doing it globally. That's an instance method that works like a static method: it sets the property for everyone.
I have a memory class loader (here) that I am using in a custom Minecraft launcher.
Memory Class Loader
Whenever I load up Minecraft (a Java LWJGL game), I am getting the following error:
27 achievements
182 recipes
Setting user
LWJGL Version: 2.4.2
java.lang.IllegalArgumentException: input == null!
at javax.imageio.ImageIO.read(Unknown Source)
at lc.<init>(SourceFile:21)
at gi.<init>(SourceFile:10)
at net.minecraft.client.Minecraft.a(SourceFile:254)
at net.minecraft.client.Minecraft.run(SourceFile:657)
at java.lang.Thread.run(Unknown Source)
I am creating the class loader like this:
Base.cLoader = new CLoader(
GameUpdater.classLoader,
new JarInputStream(new ByteArrayInputStream(jarFileBytes)));
As you can see, it manages to load up the first part then suddenly after LWJGL Version it crashes with "input == null".
Edit - Here is the new getResource method.
The error is on "URL()", as shown.
Code:
public URL getResource(final String name) {
URL url = new URL() { public InputStream openStream() {
return new ByteArrayInputStream((byte[])others.get(name));
}};
return url;
}
A wild guess... it could be this: Warning: URLs for this are not yet implemented! You cannot call getResource() or getResources()!
So your code expects to retrieve an image from the JAR using the unimplemented method. An equivalent of this is probably being executed:
ImageIO.read(memClassLoader.getResource(someString));
Except that, as we have seen, the Error thrown from getResource is getting ignored and null being used as the value. ImageIO.read goes like this:
public static BufferedImage read(URL input) throws IOException {
if (input == null) {
throw new IllegalArgumentException("input == null!");
}
InputStream istream = null;
try {
istream = input.openStream();
} catch (IOException e) {
throw new IIOException("Can't get input stream from URL!", e);
}
}
Sounds familiar? So, this is roughly what you need to implement:
public URL getResource(final String name) {
try {
return new URL("", "", 0, "",
new URLStreamHandler() { public URLConnection openConnection(URL url) {
return new URLConnection(url) {
public void connect() {}
public InputStream getInputStream() {
// return the appropriate InputStream, based on the name arg
}
};
}});
} catch (MalformedURLException e) { throw new RuntimeException(e); }
}
The MemoryClassLoader is pretty much broken. It does not implement getResource() (as stated in the comment in the source), and also it does not define Packages for the classes it loads (this may or may not break an application).
Most likely that ClassLoader was quickly hacked for testing purposes, leaving the more complicated methods out.
Implementing your own URL protocol to handle getResource() is not too difficult, in getResource() you return an URL that uses a custom protocol name (e.g. "myclassloader://resourcename"), and also a custom implementation of URLStreamHandler that handles that protocol.
That may not cover all the loopholes that might cause trouble in locating a resource, if the code loaded through the ClassLoader uses URL.toString() and converts it back it will still break.
Implementing a fully working ClassLoader that does not simple delegation to existing ClassLoaders, is not as simple as most examples make it look.
I was tinkering on android with WMS Layers. One of the services I want to load the layer from is serving them via Https. The wms layer example i found though uses:
InputStream input = null;
try {
input = url.openStream();
} catch (IOException e) {
Log.e(TAG, e.getMessage());
}
Where url is a type URL and is set to a url that uses HTTPS. This throws an error as I suspect I have to set up my certificates or something. Is there anyway to just say accept brute force this to accept the certs? I tried something similar to this in c# and was able to just basically making a call to this:
C# code to make the https work:
ServicePointManager.ServerCertificateValidationCallback += new RemoteCertificateValidationCallback(AcceptAllCertifications);
...
...
public bool AcceptAllCertifications(object sender, System.Security.Cryptography.X509Certificates.X509Certificate certification,
System.Security.Cryptography.X509Certificates.X509Chain chain, System.Net.Security.SslPolicyErrors sslPolicyErrors)
{
//this might be useful later too
//http://blog.jameshiggs.com/2008/05/01/c-how-to-accept-an-invalid-ssl-certificate-programmatically/
return true;
}
I have a WSDL file for a web service. I'm using JAX-WS/wsimport to generate a client interface to the web service. I don't know ahead of time the host that the web service will be running on, and I can almost guarantee it won't be http://localhost:8080. How to I specify the host URL at runtime, e.g. from a command-line argument?
The generated constructor MyService(URL wsdlLocation, QName serviceName) doesn't seem like what I want, but maybe it is? Perhaps one of the variants of Service.getPort(...)?
Thanks!
The constructor should work fine for your needs, when you create MyService, pass it the url of the WSDL you want i.e. http://someurl:someport/service?wsdl.
If you have a look in the generated source close to the generated constructor, you should be able to figure out what to put in it from the default constructor, should look something like:
public OrdersService() {
super(WSDL_LOCATION, new QName("http://namespace.org/order/v1", "OrdersService"));
}
You should be able to find the def of WSDL_LOCATION in the static field further up in the class.
In your generated code (eg: say "HelloWorldWebServiceImplService" ) look in to the static block on the top which will have reference to the WSDL url or wsdl file which is under META-INF.
/*
static {
URL url = null;
try {
url = new URL("http://loclahost/HelloWorld/HelloWorldWebServiceImpl?wsdl");
} catch (MalformedURLException e) {
java.util.logging.Logger.getLogger(HelloWorldWebServiceImplService.class.getName())
.log(java.util.logging.Level.INFO,
"Can not initialize the default wsdl from {0}", "http://loclahost/HelloWorld/HelloWorldWebServiceImpl?wsdl");
}
WSDL_LOCATION = url;
}
*/
Once you comment this you also need to comment out the default construtor and needless to say intialize the static WSDL_LOCATION = null; (to null)
So you will not have two constructors as shown below.
public final static URL WSDL_LOCATION = null;
public HelloWorldWebServiceImplService(URL wsdlLocation) {
super(wsdlLocation, SERVICE);
}
public HelloWorldWebServiceImplService(URL wsdlLocation, QName serviceName) {
super(wsdlLocation, serviceName);
}
Calling Webservice : Now in the client call where you instantiate this object Pass the webservice URL as an argument as shown
//You can read mywebserviceURL from property file as String.
String mywebserviceURL = "http://myqamachine.com/HelloWorld/HelloWorldWebServiceImpl?wsdl"
URL WsURL = new URL(mywebserviceURL);
HelloWorldWebServiceImplService webService = new HelloWorldWebServiceImplService(WsURL);
So here you can point the webservice url dynamically.