JUnit4: Testing System.out in multiple test methods - java

I have an application where I want to write unit tests to test the output, written to System.out (and perhaps System.err).
Each individual test works as expected, however when adding multiple tests in the same class some tests fail because JUnit4 appears to be multi-threaded (thus no guarantees exists on when exactly the stream is reset).
The same happens when I separate all test methods in their own class and use a test suite.
Any ideas?
private static final PrintStream SYS_OUT = System.out;
private static final PrintStream SYS_ERR = System.err;
private final ByteArrayOutputStream outContent = new ByteArrayOutputStream();
private final ByteArrayOutputStream errContent = new ByteArrayOutputStream();
#Before
public final void setUpStreams() throws UnsupportedEncodingException {
System.setOut(new PrintStream(this.outContent, true, CHARSET));
System.setErr(new PrintStream(this.errContent, true, CHARSET));
}
#After
public final void cleanUpStreams() {
System.setOut(SYS_OUT);
System.setErr(SYS_ERR);
}
#Test
public final void listServicesAndMethods() throws ServiceException, UnsupportedEncodingException {
com.example.Main.main(new String[]{"--list-services"});
LOG.debug(this.outContent.toString(CHARSET));
assertTrue("String not found", this.outContent.toString(CHARSET).contains("Some string"));
assertFalse("Other string found", this.outContent.toString(CHARSET).contains("Some other string"));
this.outContent.reset();
this.errContent.reset();
}
Edit: It turns out that the issue with the failing tests was not (just?) because of the streams, but due to the fact that I stored the options in static fields in my main class. This has the effect that several options stay active during consecutive tests. I realised that after implementing Arian's suggestion, I then used the second class as an instance, instead of calling static methods, thus solving my issue.
Thanks to all who replied.

If not already done so, rewrite the parts of your application to take the output as a parameter, instead of writing directly to System.out. This is usually better design regardless of testing.
In each test, create a new output stream (or spy on System.out if you must) and pass it to the code unit under test.

Related

Mockito does not return what I need

public class ConnectionPointTest {
#Test
public void testMockito() throws IOException {
DataInputStream dataInputStream = mock(DataInputStream.class);
when(dataInputStream.readUTF()).thenReturn("Test");
new JustTest(dataInputStream).doTest();
}
public class JustTest {
DataInputStream dataInputStream;
public JustTest(DataInputStream dataInputStream) {
this.dataInputStream = dataInputStream;
}
public void doTest() throws IOException {
String s = dataInputStream.readUTF();
System.out.println(s);
}
}
}
I implement JUnit testing + Mockito in the project. Then I try to mock my DataInpuStream, I have an exeption at this code :
when(dataInputStream.readUTF()).thenReturn("Test");
P.S. class JustTest is only for show you what I want to mock.
There is exeption:
java.lang.NullPointerException
at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:337)
at java.io.DataInputStream.readUTF(DataInputStream.java:589)
at java.io.DataInputStream.readUTF(DataInputStream.java:564)
...
You are trying to mock a final method:
public final String readUTF() throws IOException {
return readUTF(this);
}
Mockito (v2.x) supports this but it requires additional configuration. More details in the docs:
Mocking of final classes and methods is an incubating, opt-in feature. It uses a combination of Java agent instrumentation and subclassing in order to enable mockability of these types. As this works differently to our current mechanism and this one has different limitations and as we want to gather experience and user feedback, this feature had to be explicitly activated to be available ; it can be done via the mockito extension mechanism by creating the file src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker containing a single line:
mock-maker-inline
So, create a file named org.mockito.plugins.MockMaker with this content:
mock-maker-inline
Place this file in a folder named mockito-extensions on your test classpath (if you are using Maven just create this folder in src/test/resources)
Re run your test.
With this configuration in place I have successful run your test with:
JUnit 4.12
Mockito 2.7.19

In the System.java source, the standard input, output and error streams are declared final and initialized null?

public final static InputStream in = null;
public final static PrintStream out = null;
public final static PrintStream err = null;
But as we very well know, these streams are connected to the console by default and already open. There are also methods in the System class setIn(), setOut, and setErr() to redirect the streams. How is any of this possible when they have been declared final and set to the initialization value null?
I compiled the following code, set a breakpoint at the call to println() and debugged using netbeans. My objective was to determine exactly when the variable System.in is initialized to the standard output by stepping into the source. But it seems that the output stream out is already initialized by the time the main method is called.
public static void main(String[] args) {
System.out.println("foo");
}
This is done in order to prevent "hacking". These fields can be changed only by appropriate setters that call native methods
private static native void setIn0(InputStream in);
private static native void setOut0(PrintStream out);
private static native void setErr0(PrintStream err);
Native methods can do everything including changing final fields.
They are later on set by native methods SetIn0, SetOut0 and SetErr0
private static native void setIn0(InputStream in);
private static native void setOut0(PrintStream out);
private static native void setErr0(PrintStream err);
called from the initializeSystemClass method, which according to the JavaDoc is called after thread initialization.
FileInputStream fdIn = new FileInputStream(FileDescriptor.in);
FileOutputStream fdOut = new FileOutputStream(FileDescriptor.out);
FileOutputStream fdErr = new FileOutputStream(FileDescriptor.err);
setIn0(new BufferedInputStream(fdIn));
setOut0(new PrintStream(new BufferedOutputStream(fdOut, 128), true));
setErr0(new PrintStream(new BufferedOutputStream(fdErr, 128), true));
final fields are not necessarily constant. They can still be manipulated, it's just that manipulation is only prevented at compile-time, specifically by preventing you from using the assignment operator (=). See this question and JLS ยง17.5.3, specifically:
final fields can be changed via reflection and other implementation-dependent means.
This is necessary for things like deserialization. This can also cause some interesting caveats since compilers can optimize final fields on compile-time and run-time. The JLS linked above has an example of this.

junit test when a method is called from one test

public void createRootElement() throws FileNotFoundException, IOException
{
Properties prop = new Properties();
prop.load(new FileInputStream("/home/asdf/Desktop/test.properties"));
File file = new File(prop.getProperty("filefromroot"));
try
{
// if file doesn't exists, then create it
if (!file.exists())
{
file.createNewFile();
}
FileWriter fw = new FileWriter(file.getAbsoluteFile());
BufferedWriter bw = new BufferedWriter(fw);
bw.write("<root>"); //create the root tag for the XML File.
bw.close();
}
catch(Exception e)
{
writeLog(e.getMessage(),false);
}
}
I'm new to junit testing.I would like to know how to write test case for this and what all to be considered. how to call the method is called from this test.?
A JUnit Test Case should look like this:
import static org.junit.Assert.assertTrue;
import org.junit.Test;
public class ClassToBeTestedTest {
#Test
public void test() {
ClassToBeTested c = new ClassToBeTested();
c.createRootElement();
assertTrue(c.rootElementExists());
}
}
You mark the test method with the #Test annotation and write the code that executes what you want to test.
At this example, I created a instance of your class and called the createRootElement method.
After that, I made a assertion to verify if everything behaved like I expected.
There are many things you can assert. Read the JUnit documentation for more information.
A good practice is to write the test BEFORE you actually write the code. So the test will guide you on how to write a better code. This is called TDD. Google for it.

How to handle ordering of #Rule's when they are dependent on each other

I use embedded servers that run inside Junit test cases. Sometimes these servers require a working directory (for example the Apache Directory server).
The new #Rule in Junit 4.7 can handle these cases. The TemporaryFolder-Rule can create a temporary directory. A custom ExternalResource-Rule can be created for server. But how do I handle if I want to pass the result from one rule into another:
import static org.junit.Assert.assertEquals;
import java.io.*;
import org.junit.*;
import org.junit.rules.*;
public class FolderRuleOrderingTest {
#Rule
public TemporaryFolder folder = new TemporaryFolder();
#Rule
public MyNumberServer server = new MyNumberServer(folder);
#Test
public void testMyNumberServer() throws IOException {
server.storeNumber(10);
assertEquals(10, server.getNumber());
}
/** Simple server that can store one number */
private static class MyNumberServer extends ExternalResource {
private TemporaryFolder folder;
/** The actual datafile where the number are stored */
private File dataFile;
public MyNumberServer(TemporaryFolder folder) {
this.folder = folder;
}
#Override
protected void before() throws Throwable {
if (folder.getRoot() == null) {
throw new RuntimeException("TemporaryFolder not properly initialized");
}
//All server data are stored to a working folder
File workingFolder = folder.newFolder("my-work-folder");
dataFile = new File(workingFolder, "datafile");
}
public void storeNumber(int number) throws IOException {
dataFile.createNewFile();
DataOutputStream out = new DataOutputStream(new FileOutputStream(dataFile));
out.writeInt(number);
}
public int getNumber() throws IOException {
DataInputStream in = new DataInputStream(new FileInputStream(dataFile));
return in.readInt();
}
}
}
In this code the folder is sent as a parameter into the server so that the server can create a working directory to store data. However this does not work because Junit processes the rules in reverse order as they are defined in the file. The TemporaryFolder Rule will not be executed before the server Rule. Thus the root-folder in TempraryFolder will be null, resulting that any files are created relative to the current working directory.
If I reverse the order of the attributes in my class I get a compile error because I cannot reference a variable before it is defined.
I'm using Junit 4.8.1 (because the ordering of rules was fixed a bit from the 4.7 release)
EDIT: With the recently released Junit 4.10, you can use RuleChain to chain rules correctly (see at the end).
You could introduce another private field without the #Rule annotation, then you can reorder your code as you wish:
public class FolderRuleOrderingTest {
private TemporaryFolder privateFolder = new TemporaryFolder();
#Rule
public MyNumberServer server = new MyNumberServer(privateFolder);
#Rule
public TemporaryFolder folder = privateFolder;
#Test
public void testMyNumberServer() throws IOException {
server.storeNumber(10);
assertEquals(10, server.getNumber());
}
...
}
The cleanest solution is to have a compound rule, but the above should work.
EDIT: With the recently released Junit 4.10, you can use RuleChain to chain rules correctly:
public static class UseRuleChain {
#Rule
public TestRule chain = RuleChain
.outerRule(new LoggingRule("outer rule"))
.around(new LoggingRule("middle rule"))
.around(new LoggingRule("inner rule"));
#Test
public void example() {
assertTrue(true);
}
}
writes the log
starting outer rule
starting middle rule
starting inner rule
finished inner rule
finished middle rule
finished outer rule
To make the rules dependent, your have to initialize them first and create the dependency relationships using contructors or (depending on you rule) fluent builders. The dependency relations have to be defined in the field-initialization and could not be created in #Before methods as those are executed after rule application. To force the correct ordering of rule execution, you have to define the rule chain.
public class FolderRuleOrderingTest {
private TemporaryFolder folder = new TemporaryFolder();
//assume, we have a rule that creates a testfile in a temporary folder
//we create a dependency relationship between file and folder,
//so that file depends on folder
private TemporaryFile file = new TemporaryFile(folder, "testfile.txt");
//the rule chain ensures, the temporary folder is created before and removed
//after the testfile has been created and deleted (or whatever)
#Rule
public RuleChain chain= RuleChain.outerRule(folder).around(file));
#Test
public void testFileExist() throws IOException {
assertTrue(file.getFile().exist());
}
...
}
If you will not find normal solution, you can always create compound rule (and the only one having #Rule annotation) that contains all others and executes them in order.
You can also use the attribute order since 4.13:
You can use order() if you want to have control over the order in which the Rules are applied.
public class ThreeRules {
#Rule(order = 0)
public LoggingRule outer = new LoggingRule("outer rule");
#Rule(order = 1)
public LoggingRule middle = new LoggingRule("middle rule");
#Rule(order = 2)
public LoggingRule inner = new LoggingRule("inner rule");
// ...
}
Alternatively, you could simply offer a setter in your MyNumberServer rule instead of giving the folder in the constructor.
Furthermore, ordering amongst rules is not guaranteed the way you described. It can become a little tricky, especially when you want some communication between your rules, see e.g. Best way of logging exceptions when tests fail (e.g. using a junit rule).

JUnit test for System.out.println()

I need to write JUnit tests for an old application that's poorly designed and is writing a lot of error messages to standard output. When the getResponse(String request) method behaves correctly it returns a XML response:
#BeforeClass
public static void setUpClass() throws Exception {
Properties queries = loadPropertiesFile("requests.properties");
Properties responses = loadPropertiesFile("responses.properties");
instance = new ResponseGenerator(queries, responses);
}
#Test
public void testGetResponse() {
String request = "<some>request</some>";
String expResult = "<some>response</some>";
String result = instance.getResponse(request);
assertEquals(expResult, result);
}
But when it gets malformed XML or does not understand the request it returns null and writes some stuff to standard output.
Is there any way to assert console output in JUnit? To catch cases like:
System.out.println("match found: " + strExpr);
System.out.println("xml not well formed: " + e.getMessage());
using ByteArrayOutputStream and System.setXXX is simple:
private final ByteArrayOutputStream outContent = new ByteArrayOutputStream();
private final ByteArrayOutputStream errContent = new ByteArrayOutputStream();
private final PrintStream originalOut = System.out;
private final PrintStream originalErr = System.err;
#Before
public void setUpStreams() {
System.setOut(new PrintStream(outContent));
System.setErr(new PrintStream(errContent));
}
#After
public void restoreStreams() {
System.setOut(originalOut);
System.setErr(originalErr);
}
sample test cases:
#Test
public void out() {
System.out.print("hello");
assertEquals("hello", outContent.toString());
}
#Test
public void err() {
System.err.print("hello again");
assertEquals("hello again", errContent.toString());
}
I used this code to test the command line option (asserting that -version outputs the version string, etc etc)
Edit:
Prior versions of this answer called System.setOut(null) after the tests; This is the cause of NullPointerExceptions commenters refer to.
I know this is an old thread, but there is a nice library to do this: System Rules
Example from the docs:
public void MyTest {
#Rule
public final SystemOutRule systemOutRule = new SystemOutRule().enableLog();
#Test
public void overrideProperty() {
System.out.print("hello world");
assertEquals("hello world", systemOutRule.getLog());
}
}
It will also allow you to trap System.exit(-1) and other things that a command line tool would need to be tested for.
Instead of redirecting System.out, I would refactor the class that uses System.out.println() by passing a PrintStream as a collaborator and then using System.out in production and a Test Spy in the test. That is, use Dependency Injection to eliminate the direct use of the standard output stream.
In Production
ConsoleWriter writer = new ConsoleWriter(System.out));
In the Test
ByteArrayOutputStream outSpy = new ByteArrayOutputStream();
ConsoleWriter writer = new ConsoleWriter(new PrintStream(outSpy));
writer.printSomething();
assertThat(outSpy.toString(), is("expected output"));
Discussion
This way the class under test becomes testable by a simple refactoring, without having the need for indirect redirection of the standard output or obscure interception with a system rule.
You can set the System.out print stream via setOut() (and for in and err). Can you redirect this to a print stream that records to a string, and then inspect that ? That would appear to be the simplest mechanism.
(I would advocate, at some stage, convert the app to some logging framework - but I suspect you already are aware of this!)
Slightly off topic, but in case some people (like me, when I first found this thread) might be interested in capturing log output via SLF4J, commons-testing's JUnit #Rule might help:
public class FooTest {
#Rule
public final ExpectedLogs logs = new ExpectedLogs() {{
captureFor(Foo.class, LogLevel.WARN);
}};
#Test
public void barShouldLogWarning() {
assertThat(logs.isEmpty(), is(true)); // Nothing captured yet.
// Logic using the class you are capturing logs for:
Foo foo = new Foo();
assertThat(foo.bar(), is(not(nullValue())));
// Assert content of the captured logs:
assertThat(logs.isEmpty(), is(false));
assertThat(logs.contains("Your warning message here"), is(true));
}
}
Disclaimer:
I developed this library since I could not find any suitable solution for my own needs.
Only bindings for log4j, log4j2 and logback are available at the moment, but I am happy to add more.
If you were using Spring Boot (you mentioned that you're working with an old application, so you probably aren't but it might be of use to others), then you could use org.springframework.boot.test.rule.OutputCapture in the following manner:
#Rule
public OutputCapture outputCapture = new OutputCapture();
#Test
public void out() {
System.out.print("hello");
assertEquals(outputCapture.toString(), "hello");
}
#dfa answer is great, so I took it a step farther to make it possible to test blocks of ouput.
First I created TestHelper with a method captureOutput that accepts the annoymous class CaptureTest. The captureOutput method does the work of setting and tearing down the output streams. When the implementation of CaptureOutput's test method is called, it has access to the output generate for the test block.
Source for TestHelper:
public class TestHelper {
public static void captureOutput( CaptureTest test ) throws Exception {
ByteArrayOutputStream outContent = new ByteArrayOutputStream();
ByteArrayOutputStream errContent = new ByteArrayOutputStream();
System.setOut(new PrintStream(outContent));
System.setErr(new PrintStream(errContent));
test.test( outContent, errContent );
System.setOut(new PrintStream(new FileOutputStream(FileDescriptor.out)));
System.setErr(new PrintStream(new FileOutputStream(FileDescriptor.out)));
}
}
abstract class CaptureTest {
public abstract void test( ByteArrayOutputStream outContent, ByteArrayOutputStream errContent ) throws Exception;
}
Note that TestHelper and CaptureTest are defined in the same file.
Then in your test, you can import the static captureOutput. Here is an example using JUnit:
// imports for junit
import static package.to.TestHelper.*;
public class SimpleTest {
#Test
public void testOutput() throws Exception {
captureOutput( new CaptureTest() {
#Override
public void test(ByteArrayOutputStream outContent, ByteArrayOutputStream errContent) throws Exception {
// code that writes to System.out
assertEquals( "the expected output\n", outContent.toString() );
}
});
}
Based on #dfa's answer and another answer that shows how to test System.in, I would like to share my solution to give an input to a program and test its output.
As a reference, I use JUnit 4.12.
Let's say we have this program that simply replicates input to output:
import java.util.Scanner;
public class SimpleProgram {
public static void main(String[] args) {
Scanner scanner = new Scanner(System.in);
System.out.print(scanner.next());
scanner.close();
}
}
To test it, we can use the following class:
import static org.junit.Assert.*;
import java.io.*;
import org.junit.*;
public class SimpleProgramTest {
private final InputStream systemIn = System.in;
private final PrintStream systemOut = System.out;
private ByteArrayInputStream testIn;
private ByteArrayOutputStream testOut;
#Before
public void setUpOutput() {
testOut = new ByteArrayOutputStream();
System.setOut(new PrintStream(testOut));
}
private void provideInput(String data) {
testIn = new ByteArrayInputStream(data.getBytes());
System.setIn(testIn);
}
private String getOutput() {
return testOut.toString();
}
#After
public void restoreSystemInputOutput() {
System.setIn(systemIn);
System.setOut(systemOut);
}
#Test
public void testCase1() {
final String testString = "Hello!";
provideInput(testString);
SimpleProgram.main(new String[0]);
assertEquals(testString, getOutput());
}
}
I won't explain much, because I believe the code is readable and I cited my sources.
When JUnit runs testCase1(), it is going to call the helper methods in the order they appear:
setUpOutput(), because of the #Before annotation
provideInput(String data), called from testCase1()
getOutput(), called from testCase1()
restoreSystemInputOutput(), because of the #After annotation
I didn't test System.err because I didn't need it, but it should be easy to implement, similar to testing System.out.
Full JUnit 5 example to test System.out (replace the when part):
package learning;
import static org.assertj.core.api.BDDAssertions.then;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
class SystemOutLT {
private PrintStream originalSystemOut;
private ByteArrayOutputStream systemOutContent;
#BeforeEach
void redirectSystemOutStream() {
originalSystemOut = System.out;
// given
systemOutContent = new ByteArrayOutputStream();
System.setOut(new PrintStream(systemOutContent));
}
#AfterEach
void restoreSystemOutStream() {
System.setOut(originalSystemOut);
}
#Test
void shouldPrintToSystemOut() {
// when
System.out.println("example");
then(systemOutContent.toString()).containsIgnoringCase("example");
}
}
You don't want to redirect the system.out stream because that redirects for the ENTIRE JVM. Anything else running on the JVM can get messed up. There are better ways to test input/output. Look into stubs/mocks.
If the function is printing to System.out, you can capture that output by using the System.setOut method to change System.out to go to a PrintStream provided by you. If you create a PrintStream connected to a ByteArrayOutputStream, then you can capture the output as a String.
// Create a stream to hold the output
ByteArrayOutputStream baos = new ByteArrayOutputStream();
PrintStream ps = new PrintStream(baos);
// IMPORTANT: Save the old System.out!
PrintStream old = System.out;
// Tell Java to use your special stream
System.setOut(ps);
// Print some output: goes to your special stream
System.out.println("Foofoofoo!");
// Put things back
System.out.flush();
System.setOut(old);
// Show what happened
System.out.println("Here: " + baos.toString());
for out
#Test
void it_prints_out() {
PrintStream save_out=System.out;final ByteArrayOutputStream out = new ByteArrayOutputStream();System.setOut(new PrintStream(out));
System.out.println("Hello World!");
assertEquals("Hello World!\r\n", out.toString());
System.setOut(save_out);
}
for err
#Test
void it_prints_err() {
PrintStream save_err=System.err;final ByteArrayOutputStream err= new ByteArrayOutputStream();System.setErr(new PrintStream(err));
System.err.println("Hello World!");
assertEquals("Hello World!\r\n", err.toString());
System.setErr(save_err);
}
Although this question is very old and has already very good answers I want to provide an alternative. I liked the answer of dfa however I wanted to have something reusable in different projects without copying the configuration and so I created a library out of it and wanted to contribute back to the community. It is called Console Captor and you can add it with the following snippet:
<dependency>
<groupId>io.github.hakky54</groupId>
<artifactId>consolecaptor</artifactId>
<version>1.0.0</version>
<scope>test</scope>
</dependency>
Example class
public class FooService {
public void sayHello() {
System.out.println("Keyboard not responding. Press any key to continue...");
System.err.println("Congratulations, you are pregnant!");
}
}
Unit test
import static org.assertj.core.api.Assertions.assertThat;
import nl.altindag.console.ConsoleCaptor;
import org.junit.jupiter.api.Test;
public class FooServiceTest {
#Test
public void captureStandardAndErrorOutput() {
ConsoleCaptor consoleCaptor = new ConsoleCaptor();
FooService fooService = new FooService();
fooService.sayHello();
assertThat(consoleCaptor.getStandardOutput()).contains("Keyboard not responding. Press any key to continue...");
assertThat(consoleCaptor.getErrorOutput()).contains("Congratulations, you are pregnant!");
consoleCaptor.close();
}
}
You cannot directly print by using system.out.println or using logger api while using JUnit. But if you want to check any values then you simply can use
Assert.assertEquals("value", str);
It will throw below assertion error:
java.lang.AssertionError: expected [21.92] but found [value]
Your value should be 21.92, Now if you will test using this value like below your test case will pass.
Assert.assertEquals(21.92, str);

Categories