Many in-house solutions come to mind. Like having the properties in a database and poll it every N secs. Then also check the timestamp modification for a .properties file and reload it.
But I was looking in Java EE standards and spring boot docs and I can't seem to find some best way of doing it.
I need my application to read a properties file(or env. variables or DB parameters), then be able to re-read them. What is the best practice being used in production?
A correct answer will at least solve one scenario (Spring Boot or Java EE) and provide a conceptual clue on how to make it work on the other
After further research, reloading properties must be carefully considered. In Spring, for example, we can reload the 'current' values of properties without much problem. But. Special care must be taken when resources were initialized at the context initialization time based on the values that were present in the application.properties file (e.g. Datasources, connection pools, queues, etc.).
NOTE:
The abstract classes used for Spring and Java EE are not the best example of clean code. But it is easy to use and it does address this basic initial requirements:
No usage of external libraries other than Java 8 Classes.
Only one file to solve the problem (~160 lines for the Java EE version).
Usage of standard Java Properties UTF-8 encoded file available in the File System.
Support encrypted properties.
For Spring Boot
This code helps with hot-reloading application.properties file without the usage of a Spring Cloud Config server (which may be overkill for some use cases)
This abstract class you may just copy & paste (SO goodies :D ) It's a code derived from this SO answer
// imports from java/spring/lombok
public abstract class ReloadableProperties {
#Autowired
protected StandardEnvironment environment;
private long lastModTime = 0L;
private Path configPath = null;
private PropertySource<?> appConfigPropertySource = null;
#PostConstruct
private void stopIfProblemsCreatingContext() {
System.out.println("reloading");
MutablePropertySources propertySources = environment.getPropertySources();
Optional<PropertySource<?>> appConfigPsOp =
StreamSupport.stream(propertySources.spliterator(), false)
.filter(ps -> ps.getName().matches("^.*applicationConfig.*file:.*$"))
.findFirst();
if (!appConfigPsOp.isPresent()) {
// this will stop context initialization
// (i.e. kill the spring boot program before it initializes)
throw new RuntimeException("Unable to find property Source as file");
}
appConfigPropertySource = appConfigPsOp.get();
String filename = appConfigPropertySource.getName();
filename = filename
.replace("applicationConfig: [file:", "")
.replaceAll("\\]$", "");
configPath = Paths.get(filename);
}
#Scheduled(fixedRate=2000)
private void reload() throws IOException {
System.out.println("reloading...");
long currentModTs = Files.getLastModifiedTime(configPath).toMillis();
if (currentModTs > lastModTime) {
lastModTime = currentModTs;
Properties properties = new Properties();
#Cleanup InputStream inputStream = Files.newInputStream(configPath);
properties.load(inputStream);
environment.getPropertySources()
.replace(
appConfigPropertySource.getName(),
new PropertiesPropertySource(
appConfigPropertySource.getName(),
properties
)
);
System.out.println("Reloaded.");
propertiesReloaded();
}
}
protected abstract void propertiesReloaded();
}
Then you make a bean class that allows retrieval of property values from applicatoin.properties that uses the abstract class
#Component
public class AppProperties extends ReloadableProperties {
public String dynamicProperty() {
return environment.getProperty("dynamic.prop");
}
public String anotherDynamicProperty() {
return environment.getProperty("another.dynamic.prop");
}
#Override
protected void propertiesReloaded() {
// do something after a change in property values was done
}
}
Make sure to add #EnableScheduling to your #SpringBootApplication
#SpringBootApplication
#EnableScheduling
public class MainApp {
public static void main(String[] args) {
SpringApplication.run(MainApp.class, args);
}
}
Now you can auto-wire the AppProperties Bean wherever you need it. Just make sure to always call the methods in it instead of saving it's value in a variable. And make sure to re-configure any resource or bean that was initialized with potentially different property values.
For now, I have only tested this with an external-and-default-found ./config/application.properties file.
For Java EE
I made a common Java SE abstract class to do the job.
You may copy & paste this:
// imports from java.* and javax.crypto.*
public abstract class ReloadableProperties {
private volatile Properties properties = null;
private volatile String propertiesPassword = null;
private volatile long lastModTimeOfFile = 0L;
private volatile long lastTimeChecked = 0L;
private volatile Path propertyFileAddress;
abstract protected void propertiesUpdated();
public class DynProp {
private final String propertyName;
public DynProp(String propertyName) {
this.propertyName = propertyName;
}
public String val() {
try {
return ReloadableProperties.this.getString(propertyName);
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException(e);
}
}
}
protected void init(Path path) {
this.propertyFileAddress = path;
initOrReloadIfNeeded();
}
private synchronized void initOrReloadIfNeeded() {
boolean firstTime = lastModTimeOfFile == 0L;
long currentTs = System.currentTimeMillis();
if ((lastTimeChecked + 3000) > currentTs)
return;
try {
File fa = propertyFileAddress.toFile();
long currModTime = fa.lastModified();
if (currModTime > lastModTimeOfFile) {
lastModTimeOfFile = currModTime;
InputStreamReader isr = new InputStreamReader(new FileInputStream(fa), StandardCharsets.UTF_8);
Properties prop = new Properties();
prop.load(isr);
properties = prop;
isr.close();
File passwordFiles = new File(fa.getAbsolutePath() + ".key");
if (passwordFiles.exists()) {
byte[] bytes = Files.readAllBytes(passwordFiles.toPath());
propertiesPassword = new String(bytes,StandardCharsets.US_ASCII);
propertiesPassword = propertiesPassword.trim();
propertiesPassword = propertiesPassword.replaceAll("(\\r|\\n)", "");
}
}
updateProperties();
if (!firstTime)
propertiesUpdated();
} catch (Exception e) {
e.printStackTrace();
}
}
private void updateProperties() {
List<DynProp> dynProps = Arrays.asList(this.getClass().getDeclaredFields())
.stream()
.filter(f -> f.getType().isAssignableFrom(DynProp.class))
.map(f-> fromField(f))
.collect(Collectors.toList());
for (DynProp dp :dynProps) {
if (!properties.containsKey(dp.propertyName)) {
System.out.println("propertyName: "+ dp.propertyName + " does not exist in property file");
}
}
for (Object key : properties.keySet()) {
if (!dynProps.stream().anyMatch(dp->dp.propertyName.equals(key.toString()))) {
System.out.println("property in file is not used in application: "+ key);
}
}
}
private DynProp fromField(Field f) {
try {
return (DynProp) f.get(this);
} catch (IllegalAccessException e) {
e.printStackTrace();
}
return null;
}
protected String getString(String param) throws Exception {
initOrReloadIfNeeded();
String value = properties.getProperty(param);
if (value.startsWith("ENC(")) {
String cipheredText = value
.replace("ENC(", "")
.replaceAll("\\)$", "");
value = decrypt(cipheredText, propertiesPassword);
}
return value;
}
public static String encrypt(String plainText, String key)
throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidAlgorithmParameterException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException, InvalidKeySpecException {
SecureRandom secureRandom = new SecureRandom();
byte[] keyBytes = key.getBytes(StandardCharsets.US_ASCII);
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
KeySpec spec = new PBEKeySpec(key.toCharArray(), new byte[]{0,1,2,3,4,5,6,7}, 65536, 128);
SecretKey tmp = factory.generateSecret(spec);
SecretKey secretKey = new SecretKeySpec(tmp.getEncoded(), "AES");
byte[] iv = new byte[12];
secureRandom.nextBytes(iv);
final Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
GCMParameterSpec parameterSpec = new GCMParameterSpec(128, iv); //128 bit auth tag length
cipher.init(Cipher.ENCRYPT_MODE, secretKey, parameterSpec);
byte[] cipherText = cipher.doFinal(plainText.getBytes(StandardCharsets.UTF_8));
ByteBuffer byteBuffer = ByteBuffer.allocate(4 + iv.length + cipherText.length);
byteBuffer.putInt(iv.length);
byteBuffer.put(iv);
byteBuffer.put(cipherText);
byte[] cipherMessage = byteBuffer.array();
String cyphertext = Base64.getEncoder().encodeToString(cipherMessage);
return cyphertext;
}
public static String decrypt(String cypherText, String key)
throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidAlgorithmParameterException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException, InvalidKeySpecException {
byte[] cipherMessage = Base64.getDecoder().decode(cypherText);
ByteBuffer byteBuffer = ByteBuffer.wrap(cipherMessage);
int ivLength = byteBuffer.getInt();
if(ivLength < 12 || ivLength >= 16) { // check input parameter
throw new IllegalArgumentException("invalid iv length");
}
byte[] iv = new byte[ivLength];
byteBuffer.get(iv);
byte[] cipherText = new byte[byteBuffer.remaining()];
byteBuffer.get(cipherText);
byte[] keyBytes = key.getBytes(StandardCharsets.US_ASCII);
final Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
KeySpec spec = new PBEKeySpec(key.toCharArray(), new byte[]{0,1,2,3,4,5,6,7}, 65536, 128);
SecretKey tmp = factory.generateSecret(spec);
SecretKey secretKey = new SecretKeySpec(tmp.getEncoded(), "AES");
cipher.init(Cipher.DECRYPT_MODE, secretKey, new GCMParameterSpec(128, iv));
byte[] plainText= cipher.doFinal(cipherText);
String plain = new String(plainText, StandardCharsets.UTF_8);
return plain;
}
}
Then you can use it this way:
public class AppProperties extends ReloadableProperties {
public static final AppProperties INSTANCE; static {
INSTANCE = new AppProperties();
INSTANCE.init(Paths.get("application.properties"));
}
#Override
protected void propertiesUpdated() {
// run code every time a property is updated
}
public final DynProp wsUrl = new DynProp("ws.url");
public final DynProp hiddenText = new DynProp("hidden.text");
}
In case you want to use encoded properties you may enclose it's value inside ENC() and a password for decryption will be searched for in the same path and name of the property file with an added .key extension. In this example it will look for the password in the application.properties.key file.
application.properties ->
ws.url=http://some webside
hidden.text=ENC(AAAADCzaasd9g61MI4l5sbCXrFNaQfQrgkxygNmFa3UuB9Y+YzRuBGYj+A==)
aplication.properties.key ->
password aca
For the encryption of property values for the Java EE solution I consulted Patrick Favre-Bulle excellent article on Symmetric Encryption with AES in Java and Android. Then checked the Cipher, block mode and padding in this SO question about AES/GCM/NoPadding. And finally I made the AES bits be derived from a password from #erickson excellent answer in SO about AES Password Based Encryption. Regarding encryption of value properties in Spring I think they are integrated with Java Simplified Encryption
Wether this qualify as a best practice or not may be out of scope. This answer shows how to have reloadable properties in Spring Boot and Java EE.
This functionality can be achieved by using a Spring Cloud Config Server and a refresh scope client.
Server
Server (Spring Boot app) serves the configuration stored, for example, in a Git repository:
#SpringBootApplication
#EnableConfigServer
public class ConfigServer {
public static void main(String[] args) {
SpringApplication.run(ConfigServer.class, args);
}
}
application.yml:
spring:
cloud:
config:
server:
git:
uri: git-repository-url-which-stores-configuration.git
configuration file configuration-client.properties (in a Git repository):
configuration.value=Old
Client
Client (Spring Boot app) reads configuration from the configuration server by using #RefreshScope annotation:
#Component
#RefreshScope
public class Foo {
#Value("${configuration.value}")
private String value;
....
}
bootstrap.yml:
spring:
application:
name: configuration-client
cloud:
config:
uri: configuration-server-url
When there is a configuration change in the Git repository:
configuration.value=New
reload the configuration variable by sending a POST request to the /refresh endpoint:
$ curl -X POST http://client-url/actuator/refresh
Now you have the new value New.
Additionally Foo class can serve the value to the rest of application via RESTful API if its changed to RestController and has a corresponding endpont.
I used #David Hofmann concept and made some changes because of not all was good.
First of all, in my case I no need auto-reload, I just call the REST controller for updating properties.
The second case #David Hofmann's approach not workable for me with outside files.
Now, this code can work with application.properties file from resources(inside the app) and from an outside place. The outside file I put near jar, and I use this --spring.config.location=app.properties argument when the application starts.
#Component
public class PropertyReloader {
private final Logger logger = LoggerFactory.getLogger(getClass());
#Autowired
private StandardEnvironment environment;
private long lastModTime = 0L;
private PropertySource<?> appConfigPropertySource = null;
private Path configPath;
private static final String PROPERTY_NAME = "app.properties";
#PostConstruct
private void createContext() {
MutablePropertySources propertySources = environment.getPropertySources();
// first of all we check if application started with external file
String property = "applicationConfig: [file:" + PROPERTY_NAME + "]";
PropertySource<?> appConfigPsOp = propertySources.get(property);
configPath = Paths.get(PROPERTY_NAME).toAbsolutePath();
if (appConfigPsOp == null) {
// if not we check properties file from resources folder
property = "class path resource [" + PROPERTY_NAME + "]";
configPath = Paths.get("src/main/resources/" + PROPERTY_NAME).toAbsolutePath();
}
appConfigPsOp = propertySources.get(property);
appConfigPropertySource = appConfigPsOp;
}
// this method I call into REST cintroller for reloading all properties after change
// app.properties file
public void reload() {
try {
long currentModTs = Files.getLastModifiedTime(configPath).toMillis();
if (currentModTs > lastModTime) {
lastModTime = currentModTs;
Properties properties = new Properties();
#Cleanup InputStream inputStream = Files.newInputStream(configPath);
properties.load(inputStream);
String property = appConfigPropertySource.getName();
PropertiesPropertySource updatedProperty = new PropertiesPropertySource(property, properties);
environment.getPropertySources().replace(property, updatedProperty);
logger.info("Configs {} were reloaded", property);
}
} catch (Exception e) {
logger.error("Can't reload config file " + e);
}
}
}
I hope that my approach will help somebody
As mentioned by #Boris, Spring Cloud Config is the way to go to avoid patchy solution. To keep the setup minimum, I will suggest the Embedding the Config Server Approach with native type (file type).
To support automatic config refresh without calling the actuator endpoint manually, I have created a directory listener to detect file changes and to dispatch refresh scope event.
Proof Of Concept repo (git)
For spring boot, there's a really good article on this topic here, but for multiple property files it doesn't work perfectly.
In my case I had 2 property files, one non sensitive and one containing the passwords. I proceeded with the following:
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.10</version>
</dependency>
Extend the spring's PropertySource so that you can add the reloadable version to the environment.
public class ReloadablePropertySource extends PropertySource {
private final PropertiesConfiguration propertiesConfiguration;
public ReloadablePropertySource(String name, String path, ConfigurationListener listener) {
super(StringUtils.hasText(name) ? name : path);
try {
this.propertiesConfiguration = getConfiguration(path, listener);
} catch (Exception e) {
throw new MissingRequiredPropertiesException();
}
}
#Override
public Object getProperty(String s) {
return propertiesConfiguration.getProperty(s);
}
private PropertiesConfiguration getConfiguration(String path, ConfigurationListener listener) throws ConfigurationException {
PropertiesConfiguration configuration = new PropertiesConfiguration(path);
FileChangedReloadingStrategy reloadingStrategy = new FileChangedReloadingStrategy();
reloadingStrategy.setRefreshDelay(5000);
configuration.setReloadingStrategy(reloadingStrategy);
configuration.addConfigurationListener(listener);
return configuration;
}
}
Now add all of your properties files (now reloadable) inside the spring's env
#Configuration
public class ReloadablePropertySourceConfig {
private final ConfigurableEnvironment env;
#Value("${spring.config.location}")
private String appConfigPath;
#Value("${spring.config.additional-location}")
private String vaultConfigPath;
public ReloadablePropertySourceConfig(ConfigurableEnvironment env) {
this.env = env;
}
#Bean
#ConditionalOnProperty(name = "spring.config.location")
public ReloadablePropertySource getAppConfigReloadablePropertySource(){
ReloadablePropertySource rps = new ReloadablePropertySource("dynamicNonSensitive", appConfigPath, new PropertiesChangeListener());
MutablePropertySources sources = env.getPropertySources();
sources.addFirst(rps);
return rps;
}
#Bean
#ConditionalOnProperty(name = "spring.config.additional-location")
public ReloadablePropertySource getVaultReloadablePropertySource(){
ReloadablePropertySource rps = new ReloadablePropertySource("dynamicVault", vaultConfigPath, new PropertiesChangeListener());
MutablePropertySources sources = env.getPropertySources();
sources.addFirst(rps);
return rps;
}
private static class PropertiesChangeListener implements ConfigurationListener{
#Override
public void configurationChanged(ConfigurationEvent event) {
if (!event.isBeforeUpdate()){
System.out.println("config refreshed!");
}
}
}
}
From the article
We've added the new property source as the first item because we want it to override any existing property with the same key
In our case, we have 2 "reloadable" property sources and both will be looked up first.
Finally create one more class from which we can access the env's properties
#Component
public class ConfigProperties {
private final Environment environment;
public ConfigProperties(Environment environment) {
this.environment = environment;
}
public String getProperty(String name){
return environment.getProperty(name);
}
}
Now you can autowire ConfigProperties and always get the latest property in the files without requiring to restart the application.
#RestController
#Slf4j
public class TestController {
#Autowired
private ConfigProperties env;
#GetMapping("/refresh")
public String test2() {
log.info("hit");
String updatedProperty = env.getProperty("test.property");
String password = env.getProperty("db.password");
return updatedProperty + "\n" + password;
}
}
where test.property is coming from 1st file and db.password is coming from another.
If you want to change the properties at realtime and don't want to restart the server then follow the below steps:
1). Application.properties
app.name= xyz
management.endpoints.web.exposure.include=*
2). Add below dependencies in pom.xml
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-rest</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-context</artifactId>
<version>2.1.1.RELEASE</version>
</dependency>
3).Place application.properties in /target/config folder. Create the jar in /target folder
4).add a classas below ApplcationProperties.java
#Component
#RefreshScope
#ConfigurationProperties(prefix = "app")
public class ApplicationProperties {
private String name;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
5). Write Controller.java and inject ApplcationProperties
#RestController
public class TestController {
#Autowired
private ApplicationProperties applcationProperties;
#GetMapping("/test")
public String getString() {
return applcationProperties.getName();
}
}
6).Run the spring boot application
Call localhost:XXXX/test from your browser
Output : xyz
7). Change the value in application.properties from xyz to abc
8). Using postman send a POST request to localhost:XXXX/actuator/refresh
response: ["app.name"]
9). Call localhost:XXXX/find from your browser
Output : abc
I have a metrics collector that store data on InfluxDB, I want to test the methods to store that metrics. I'm trying it but I'm not able to mock the InfluxDB client. I don't want to point to a real InfluxDB on the test environment.
Everything I've achieved so far are some "null pointer exceptions" and conection refused.
This is my test (using TestNG). What am I doing wrong?
#Test
public void validateMetrics() {
String influxHost = "http://localhost";
String credentials = "admin:admin";
String influxDatabaseName = "testDataBase";
influxDB = InfluxDBFactory.connect(influxHost, credentials.split(":")[0], credentials.split(":")[1]);
MetricsCollector metricsCollector = null;
try {
String hostName = "test-server-01";
int statusValue = 1;
metricsCollector = new MetricsCollector(influxDB);
BatchPoints metrics = metricsCollector.initBatchPoint(influxDatabaseName);
Point point = metricsCollector.setMetric(hostName, "status", statusValue);
metrics = metricsCollector.addToMetrics(metrics, point);
Assert.assertTrue(metrics.getPoints().get(0).lineProtocol().contains(hostName));
Assert.assertTrue(metrics.getPoints().get(0).lineProtocol().contains("status=" + statusValue));
} finally {
if (metricsCollector != null) {
metricsCollector.closeConnection();
}
}
}
I suspect the reason you cannot mock the InfluxDB client is that it is created by a static method: InfluxDBFactory.connect(). To mock this you will need PowerMock.
Something like this:
#PrepareForTest({InfluxDBFactory.class})
public class ATestClass {
#Test
public void validateMetrics() {
// this allows you to mock static methods on InfluxDBFactory
PowerMockito.mockStatic(InfluxDBFactory.class);
String influxHost = "http://localhost";
String credentials = "admin:admin";
String influxDatabaseName = "testDataBase";
InfluxDB influxDB = Mockito.mock(InfluxDB.class);
// when the connect() method is invoked in your test run it will return a mocked InfluxDB rather than a _real_ InfluxDB
PowerMockito.when(InfluxDBFactory.connect(influxHost, credentials.split(":")[0], credentials.split(":")[1])).thenReturn(influxDB);
// you won't do this in your test, I've only included it here to show you that InfluxDBFactory.connect() returns the mocked instance of InfluxDB
InfluxDB actual = InfluxDBFactory.connect(influxHost, credentials.split(":")[0], credentials.split(":")[1]);
Assert.assertSame(influxDB, actual);
// the rest of your test
// ...
}
}
Note: there are specific compatability requirements for TestNG, Mockito and PowerMock described here and here.
I totally misunderstanding how mockito works. Here is my fixed code:
#Mock private InfluxDB influxDB;
#Test
public void validateMetrics() {
MetricsCollector metricsCollector = null;
String influxHost = "http://localhost";
String credentials = "admin:admin";
String influxDatabaseName = "testDataBase";
influxDB = InfluxDBFactory.connect(influxHost, credentials.split(":")[0], credentials.split(":")[1]);
try {
String hostName = "test-server-01";
int statusValue = 1;
metricsCollector = new MetricsCollector(influxDB);
BatchPoints metrics = metricsCollector.initBatchPoint(influxDatabaseName);
Point point = metricsCollector.setMetric(hostName, "status", statusValue);
metrics = metricsCollector.addToMetrics(metrics, point);
Assert.assertTrue(metrics.getPoints().get(0).lineProtocol().contains(hostName));
Assert.assertTrue(metrics.getPoints().get(0).lineProtocol().contains("status=" + statusValue));
} finally {
if (metricsCollector != null) {
metricsCollector.closeConnection();
}
}
}
Yep, added that simple "mock" annotation and all works fine.
Hi I am working on a project and using PrintWriter class for opening and writing in the file. But when I am writing the test case for same it gives following error at Line 153
Wanted but not invoked:
mockPrintWriter.println("ID url1
");
-> at x.y.z.verify(ProcessImageDataTest.java:153)
Actually, there were zero interactions with this mock.
Code: (Uses Lombok Library)
ProcessImageData.java
#Setter
#RequiredArgsConstructor
public class ProcessImageData implements T {
private final File newImageDataTextFile;
#Override
public void execute() {
LineIterator inputFileIterator = null;
try {
File filteredImageDataTextFile = new File(filteredImageDataTextFilepath);
PrintWriter writer = new PrintWriter(newImageDataTextFile);
inputFileIterator = FileUtils.lineIterator(filteredImageDataTextFile, StandardCharsets.UTF_8.displayName());
while (inputFileIterator.hasNext()) {
if(someCondition)
**Line51** writer.println(imageDataFileLine);
//FileUtils.writeStringToFile(newImageDataTextFile, imageDataFileLine + NEWLINE, true);
}
}
} catch (Exception e) {
} finally {
LineIterator.closeQuietly(inputFileIterator);
**LINE63** writer.close();
}
}
ProcessImageDataTest.java
#RunWith(PowerMockRunner.class)
#PrepareForTest({ ProcessImageData.class, FileUtils.class, Printwriter.class })
public class ProcessImageDataTest {
private ProcessImageData processImageData;
private static final String FILTERED_IMAGE_DATA_TEXT_FILE_PATH = "filteredFilepath";
private File FILTEREDFILE = new File(FILTERED_PATH);
private static final File IMAGE__FILE = new File("imageFilePath");
private LineIterator lineIterator;
#Mock
private PrintWriter mockPrintWriter;
#Before
public void init() throws Exception {
MockitoAnnotations.initMocks(this);
processImageData = new ProcessImageData(Palettes_file, FILTERED_PATH, IMAGE_FILE);
PowerMockito.mockStatic(FileUtils.class);
PowerMockito.whenNew(PrintWriter.class).withArguments(IMAGE_FILE).thenReturn(mockPrintWriter);
PowerMockito.when(FileUtils.lineIterator(FILTERED_FILE, StandardCharsets.UTF_8.displayName())).thenReturn(lineIterator);
PowerMockito.when(lineIterator.hasNext()).thenReturn(true, true, false);
}
#Test
public void testTaskWhenIDInDBAndStale() throws IOException {
PowerMockito.when(lineIterator.nextLine()).thenReturn(ID2 + SPACE + URL1, ID1 + SPACE + URL2);
processImageData.execute();
List<String> exepctedFileContentOutput = Arrays.asList(ID2 + SPACE + URL1 + NEWLINE);
verify(exepctedFileContentOutput, 1, 1);
}
#Test
public void testTaskWhenIDNotInDB() throws IOException {
PowerMockito.when(lineIterator.nextLine()).thenReturn(ID2 + SPACE + URL1, ID3 + SPACE + URL2);
processImageData.execute();
List<String> exepctedFileContentOutput = Arrays.asList(ID3 + SPACE + URL2 + NEWLINE);
verify(exepctedFileContentOutput, 1, 1);
}
private void verify(List<String> exepctedFileContentOutput, int fileWriteTimes, int fileReadTimes) throws IOException {
for (String line : exepctedFileContentOutput){
**Line153** Mockito.verify(mockPrintWriter, Mockito.times(fileWriteTimes)).print(line);
}
PowerMockito.verifyStatic(Mockito.times(fileReadTimes));
FileUtils.lineIterator(FILTERED_IMAGE_DATA_TEXT_FILE, StandardCharsets.UTF_8.displayName());
}
}
I am mocking a new operator for PrintWriter also, injecting using beans. What is the mistake I am doing?? I am stuck on it from long time and not getting the error?
Any help is appreciated.
Updated :
I did changes suggested below and updated the code, but now I get the error:
Wanted but not invoked: mockPrintWriter.print("ASIN2 url1 "); ->
at softlines.ctl.ruleExecutor.tasks.ProcessImageDataTest.verify(ProcessImageDataTest.java:153)
However, there were other interactions with this mock: -> at softlines.ctl.ruleExecutor.tasks.ProcessImageData.execute(ProcessImageData.java:51) ->
at softlines.ctl.ruleExecutor.tasks.ProcessImageData.execute(ProcessImageData.java:51) ->
at softlines.ctl.ruleExecutor.tasks.ProcessImageData.execute(ProcessImageData.java:58) –
I see 3 issues in your test:
You don't try to mock the correct constructor, indeed in the method execute, you create your PrintWriter with only one argument of type File while you try to mock the constructor with 2 arguments one of type File and the other one of type String.
So the code should rather be:
PowerMockito.whenNew(PrintWriter.class)
.withArguments(IMAGE_FILE)
.thenReturn(mockPrintWriter);
To be able to mock a constructor you need to prepare the class creating the instance which is ProcessImageData in this case, so you need to add ProcessImageData.class in the annotation #PrepareForTest. (I'm not sure ProcessImageDataTest.class is needed there)
The field lineIterator should be annotated with #Mock.
Instead of verifying print with a new line, you should verify directly println without new line it is much less error prone.
I simplified your code to show the idea.
Assuming that ProcessImageData is:
public class ProcessImageData {
private final File newImageDataTextFile;
public ProcessImageData(final File newImageDataTextFile) {
this.newImageDataTextFile = newImageDataTextFile;
}
public void execute() throws Exception{
try (PrintWriter writer = new PrintWriter(newImageDataTextFile)) {
LineIterator inputFileIterator = FileUtils.lineIterator(
newImageDataTextFile, StandardCharsets.UTF_8.displayName()
);
while (inputFileIterator.hasNext()) {
writer.println(inputFileIterator.nextLine());
}
}
}
}
My unit test would then be:
#RunWith(PowerMockRunner.class)
#PrepareForTest({ProcessImageData.class, FileUtils.class})
public class ProcessImageDataTest {
private File file = new File("imageFilePath");
private ProcessImageData processImageData;
#Mock
private PrintWriter mockPrintWriter;
#Mock
private LineIterator lineIterator;
#Before
public void init() throws Exception {
MockitoAnnotations.initMocks(this);
processImageData = new ProcessImageData(file);
PowerMockito.whenNew(PrintWriter.class)
.withArguments(file)
.thenReturn(mockPrintWriter);
PowerMockito.mockStatic(FileUtils.class);
PowerMockito.when(
FileUtils.lineIterator(file, StandardCharsets.UTF_8.displayName())
).thenReturn(lineIterator);
PowerMockito.when(lineIterator.hasNext()).thenReturn(true, true, false);
}
#Test
public void testExecute() throws Exception {
PowerMockito.when(lineIterator.nextLine()).thenReturn("Foo", "Bar");
processImageData.execute();
Mockito.verify(mockPrintWriter, Mockito.times(1)).println("Foo");
Mockito.verify(mockPrintWriter, Mockito.times(1)).println("Bar");
}
}
For more details please refer to How to mock construction of new objects.
how can I add verification in unit test for writer.close?
One way could be to simply check that close() at be called once by adding the next line to your unit test:
Mockito.verify(mockPrintWriter, Mockito.times(1)).close();
Your construction of the PrintWriter doesn't match the mock. You told PowerMockito to return your mock like this:
PowerMockito.whenNew(PrintWriter.class).withArguments(IMAGE_FILE , StandardCharsets.UTF_8.name()).thenReturn(mockPrintWriter);
So you would have to say:
new PrintWriter(IMAGE_FILE, "UTF-8"); // 2 arguments
But instead in your execute method in the code that is being tested, you do:
PrintWriter writer = new PrintWriter(newImageDataTextFile); // only 1 argument
So you either need to change the PowerMockito withArguments clause, or you need to add "UTF-8" to the constructor invocation in the execute method.
I have tried to write a little code using libGDX to work with network. Here's a code:
public class Test {
public static void main(String[] args) {
String accessToken = "********"; //a set of symbols, not important because it is specific of request target
String userID = "*********"; //also not important
String message = "Hello World";
String uri = "method/wall.post?owner_id=" + userID + "&message=" + message + "&access_token=" + accessToken;
HttpRequestBuilder requestBuilder = new HttpRequestBuilder();
HttpRequest httpRequest = requestBuilder.newRequest().method(HttpMethods.GET).url("https://api.vk.com/").content(uri).build();
Gdx.net.sendHttpRequest(httpRequest, //Here Eclipse shows NullPointerException
null); //But not here
}
If I write this URL in browser, it works right. It means, that the problem on my side. How to fix it?
Summary of values of the object which causes NullPointerException:
You are writing this code in the static main entry point of your program. Your Gdx is not yet loaded,so Gdx is still null at this point.
Create a none static class and put your code in it's constructor and initialize that class within this static entry point.
public class WebTest()
{
public WebTest()
{
String accessToken = "********"; //a set of symbols, not important because it is specific of request target
String userID = "*********"; //also not important
String message = "Hello World";
String uri = "method/wall.post?owner_id=" + userID + "&message=" + message + "&access_token=" + accessToken;
HttpRequestBuilder requestBuilder = new HttpRequestBuilder();
HttpRequest httpRequest = requestBuilder.newRequest().method(HttpMethods.GET).url("https://api.vk.com/").content(uri).build();
Gdx.net.sendHttpRequest(httpRequest, //Here Eclipse shows NullPointerException
null); //But not here
}
}
public class Test {
public static void main(String[] args) {
new WebTest();
}
}
I have a simple entity class and it is supposed to include unique names on it.
#Entity
class Package {
#PrimaryKey(sequence = "ID")
public Long id;
#SecondaryKey(relate = Relationship.ONE_TO_ONE)
public String name;
private Package() {}
public Package(String name) { this.name = name; }
#Override
public String toString() { return id + " : " + name; }
}
I want to use deferred writing option because of extensive modification. Here is the test i tried and its output.
final String dbfilename = "test01";
new File(dbfilename).mkdirs();
EnvironmentConfig config = new EnvironmentConfig().setAllowCreate(true);
Environment environment = new Environment(new File(dbfilename), config);
StoreConfig storeConfig = new StoreConfig().setAllowCreate(true).setDeferredWrite(true);
EntityStore store = new EntityStore(environment, "", storeConfig);
PrimaryIndex<Long, Package> primaryIndex = store.getPrimaryIndex(Long.class, Package.class);
try {
primaryIndex.put(new Package("package01")); // will be put.
primaryIndex.put(new Package("package01")); // throws exception.
} catch (UniqueConstraintException ex) {
System.out.println(ex.getMessage());
}
store.sync(); // flush them all
// expecting to find one element
SortedMap<Long,Package> sortedMap = primaryIndex.sortedMap();
for (Package entity : sortedMap.values()) {
System.out.println(entity);
}
Output
(JE 5.0.73) Unique secondary key is already present
1 : package01
2 : package01
So my question is that even if it throws exception while putting second package, why does it lists two packages. Any way to avoid this without using transactions?
Thanks.