Spring Boot #Value is not being populated..why? - java

I am trying to get a value from my application.properties in Spring Boot using #Value but no matter what I do it is always null.
I am doing this in my HTTPClient test class. I have tried using environment variables, propertySource, PostConstruct, using getters and setters, and anything else I could find online but it does not seem to be populating at all... My test class is in src/test/java and the application.properties is in src/test/resources. I do also have a application.properties in my src/main/java but that is in a completely different folder so it should not be affecting it.
Here is my code:
import static org.junit.Assert.*;
import java.io.IOException;
import java.util.Map;
import javax.annotation.PostConstruct;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.fluent.Request;
import org.apache.http.entity.ContentType;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.ConfigurableApplicationContext;
import com.google.gson.Gson;
import com.nulogix.billing.configuration.EndPointTestConfiguration;
import com.nulogix.billing.mockserver.MockServerApp;
#SpringBootTest(classes = EndPointTestConfiguration.class)
public class HttpClientTest {
#Value("${billing.engine.address}")
private String billingEngineAddress;
#Value("${billing.engine.port}")
private String billingEnginePort;
#PostConstruct
private void customInit() {
System.out.print(billingEngineAddress);
System.out.print(billingEnginePort);
}
public static final String request_bad = "ncs|56-2629193|1972-03-28|20190218|77067|6208|3209440|self|";
public static final String request_good = "ncs|56-2629193|1972-03-28|20190218|77067|6208|3209440|self|-123|-123|-123|0.0|0.0|0.0|0.0|0.0|0.0|0.0";
#Test
public void test_bad() throws ClientProtocolException, IOException {
// missing parameter
String result = Request.Post("http://" + billingEngineAddress + ":" + billingEnginePort)
.connectTimeout(2000)
.socketTimeout(2000)
.bodyString(request_bad, ContentType.TEXT_PLAIN)
.execute().returnContent().asString();
Map<?, ?> resultJsonObj = new Gson().fromJson(result, Map.class);
// ensure the key exists
assertEquals(resultJsonObj.containsKey("status"), true);
assertEquals(resultJsonObj.containsKey("errorMessage"), true);
// validate values
Boolean status = (Boolean) resultJsonObj.get("status");
assertEquals(status, false);
String errorMessage = (String) resultJsonObj.get("errorMessage");
assertEquals(errorMessage.contains("Payload has incorrect amount of parts"), true);
}
#Test
public void test_good() throws ClientProtocolException, IOException {
String result = Request.Post("http://" + billingEngineAddress + ":" + billingEnginePort)
.connectTimeout(2000)
.socketTimeout(2000)
.bodyString(request_good, ContentType.TEXT_PLAIN)
.execute().returnContent().asString();
Map<?, ?> resultJsonObj = new Gson().fromJson(result, Map.class);
// ensure the key exists
assertEquals(resultJsonObj.containsKey("status"), true);
assertEquals(resultJsonObj.containsKey("errorMessage"), false);
assertEquals(resultJsonObj.containsKey("HasCopay"), true);
assertEquals(resultJsonObj.containsKey("CopayAmount"), true);
assertEquals(resultJsonObj.containsKey("HasCoinsurance"), true);
assertEquals(resultJsonObj.containsKey("CoinsuranceAmount"), true);
assertEquals(resultJsonObj.containsKey("version"), true);
// validate values
Boolean status = (Boolean) resultJsonObj.get("status");
assertEquals(status, true);
String version = (String) resultJsonObj.get("version");
assertEquals(version, "0.97");
}
}
I am getting the values from my application.properties to get the IP address and port and test my Request.post.
Here is my application.properties
server.port=9119
server.ssl.enabled=false
logging.config=classpath:logback-spring.xml
logging.file=messages
logging.file.max-size=50MB
logging.level.com.nulogix=DEBUG
billing.engine.address=127.0.0.1
billing.engine.port=9119
billing.engine.api.version=0.97
billing.engine.core.name=Patient_Responsibility

So what solved my issue is copying over the application.properties from my main to my test/resources. I then used #PropertySource to change the value to test.properties, separating it from application.properties in main. I created a bean of values in my endpoint config and then auto wired it into my httpclient test class.
I also added in web environment in my SpringBootTest annotation to use the defined port in test.properties and to run the test with my endpoint config class and main app class. This caused the #value strings to be populated and not be null.
#Configuration
public class EndPointTestConfiguration {
#Value("${billing.engine.address}")
private String mockServerIP;
#Value("${billing.engine.port}")
private String mockServerPort;
#Bean
public String mockAddress() {
String mockServerAddress = "http://" + mockServerIP + ":" + mockServerPort;
return mockServerAddress;
}
#Bean
public GetVersionEndPoint getVersionEndPoint() {
return new GetVersionEndPoint();
}
#Bean
public AnalyzeEndPoint analyzeEndPoint() throws JAXBException {
return new AnalyzeEndPoint();
}
#Bean
public PredictionEngineService predictionEngineService() {
return new PredictionEngineService();
}
#Bean
public String studyDetailDemo() throws IOException {
File file = ResourceUtils.getFile("src/test/resources/study-details-demo.txt");
String content = new String(Files.readAllBytes(file.toPath()));
return content;
}
#Bean
public String studyDetailSampleNormal() throws IOException {
File file = ResourceUtils.getFile("src/test/resources/study-details-normal.txt");
String content = new String(Files.readAllBytes(file.toPath()));
return content;
}
#Bean
public String studyDetailSampleCms() throws IOException {
File file = ResourceUtils.getFile("src/test/resources/study-details-cms.txt");
String content = new String(Files.readAllBytes(file.toPath()));
return content;
}
}
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT,classes = {EndPointTestConfiguration.class,MockServerApp.class
})
#PropertySource(value={"classpath:test.properties"}, ignoreResourceNotFound = true)
public class HttpClientTest {
#Autowired
private String mockAddress;
public static final String request_bad = "ncs|56-2629193|1972-03-28|20190218|77067|6208|3209440|self|";
public static final String request_good = "ncs|56-2629193|1972-03-28|20190218|77067|6208|3209440|self|-123|-123|-123|0.0|0.0|0.0|0.0|0.0|0.0|0.0";
#Test
public void test_bad() throws ClientProtocolException, IOException {
// missing parameter
String result = Request.Post(mockAddress)
.connectTimeout(2000)
.socketTimeout(2000)
.bodyString(request_bad, ContentType.TEXT_PLAIN)
.execute().returnContent().asString();
Map<?, ?> resultJsonObj = new Gson().fromJson(result, Map.class);
// ensure the key exists
assertEquals(resultJsonObj.containsKey("status"), true);
assertEquals(resultJsonObj.containsKey("errorMessage"), true);
// validate values
Boolean status = (Boolean) resultJsonObj.get("status");
assertEquals(status, false);
String errorMessage = (String) resultJsonObj.get("errorMessage");
assertEquals(errorMessage.contains("Payload has incorrect amount of parts"), true);
}
#Test
public void test_good() throws ClientProtocolException, IOException {
String result = Request.Post(mockAddress)
.connectTimeout(2000)
.socketTimeout(2000)
.bodyString(request_good, ContentType.TEXT_PLAIN)
.execute().returnContent().asString();
Map<?, ?> resultJsonObj = new Gson().fromJson(result, Map.class);
// ensure the key exists
assertEquals(resultJsonObj.containsKey("status"), true);
assertEquals(resultJsonObj.containsKey("errorMessage"), false);
assertEquals(resultJsonObj.containsKey("HasCopay"), true);
assertEquals(resultJsonObj.containsKey("CopayAmount"), true);
assertEquals(resultJsonObj.containsKey("HasCoinsurance"), true);
assertEquals(resultJsonObj.containsKey("CoinsuranceAmount"), true);
assertEquals(resultJsonObj.containsKey("version"), true);
// validate values
Boolean status = (Boolean) resultJsonObj.get("status");
assertEquals(status, true);
String version = (String) resultJsonObj.get("version");
assertEquals(version, "0.97");
}
}

You need to move your application.properties to src/test/resource folder. For test, spring load application properties from this path.

You can do as follows :
#RunWith(SpringRunner.class)
#SpringBootTest(classes={Application.class})
public class HttpClientTest {
.............
.................
}

Related

spring-integration-mail Receiver mail for read email program in spring boot

I copied code from git for read emails I tired many but facing same as below errors in console while running the application. Please help me resolve this:
enter image description here
This is my Application.yaml : Here my specified username and password are correct only
logging:
level:
root: INFO
org.ygaros: DEBUG
spring:
mail:
username: arunofficial302#gmail.com
password: ***************
properties:
protocol: imaps
host: imap.gmail.com
port: 993
This is my config file:
package com.htc.mailOperation.config;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.annotation.InboundChannelAdapter;
import org.springframework.integration.annotation.Poller;
import org.springframework.integration.annotation.ServiceActivator;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.config.EnableIntegration;
import org.springframework.integration.mail.ImapMailReceiver;
import org.springframework.integration.mail.MailReceiver;
import org.springframework.integration.mail.MailReceivingMessageSource;
import org.springframework.messaging.Message;
import com.htc.mailOperation.service.MailReceiverService;
import javax.mail.internet.MimeMessage;
#Configuration
#EnableIntegration
public class MailConfig {
private static final Logger log = LoggerFactory.getLogger(MailConfig.class);
#Autowired
private MailReceiverService receiverService;
#Value("${spring.mail.username}")
private String USERNAME;
#Value("${spring.mail.password}")
private String PASSWORD;
#Value("${spring.mail.properties.port}")
private int IMAP_PORT;
#Value("${spring.mail.properties.protocol}")
private String IMAP_PROTOCOL;
#Value("${spring.mail.properties.host}")
private String IMAP_HOST;
/*
Define a channel where we send and get the messages (mails in this example).
MessageSource is sending messages to this channel.
DirectChannel is a SubscribableChannel.
*/
#Bean("incomingEmailsChannel")
public DirectChannel defaultChannel() {
DirectChannel directChannel = new DirectChannel();
directChannel.setDatatypes(MimeMessage.class);
return directChannel;
}
#Bean
public MailReceiver imapMailReceiver() {
String url = String.format(
"%s://%s:%s#%s:%s/INBOX", IMAP_PROTOCOL, USERNAME, PASSWORD, IMAP_HOST, IMAP_PORT
);
log.debug("IMAP url: {}", url.replace(PASSWORD, "x".repeat(8)));
ImapMailReceiver imapMailReceiver = new ImapMailReceiver(url);
imapMailReceiver.setShouldMarkMessagesAsRead(true);
imapMailReceiver.setShouldDeleteMessages(false);
/*
Attach content to message because by default the MimeMessage
doesn't contain content body.
*/
imapMailReceiver.setSimpleContent(true);
//imapMailReceiver.setMaxFetchSize(1);
return imapMailReceiver;
}
/*
Provide MessageSource of Mails as spring integration Messages from ImapMailReceiver to be sent
through incomingEmailsChannel.
Poller with defined rate at which the messages are pushed to the channel
(if there are any) every 5 sec.
*/
#Bean
#InboundChannelAdapter(
channel = "incomingEmailsChannel",
poller = #Poller(fixedDelay = "5000")
)
public MailReceivingMessageSource mailMessageSource(MailReceiver mailReceiver) {
return new MailReceivingMessageSource(mailReceiver);
}
/*
Connect (subscribe) to incomingEmailsChannel channel and provide method to
handle each individual messages.
*/
#ServiceActivator(inputChannel = "incomingEmailsChannel")
public void receive(Message<MimeMessage> message) {
receiverService.handleReceivedMail(message.getPayload());
}
}
This is my service file:
package com.htc.mailOperation.service;
import org.apache.commons.mail.util.MimeMessageParser;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.core.io.FileSystemResource;
import org.springframework.stereotype.Service;
import javax.activation.DataSource;
import javax.mail.internet.MimeMessage;
import java.io.*;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
#Service
public class MailReceiverService {
private static final Logger log = LoggerFactory.getLogger(MailReceiverService.class);
private static final String ATTACHMENTS_FOLDER_STORE = "data";
public void handleReceivedMail(MimeMessage receivedMessage){
try {
MimeMessageParser mimeMessageParser = new MimeMessageParser(receivedMessage);
mimeMessageParser.parse();
logMail(mimeMessageParser);
saveAttachments(mimeMessageParser);
/*
To delete downloaded email
Setting those flag will delete messages only when we use folder.close(true);
message.setFlag(Flags.Flag.DELETED, true);
*/
} catch (Exception e) {
log.error(e.getMessage(), e);
}
}
private void logMail(MimeMessageParser mimeMessageParser) throws Exception {
log.debug("From: {} To: {} Subject: {}",
mimeMessageParser.getFrom(), mimeMessageParser.getTo(), mimeMessageParser.getSubject());
log.debug("Mail content: {}", mimeMessageParser.getPlainContent());
}
private void saveAttachments(MimeMessageParser parser){
List<DataSource> attachments = parser.getAttachmentList();
log.debug("Email has {} attachment files", attachments.size());
attachments.forEach(dataSource -> {
String fileName = dataSource.getName();
if(fileName != null && fileName.length() > 0){
String rootDirPath = new FileSystemResource("").getFile().getAbsolutePath();
createDirectoryIfNotExists(rootDirPath);
String attachmentFilePath = rootDirPath +
File.separator +
ATTACHMENTS_FOLDER_STORE +
File.separator +
fileName;
File attachmentFile = new File(attachmentFilePath);
log.info("Attachment file saved to: {}", attachmentFilePath);
try(
InputStream in = dataSource.getInputStream();
OutputStream out = new FileOutputStream(attachmentFile)
){
copyStream(in, out);
}catch(IOException e){
log.error("Failed to save file.", e);
}
}
});
}
private void createDirectoryIfNotExists(String directoryPath) {
Path of = Path.of(directoryPath);
if (!Files.exists(of)) {
try {
Files.createDirectories(of);
} catch (IOException e) {
log.error("An error: {} on creating: {}", e, directoryPath);
}
}
}
private void copyStream(InputStream in, OutputStream out) throws IOException {
byte[] buffer = new byte[8192]; //just my standard base buffer
int readByteCount;
while(-1 != (readByteCount = in.read(buffer))){
out.write(buffer, 0, readByteCount);
}
}
}
can someone please help me with this?
You need to convert your username and password to x-www-form-urlencoded format before build url.
For example:
...
public MailReceiver imapMailReceiver() {
USERNAME = URLEncoder.encode(USERNAME, StandardCharsets.UTF_8.name());
PASSWORD = URLEncoder.encode(PASSWORD, StandardCharsets.UTF_8.name());
String url = String.format(
"%s://%s:%s#%s:%s/INBOX", IMAP_PROTOCOL, USERNAME, PASSWORD, IMAP_HOST, IMAP_PORT
);
...

Updating specific part of value in hash map dynamically

I have a spring boot app with a HTTP post request handler. It accepts a payload that I parse and outputs a JSON. I have handled it that it needs to accept a payload of certain parameters(18).
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;
import com.google.gson.Gson;
#Validated
#RestController
public class MockController {
#Autowired
MockConfig mockconfig;
private static final Logger LOGGER = LoggerFactory.getLogger(MockController.class);
#RequestMapping(value = "/", method = RequestMethod.GET)
public String index() {
return "hello!";
}
String[] parse;
#PostMapping(value = "/")
public String payloader(#RequestBody String params ) throws IOException{
LOGGER.debug("code is hitting");
parse = params.split("\\|");
String key;
String value;
String dyn;
Map<String, String> predictionFeatureMap = mockconfig.getPredictionFeatureMap();
if(parse.length!=18) {
key = "Not_enough_parameters";
value = predictionFeatureMap.get(key);
Map<?, ?> resultJsonObj = new Gson().fromJson(value, Map.class);
}
else {
key = params;
value = predictionFeatureMap.get(key);
}
return value;
}
}
My config class is where I get the input and output from a file and put them into a hashmap.
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.Scanner;
import org.springframework.context.annotation.Configuration;
#Configuration
public class MockConfig {
private Map<String, String> predictionFeatureMap = new HashMap<String, String>();
public Map<String,String> getPredictionFeatureMap() throws IOException {
return predictionFeatureMap;
}
public MockConfig() throws IOException {
init();
}
private Map<String, String> init() throws IOException {
Scanner sc = new Scanner (new File("src/test/resources/Payload1.txt"));
int counter = 1;
String key = "";
while (sc.hasNextLine()) {
if(counter % 2 != 0) {
key = sc.nextLine();
}else {
predictionFeatureMap.put(key, sc.nextLine());
}
counter++;
}
sc.close();
return predictionFeatureMap;
}
}
This is the key and value in the file that I'm trying to work with specifically.
Not_enough_parameters
{"status": false, "errorMessage": "Payload has incorrect amount of parts: expecting: 18, actual:8", "version": "0.97", "coreName": "Patient_Responsibility"}
(The JSON string is the response to too much or too little parameters... the paramter length should be 18.)
Example input:
ncs|56-2629193|1972-03-28|20190218|77067|6208|3209440|self|-123|-123|-123|0.0|0.0|0.0|0.0|0.0|0.0|0.0
This input would pass because it has 18 parameters...
What I want to do is if a user curls for example 5 parameters
ncs|56-2629193|1972-03-28|20190218|77067
I want the value(JSON error message) to have the 'actual' field updated like:
{"status": false, "errorMessage": "Payload has incorrect amount of parts: expecting: 18, actual:5", "version": "0.97", "coreName": "Patient_Responsibility"}
without hardcoding it into the txt file or hashmap...
I have tried getting the index of the string and replacing the '8' character with parse.length() and casting it as a char but it just gives me this output:
{"status": false, "errorMessage": "Payload has incorrect amount of parts: expecting:1 actual:", "version": "0.97", "coreName": "Nulogix_Patient_Responsibility"}
How do I parse or index the JSON to update this value? Or is there a hashmap method to deal with this?
When working with a framework, you usually handle errors using the frameworks way of handling errors.
To handle errors in spring boot you typically use a controller advice that will assist in handling errors. This is created by annotating a class with #ControllerAdvice.
There you can catch thrown exceptions and build responses that will be returned to the calling client.
#PostMapping(value = "/")
public String payloader(#RequestBody String params ) throws IOException{
LOGGER.debug("code is hitting");
parse = params.split("\\|");
String key;
String value;
String dyn;
Map<String, String> predictionFeatureMap = mockconfig.getPredictionFeatureMap();
if(parse.length!=18) {
// Here you throw a custom runtime exception and pass what
// parameters you want that will help you build your error
// message you are passing to your client.
final String errorMessage = String.format(
"Payload has incorrect amount of parts: expecting:%d actual:%d",
predictionFeatureMap.size(),
parse.length);
throw new MyException(errorMessage);
}
else {
key = params;
value = predictionFeatureMap.get(key);
}
return value;
}
Then in a controller advice class
#ControllerAdvice
public class Foo extends ResponseEntityExceptionHandler {
#ExceptionHandler(MyException.class)
public ResponseEntity<MyCustomErrorBody> handleControllerException(HttpServletRequest request, MyException ex) {
// Build your error response here and return it.
// Create a class that will represent the json error body
// and pass it as body and jackson will deserialize it for
// you into json automatically.
final MyCustomErrorBody body = new MyCustomErrorBody(false, ex.getMessage(), "0.97", "Patient_Responsibility")
return ResponseEntity.unprocessableEntity().body(myCustomErrorBody).build();
}
}
public class MyCustomErrorBody {
private boolean status;
private String errorMessage;
private String version;
private String coreName;
// constructor and getters setters ommitted
}
Links about spring boot error handling:
Official spring boot documentation
Baeldung exception-handling-for-rest-with-spring

How to get list of files managed by Spring-Cloud-Config-Server

I have 3 Spring-Boot applications running:
Eureka:8761
Spring-Cloud-Config:8080
myMicroService:8181
For Spring-Cloud-Config, I am using a local git URI to populate data. The local repo is on branch master and has a file structure like this:
./myMicroService
|-- application.properties
|-- foo.txt
|-- bar.txt
According to the documentation, I can access the text files like this:
http://localhost:8080/myMicroService/default/master/foo.txt
http://localhost:8080/myMicroService/default/master/bar.txt
Which works, but how do I get the full list of available *.txt files served up by Spring-Cloud-Config server?
I tried this:
http://localhost:8080/myMicroService/default/master
Which only returns application.properties and its values.
Given there is no OOTB solution for this, I added a new request mapping to my config-server:
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Collection;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import javax.servlet.http.HttpServletRequest;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.config.server.EnableConfigServer;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
import org.springframework.core.io.Resource;
import org.springframework.core.io.ResourceLoader;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
#SpringBootApplication
#EnableConfigServer
#EnableEurekaClient
#RestController
public class ConfigServer {
public static void main(String[] args) {
SpringApplication.run(ConfigServer.class, args);
}
#Value("${spring.cloud.config.server.git.uri}")
private String uri;
#Autowired private ResourceLoader resourceLoader;
#GetMapping("/{name}/{profile}/{label}/listFiles")
public Collection<String> retrieve(
#PathVariable String name,
#PathVariable String profile,
#PathVariable String label,
HttpServletRequest request)
throws IOException {
Resource resource = resourceLoader.getResource(uri);
String uriPath = resource.getFile().getPath();
Path namePath = Paths.get(uriPath, name);
String baseUrl =
String.format(
"http://%s:%d/%s/%s/%s",
request.getServerName(), request.getServerPort(), name, profile, label);
try (Stream<Path> files = Files.walk(namePath)) {
return files
.map(Path::toFile)
.filter(File::isFile)
.map(File::getName)
.map(filename -> baseUrl + "/" + filename)
.collect(Collectors.toList());
}
}
}
getting list of files for myMicroService:
curl http://localhost:8888/myMicroService/default/master/listFiles
result:
[
"http://localhost:8888/myMicroService/default/master/application.properties",
"http://localhost:8888/myMicroService/default/master/foo.txt",
"http://localhost:8888/myMicroService/default/master/bar.txt"
]
The previous solution from assumes that your spring.cloud.config.server.git.uri is of https type. The solution below will reference the local filesystem that contains the files that have been pulled from the repo and will work with any repo type. The endpoints also accept a search path and take the format of:
/{name}/{profile}/{label}/{path}/listFiles
/{name}/{profile}/{path}/listFiles?useDefaultLabel
#Configuration
#EnableAutoConfiguration
#EnableConfigServer
#RestController
public class CloudConfigApplication {
private UrlPathHelper helper = new UrlPathHelper();
private SearchPathLocator service;
public static void main(String[] args) {
SpringApplication.run(CloudConfigApplication.class, args);
}
public CloudConfigApplication(SearchPathLocator service) {
this.service = service;
}
#RequestMapping("/{name}/{profile}/{label}/**/listFiles")
public List<String> retrieve(#PathVariable String name, #PathVariable String profile,
#PathVariable String label, ServletWebRequest request,
#RequestParam(defaultValue = "true") boolean resolvePlaceholders)
throws IOException {
String path = getDirPath(request, name, profile, label);
return listAll(request, name, profile, label, path);
}
#RequestMapping(value = "/{name}/{profile}/**/listFiles", params = "useDefaultLabel")
public List<String> retrieve(#PathVariable String name, #PathVariable String profile,
ServletWebRequest request,
#RequestParam(defaultValue = "true") boolean resolvePlaceholders)
throws IOException {
String path = getDirPath(request, name, profile, null);
return listAll(request, name, profile, null, path);
}
private String getDirPath(ServletWebRequest request, String name, String profile, String label) {
String stem;
if (label != null) {
stem = String.format("/%s/%s/%s/", name, profile, label);
} else {
stem = String.format("/%s/%s/", name, profile);
}
String path = this.helper.getPathWithinApplication(request.getRequest());
path = path.substring(path.indexOf(stem) + stem.length()).replace("listFiles", "");
return path;
}
public synchronized List<String> listAll(ServletWebRequest request, String application, String profile, String label, String path) {
if (StringUtils.hasText(path)) {
String[] locations = this.service.getLocations(application, profile, label).getLocations();
List<String> fileURIs = new ArrayList<>();
try {
int i = locations.length;
while (i-- > 0) {
String location = String.format("%s%s", locations[i], path).replace("file:", "");
Path filePath = new File(location).toPath();
if(Files.exists(filePath)) {
fileURIs.addAll(Files.list(filePath).filter(file -> !Files.isDirectory(file)).map(file -> {
String URL =
String.format(
"%s://%s:%d%s/%s/%s/%s%s?useDefaultLabel",
request.getRequest().getScheme(), request.getRequest().getServerName(),
request.getRequest().getServerPort(), request.getRequest().getContextPath(),
application, profile, path,
file.getFileName().toString());
if(label != null) {
URL = String.format(
"%s://%s:%d%s/%s/%s/%s/%s%s",
request.getRequest().getScheme(), request.getRequest().getServerName(),
request.getRequest().getServerPort(), request.getRequest().getContextPath(),
application, profile, label, path,
file.getFileName().toString());
}
return URL;
}).collect(Collectors.toList()));
}
}
} catch (IOException var11) {
throw new NoSuchResourceException("Error : " + path + ". (" + var11.getMessage() + ")");
}
return fileURIs;
}
throw new NoSuchResourceException("Not found: " + path);
}
}
Getting the list:
curl http://localhost:8080/context/appName/profile/label/some/path/listFiles
Example Results:
[
"http://localhost:8080/context/appName/profile/label/some/path/robots.txt"
]

Spring Integration File reading

I am newbie to Spring Integration. I am working on solution, but I am stuck on a specific issue while using inbound file adapter ( FileReadingMessageSource ).
I have to read files from different directories and process them and save the files in different directories. As I understand, the directory name is fixed at the start of the flow.
Can some one help me on changing the directory name for different requests.
I attempted the following. First of all, I am not sure whether it is correct way to about and although it worked for only one directory. I think Poller was waiting for more files and never came back to read another directory.
#SpringBootApplication
#EnableIntegration
#IntegrationComponentScan
public class SiSampleFileProcessor {
#Autowired
MyFileProcessor myFileProcessor;
#Value("${si.outdir}")
String outDir;
#Autowired
Environment env;
public static void main(String[] args) throws IOException {
ConfigurableApplicationContext ctx = new SpringApplication(SiSampleFileProcessor.class).run(args);
FileProcessingService gateway = ctx.getBean(FileProcessingService.class);
boolean process = true;
while (process) {
System.out.println("Please enter the input Directory: ");
String inDir = new Scanner(System.in).nextLine();
if ( inDir.isEmpty() || inDir.equals("exit") ) {
process=false;
} else {
System.out.println("Processing... " + inDir);
gateway.processFilesin(inDir);
}
}
ctx.close();
}
#MessagingGateway(defaultRequestChannel="requestChannel")
public interface FileProcessingService {
String processFilesin( String inputDir );
}
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() {
return Pollers.fixedDelay(1000).get();
}
#Bean
public MessageChannel requestChannel() {
return new DirectChannel();
}
#ServiceActivator(inputChannel = "requestChannel")
#Bean
GenericHandler<String> fileReader() {
return new GenericHandler<String>() {
#Override
public Object handle(String p, Map<String, Object> map) {
FileReadingMessageSource fileSource = new FileReadingMessageSource();
fileSource.setDirectory(new File(p));
Message<File> msg;
while( (msg = fileSource.receive()) != null ) {
fileInChannel().send(msg);
}
return null; // Not sure what to return!
}
};
}
#Bean
public MessageChannel fileInChannel() {
return MessageChannels.queue("fileIn").get();
}
#Bean
public IntegrationFlow fileProcessingFlow() {
return IntegrationFlows.from(fileInChannel())
.handle(myFileProcessor)
.handle(Files.outboundAdapter(new File(outDir)).autoCreateDirectory(true).get())
.get();
}
}
EDIT: Based on Gary's response replaced some methods as
#MessagingGateway(defaultRequestChannel="requestChannel")
public interface FileProcessingService {
boolean processFilesin( String inputDir );
}
#ServiceActivator(inputChannel = "requestChannel")
public boolean fileReader(String inDir) {
FileReadingMessageSource fileSource = new FileReadingMessageSource();
fileSource.setDirectory(new File(inDir));
fileSource.afterPropertiesSet();
fileSource.start();
Message<File> msg;
while ((msg = fileSource.receive()) != null) {
fileInChannel().send(msg);
}
fileSource.stop();
System.out.println("Sent all files in directory: " + inDir);
return true;
}
Now it is working as expected.
You can use this code
FileProcessor.java
import org.springframework.messaging.Message;
import org.springframework.stereotype.Component;
#Component
public class FileProcessor {
private static final String HEADER_FILE_NAME = "file_name";
private static final String MSG = "%s received. Content: %s";
public void process(Message<String> msg) {
String fileName = (String) msg.getHeaders().get(HEADER_FILE_NAME);
String content = msg.getPayload();
//System.out.println(String.format(MSG, fileName, content));
System.out.println(content);
}
}
LastModifiedFileFilter.java
package com.example.demo;
import org.springframework.integration.file.filters.AbstractFileListFilter;
import java.io.File;
import java.util.HashMap;
import java.util.Map;
public class LastModifiedFileFilter extends AbstractFileListFilter<File> {
private final Map<String, Long> files = new HashMap<>();
private final Object monitor = new Object();
#Override
protected boolean accept(File file) {
synchronized (this.monitor) {
Long previousModifiedTime = files.put(file.getName(), file.lastModified());
return previousModifiedTime == null || previousModifiedTime != file.lastModified();
}
}
}
Main Class= DemoApplication.java
package com.example.demo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
import org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration;
import java.io.File;
import java.io.IOException;
import java.nio.charset.Charset;
import org.apache.commons.io.FileUtils;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.annotation.Aggregator;
import org.springframework.integration.annotation.InboundChannelAdapter;
import org.springframework.integration.annotation.Poller;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.channel.QueueChannel;
import org.springframework.integration.core.MessageSource;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.dsl.channel.MessageChannels;
import org.springframework.integration.dsl.core.Pollers;
import org.springframework.integration.file.FileReadingMessageSource;
import org.springframework.integration.file.filters.CompositeFileListFilter;
import org.springframework.integration.file.filters.SimplePatternFileListFilter;
import org.springframework.integration.file.transformer.FileToStringTransformer;
import org.springframework.integration.scheduling.PollerMetadata;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.PollableChannel;
import org.springframework.stereotype.Component;
#SpringBootApplication
#Configuration
public class DemoApplication {
private static final String DIRECTORY = "E:/usmandata/logs/input/";
public static void main(String[] args) throws IOException, InterruptedException {
SpringApplication.run(DemoApplication.class, args);
}
#Bean
public IntegrationFlow processFileFlow() {
return IntegrationFlows
.from("fileInputChannel")
.transform(fileToStringTransformer())
.handle("fileProcessor", "process").get();
}
#Bean
public MessageChannel fileInputChannel() {
return new DirectChannel();
}
#Bean
#InboundChannelAdapter(value = "fileInputChannel", poller = #Poller(fixedDelay = "1000"))
public MessageSource<File> fileReadingMessageSource() {
CompositeFileListFilter<File> filters =new CompositeFileListFilter<>();
filters.addFilter(new SimplePatternFileListFilter("*.log"));
filters.addFilter(new LastModifiedFileFilter());
FileReadingMessageSource source = new FileReadingMessageSource();
source.setAutoCreateDirectory(true);
source.setDirectory(new File(DIRECTORY));
source.setFilter(filters);
return source;
}
#Bean
public FileToStringTransformer fileToStringTransformer() {
return new FileToStringTransformer();
}
#Bean
public FileProcessor fileProcessor() {
return new FileProcessor();
}
}
The FileReadingMessageSource uses a DirectoryScanner internally; it is normally set up by Spring after the properties are injected. Since you are managing the object outside of Spring, you need to call Spring bean initialization and lifecycle methods afterPropertiesSet() , start() and stop().
Call stop() when the receive returns null.
> return null; // Not sure what to return!
If you return nothing, your calling thread will hang in the gateway waiting for a response. You could change the gateway to return void or, since your gateway is expecting a String, just return some value.
However, your calling code is not looking at the result anyway.
> gateway.processFilesin(inDir);
Also, remove the #Bean from the #ServiceActivator; with that style, the bean type must be MessageHandler.

java.lang.IllegalStateException: Autowired annotation requires at least one argument

Cannot Autowire and run Spring web application.
Error:
java.lang.IllegalStateException: Autowired annotation requires at least one argument: public main.java.com.springapp.mvc.controller.DSLRServletController()
DSLRServletController:
package main.java.com.springapp.mvc.controller;
import main.java.com.springapp.mvc.dao.DSLRDAO;
import main.java.com.springapp.mvc.model.DSLR;
import main.java.com.springapp.mvc.pckg.DSLRForm;
import main.java.com.springapp.mvc.pckg.DSLRValidaor;
import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.Serializable;
import java.util.*;
#Controller
public class DSLRServletController {
static Logger logger = Logger.getLogger(DSLRServletController.class);
private DSLR DSLR;
private DSLRDAO dslrDAO;
private DSLR dslr;
#Autowired
public DSLRServletController() {
this.dslrDAO = new DSLRDAO();
}
public void init() {logger.error("DSLRServlet.init(): just started"); }
#RequestMapping(value = "/s", method = RequestMethod.GET)
public String showHTMLResponse(#ModelAttribute("dslrs") DSLR dslrs[],
#ModelAttribute("dslr") DSLR dslr,
#ModelAttribute("dslrErrors") HashMap dslrErrors,
#ModelAttribute ("dslrform") DSLRForm dslrForm,
#RequestParam("id") String paramId,
#RequestParam("action") String paramAction,
Model model){
if(paramId == null || paramId.equals("")){
//show_all_dslrs
dslrs = getAllDslrs(); // DSLR adslrs[] -> to MODEL; HOW?
return "dslrs";
}else{
//show_this_dslr;
HashMap<String,Object> dslrHashMap = getDSLRById(paramId);
dslr = (DSLR) dslrHashMap.get("dslr");
dslrForm = (DSLRForm)dslrHashMap.get("dslrForm");
dslrErrors = (HashMap)dslrHashMap.get("dslrErrors");
if(dslr != null){
return "dslr";
}
else{
return "error";
}
}
}
#RequestMapping(value = "/s", method = RequestMethod.POST)
public String showHTMLResponsePOST(#ModelAttribute("dslrs") DSLR dslrs[],
#ModelAttribute("dslrErrors") HashMap<?,?> dslrErrors,
#ModelAttribute ("dslrform") DSLRForm dslrForm,
#RequestParam("id") String paramId,
#RequestParam("action") String paramAction,
#RequestParam("dslr_model") String paramModel,
#RequestParam("price") String paramPrice,
#RequestParam("description") String paramDescription,
Model model){
int iStatusCode = 0;
if(paramAction.equals("save") )
iStatusCode = saveDSLR(paramId, paramModel, paramPrice, paramDescription, dslrErrors, dslrForm); // POST
return "dslrs";
}
private int saveDSLR(String paramId,
String paramModel,
String paramPrice,
String paramDescription,
HashMap<?,?> context_dslrErrors,
DSLRForm context_dslrForm
) {
int byte0 = 1;
try {
DSLRValidaor dslrValidaor = new DSLRValidaor();
DSLRForm dslrForm = new DSLRForm();
dslrForm.setDslrId(paramId);
dslrForm.setModel(paramModel);
dslrForm.setPrice(paramPrice);
dslrForm.setDescription(paramDescription);
HashMap hashmap = dslrValidaor.Validate(dslrForm);
if(hashmap.size() > 0) {
context_dslrForm = dslrForm;
context_dslrErrors = hashmap;
byte0 = -1;
} else{
DSLRDAO planedao = new DSLRDAO();
DSLR dslr = new DSLR();
dslr.setDslrId(Integer.parseInt(paramId));
dslr.setModel(paramModel);
dslr.setPrice(Integer.parseInt(paramPrice));
dslr.setDescription(paramDescription);
planedao.update(dslr);
}
}
catch(Exception exception)
{
logger.error((new StringBuilder()).append("DSLRServlet.saveDSLR():").append(exception.getMessage()).toString());
byte0 = -1;
}
return byte0;
}
private DSLR[] getAllDslrs(){
DSLR adslrs[] = null;
try
{
DSLRDAO DSLRDAO = new DSLRDAO();
adslrs = (DSLR[])DSLRDAO.findAll();
}
catch(Exception exception)
{
logger.error((new StringBuilder()).append("PlaneServlet.getAllPlanes():").append(exception.getMessage()).toString());
}
// request.setAttribute("dslrs", adslrs);
return adslrs;
}
private HashMap<String, Object> getDSLRById(String s)
{
HashMap<String,Object> map = new HashMap<String, Object>();
DSLR dslr = null;
try {
int i = Integer.parseInt(s);
DSLRDAO DSLRDAO = new DSLRDAO();
dslr = (DSLR)DSLRDAO.findById(i);
DSLRForm dslrForm = new DSLRForm();
dslrForm.setDslrId(Integer.toString(dslr.getDslrId()));
dslrForm.setModel(dslr.getModel());
dslrForm.setPrice(Integer.toString(dslr.getPrice()));
dslrForm.setDescription(dslr.getDescription());
map.put("dslr", dslr);
map.put("dslrform", dslrForm);
map.put("dslrErrors", new HashMap());
}
catch(Exception exception)
{
logger.error((new StringBuilder()).append("DSLRServlet.getDSLRById():").append(exception.getMessage()).toString());
}
return map;
}
#Autowired
public void setDslrDAO(DSLRDAO dslrDAO) {
this.dslrDAO = dslrDAO;
}
public DSLRDAO getDslrDAO() {
return dslrDAO;
}
#Autowired
public void setDSLR(DSLR DSLR) {
dslr = DSLR;
}
public DSLR getDSLR() {
return dslr;
}
}
Why #Autowired annotation returns error? How to fix it?
In addition to reimeus answer, you can't use #Autowired on a default constructor.
It's often considered better to autowire the constructor rather than a field.
#Autowired
public DSLRServletController(DSLRDAO dslrDAO) {
this.dslrDAO = dslrDAO;
}
The constructor isnt a valid setter method or instance variable
#Autowired
private DSLRDAO dslrDAO;
I was facing this same issue and found the root cause, which is different than the original question asked, but may help someone for sure.
We were using Lombok jar to generate default code like Getters/Setters, similarly we were using Lombok annotation to generate a Constructor for the Required Argument and mark it with #Autowired behind the scene.
#RequiredArgsConstructor(onConstructor = #__(#Autowired))
public class MyClass {
private MyDependency myDependency;
}
The issue which was causing the problem was a missing final keyword while defining the dependency. #RequiredArgsConstructor identifies the required argument as the one which are marked final and must be populated during construction.
Another annotation which could have been used is #AllArgsConstructor (this will create a constructor for all the dependencies so final is not really needed)
So, the correct solution to resolve the said error is to use final
#RequiredArgsConstructor(onConstructor = #__(#Autowired))
public class MyClass {
private final MyDependency myDependency;
}

Categories