We Keep Coding

Spring Boot REST API return list of files

I am trying to return a list of files from a directory. Here's my code:
package com.demo.web.api.file;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import com.demo.core.Logger;
import io.swagger.v3.oas.annotations.Operation;
#RestController
#RequestMapping(value = "/files")
public class FileService {
private static final Logger logger = Logger.factory(FileService.class);
#Value("${file-upload-path}")
public String DIRECTORY;
#Value("${file-upload-check-subfolders}")
public boolean CHECK_SUBFOLDERS;
#GetMapping(value = "/list")
#Operation(summary = "Get list of Uploaded files")
public ResponseEntity<List<File>> list() {
List<File> files = new ArrayList<>();
if (CHECK_SUBFOLDERS) {
// Recursive check
try (Stream<Path> walk = Files.walk(Paths.get(DIRECTORY))) {
List<Path> result = walk.filter(Files::isRegularFile).collect(Collectors.toList());
for (Path p : result) {
files.add(p.toFile().getAbsoluteFile());
}
} catch (Exception e) {
logger.error(e.getMessage());
}
} else {
// Checks the root directory only.
try (Stream<Path> walk = Files.walk(Paths.get(DIRECTORY), 1)) {
List<Path> result = walk.filter(Files::isRegularFile).collect(Collectors.toList());
for (Path p : result) {
files.add(p.toFile().getAbsoluteFile());
}
} catch (Exception e) {
logger.error(e.getMessage());
}
}
return ResponseEntity.ok().body(files);
}
}
As seen in the code, I am trying to return a list of files.
However, when I test in PostMan, I get a list of string instead.
How can I make it return the file object instead of the file path string? I need to get the file attributes (size, date, etc.) to display in my view.

I would recommend that you change your ResponseEntity<> to return not a List of File but instead, a List of Resource, which you can then use to obtain the file metadata that you need.
public ResponseEntity<List<Resource>> list() {}
You can also try specifying a produces=MediaType... param in your #GetMapping annotation so as to tell the HTTP marshaller which kind of content to expect.

You'd have to Create a separate payload with the details you wanna respond with.
public class FilePayload{
private String id;
private String name;
private String size;
public static fromFile(File file){
// create FilePayload from File object here
}
}
And convert it using a mapper from your internal DTO objects to payload ones.
final List<FilePayload> payload = files.forEach(FilePayload::fromFile).collect(Collectors.toList());
return new ResponseEntity<>(payload, HttpStatus.OK);
I think you should not return a body in this case as you may be unaware of the size.
Better to have another endpoint to GET /files/{id}

I did give this another thought. What I just needed was the filename, size and date of the file. From there, I can get the file extension and make my list display look good already.
Here's the refactored method:
#GetMapping(value = "/list")
#Operation(summary = "Get list of Uploaded files")
public ResponseEntity<String> list() {
JSONObject responseObj = new JSONObject();
List<JSONObject> files = new ArrayList<>();
// If CHECK_SUBFOLDERS is true, pass MAX_VALUE to make it recursive on all
// sub-folders. Otherwise, pass 1 to use the root directory only.
try (Stream<Path> walk = Files.walk(Paths.get(DIRECTORY), CHECK_SUBFOLDERS ? MAX_VALUE : 1)) {
List<Path> result = walk.filter(Files::isRegularFile).collect(Collectors.toList());
for (Path p : result) {
JSONObject file = new JSONObject();
file.put("name", p.toFile().getName());
file.put("size", p.toFile().length());
file.put("lastModified", p.toFile().lastModified());
files.add(file);
}
responseObj.put("data", files);
} catch (Exception e) {
String errMsg = CoreUtils.formatString("%s: Error reading files from the directory: \"%s\"",
e.getClass().getName(), DIRECTORY);
logger.error(e, errMsg);
responseObj.put("errors", errMsg);
}
return ResponseEntity.ok().body(responseObj.toString());
}
The above was what I ended up doing. I created a JSONObject of the props I need, and then returned the error if it did not succeed. This made it a lot better for me.

Read /Edit / Write jpg IPTC metadata in Java

I am using Apache Commons but it is not enough for me because it is so old technology. So ,i found iCafe and it seems better but I am having the error below. Any idea what i am doing wrong?
private static List<IPTCDataSet> createIPTCDataSet() {
List<IPTCDataSet> iptcs = new ArrayList<IPTCDataSet>();
iptcs.add(new IPTCDataSet(IPTCApplicationTag.COPYRIGHT_NOTICE, "Copyright 2014-2016, yuwen_66#yahoo.com"));
iptcs.add(new IPTCDataSet(IPTCApplicationTag.CATEGORY, "ICAFE"));
iptcs.add(new IPTCDataSet(IPTCApplicationTag.KEY_WORDS, "Welcome 'icafe' user!"));
return iptcs;
}
private static IPTC createIPTC() {
IPTC iptc = new IPTC();
iptc.addDataSets(createIPTCDataSet());
return iptc;
}
public static void main(String[] args) throws IOException {
FileInputStream fin = new FileInputStream("C:/Users/rajab/Desktop/test/ibo.jpeg");
FileOutputStream fout = new FileOutputStream("C:/Users/rajab/Desktop/test/ibo/ibo.jpeg");
List<Metadata> metaList = new ArrayList<Metadata>();
//metaList.add(populateExif(TiffExif.class));
metaList.add(createIPTC());
metaList.add(new Comments(Arrays.asList("Comment1", "Comment2")));
Metadata.insertMetadata(metaList, fin, fout);
}
}
and my EXCEPTION
run:
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at com.icafe4j.image.meta.Metadata.(Unknown Source)
at vectorcleaner.Metadata1.populateExif(Metadata1.java:41)
at vectorcleaner.Metadata1.main(Metadata1.java:127)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 3 more
C:\Users\rajab\AppData\Local\NetBeans\Cache\8.2\executor-snippets\run.xml:53:
Java returned: 1
BUILD FAILED (total time: 0 seconds)

Not sure what exactly you want but here is what ICAFE can do with IPTC metadata:
Read IPTC from image.
Insert IPTC into image or update IPTC of existing image.
Delete IPTC from image.
For reading IPTC, here is an example:
import java.io.IOException;
import java.util.List;
import java.util.Map;
import java.util.Iterator;
import com.icafe4j.image.meta.Metadata;
import com.icafe4j.image.meta.MetadataEntry;
import com.icafe4j.image.meta.MetadataType;
import com.icafe4j.image.meta.iptc.IPTC;
public class ExtractIPTC {
public static void main(String[] args) throws IOException {
Map<MetadataType, Metadata> metadataMap = Metadata.readMetadata(args[0]);
IPTC iptc = (IPTC)metadataMap.get(MetadataType.IPTC);
if(iptc != null) {
Iterator<MetadataEntry> iterator = iptc.iterator();
while(iterator.hasNext()) {
MetadataEntry item = iterator.next();
printMetadata(item, "", " ");
}
}
}
private void printMetadata(MetadataEntry entry, String indent, String increment) {
logger.info(indent + entry.getKey() (StringUtils.isNullOrEmpty(entry.getValue())? "" : ": " + entry.getValue()));
if(entry.isMetadataEntryGroup()) {
indent += increment;
Collection<MetadataEntry> entries = entry.getMetadataEntries();
for(MetadataEntry e : entries) {
printMetadata(e, indent, increment);
}
}
}
}
For insert/update IPTC, here is an example:
import java.io.IOException;
import java.util.List;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import com.icafe4j.image.meta.Metadata;
import com.icafe4j.image.meta.MetadataEntry;
import com.icafe4j.image.meta.MetadataType;
import com.icafe4j.image.meta.iptc.IPTCDataSet;
import com.icafe4j.image.meta.iptc.IPTCApplicationTag;
public class InsertIPTC {
public static void main(String[] args) throws IOException {
FileInputStream fin = new FileInputStream("C:/Users/rajab/Desktop/test/ibo.jpeg");
FileOutputStream fout = new FileOutputStream("C:/Users/rajab/Desktop/test/ibo/ibo.jpeg");
Metadata.insertIPTC(fin, fout, createIPTCDataSet(), true);
}
private static List<IPTCDataSet> createIPTCDataSet() {
List<IPTCDataSet> iptcs = new ArrayList<IPTCDataSet>();
iptcs.add(new IPTCDataSet(IPTCApplicationTag.COPYRIGHT_NOTICE, "Copyright 2014-2016, yuwen_66#yahoo.com"));
iptcs.add(new IPTCDataSet(IPTCApplicationTag.OBJECT_NAME, "ICAFE"));
iptcs.add(new IPTCDataSet(IPTCApplicationTag.KEY_WORDS, "Welcome 'icafe' user!"));
return iptcs;
}
}
The above example uses Metadata.insertIPTC instead of Metadata.insertMetadata because we need one more boolean parameter. If set to true, it will keep the existing IPTC data and only update the those we want. Some of the IPTC entries allow multiple values. In that case, we only append/add the new ones. For other unique entries, they will be replaced by the new ones.
Looks like you want to add key words and title. In your question, you already showed code to insert key words and in order to insert title, use OBJECT_NAME which can be found in the example above.
Note: you can add multiple key words as well. Some softwares can only handle one key words record. In that case, you can put all the key words in one record separated by semi-colon instead of insert multiple entries.

Exception while calling Parser method outside main class

In my application I have a method which I cant execute without main method. It only runs inside the main method. When I call that method inside my servlet class. It show an exception
My class with Main Method
package com.books.servlet;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.net.URL;
import java.nio.channels.Channels;
import java.nio.channels.ReadableByteChannel;
import java.util.HashSet;
import java.util.Set;
import opennlp.tools.cmdline.parser.ParserTool;
import opennlp.tools.parser.Parse;
import opennlp.tools.parser.Parser;
import opennlp.tools.parser.ParserFactory;
import opennlp.tools.parser.ParserModel;
public class ParserTest {
// download
public void download(String url, File destination) throws IOException, Exception {
URL website = new URL(url);
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream(destination);
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
fos.close();
rbc.close();
}
public static Set<String> nounPhrases = new HashSet<>();
private static String line = "The Moon is a barren, rocky world ";
public void getNounPhrases(Parse p) {
if (p.getType().equals("NN") || p.getType().equals("NNS") || p.getType().equals("NNP")
|| p.getType().equals("NNPS")) {
nounPhrases.add(p.getCoveredText());
}
for (Parse child : p.getChildren()) {
getNounPhrases(child);
}
}
public void parserAction() throws Exception {
// InputStream is = new FileInputStream("en-parser-chunking.bin");
File modelFile = new File("en-parser-chunking.bin");
if (!modelFile.exists()) {
System.out.println("Downloading model.");
download("https://drive.google.com/uc?export=download&id=0B4uQtYVPbChrY2ZIWmpRQ1FSVVk", modelFile);
}
ParserModel model = new ParserModel(modelFile);
Parser parser = ParserFactory.create(model);
Parse topParses[] = ParserTool.parseLine(line, parser, 1);
for (Parse p : topParses) {
// p.show();
getNounPhrases(p);
}
}
public static void main(String[] args) throws Exception {
new ParserTest().parserAction();
System.out.println("List of Noun Parse : " + nounPhrases);
}
}
It gives me below output
List of Noun Parse : [barren,, world, Moon]
Then I commented the main method and. Called the ParserAction() method in my servlet class
if (name.equals("bkDescription")) {
bookDes = value;
try {
new ParserTest().parserAction();
System.out.println("Nouns Are"+ParserTest.nounPhrases);
} catch (Exception e) {
}
It gives me the below exceptions
And below error in my Browser
Why is this happening ? I can run this with main method. But when I remove main method and called in my servlet. it gives an exception. Is there any way to fix this issue ?
NOTE - I have read below instructions in OpenNLP documentation , but I have no clear idea about it. Please help me to fix his issue.
Unlike the other components to instantiate the Parser a factory method
should be used instead of creating the Parser via the new operator.
The parser model is either trained for the chunking parser or the tree
insert parser the parser implementation must be chosen correctly. The
factory method will read a type parameter from the model and create an
instance of the corresponding parser implementation.

Either create an object of ParserTest class or remove new keyword in this line new ParserTest().parserAction();

API to Validate .wsdl files using java

I need to build an utility program that accepts .wsdl file as input. Validates the file and gives output as weather file is valid(in terms of semantics). And if possible i also want to show error's in file (if any).
I know there are utilities available online, but i cannot upload incoming .wsdl files over internet(for security purpose). Hence i wan to do this operation programatically using Java.
Please suggest me if there are any API's available for this in java?

Credit to this blog usign wsdl4j
www.vorburger.ch/files/WSDLValidationTask.java‎
package wsdlvalidation;
import java.io.File;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.StringTokenizer;
import java.util.Map.Entry;
import javax.wsdl.Definition;
import javax.wsdl.Fault;
import javax.wsdl.Message;
import javax.wsdl.Operation;
import javax.wsdl.Part;
import javax.wsdl.PortType;
import javax.wsdl.WSDLException;
import javax.wsdl.factory.WSDLFactory;
import javax.wsdl.xml.WSDLReader;
import javax.xml.namespace.QName;
import org.apache.tools.ant.BuildException;
import org.apache.tools.ant.taskdefs.MatchingTask;
import org.apache.tools.ant.types.FileSet;
/**
* Ant Task to validate a WDSL for an Axis1 bug.
*
*/
public class WSDLValidationTask extends MatchingTask
{
private FileSet configuredWsdl;
public void execute() throws BuildException
{
super.execute();
try {
WSDLFactory wsdlFactory = WSDLFactory.newInstance();
WSDLReader reader = wsdlFactory.newWSDLReader();
Iterator it = getWSDLFileNamesList().iterator();
while (it.hasNext()) {
String wsdl = (String) it.next();
Definition theWSDL = reader.readWSDL(wsdl);
// This is a Bag of all Messages in the WSDL that are used in some <wsdl:fault> of any Operation of any PortType
Set faultMessages = new HashSet();
Map allPortTypes = theWSDL.getPortTypes();
Iterator portTypeIt = allPortTypes.entrySet().iterator();
while (portTypeIt.hasNext()) {
Map.Entry entry = (Entry) portTypeIt.next();
PortType portType = (PortType) entry.getValue();
List allOperations = portType.getOperations();
Iterator listIt = allOperations.iterator();
while (listIt.hasNext()) {
Operation operation = (Operation)listIt.next();
Iterator faultIt = operation.getFaults().values().iterator();
while (faultIt.hasNext()) {
Fault fault = (Fault) faultIt.next();
faultMessages.add(fault.getMessage());
}
}
}
Map allMessages = theWSDL.getMessages();
Iterator messageIt = allMessages.entrySet().iterator();
while (messageIt.hasNext()) {
Map.Entry entry = (Entry) messageIt.next();
QName messageNameQName = (QName) entry.getKey();
String messageName = messageNameQName.getLocalPart();
Message message = (Message) entry.getValue();
Map parts = message.getParts();
validate(parts.size() == 1, wsdl,
"wsdl:message has more than one part: " + messageNameQName.toString());
Part messagePart = (Part) parts.values().iterator().next();
validate(messagePart.getTypeName() == null, wsdl, "wsdl:part should not have a 'type' attribute: " + messagePart.getName());
// Only for Messages that are used in Fault:
if (faultMessages.contains(message)) {
validate(!messagePart.getElementName().getLocalPart().equals(messageName), wsdl,
"Due to an Axis1 bug, please do NOT use the same name for <wsdl:message name=\"" + messageName + "\"> and <xsd:element name=\"" + messagePart.getElementName().getLocalPart()+"\">");
}
}
}
} catch (WSDLException e) {
throw new BuildException(e);
}
}
private void validate(boolean condition, String wsdlFilename, String failureMessage) throws BuildException {
if (!condition) {
throw new BuildException(wsdlFilename + ": " + failureMessage);
}
}
// TODO Doesn't code like this already exist in ant??
private List getWSDLFileNamesList() {
List/*<String>*/ wsdlList = new ArrayList/*<String>*/();
File dir = configuredWsdl.getDir(configuredWsdl.getProject());
StringTokenizer tokenizer = new StringTokenizer(configuredWsdl.toString(), ";");
while (tokenizer.hasMoreTokens()) {
String token = tokenizer.nextToken();
wsdlList.add(new File(dir, token).toString());
}
return wsdlList;
}
public void addConfiguredWsdl(FileSet fileSet) {
configuredWsdl = fileSet;
}
}

Another way is to use SOAPUI tool API. Refer to below link.
Try to import WSDL and catch any exception to know whether WSDL is correct or not.
WsdlInterface iface = WsdlInterfaceFactory.importWsdl( "WSDl_LOCATION", true )[0];
Integrating With SoapUI

Validate every field in a single pass with SuperCSV

I'm trying to write a large number of rows (~2 million) from a database to a CSV file using SuperCSV. I need to perform validation on each cell as it is written, and the built-in CellProcessors do very nicely. I want to capture all the exceptions that are thrown by the CellProcessors so that I can go back to the source data and make changes.
The problem is that when there are multiple errors in a single row (e.g. The first value is out of range, the second value is null but shouldn't be), only the first CellProcessor will execute, and so I'll only see one of the errors. I want to process the whole file in a single pass, and have a complete set of exceptions at the end of it.
This is the kind of approach I'm trying:
for (Row row : rows) {
try {
csvBeanWriter.write(row, HEADER_MAPPINGS, CELL_PROCESSORS);
} catch (SuperCsvCellProcessorException e) {
log(e);
}
}
How can I achieve this? Thanks!
EDIT: Here is the code I wrote that's similar to Hound Dog's, in case it helps anyone:
import java.util.List;
import org.supercsv.cellprocessor.CellProcessorAdaptor;
import org.supercsv.cellprocessor.ift.CellProcessor;
import org.supercsv.exception.SuperCsvCellProcessorException;
import org.supercsv.util.CsvContext;
public class ExceptionCapturingCellProcessor extends CellProcessorAdaptor {
private final List<Exception> exceptions;
private final CellProcessor current;
public ExceptionCapturingCellProcessor(CellProcessor current, CellProcessor next, List<Exception> exceptions) {
super(next);
this.exceptions = exceptions;
this.current = current;
}
#Override
public Object execute(Object value, CsvContext context) {
// Check input is not null
try {
validateInputNotNull(value, context);
} catch (SuperCsvCellProcessorException e) {
exceptions.add(e);
}
// Execute wrapped CellProcessor
try {
current.execute(value, context);
} catch (SuperCsvCellProcessorException e) {
exceptions.add(e);
}
return next.execute(value, context);
}
}

I'd recommend writing a custom CellProcessor to achieve this. The following processor can be placed at the start of each CellProcessor chain - it will simply delegate to the processor chained after it, and will suppress any cell processing exceptions.
package example;
import java.util.ArrayList;
import java.util.List;
import org.supercsv.cellprocessor.CellProcessorAdaptor;
import org.supercsv.cellprocessor.ift.CellProcessor;
import org.supercsv.exception.SuperCsvCellProcessorException;
import org.supercsv.util.CsvContext;
public class SuppressException extends CellProcessorAdaptor {
public static List<SuperCsvCellProcessorException> SUPPRESSED_EXCEPTIONS =
new ArrayList<SuperCsvCellProcessorException>();
public SuppressException(CellProcessor next) {
super(next);
}
public Object execute(Object value, CsvContext context) {
try {
// attempt to execute the next processor
return next.execute(value, context);
} catch (SuperCsvCellProcessorException e) {
// save the exception
SUPPRESSED_EXCEPTIONS.add(e);
// and suppress it (null is written as "")
return null;
}
}
}
And here it is in action:
package example;
import java.io.StringWriter;
import java.util.Arrays;
import java.util.List;
import org.supercsv.cellprocessor.constraint.NotNull;
import org.supercsv.cellprocessor.constraint.StrMinMax;
import org.supercsv.cellprocessor.ift.CellProcessor;
import org.supercsv.exception.SuperCsvCellProcessorException;
import org.supercsv.io.CsvBeanWriter;
import org.supercsv.io.ICsvBeanWriter;
import org.supercsv.prefs.CsvPreference;
public class TestSuppressExceptions {
private static final CellProcessor[] PROCESSORS = {
new SuppressException(new StrMinMax(0, 4)),
new SuppressException(new NotNull()) };
private static final String[] HEADER = { "name", "age" };
public static void main(String[] args) throws Exception {
final StringWriter stringWriter = new StringWriter();
ICsvBeanWriter beanWriter = null;
try {
beanWriter = new CsvBeanWriter(stringWriter,
CsvPreference.STANDARD_PREFERENCE);
beanWriter.writeHeader(HEADER);
// set up the data
Person valid = new Person("Rick", 43);
Person nullAge = new Person("Lori", null);
Person totallyInvalid = new Person("Shane", null);
Person valid2 = new Person("Carl", 12);
List<Person> people = Arrays.asList(valid, nullAge, totallyInvalid,
valid2);
for (Person person : people) {
beanWriter.write(person, HEADER, PROCESSORS);
if (!SuppressException.SUPPRESSED_EXCEPTIONS.isEmpty()) {
System.out.println("Suppressed exceptions for row "
+ beanWriter.getRowNumber() + ":");
for (SuperCsvCellProcessorException e :
SuppressException.SUPPRESSED_EXCEPTIONS) {
System.out.println(e);
}
// clear ready for next row
SuppressException.SUPPRESSED_EXCEPTIONS.clear();
}
}
} finally {
beanWriter.close();
}
// CSV will have empty columns for invalid data
System.out.println(stringWriter);
}
}
Here's the suppressed exceptions output (row 4 has two exceptions, one for each column):
Suppressed exceptions for row 3:
org.supercsv.exception.SuperCsvConstraintViolationException: null value
encountered processor=org.supercsv.cellprocessor.constraint.NotNull
context={lineNo=3, rowNo=3, columnNo=2, rowSource=[Lori, null]}
Suppressed exceptions for row 4:
org.supercsv.exception.SuperCsvConstraintViolationException: the length (5)
of value 'Shane' does not lie between the min (0) and max (4) values (inclusive)
processor=org.supercsv.cellprocessor.constraint.StrMinMax
context={lineNo=4, rowNo=4, columnNo=2, rowSource=[Shane, null]}
org.supercsv.exception.SuperCsvConstraintViolationException: null value
encountered processor=org.supercsv.cellprocessor.constraint.NotNull
context={lineNo=4, rowNo=4, columnNo=2, rowSource=[Shane, null]}
And the CSV output
name,age
Rick,43
Lori,
,
Carl,12
Notice how the invalid values were written as "" because the SuppressException processor returned null for those values (not that you'd use the CSV output anyway, as it's not valid!).