We Keep Coding

Micronaut test fileupload

I want to test a method, which receives a CompletedFileUpload.
public void uploadZip(String orderId, CompletedFileUpload fileUpload) throws IOException {
Path path = Files.createTempFile("file-", "-temp");
File tempFile = path.toFile();
try (FileOutputStream outputStream = new FileOutputStream(tempFile)) {
outputStream.write(fileUpload.getBytes());
someMethod(orderId, tempFile);
}
}
The CompletedFileUpload is an interface:
package io.micronaut.http.multipart;
public interface CompletedFileUpload extends FileUpload, CompletedPart {
}
I want to find a way to Mock the File upload, for spring boot i found :
MockMultipartFile multipartFile =
new MockMultipartFile("springboot.png", "springboot.png", "image/png", "".getBytes());
Is there something similar that I can use in Micronaut?

You can build your own MockFileUpload test class the way Spring does:
public class MockFileUpload implements CompletedFileUpload {
private final String filename;
private final MediaType mediaType;
private final byte[] content;
public MockFileUpload(String filename, MediaType mediaType) {
this(filename, mediaType, null);
}
public MockFileUpload(String filename, MediaType mediaType, #Nullable byte[] content) {
this.filename = filename;
this.mediaType = mediaType;
this.content = (content != null ? content : new byte[0]);
}
#Override
public InputStream getInputStream() throws IOException {
return new ByteArrayInputStream(content);
}
#Override
public byte[] getBytes() throws IOException {
return content;
}
#Override
public ByteBuffer getByteBuffer() throws IOException {
return ByteBuffer.wrap(content);
}
#Override
public Optional<MediaType> getContentType() {
return Optional.of(mediaType);
}
#Override
public String getName() {
return filename;
}
#Override
public String getFilename() {
return filename;
}
#Override
public long getSize() {
return content.length;
}
#Override
public long getDefinedSize() {
return content.length;
}
#Override
public boolean isComplete() {
return true;
}
}
and then in your tests you can use the class this way:
CompletedFileUpload file = new MockFileUpload("invoice.pdf", MediaType.APPLICATION_PDF_TYPE);

GraphQL Error: Expected a user-defined GraphQL scalar type with name 'FileUpload' but found none

for several days I have been trying to implement the upload file in Java-GraphQL. I found this topic: How to upload files with graphql-java? I implemented second solutions.
public class FileUpload {
private String contentType;
private byte[] content;
public FileUpload(String contentType, byte[] content) {
this.contentType = contentType;
this.content = content;
}
public String getContentType() {
return contentType;
}
public byte[] getContent() {
return content;
}
}
public class MyScalars {
public static final GraphQLScalarType FileUpload = new GraphQLScalarType(
"FileUpload",
"A file part in a multipart request",
new Coercing<FileUpload, Void>() {
#Override
public Void serialize(Object dataFetcherResult) {
throw new CoercingSerializeException("Upload is an input-only type");
}
#Override
public FileUpload parseValue(Object input) {
if (input instanceof Part) {
Part part = (Part) input;
try {
String contentType = part.getContentType();
byte[] content = new byte[part.getInputStream().available()];
part.delete();
return new FileUpload(contentType, content);
} catch (IOException e) {
throw new CoercingParseValueException("Couldn't read content of the uploaded file");
}
} else if (null == input) {
return null;
} else {
throw new CoercingParseValueException(
"Expected type " + Part.class.getName() + " but was " + input.getClass().getName());
}
}
#Override
public FileUpload parseLiteral(Object input) {
throw new CoercingParseLiteralException(
"Must use variables to specify Upload values");
}
});
}
public class FileUploadResolver implements GraphQLMutationResolver {
public Boolean uploadFile(FileUpload fileUpload) {
String fileContentType = fileUpload.getContentType();
byte[] fileContent = fileUpload.getContent();
// Do something in order to persist the file :)
return true;
}
}
scalar FileUpload
type Mutation {
uploadFile(fileUpload: FileUpload): Boolean
}
I get this error during compilation:
Caused by: com.coxautodev.graphql.tools.SchemaClassScannerError: Expected a user-defined GraphQL scalar type with name 'FileUpload' but found none!

Have you registered it via RuntimeWiring?
Take a look here: Custom Scalar in Graphql-java

You have to extend GraphQLScalarType in your MyScalars class

How to Implement Factory Design Pattern for CsvProcessing based on Key

I have written a controller which is a default for MototuploadService(for Motor Upload), but I need to make one Factory Design so that
based on parentPkId, need to call HealUploadService, TempUploadService, PersonalUploadService etc which will have separate file processing stages.
controller is below.
#RequestMapping(value = "/csvUpload", method = RequestMethod.POST)
public List<String> csvUpload(#RequestParam String parentPkId, #RequestParam List<MultipartFile> files)
throws IOException, InterruptedException, ExecutionException, TimeoutException {
log.info("Entered method csvUpload() of DaoController.class");
List<String> response = new ArrayList<String>();
ExecutorService executor = Executors.newFixedThreadPool(10);
CompletionService<String> compService = new ExecutorCompletionService<String>(executor);
List< Future<String> > futureList = new ArrayList<Future<String>>();
for (MultipartFile f : files) {
compService.submit(new ProcessMutlipartFile(f ,parentPkId,uploadService));
futureList.add(compService.take());
}
for (Future<String> f : futureList) {
long timeout = 0;
System.out.println(f.get(timeout, TimeUnit.SECONDS));
response.add(f.get());
}
executor.shutdown();
return response;
}
Here is ProcessMutlipartFile class which extends the callable interface, with CompletionService's compService.submit() invoke this class, which in turn executes call() method, which will process a file.
public class ProcessMutlipartFile implements Callable<String>
{
private MultipartFile file;
private String temp;
private MotorUploadService motUploadService;
public ProcessMutlipartFile(MultipartFile file,String temp, MotorUploadService motUploadService )
{
this.file=file;
this.temp=temp;
this.motUploadService=motUploadService;
}
public String call() throws Exception
{
return motUploadService.csvUpload(temp, file);
}
}
Below is MotorUploadService class, where I'm processing uploaded CSV file, line by line and then calling validateCsvData() method to validate Data,
which returns ErrorObject having line number and Errors associated with it.
if csvErrorRecords is null, then error-free and proceed with saving to Db.
else save errorList to Db and return Upload Failure.
#Component
public class MotorUploadService {
#Value("${external.resource.folder}")
String resourceFolder;
public String csvUpload(String parentPkId, MultipartFile file) {
String OUT_PATH = resourceFolder;
try {
DateFormat df = new SimpleDateFormat("yyyyMMddhhmmss");
String filename = file.getOriginalFilename().split(".")[0] + df.format(new Date()) + file.getOriginalFilename().split(".")[1];
Path path = Paths.get(OUT_PATH,fileName)
Files.copy(file.getInputStream(), path, StandardCopyOption.REPLACE_EXISTING);
}
catch(IOException e){
e.printStackTrace();
return "Failed to Upload File...try Again";
}
List<TxnMpMotSlaveRaw> txnMpMotSlvRawlist = new ArrayList<TxnMpMotSlaveRaw>();
try {
BufferedReader br = new BufferedReader(new InputStreamReader(file.getInputStream()));
String line = "";
int header = 0;
int lineNum = 1;
TxnMpSlaveErrorNew txnMpSlaveErrorNew = new TxnMpSlaveErrorNew();
List<CSVErrorRecords> errList = new ArrayList<CSVErrorRecords>();
while ((line = br.readLine()) != null) {
// TO SKIP HEADER
if (header == 0) {
header++;
continue;
}
lineNum++;
header++;
// Use Comma As Separator
String[] csvDataSet = line.split(",");
CSVErrorRecords csvErrorRecords = validateCsvData(lineNum, csvDataSet);
System.out.println("Errors from csvErrorRecords is " + csvErrorRecords);
if (csvErrorRecords.equals(null) || csvErrorRecords.getRecordNo() == 0) {
//Function to Save to Db
} else {
// add to errList
continue;
}
}
if (txnMpSlaveErrorNew.getErrRecord().size() == 0) {
//save all
return "Successfully Uploaded " + file.getOriginalFilename();
}
else {
// save the error in db;
return "Failure as it contains Faulty Information" + file.getOriginalFilename();
}
} catch (IOException ex) {
ex.printStackTrace();
return "Failure Uploaded " + file.getOriginalFilename();
}
}
private TxnMpMotSlaveRaw saveCsvData(String[] csvDataSet, String parentPkId) {
/*
Mapping csvDataSet to PoJo
returning Mapped Pojo;
*/
}
private CSVErrorRecords validateCsvData(int lineNum, String[] csvDataSet) {
/*
Logic for Validation goes here
*/
}
}
How to make it as a factory design pattern from controller,
so that if
parentPkId='Motor' call MotorUploadService,
parentPkId='Heal' call HealUploadService
I'm not so aware of the Factory Design pattern, please help me out.
Thanks in advance.

If I understood the question, in essence you would create an interface, and then return a specific implementation based upon the desired type.
So
public interface UploadService {
void csvUpload(String temp, MultipartFile file) throws IOException;
}
The particular implementations
public class MotorUploadService implements UploadService
{
public void csvUpload(String temp, MultipartFile file) {
...
}
}
public class HealUploadService implements UploadService
{
public void csvUpload(String temp, MultipartFile file) {
...
}
}
Then a factory
public class UploadServiceFactory {
public UploadService getService(String type) {
if ("Motor".equals(type)) {
return new MotorUploadService();
}
else if ("Heal".equals(type)) {
return new HealUploadService();
}
}
}
The factory might cache the particular implementations. One can also use an abstract class rather than an interface if appropriate.
I think you currently have a class UploadService but that is really the MotorUploadService if I followed your code, so I would rename it to be specific.
Then in the controller, presumably having used injection for the UploadServiceFactory
...
for (MultipartFile f : files) {
UploadService uploadSrvc = uploadServiceFactory.getService(parentPkId);
compService.submit(new ProcessMutlipartFile(f ,parentPkId,uploadService));
futureList.add(compService.take());
}
So with some additional reading in your classes:
public class ProcessMutlipartFile implements Callable<String>
{
private MultipartFile file;
private String temp;
private UploadService uploadService;
// change to take the interface UploadService
public ProcessMutlipartFile(MultipartFile file,String temp, UploadService uploadService )
{
this.file=file;
this.temp=temp;
this.uploadService=uploadService;
}
public String call() throws Exception
{
return uploadService.csvUpload(temp, file);
}
}

Java: accessing a List of Strings as an InputStream

Is there any way InputStream wrapping a list of UTF-8 String? I'd like to do something like:
InputStream in = new XyzInputStream( List<String> lines )

You can read from a ByteArrayOutputStream and you can create your source byte[] array using a ByteArrayInputStream.
So create the array as follows:
List<String> source = new ArrayList<String>();
source.add("one");
source.add("two");
source.add("three");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
for (String line : source) {
baos.write(line.getBytes());
}
byte[] bytes = baos.toByteArray();
And reading from it is as simple as:
InputStream in = new ByteArrayInputStream(bytes);
Alternatively, depending on what you're trying to do, a StringReader might be better.

You can concatenate all the lines together to create a String then convert it to a byte array using String#getBytes and pass it into ByteArrayInputStream. However this is not the most efficient way of doing it.

In short, no, there is no way of doing this using existing JDK classes. You could, however, implement your own InputStream that read from a List of Strings.
EDIT: Dave Web has an answer above, which I think is the way to go. If you need a reusable class, then something like this might do:
public class StringsInputStream<T extends Iterable<String>> extends InputStream {
private ByteArrayInputStream bais = null;
public StringsInputStream(final T strings) throws IOException {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
for (String line : strings) {
outputStream.write(line.getBytes());
}
bais = new ByteArrayInputStream(outputStream.toByteArray());
}
#Override
public int read() throws IOException {
return bais.read();
}
#Override
public int read(byte[] b) throws IOException {
return bais.read(b);
}
#Override
public int read(byte[] b, int off, int len) throws IOException {
return bais.read(b, off, len);
}
#Override
public long skip(long n) throws IOException {
return bais.skip(n);
}
#Override
public int available() throws IOException {
return bais.available();
}
#Override
public void close() throws IOException {
bais.close();
}
#Override
public synchronized void mark(int readlimit) {
bais.mark(readlimit);
}
#Override
public synchronized void reset() throws IOException {
bais.reset();
}
#Override
public boolean markSupported() {
return bais.markSupported();
}
public static void main(String[] args) throws Exception {
List source = new ArrayList();
source.add("foo ");
source.add("bar ");
source.add("baz");
StringsInputStream<List<String>> in = new StringsInputStream<List<String>>(source);
int read = in.read();
while (read != -1) {
System.out.print((char) read);
read = in.read();
}
}
}
This basically an adapter for ByteArrayInputStream.

You can create some kind of IterableInputStream
public class IterableInputStream<T> extends InputStream {
public static final int EOF = -1;
private static final InputStream EOF_IS = new InputStream() {
#Override public int read() throws IOException {
return EOF;
}
};
private final Iterator<T> iterator;
private final Function<T, byte[]> mapper;
private InputStream current;
public IterableInputStream(Iterable<T> iterable, Function<T, byte[]> mapper) {
this.iterator = iterable.iterator();
this.mapper = mapper;
next();
}
#Override
public int read() throws IOException {
int n = current.read();
while (n == EOF && current != EOF_IS) {
next();
n = current.read();
}
return n;
}
private void next() {
current = iterator.hasNext()
? new ByteArrayInputStream(mapper.apply(iterator.next()))
: EOF_IS;
}
}
To use it
public static void main(String[] args) throws IOException {
Iterable<String> strings = Arrays.asList("1", "22", "333", "4444");
try (InputStream is = new IterableInputStream<String>(strings, String::getBytes)) {
for (int b = is.read(); b != -1; b = is.read()) {
System.out.print((char) b);
}
}
}

In my case I had to convert a list of string in the equivalent file (with a line feed for each line).
This was my solution:
List<String> inputList = Arrays.asList("line1", "line2", "line3");
byte[] bytes = inputList.stream().collect(Collectors.joining("\n", "", "\n")).getBytes();
InputStream inputStream = new ByteArrayInputStream(bytes);

You can do something similar to this:
https://commons.apache.org/sandbox/flatfile/xref/org/apache/commons/flatfile/util/ConcatenatedInputStream.html
It just implements the read() method of InputStream and has a list of InputStreams it is concatenating. Once it reads an EOF it starts reading from the next InputStream. Just convert the Strings to ByteArrayInputStreams.

you can also do this way create a Serializable List
List<String> quarks = Arrays.asList(
"up", "down", "strange", "charm", "top", "bottom"
);
//serialize the List
//note the use of abstract base class references
try{
//use buffering
OutputStream file = new FileOutputStream( "quarks.ser" );
OutputStream buffer = new BufferedOutputStream( file );
ObjectOutput output = new ObjectOutputStream( buffer );
try{
output.writeObject(quarks);
}
finally{
output.close();
}
}
catch(IOException ex){
fLogger.log(Level.SEVERE, "Cannot perform output.", ex);
}
//deserialize the quarks.ser file
//note the use of abstract base class references
try{
//use buffering
InputStream file = new FileInputStream( "quarks.ser" );
InputStream buffer = new BufferedInputStream( file );
ObjectInput input = new ObjectInputStream ( buffer );
try{
//deserialize the List
List<String> recoveredQuarks = (List<String>)input.readObject();
//display its data
for(String quark: recoveredQuarks){
System.out.println("Recovered Quark: " + quark);
}
}
finally{
input.close();
}
}
catch(ClassNotFoundException ex){
fLogger.log(Level.SEVERE, "Cannot perform input. Class not found.", ex);
}
catch(IOException ex){
fLogger.log(Level.SEVERE, "Cannot perform input.", ex);
}

I'd like to propose my simple solution:
public class StringListInputStream extends InputStream {
private final List<String> strings;
private int pos = 0;
private byte[] bytes = null;
private int i = 0;
public StringListInputStream(List<String> strings) {
this.strings = strings;
this.bytes = strings.get(0).getBytes();
}
#Override
public int read() throws IOException {
if (pos >= bytes.length) {
if (!next()) return -1;
else return read();
}
return bytes[pos++];
}
private boolean next() {
if (i + 1 >= strings.size()) return false;
pos = 0;
bytes = strings.get(++i).getBytes();
return true;
}
}

Configuring Spring MVC controller to send file to client

I think my scenario is pretty common. I have a database and I want my Spring MVC app to accept a request in the controller, invoke the DB service to get data and send that data to the client as a CSV file. I'm using the JavaCSV library found here to assist in the process: http://sourceforge.net/projects/javacsv/
I've found several examples of people doing similar things and cobbled together something that looks correct-ish. When I hit the method, though, nothing is really happening.
I thought writing the data to the HttpServletResponse's outputStream would be sufficient, but apparently, I'm missing something.
Here's my controller code:
#RequestMapping(value="/getFullData.html", method = RequestMethod.GET)
public void getFullData(HttpSession session, HttpServletRequest request, HttpServletResponse response) throws IOException{
List<CompositeRequirement> allRecords = compReqServ.getFullDataSet((String)session.getAttribute("currentProject"));
response.setContentType("data:text/csv;charset=utf-8");
response.setHeader("Content-Disposition","attachment; filename=\yourData.csv\"");
OutputStream resOs= response.getOutputStream();
OutputStream buffOs= new BufferedOutputStream(resOs);
OutputStreamWriter outputwriter = new OutputStreamWriter(buffOs);
CsvWriter writer = new CsvWriter(outputwriter, '\u0009');
for(int i=1;i <allRecords.size();i++){
CompositeRequirement aReq=allRecords.get(i);
writer.write(aReq.toString());
}
outputwriter.flush();
outputwriter.close();
};
What step am I missing here? Basically, the net effect is... nothing. I would have thought setting the header and content type would cause my browser to pick up on the response and trigger a file download action.

It seems to be because your Content-type is set incorrectly, it should be response.setContentType("text/csv;charset=utf-8") instead of response.setContentType("data:text/csv;charset=utf-8").
Additionally, if you are using Spring 3, you should probably use a #ResponseBody HttpMessageConverter for code reuse. For example:
In the controller:
#RequestMapping(value = "/getFullData2.html", method = RequestMethod.GET, consumes = "text/csv")
#ResponseBody // indicate to use a compatible HttpMessageConverter
public CsvResponse getFullData(HttpSession session) throws IOException {
List<CompositeRequirement> allRecords = compReqServ.getFullDataSet((String) session.getAttribute("currentProject"));
return new CsvResponse(allRecords, "yourData.csv");
}
plus a simple HttpMessageConverter:
public class CsvMessageConverter extends AbstractHttpMessageConverter<CsvResponse> {
public static final MediaType MEDIA_TYPE = new MediaType("text", "csv", Charset.forName("utf-8"));
public CsvMessageConverter() {
super(MEDIA_TYPE);
}
protected boolean supports(Class<?> clazz) {
return CsvResponse.class.equals(clazz);
}
protected void writeInternal(CsvResponse response, HttpOutputMessage output) throws IOException, HttpMessageNotWritableException {
output.getHeaders().setContentType(MEDIA_TYPE);
output.getHeaders().set("Content-Disposition", "attachment; filename=\"" + response.getFilename() + "\"");
OutputStream out = output.getBody();
CsvWriter writer = new CsvWriter(new OutputStreamWriter(out), '\u0009');
List<CompositeRequirement> allRecords = response.getRecords();
for (int i = 1; i < allRecords.size(); i++) {
CompositeRequirement aReq = allRecords.get(i);
writer.write(aReq.toString());
}
writer.close();
}
}
and a simple object to bind everything together:
public class CsvResponse {
private final String filename;
private final List<CompositeRequirement> records;
public CsvResponse(List<CompositeRequirement> records, String filename) {
this.records = records;
this.filename = filename;
}
public String getFilename() {
return filename;
}
public List<CompositeRequirement> getRecords() {
return records;
}
}

Based on Pierre answer, i did a converter. Here is the full code, that works with any Object passed:
TsvMessageConverter.java
public class TsvMessageConverter extends AbstractHttpMessageConverter<TsvResponse> {
public static final MediaType MEDIA_TYPE = new MediaType("text", "tsv", Charset.forName("utf-8"));
private static final Logger logger = LoggerFactory.getLogger(TsvMessageConverter.class);
public TsvMessageConverter() {
super(MEDIA_TYPE);
}
protected boolean supports(Class<?> clazz) {
return TsvResponse.class.equals(clazz);
}
#Override
protected TsvResponse readInternal(Class<? extends TsvResponse> clazz, HttpInputMessage inputMessage) throws IOException, HttpMessageNotReadableException {
return null;
}
protected void writeInternal(TsvResponse tsvResponse, HttpOutputMessage output) throws IOException, HttpMessageNotWritableException {
output.getHeaders().setContentType(MEDIA_TYPE);
output.getHeaders().set("Content-Disposition", "attachment; filename=\"" + tsvResponse.getFilename() + "\"");
final OutputStream out = output.getBody();
writeColumnTitles(tsvResponse, out);
if (tsvResponse.getRecords() != null && tsvResponse.getRecords().size() != 0) {
writeRecords(tsvResponse, out);
}
out.close();
}
private void writeRecords(TsvResponse response, OutputStream out) throws IOException {
List<String> getters = getObjectGetters(response);
for (final Object record : response.getRecords()) {
for (String getter : getters) {
try {
Method method = ReflectionUtils.findMethod(record.getClass(), getter);
out.write(method.invoke(record).toString().getBytes(Charset.forName("utf-8")));
out.write('\t');
} catch (IllegalAccessException | InvocationTargetException e) {
logger.error("Erro ao transformar em CSV", e);
}
}
out.write('\n');
}
}
private List<String> getObjectGetters(TsvResponse response) {
List<String> getters = new ArrayList<>();
for (Method method : ReflectionUtils.getAllDeclaredMethods(response.getRecords().get(0).getClass())) {
String methodName = method.getName();
if (methodName.startsWith("get") && !methodName.equals("getClass")) {
getters.add(methodName);
}
}
sort(getters);
return getters;
}
private void writeColumnTitles(TsvResponse response, OutputStream out) throws IOException {
for (String columnTitle : response.getColumnTitles()) {
out.write(columnTitle.getBytes());
out.write('\t');
}
out.write('\n');
}
}
TsvResponse.java
public class TsvResponse {
private final String filename;
private final List records;
private final String[] columnTitles;
public TsvResponse(List records, String filename, String ... columnTitles) {
this.records = records;
this.filename = filename;
this.columnTitles = columnTitles;
}
public String getFilename() {
return filename;
}
public List getRecords() {
return records;
}
public String[] getColumnTitles() {
return columnTitles;
}
}
And on SpringContext.xml add the following:
<mvc:annotation-driven>
<mvc:message-converters register-defaults="true">
<bean class="com.mypackage.TsvMessageConverter"/>
</mvc:message-converters>
</mvc:annotation-driven>
So, you can use on your controller like this:
#RequestMapping(value="/tsv", method= RequestMethod.GET, produces = "text/tsv")
#ResponseBody
public TsvResponse tsv() {
return new TsvResponse(myListOfPojos, "fileName.tsv",
"Name", "Email", "Phone", "Mobile");
}