I have DB2 database with XML column. I would like to read data from it and save each XML to separate file.
Here is a part of my code:
final List<Map<String, Object>> myList = dbcManager.createQuery(query).getResultList();
int i=0;
for (final Map<String, Object> element : myList) {
i++;
String filePath = "C://elements//elem_" + i + ".xml";
File file = new File(filePath);
if(!file.exists()){
file.createNewFile();
}
BufferedWriter out = new BufferedWriter(new FileWriter(filePath));
out.write(element.get("columnId"));
out.close();
}
Now, I have error in line out.write(element.get("columnId"));, because element.get("columnId") is an object type and it should be for example string.
And my question is: To which type should I convert (cast) element.get("columnId") to save it in xml file?
You should use the ResultSet.getSQLXML() method to read the XML column value, then use an appropriate method of the SQLXML class, e.g. getString() or getCharacterStream(). More info here.
Related
I am trying to implement an DMN (Decision Model and Notation) evaluation service, where the user can upload a csv file with test cases to be evaluated and receive results also as a csv file for every test cases in the input file.
Reading the input csv file and evaluating the test cases works without problems. But I have some issues in writing the results to a csv file using OpenCsv.
Here is the mapped bean, which should be converted to csv row:
#Data
#AllArgsConstructor
#NoArgsConstructor
public class DmnTestCaseResult {
private Map<String, Object> testInput;
private Map<String, Object> expectedOutput;
private List<Map<String, Object>> testOutput;
private String errorMessage;
}
As you can see here, the test case result can have in some situations multiple testOutputs, defined as a list of map.
What I want is to write for every map entry in the testOutput, a seperate row in the csv file. But with the code I wrote below, only the first entry of the testOutput is written as only one row in the csv file.
public String convertDmnRuleTestResultToCsv(DmnRuleTestResult result) {
List<DmnTestCaseResult> results = result.getTestCases();
try(StringWriter sw = new StringWriter(); CSVWriter writer = new CSVWriter(sw, CSVWriter.DEFAULT_SEPARATOR, CSVWriter.NO_QUOTE_CHARACTER, CSVWriter.NO_ESCAPE_CHARACTER, CSVWriter.DEFAULT_LINE_END)) {
StatefulBeanToCsv<DmnTestCaseResult> beanToCsv = new StatefulBeanToCsvBuilder<DmnTestCaseResult>(writer)
.withApplyQuotesToAll(false)
.build();
beanToCsv.write(results);
return sw.toString();
} catch(Exception ex){
throw new CsvParseException(ex.getMessage());
}
}
How can I tell the OpenCsv that it should create seperate row for each entry in the testOutputs ?
EDIT: Added more information
UI:
Resulted incorrect CSV:
Expected correct CSV:
As you can see from the screenshots, one input can have multiple test outputs. Therefore I want to create for every test output a seperate line in csv file.
As StatefulBeanToCsv does not seem to be capable to generating multiple lines for a single bean, I suggest implementing a custom mapping function. This also requires you to manually print the header line as well.
public static String convertDmnRuleTestResultToCsv(DmnRuleTestResult result) {
List<DmnTestCaseResult> results = result.getTestCases();
try (StringWriter sw = new StringWriter();
CSVWriter writer = new CSVWriter(sw, CSVWriter.DEFAULT_SEPARATOR,
CSVWriter.NO_QUOTE_CHARACTER, CSVWriter.NO_ESCAPE_CHARACTER,
CSVWriter.DEFAULT_LINE_END)) {
writeHeader(writer);
for (DmnTestCaseResult r : results) {
for (Map<String, Object> map : r.getTestOutput())
writer.writeNext(map(r, map));
}
return sw.toString();
} catch (Exception ex) {
throw new RuntimeException(ex.getMessage());
}
}
private static void writeHeader(CSVWriter writer) {
List<String> header = new ArrayList<>();
header.add("ERRORMESSAGE");
header.add("EXPECTEDOUTPUT");
header.add("INPUT");
header.add("OUTPUT");
writer.writeNext(header.toArray(new String[] {}));
}
private static String[] map(DmnTestCaseResult r, Map<String, Object> testOutput) {
// you can manually adjust formats here as well; entrySet() call can be left out, it does change the format. do what you like more
List<String> line = new ArrayList<>();
line.add(r.getErrorMessage());
line.add(r.getExpectedOutput().entrySet().toString());
line.add(r.getTestInput().entrySet().toString());
line.add(testOutput.entrySet().toString());
return line.toArray(new String[] {});
}
And this prints:
ERRORMESSAGE,EXPECTEDOUTPUT,INPUT,OUTPUT
errorMessage,[expectedOutput1=expectedOutput1, expectedOutput2=expectedOutput2],[input2=testInput2, input1=testInput1],[testOut2=testOut2, testOut=testOut1]
errorMessage,[expectedOutput1=expectedOutput1, expectedOutput2=expectedOutput2],[input2=testInput2, input1=testInput1],[testOut3=testOut3, testOut4=testOut4]
i write a nifi custom processor that handle xlsx to csv , and in my code i generate a string csv like this from xlsx, but i don´t know how to send it as csv, and the other processors like inferavroschema
from csv can recognize the inpustream as csv.
final AtomicReference<String> value = new AtomicReference<>();
String csvToString= "
name,age,info
javo,23,wasa
pepe,34,lima"
value.set(csvToString);
And in my code i send it in this way, but when i see the data provenance it apears as content type text:
String results = value.get();
if(results != null && !results.isEmpty()){
flowFile = session.putAttribute(flowFile, "csv", results);
}
flowFile = session.write(flowFile, new OutputStreamCallback() {
#Override
public void process(OutputStream out) throws IOException {
out.write(value.get().getBytes());
}
});
session.transfer(flowFile, SUCCESS);
The attribute name for content type in nifi named mime.type
So, you have to set its value to csv.
flowFile = session.putAttribute(flowFile, "mime.type" "csv");
I have a class that try to read a CSV file using Apache Common CSV, so far my code is working fine except that am not getting the result am expecting.
My code is displaying a duplicate of the second column in the csv file as below:
support#gmail.com
google
google.com
support#gmail.com
google
tutorialspoint
info#tuto.com
google
My CSV File
Name,User Name,Password
google.com,support#gmail.com,google
tutorialspoint,info#tuto.com,google
i expect to get something like this:
google.com
support#gmail.com
google
tutorialspoint
info#tuto.com
google
Here is my block that parses the csv using Apache CSV
public List<String> readCSV(String[] fields) {
// HERE WE START PROCESSING THE READ CSV CONTENTS
List<String> contents = new ArrayList<String>();
FileReader fileReader = null;
CSVParser csvFileParser = null;
// HERE WE START PROCESSING
if(fields!=null){
//Create the CSVFormat object with the header mapping
CSVFormat csvFileFormat = CSVFormat.DEFAULT.withHeader(FILE_HEADER_MAPPING);
try {
//Create a new list of student to be filled by CSV file data
List<String> content=new ArrayList<String>();
//initialize FileReader object
fileReader = new FileReader(FilePath);
//initialize CSVParser object
csvFileParser = new CSVParser(fileReader, csvFileFormat);
//Get a list of CSV file records
List<CSVRecord> csvRecords = csvFileParser.getRecords();
//Read the CSV file records starting from the second record to skip the header
for (int i = 1; i < csvRecords.size(); i++) {
CSVRecord record = csvRecords.get(i);
//Create a new student object and fill his data
for(int j=0; j<fields.length; j++){
content.add(record.get(fields[j]));
}
// Here we submit to contents
contents.addAll(content);
System.out.println(contents.size());
} // end of loop
}
catch (Exception e) {
e.printStackTrace();
} finally {
try {
fileReader.close();
csvFileParser.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
// Here we return
return contents;
}
I cant just figure out what am missing here, any help will be welcomed.
The reason is that you're adding the String list content each iteration
contents.addAll(content);
Either clear content on each iteration or just change
content.add(record.get(fields[j]));
to
contents.add(record.get(fields[j]));
and remove the
contents.addAll(content);
line
I'm inclined to use CSVRecord because it can be used to map with a header and get the corresponding value. My application frequently uses CSVRecord class. However, I cannot instantiate the CSVRecord. I would prefer not to modify the source/create a new class since it already provides a parser that returns CSVRecord. I have got a list of strings (header as well as the values) that needed to be converted to the CSVRecord type. Is there a direct way that this can be done without going around with formatting and then parsing back? Like the one below:
private CSVRecord format(List<String> header, List<String> values)
{
CSVFormat csvFormat = CSVFormat.DEFAULT.withRecordSeparator(System.lineSeparator())
.withQuoteMode(QuoteMode.ALL);
CSVRecord csvRecord = null;
final StringWriter out = new StringWriter();
try (CSVPrinter csvPrinter = new CSVPrinter(out, csvFormat);)
{
csvPrinter.printRecord(values);
String value = out.toString().trim();
for (CSVRecord r : CSVParser.parse(value, csvFormat.withHeader(header.toArray(new String[header.size()]))))
csvRecord = r;
}
catch (IOException e)
{
logger.error("Unable to format the Iterable to CSVRecord. Header: [{}]; Values: [{}]", e,
String.join(", ", header), String.join(", ", values));
}
return csvRecord;
}
private void testMethod() throws Exception
{
List<String> header = Arrays.asList("header1", "header2", "header3");
List<String> record = Arrays.asList("val1", "val2", "val3");
CSVRecord csvRecord = format(header, record);
logger.info("{}", csvRecord.get("header2"));
}
You could pass the list as a string directly into the CSVParser instead of creating a writer.
CSVRecord csvr = CSVParser.parse(
values.stream().collect(Collectors.joining(","))
,csvFormat.withHeader(header.toArray(new String[header.size()])))
.getRecords().get(0);
The BeanIO and SimpleFlatMapper are way better at solving this problem. BeanIO uses a Map data structure and a config file to declare how the CSV file should be structured so it is very powerful. SimpleFlatMapper will take you POJO properties as the heading names by default and output the property values are column values.
BeanIO
http://beanio.org/2.1/docs/reference/index.html#CSVStreamFormat
SimpleFlatMapper
http://simpleflatmapper.org/
CsvParser
.mapTo(MyObject.class)
.stream(reader)
.forEach(System.out::println);
I have a CSV files (excel) which has data in it and i need to parse the data using java.
the data in those files doesn't separated using comma,the CSV files has number of columns and number of rows(each cell has data) where all the data is written.
i need to go through on all the files until i get to the EOF(end of file)of each file and parse the data.
the files contains also empty rows in it so empty row is not a criteria to stop parsing,i think only EOF will indicate that i've reached to the end of the specific file.
many thanks.
You can use opencsv to parse the excel CSV. I've used this myself, all you need to do is split on the ';'. Empty cells will be parsed aswell.
You can find info here : http://opencsv.sourceforge.net/
And to parse the excelCSV you can do:
CSVReader reader = new CSVReader(new FileReader("yourfile.csv"), ';');
Aside from other suggestions, I would offer Jackson CSV module. Jackson has very powerful data-binding functionality, and CSV module allows reading/writing as CSV as an alternative to JSON (or XML, YAML, and other supported formats). So you can also do conversions between other data formats, in addition to powerful CSV-to/from-POJO binding.
Please have a Stream Object to read the CSV file.
FileInputStream fis = new FileInputStream("FileName.CSV");
BufferedInputStream bis = new BufferedInputStream(fis);
InputStreamReader isr = new InputStreamReader(bis);
Read an inputstream Object and store the file in String object.
Then using StringTokenizer with ,[comma] as delimeter -->you will get the tokens
Please manipulate the token to get the value.
String str = "This is String , split by StringTokenizer, created by mkyong";
StringTokenizer st = new StringTokenizer(str);
System.out.println("---- Split by space ------");
while (st.hasMoreElements()) {
System.out.println(st.nextElement());
}
System.out.println("---- Split by comma ',' ------");
StringTokenizer st2 = new StringTokenizer(str, ",");
while (st2.hasMoreElements()) {
System.out.println(st2.nextElement());
}
Thanks,
Pavan
Suppose you have a csv fileContent in form of string:
String fileContent;
Generally, the CSV fileContent are parsed into List>.
final List<String> rows = new ArrayList<String>(Lists.newArraysList(fileContent.split("[\\r\\n]+")));
Split the file into List of rows.
Then use CSVParser of OpenCSV and parse the comma separated line into List
final CSVParser parser = new CSVParser();
final List<List<String>> csvDetails = new ArrayList<List<String>>();
rows.forEach(t -> {
try {
csvDetails.add(Lists.newArrayList(parser.parseLine(t)));
} catch (Exception e) {
throw new RunTimeException("Exception occurred while parsing the data");
}
});