We Keep Coding

Write ResultList from DB into a CSV file in Java

I need to write my query.getResultList() into a .CSV file.
I call the query over this:
final Query q = em.createNamedQuery("getalljobs");
final List<Job> joblist = q.getResultList();
and the Namequery just do SELECT * FROM TABLE, the result of query.getResultList() looks like this:
[id;name, id;name, ... ]
I can't use OpenCSV.
The CSV file needs to have headers.

You can use something like
Query q = em.createNamedQuery("getalljobs");
List<Job> jobList = q.getResultList();
String csvHeader = getHeader();
try (PrintWriter fw = new PrintWriter(new FileWriter("output.csv"))) {
fw.println(csvHeader);
for(String line : jobList){
fw.println(line);
}
}

Solution
Write line for line in the file with StringBuidler.append. To this add your comma and a new line after each row.

Make custom code to reduce number of repetitive lines

I have to get 'tags' from the database and store them in an array so I could check if my document contains them. Due to the number of tag categories (customers, system_dependencies, keywords) I have multiple arrays to compare my document with. Is there an easy way to simplify and make my code look nicer?
This is my approach but it looks terrible with all the repetitive for loops.
ArrayList<String> KEYWORDS2 = new ArrayList<String>();
ArrayList<String> CUSTOMERS = new ArrayList<String>();
ArrayList<String> SYSTEM_DEPS = new ArrayList<String>();
ArrayList<String> MODULES = new ArrayList<String>();
ArrayList<String> DRIVE_DEFS = new ArrayList<String>();
ArrayList<String> PROCESS_IDS = new ArrayList<String>();
while (resultSet2.next()) {
CUSTOMERS.add(resultSet2.getString(1));
}
sql = "SELECT da_tag_name FROM da_tags WHERE da_tag_type_id = 6";
stmt = conn.prepareStatement(sql);
resultSet2 = stmt.executeQuery();
while (resultSet2.next()) {
SYSTEM_DEPS.add(resultSet2.getString(1));
}
while (resultSet.next()) {
String da_document_id = resultSet.getString(1);
String file_name = resultSet.getString(2);
try {
if(file_name.endsWith(".docx") || file_name.endsWith(".docm")) {
System.out.println(file_name);
XWPFDocument document = new XWPFDocument(resultSet.getBinaryStream(3));
XWPFWordExtractor wordExtractor = new XWPFWordExtractor(document);
//Return what's inside the document
System.out.println("Keywords found in the document:");
for (String keyword : KEYWORDS) {
if (wordExtractor.getText().contains(keyword)) {
System.out.println(keyword);
}
}
System.out.println("\nCustomers found in the document:");
for (String customer : CUSTOMERS) {
if (wordExtractor.getText().contains(customer)) {
System.out.println(customer);
}
}
System.out.println("\nSystem dependencies found in the document:");
for (String systemDeps : SYSTEM_DEPS) {
if (wordExtractor.getText().contains(systemDeps)) {
System.out.println(systemDeps);
}
}
System.out.println("Log number: " + findLogNumber(wordExtractor));
System.out.println("------------------------------------------");
wordExtractor.close();
}
As you can see there are 3 more to come and this doesn't look good already. Maybe there's a way to compare all of them at the same time.
I have made another attempt at this creating this method:
public void genericForEachLoop(ArrayList<String> al, POITextExtractor te) {
for (String item : al) {
if (te.getText().contains(item)) {
System.out.println(item);
}
}
}
Then calling it like so: genericForEachLoop(MODULES, wordExtractor);
Any better solutions?

I've got two ideas to shorten this: first of all you can write a general for-loop in a separate method that has an ArrayList as a parameter. Then you pass it each of your ArrayLists successively, which would mean that at least you do not have to repeat the for-loops. Secondly, you can create an ArrayList of type ArrayList and store your ArrayLists inside it. Then you can iterate over the whole thing. Only apparent disadvantage of both ideas (or a combination of them) would be, that you need to name the variable for your query string alike for the search of each ArrayList.

What you could do is use a Map and an enum like this:
enum TagType {
KEYWORDS2(2), // or whatever its da_tag_type_id is
CUSTOMERS(4),
SYSTEM_DEPS(6),
MODULES(8),
DRIVE_DEFS(10),
PROCESS_IDS(12);
public final daTagTypeId; // this will be used in queries
TagType(int daTagTypeId) {
this.daTagTypeId = daTagTypeId;
}
}
Map<TagType, List<String>> tags = new HashMap<>();
XWPFDocument document = new XWPFDocument(resultSet.getBinaryStream(3));
XWPFWordExtractor wordExtractor = new XWPFWordExtractor(document);
for(TagType tagType : TagType.values()) {
tags.put(tagType, new ArrayList<>()); // initialize
String sql = String.format("SELECT da_tag_name FROM da_tags WHERE da_tag_type_id = %d", tagType.daTagTypeId); // build query
stmt = conn.prepareStatement(sql);
resultSet2 = stmt.executeQuery();
while(resultSet2.next()) { // fill from DB
tags.get(tagType).add(.add(resultSet2.getString(1)));
}
System.out.println(String.format("%s found in the document:", tags.get(tagType).name());
for (String tag : tags.get(tagType)) { // search in text
if (wordExtractor.getText().contains(tag)) {
System.out.println(keyword);
}
}
}
But at this point I'm not sure you need those lists at all:
enum TagType {
KEYWORDS2(2), // or whatever its da_tag_type_id is
CUSTOMERS(4),
SYSTEM_DEPS(6),
MODULES(8),
DRIVE_DEFS(10),
PROCESS_IDS(12);
public final daTagTypeId; // this will be used in queries
TagType(int daTagTypeId) {
this.daTagTypeId = daTagTypeId;
}
}
XWPFDocument document = new XWPFDocument(resultSet.getBinaryStream(3));
XWPFWordExtractor wordExtractor = new XWPFWordExtractor(document);
for(TagType tagType : TagType.values()) {
String sql = String.format("SELECT da_tag_name FROM da_tags WHERE da_tag_type_id = %d", tagType.daTagTypeId); // build query
stmt = conn.prepareStatement(sql);
resultSet2 = stmt.executeQuery();
System.out.println(String.format("%s found in the document:", tags.get(tagType).name());
while(result2.next()) {
String tag = resultSet2.getString(1);
if (wordExtractor.getText().contains(tag)) {
System.out.println(keyword);
}
}
}
This given I don't know where those resultSet is declared and initialised, nor where that resultSet2 is initialised.
Basically you just fetch tags for each type from DB and then directly search them in the text without storing them at first and then re-iterating the stored ones... I mean that's what the DB is there for.

Write List of Maps to a CSV

I have a standalone application, which connects to a SQL database and saves ResultSet in a list of Map. This is what I have so far:
List<Map<String, Object>> rows;
stmt = conn.createStatement();
Resultset rs = stmt.executeQuery(queryString);
ResultSetMetaData rsmd; //Properties of Resultset object and column count
while(rs.next){
Map<String, Object> rowResult = new HashMap<String, Object>(columnCount);
for(int i =1; i <=columnCount; i++){
rowResult.put(rsmd.getColumnName(i), rs.getObject(i));
}
rows.add(rowResult);
}
//WRITE TO CSV
String csv = "C:\\Temp\\data.csv";
CSVWriter writer = new CSVWriter(new FileWriter(csv));
//Write the record to file
writer.writeNext(rows);
//close the writer
writer.close();
How do I add this "rows" of List to a csv with columns? Any clues and suggestions. Your help is appreciated.

Since every record will have the same columns in the same order, then I would just use a List<List<Object>> for the rows.
For the headers, you don't need to get them on every row. Just get them once like so:
List<String> headers = new ArrayList<>();
for (int i = 1; i <= columnCount; i++ )
{
String colName = rsmd.getColumnName(i);
headers.add(colName);
}
Next, you can get the rows like this:
List<List<Object>> rows = new ArrayList<>();
while(rs != null && rs.next)
{
List<Object> row = new ArrayList<>();
for(int i =1; i <=columnCount; i++)
{
row.add(rs.getObject(i));
}
rows.add(row);
}
Finally, to create the CSV file, you can do this:
// create the CSVWriter
String csv = "C:\\Temp\\data.csv";
CSVWriter writer = new CSVWriter(new FileWriter(csv));
// write the header line
for (String colName : headers)
{
writer.write(colName);
}
writer.endRecord();
// write the data records
for (List<Object> row : rows)
{
for (Object o : row)
{
// handle nulls how you wish here
String val = (o == null) ? "null" : o.toString();
writer.write(val);
}
writer.endRecord();
}
// you should close the CSVWriter in a finally block or use a
// try-with-resources Statement
writer.close;
Note: In my code examples, I'm using Type Inference
See: Try-With-Resources Statement.

Honestly for what you are trying to do I would recommend you use the writeAll method in CSVWriter and pass in the ResultSet.
writer.writeAll(rs, true);
The second parameter is the include column names so the first row in your csv file will be the column names. Then when you read the file you can translate that back into your Map if you want to (though it will be all strings unless you know when you are reading it what the types are).
Hope that helps.
Scott :)

Java returning null pointer exception for SQL query that gets passed to JSP

I am working on a school assignment that required us to use SQL statements in Java code as well as use the LIKE operator for a search. In order to properly search I have to get a string from the user, and split the string by any delimiter, and then run the query like so:
SELECT * FROM movies WHERE (movies.title LIKE '%userInput%');
I then return this query in the form of an ArrayList.
Now, when I was testing it out. I originally tested it with no user input, and my query became: SELECT * FROM movies WHERE (movies.title LIKE '%%');. This gave me the correct results.
However when I put a title in there, all of the sudden I get a NullPointerException on this line:
if(title.equals("")) { return "(movies.title LIKE '%%') "; from this section of my code:
public String getSearchString(String title) {
if(title.equals("")) { return "(movies.title LIKE '%%') "; }
String ret = "(";
ArrayList<String> titleArray = Util.splitSearch(title);
for(int i = 0; i < titleArray.size() - 1; ++i) {
String temp = titleArray.get(i);
String stmt = "movies.title LIKE '%" + temp + "%' OR ";
ret += stmt;
}
String temp = "movies.title LIKE '%" + titleArray.get(titleArray.size() - 1) + "%')";
ret += temp;
return ret;
}
This is then called like so:
public List<Movie> listMovies(String title) throws SQLException {
List<Movie> search = new ArrayList<Movie>();
if(null != title && title.isEmpty()) { title = ""; }
ResultSet res = queryMovies(getSearchString(title));
while(res.next()) {
Movie mov = new Movie();
mov.setTitle(res.getString("title"));
search.add(mov);
}
return search;
}
private static queryMovies(String st) throws SQLException {
ResultSet res = null;
try {
PreparedStatement ps = dbcon.prepareStatement(st);
res = ps.executeQuery();
} catch(SQLException e) {
e.printStackTrace();
}
return res;
}
I unfortunately have to do this since I won't know how much a user will enter. And I am also not allowed to use external libraries that make the formatting easier. For reference my Util.splitSearch(...) method looks like this. It should be retrieving anything that is a alphanumeric character and should be splitting on anything that is not alphanumeric:
public static ArrayList<String> splitSearch(String str) {
String[] strArray = str.split("[^a-zA-Z0-9']");
return new ArrayList(Arrays.asList(strArray));
}
What is interesting is when I pass in getSearchString(""); explicitly, I do not get a NullPointerException. It is only when I allows the variable title to be used do I get one. And I still get one when no string is entered.
Am I splitting the String wrong? Am I somehow giving SQL the wrong statement? Any help would be appreciated, as I am very new to this.

the "title" which is passed from input is null, hence you're getting nullpointerexception when you do title.equals("").
Best practices suggest you do a null check like (null != title && title.equals("")).
You can also do "".equals(title)

GAE Query with Collection Parameter

I have verified that the entity I am looking for is in the datastore. I have verified that the list I pass as a method parameter contains this entity. I am trying to find all objects that have their 'userGmail' contained in the list of strings I pass.
Here is my code
#SuppressWarnings("unchecked")
#ApiMethod(name = "findFriendsByEmailList")
public CollectionResponse<ZeppaUser> findFriendsByEmailList(
#Named("emailsList") List<String> emailsList, User user)
throws OAuthRequestException {
if (user == null) {
throw new OAuthRequestException(
"Null User Authorization Exception, findFriendsByEmailList");
}
PersistenceManager mgr = null;
List<ZeppaUser> execute = null;
Query query = null;
try {
mgr = getPersistenceManager();
query = mgr.newQuery(ZeppaUser.class);
query.declareParameters("java.util.List emailListParam");
query.setFilter("emailListParam.contains( userGmail )");
execute = (List<ZeppaUser>) query.execute(emailsList);
query.closeAll();
} finally {
mgr.close();
}
return CollectionResponse.<ZeppaUser> builder().setItems(execute)
.build();
}
This is the stack trace I receive from it:
Something worth noting: I do not receive this error on lists I pass in that to not contain an element found in the datastore. Just when it does exist which leads me to believe that the Query has located the element but has not been closed or executed into a return parameter correctly. If it is preferable to return List that is more than ok. I have tried multiple variations of this with no success thus far. It is getting quite frustrating.

Ok so I found a way around it.
Lists cannot be passed into ApiEndpoints. That or I didn't figure out the correct way to do it and would LOVE an update on the proper way to do this.
Instead, in my client, I construct a String of emails seperated by a comma and send a string into the parameter as an 'encoded' string list then 'decode' it upon execution. Works well but seems hacky.
here are the methods I used. This is convenient though because it works with iOS as well.
public static String encodeListString(ArrayList<String> stringList){
StringBuilder stringbuilder = new StringBuilder();
stringbuilder.append(stringList.get(0));
if(stringList.size() > 1){
for( int i = 0; i < stringList.size(); i++){
stringbuilder.append(",");
stringbuilder.append(stringList.get(i));
}
}
return stringbuilder.toString();
}
public static List<String> decodeListString(String encodedString){
char[] characters = encodedString.toCharArray();
StringBuilder stringbuilder = new StringBuilder();
int position = 0;
ArrayList<String> stringList = new ArrayList<String>();
while(true){
try {
char character = characters[position];
if(character == ','){
String resultString = stringbuilder.toString();
stringList.add(resultString);
stringbuilder = new StringBuilder(); // clear it
} else {
stringbuilder.append(character);
}
position++;
} catch (ArrayIndexOutOfBoundsException aiex){
// List ended
String resultString = stringbuilder.toString();
if(!resultString.isEmpty())
stringList.add(resultString);
break;
}
}
return stringList;
}