In my application I use a lot of CSV files which I have to read and build a lists based on them. I'd like to discover an easy way to do this. Do you know any easy framework which does it without using number of config files etc?
For instance, I have got a class Person:
public class Person {
String name;
String surname;
double shoeSize;
boolean sex; // true: male, false:female
public Person() {
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getSurname() {
return surname;
}
public void setSurname(String surname) {
this.surname = surname;
}
public double getShoeSize() {
return shoeSize;
}
public void setShoeSize(double shoeSize) {
this.shoeSize = shoeSize;
}
public boolean isSe) {
return sex;
}
public void setSeboolean sex) {
this.sex = sex;
}
}
For this class, I have prepared CSV file:
name,surname,shoesize,sex
Tom,Tommy,32,true
Anna,Anny,27,false
How can I do it easily?
One of the simplest ways to read and serialize data is by using the Jackson library.
It also has an extension for CSV, you can find the wiki here
Let's say you have a Pojo like this:
#JsonPropertyOrder({ "name", "surname", "shoesize", "gender" })
public class Person {
public String name;
public String surname;
public int shoesize;
public String gender;
}
And a CSV like this:
Tom,Tommy,32,m
Anna,Anny,27,f
Then reading it is done like so:
MappingIterator<Person> personIter = new CsvMapper().readerWithTypedSchemaFor(Person.class).readValues(csvFile);
List<Person> people = personIter.readAll();
This is simple enough for my taste, basically all you need to do is add the column order in your CSV file using the #JsonPropertyOrder annotation and then just read the file using the above 2 lines.
There are lot of good frameworks written in Java to parse a CSV file and form a List of Objects. OpenCSV, JSefa & jCSV are to name a few of them.
For your requirement, I believe jCSV suits the best. Below is the sample code from jCSV which you can make use of easily.
Reader reader = new FileReader("persons.csv");
CSVReader<Person> csvPersonReader = ...;
// read all entries at once
List<Person> persons = csvPersonReader.readAll();
// read each entry individually
Iterator<Person> it = csvPersonReader.iterator();
while (it.hasNext()) {
Person p = it.next();
// ...
}
Moreover, parsing a CSV file and converting it to a List isn't a big deal and it can be achieved without using any framework, as shown below.
br = new BufferedReader(new FileReader(csvFileToRead));
List<Person> personList = new ArrayList<>();
while ((line = br.readLine()) != null) {
// split on comma(',')
String[] personCsv = line.split(splitBy);
// create car object to store values
Person personObj = new Person();
// add values from csv to car object
personObj.setName(personCsv[0]);
personObj.setSurname(personCsv[1]);
personObj.setShoeSize(personCsv[2]);
personObj.setGender(personCsv[3]);
// adding car objects to a list
personList.add(personObj);
}
If the mapping of CSV columns to bean object is complex, repetitive or large in real case scenario, then it can be done easily by using DozerBeanMapper.
Hope this will help you.
Shishir
Not sure if you need to go as far as using an external library (and taking the usually implied performance hit). It's a pretty simple thing to implement. And if nothing else, it always helps to know what's going on behind the scenes in such a library:
public List<Person> readFile(String fileName) throws IOException {
List<Person> result = new ArrayList<Person>();
BufferedReader br = new BufferedReader(new FileReader(new File(fileName)));
try {
// Read first line
String line = br.readLine();
// Make sure file has correct headers
if (line==null) throw new IllegalArgumentException("File is empty");
if (!line.equals("name,surname,shoesize,sex"))
throw new IllegalArgumentException("File has wrong columns: "+line);
// Run through following lines
while ((line = br.readLine()) != null) {
// Break line into entries using comma
String[] items = line.split(",");
try {
// If there are too many entries, throw a dummy exception, if
// there are too few, the same exception will be thrown later
if (items.length>4) throw new ArrayIndexOutOfBoundsException();
// Convert data to person record
Person person = new Person();
person.setName ( items[0] );
person.setSurname ( items[1] );
person.setShoeSize(Double .parseDouble (items[2]));
person.setSex (Boolean.parseBoolean(items[3]));
result.add(person);
} catch (ArrayIndexOutOfBoundsException|NumberFormatException|NullPointerException e) {
// Caught errors indicate a problem with data format -> Print warning and continue
System.out.println("Invalid line: "+ line);
}
}
return result;
} finally {
br.close();
}
}
Note that the catch statement uses Java 7 multi-catch. For older Java versions, either split it into 3 catch blocks or replace ArrayIndexOutOfBoundsException|NumberFormatException|NullPointerException with Exception. The latter is usually discouraged as it masks and ignores all other exceptions as well, but in a simple example like this, the risk is probably not too high.
This answer, unfortunately, is specific to your problem, but given that it is very straight forward, it should be easy to adapt to other situations as well...
Another neat thing you can do is to match line inside the while loop with a regular expression rather than simply splitting it based on a comma. That way you could also implement data validation in one shot (for example only match a sensible number for shoe size).
Note that the above implementation doesn't work if you have names that contain commas which are then enclosed in quotes (like "Jackson, Jr." as a last name). You can cover this case "easily" if you use regular expressions as described above, or by checking the first letter of the last name and if it is a quotation mark, combine item[1] with item[2] and use item[3] and item[4] instead for the shoesize and sex. This special case will likely be covered by most of the external libraries suggested here, so if you're not worried about any dependencies, licensing issues, and performance hits, those might be the easier way out...
opencsv is a good and simple solution. It is a small but powerful library. You can download it from the opencsv website (direct download from sourceforge, use the jar in the deploy directory) or use maven.
The java bean mapping feature makes it really simple because your CSV column names are matching the property names of your class (it ignores the different capitalisation).
How to use it:
Reader reader = // ... reader for the input file
// let it map the csv column headers to properties
CsvToBean<Person> csvPersons = new CsvToBean<Person>();
HeaderColumnNameMappingStrategy<Person> strategy = new HeaderColumnNameMappingStrategy<Person>();
strategy.setType(Person.class);
// parse the file and get a list of persons
List<Person> persons = csvPersons.parse(strategy, reader);
That's all.
I solved this recently by using Immutables and Jackson, and I think it's a great way to go if you're willing to use these libraries.
Immutables and Jackson integrate very well. To take OP's example, all you'd have to do is specify the Immutables class like so (annotations qualified for snippet explicitness):
#org.immutables.value.Value.Immutable
#com.fasterxml.jackson.databind.annotation.JsonDeserialize(as = ImmutablePerson.class)
public interface Person {
String getName();
String getSurname();
double getShoeSize();
boolean getSex();
}
Then, using the Jackson CSV module, you can easily deserialize each row of the CSV into the class Immutables has generated for you:
List<Person> loadPeople(File personsCsvFile) throws IOException {
CsvSchema schema = CsvSchema.emptySchema().withHeader();
MappingIterator<Person> personsIterator = new CsvMapper()
.readerFor(Person.class)
.with(schema)
.readValues(personsCsvFile);
return personsIterator.readAll();
}
Use OpenCSV
Here is a complete example that reads entries and adds them to a List:
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.List;
import au.com.bytecode.opencsv.CSVReader;
public class CSVReaderImplementor {
private String fileName;
private CSVReader reader;
private List<String[]> entries;
public CSVReaderImplementor(String fileName) throws IOException, FileNotFoundException {
this.fileName = fileName;
reader = new CSVReader(new FileReader(this.fileName));
entries = reader.readAll();
}
public List getEntries() {
return entries;
}
public static void main(String[] args) throws FileNotFoundException, IOException {
CSVReaderImplementor cri = new CSVReaderImplementor("yourfile.csv");
for(int i = 0; i < 50; i++) {
System.out.println(cri.getEntries().get(i).toString());
}
}
}
A List of type String[] is returned. You can iterate through the String array for each entry in the list and use the values at each index to populate your Bean constructor.
I think SuperCSV + Dozer easy to use and quite robust for java bean CSV serialization
http://supercsv.sourceforge.net/dozer.html
Related
I'm working on a program for my Java class where I'm using a file of objects (clothing items) that represents inventory for a store. Each Retail_Item has four attributes: int itemNumber, String description, int numInInventory, and double price.
What I'm trying to figure out is how to read in each line from the file and turn each line into an object. My first thought was to create a while loop with vars like currentItemNumber, currentDescription, etc. So I tried this:
while (file.hasNextLine()) {
currentItemNumber = file.nextInt();
currentDescription = file.next
} // end while
But I got stuck there because every other time I've read in a String to a Scanner, I've always used nextLine. Can't use that here though, because each line contains multiple attributes of the object, not a String within a line. Is there a way to do this in the structure I'm trying to use, or should I be doing this a different way? I know I've seen and done some things where I parsed a String into separate pieces which I've seen people refer to as "tokens." Would people recommend reading each line in and then parsing it into separate tokens, then assigning each token to its appropriate attribute? Then I guess I'd have to cast those tokens into the appropriate object, since I think reading the whole line in and then parsing it would make each piece a String.
Here's a sample of what's in the text file (which can't be changed in any way, per the professor's instructions):
1000 Pants 10 19.99
2000 Jeans 2 25.95
3000 Shirt 12 12.50
Thanks in advance for your sage wisdom if you've got it.
The following code fulfills your requirement as stated in your question, namely how to create an instance of class RetailItem from a line of text from your text file. I presume it uses things that you may not have learned yet, like class Paths and try-with-resources. This is just used to scan through your file.
First, class RetailItem contains the members you described in your question. Next, I wrote a constructor for class RetailItem that creates a new instance and initializes the instance members. Then I wrote a toString() method that displays the contents of a RetailItem object in "human readable" form. Finally a main() method that reads your text file (which I named "clothes.txt"), line by line - using a Scanner. For each line read, the code splits it using a delimiter which consists of at least one whitespace character. (I presume you haven't yet learned about regular expressions in java.) Then I convert the elements of the String array returned by method split() into appropriate data types that are required by the RetailItem constructor. Then I call the constructor, thus creating an instance of class RetailItem (as you requested) and I print the created instance.
import java.io.IOException;
import java.nio.file.Paths;
import java.util.Scanner;
public class RetailItem {
private static final int FIELDS = 4;
private int itemNumber;
private String description;
private int numInInventory;
private double price;
public RetailItem(int itemNumber, String description, int numInInventory, double price) {
this.itemNumber = itemNumber;
this.description = description;
this.numInInventory = numInInventory;
this.price = price;
}
#Override // java.lang.Object
public String toString() {
return String.format("%4d %-5s %2d %2.2f", itemNumber, description, numInInventory, price);
}
public static void main(String[] args) {
try (Scanner file = new Scanner(Paths.get("clothes.txt"))) {
while (file.hasNextLine()) {
String record = file.nextLine();
String[] fields = record.split("\\s+");
if (fields.length == FIELDS) {
int itemNumber = Integer.parseInt(fields[0]);
String description = fields[1];
int numInInventory = Integer.parseInt(fields[2]);
double price = Double.parseDouble(fields[3]);
RetailItem item = new RetailItem(itemNumber, description, numInInventory, price);
System.out.println(item);
}
}
}
catch (IOException xIo) {
xIo.printStackTrace();
}
}
}
I think the way that I would do is, like you said, parse each line into separate strings and then assign each piece to instance variables of the object you are building.
I have done something like this before, maybe it can be helpful.
Scanner fileScan;
File babyNameFile = new File("yob2015.txt");
try {
fileScan = new Scanner(babyNameFile);
} catch (FileNotFoundException e) {
System.out.println("File does not exist");
return;
}
String currentLine;
int numberOfGirlsNames = 0;
while (fileScan.hasNextLine()) {
String[] values;
currentLine = fileScan.nextLine();
values = currentLine.split(",");
if (values[1].equals("F")) {
numberOfGirlsNames = numberOfGirlsNames+1;
}
}
System.out.println("Number of female names was "+numberOfGirlsNames);
I'm currently in a High School level Java course. I've been doing plenty of research here, on Stack Overflow, trying to work through a project i'm currently assigned. The project consists of making modifications to, and searching through, various words pulled from an encyclopedia file. This is what I am having trouble with, the very basic form of this project. I already found the method in which to solve this problem, but i wasn't able to find a good way of implementing it. This is a copy of the code i found here: (the third method down contains the portion i took from this site)
class word
{
public String newString;
EasyReader fileIn = new EasyReader("Encyclopedia.txt");
EasyWriter fileOut = new EasyWriter("writeHere.txt");
String fileName="Encyclopedia.txt";
private String onFile;
public word()
{
onFile="";
}
public word(String s)
{
onFile=s;
}
String file = "Encyclopedia.txt";
private String readFile(String file) throws IOException
{
BufferedReader reader = new BufferedReader(new FileReader(file));
String line=null;
StringBuilder stringBuilder = new StringBuilder();
String is=System.getProperty("line.seperator");
while((line=reader.readLine())!=null)
{
stringBuilder.append(line);
stringBuilder.append(is);
}
newString=stringBuilder.toString();
return stringBuilder.toString();
}
}
So, the question: how do i use this method? i know it sounds silly, but how do run this method and then use the data later? It is supposed to take a given text file and return a string, but i'm not even sure how to get the return value after it has processed.
Any help would be greatly appreciated. I made an account here just to ask this question. If i need to post this somewhere else, or if there is a better site to use to find an answer and some more basic help, please let me know. Thanks,
-Ethan
The readFile method seems to be doing multiple things at once. It accepts a file argument which overrides the member variable with the same name. Then it reads the file into a String and sets the newString member variable to the result before returning the same result.
So I would recommend first deciding whether the method should return the data or set the member variable. If multiple methods are going to be using the result, it might be useful to use the member variable, otherwise go the return route. Also, you can probably remove the file member variable since it is ignored by the method.
You can rewrite the method to look like this (I just removed the newString=stringBuilder.toString(); line, and I changed it to static since it can be):
private static String readFile(String file) throws IOException
{
BufferedReader reader = new BufferedReader(new FileReader(file));
String line=null;
StringBuilder stringBuilder = new StringBuilder();
String is=System.getProperty("line.seperator");
while((line=reader.readLine())!=null)
{
stringBuilder.append(line);
stringBuilder.append(is);
}
return stringBuilder.toString();
}
And wherever you need to use it call it like this (remember to catch the IOException):
try {
String someString = readFile("filename.txt");
} catch(IOException e) {
// handle error
}
Remember it must be called from inside the same class unless you change private to public.
Also, it might be worth reading and following a standard code style. It can really help by distinguishing between different types of variable for example.
Actual method that read string from file is: "readFile". And in your class, you are not calling that.
private String readFile(String file) throws IOException
You can pass file name as parameter, and It will returns read string.
So, how about modify your word(String s), and add method that will return actual result of read file?
public word(String s)
{
onFile=s;
newString = readfile(onFile);
}
public getNewString(){
return newString;
}
Try this:
String file ="/path/to/file.csv"
word myWord = new word();
o = myWord.getClass().getDeclaredMethod("readFile");
o.setAccessible(true);
Object r = o.invoke(myWord);
//print result
I have an Excel file that looks like this:
What I'm trying to do now is supply the DEST_NAME_DISPLAY and retrieve the DEST_ID.
As you might have noticed, this file contains a lot of destination, so I don't want to loop over them all to find a DEST_ID. Is this possible? If it is, how?
Since the file rarely changes you can read it into a Map.
For example like this: How to group the values which are in excel to a HashMap
Afterwards, you can get the value like this:
Map<String, Integer> map = ... // from file
Integer value = map.get(key);
Edit: BackSlash points out, a database is better because the data will remain after the application quits. Of course it is a little harder to set up, but worth considering if you need persistance.
In java8 you can use the functional programming features to get it done.Following is the psuedo code written in javascript.
var myColumnDefs = [
{key:"Tokyo", value:"2211"},
{key:"Akihabara", value:"2211"},
{key:"Kyoto",value:"3344"}]
var input = "Kyoto"
var p = myColumnDefs.filter(function(x){ return x.key==input }).map(function(x){return x.value});
alert(p);
The following is the longer version which is written in Java 8
import java.util.Arrays;
class Location{
public String areaCode;
public String areaName;
public String otherInfo;
public Location(String areaName,String areaCode,String otherInfo){
this.areaCode = areaCode;
this.areaName = areaName;
this.otherInfo = otherInfo;
}
}
public class Main {
public static void main(String[] args) {
Location[] locations = {
new Location("Tokyo","2211","TK"),
new Location("Akihabara","2211","AHR"),
new Location("Kyoto","3344","KT")
};
String input = "Tokyo";
Location[] output =(Location[]) Arrays.stream(locations).filter(x->x.areaName.equals(input)).toArray(size -> new Location[size]);
System.out.println(output[0].areaCode);
}
}
I'm creating a race results program and I need to store all the info from a text file that looks like this:
------1 Jackson Bertoli 11 Jasper 15.29-----------------------------------------------------------------------------------------
As you can see, it contains the place, a first name, a last name, a grade, a school name, and a time.
This information needs to be stored together, so I figured the best way to do this was storing it in an object "runner", and then putting all of those "runner" objects inside an arraylist. The only problem with this is that I can't find how to make a different object name for each object in my while loo (which is going through each line to read the elements from the text file) for each separate runner. Is it even necessary to have a different object name for each object? And if I am allowed to have many separate objects with the same name, how do I distinguish them apart? Here's a bit of the code behind it
public void readfile() {
while (initialresults.hasNext()){
strplace = initialresults.next();
dataplace = Integer.parseInt(strplace);
datafirstname = initialresults.next();
datalastname = initialresults.next();
strgrade = initialresults.next();
datagrade = Integer.parseInt(strgrade);
dataschool = initialresults.next();
strorigtime = initialresults.next();
dataorigtime = Double.parseDouble(strorigtime);
runner newrunner = new runner(datafirstname,datalastname, datagrade, dataschool, dataorigtime);
amountofrunners.add(newrunner);
counter++;
}
}
So you can see I'm reading each element from the text file, and then trying to make a new "runner" object, store those elements from the text file in that object, but then I'm stuck putting the next line's elements in that same object name. How can I create a new object name every time to store those elements in? The only reason I'm using an object is because it seems like the best way to keep data organized for a runner. But if I used an array or a list, wouldn't that get disorganized and difficult to sort through? Thank you for your help in advance.
When your loop runs, even though the runner class is being used multiple times, it creates a new (and different) runner object each time the loop iterates (runs through a cycle).
In the line amountofrunners.add(newrunner);, Java basically copies the value(s) stored in newrunner and stores it/them seperately. Every object is different in this case, even if it has the same variable name when it's created.
Using an ArrayList actually makes it easier to sort through because you can use indexes, foreach loops, and other features to help manage the objects stored in the ArrayList.
I think you were trying to do something like this:
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;
public static class Runner{
int place;
String firstName, lastName, school;
double origTime, grade;
Runner(int place,String firstName,String lastName,double grade,String school,double origTime){
this.place = place;
this.firstName = firstName;
this.lastName = lastName;
this.grade = grade;
this.school = school;
this.origTime = origTime;
}
}
public static List<Runner> readfile(Scanner in) {
List<Runner> runners = new ArrayList<Runner>();
while (in.hasNext()){
runners.add(new Runner(
in.nextInt(), // place
in.next(), in.next(), //first, last name
in.nextDouble(), // grade
in.next(), // school
in.nextDouble() // original time
));
}
return runners;
}
public static void main(String[] args) throws IOException {
List<Runner> runners;
try (Scanner s = new Scanner(new BufferedReader(new FileReader("file.txt")))) {
runners = readfile(s);
}
// Do something with runners
}
I've got a table with some factors that I need to incorporate into a Java program. At first I was thinking of hardcoding the number but it seems like a pain trying to create a data structure that will fit the factors. So I wanted to ask around and see if it would be better to implement this as reference data in a database, a flat file or in java. The number would change every six months and would be used for mathematical computations.
Thoughts?
For slow-changing data like this, I would use an external config file. Based on the structure of your data, it seems that a CSV would work well, and would be easy for a business user to edit using Excel.
If it will change more often, you need to generate the data programmatically, or you want to provide a UI for editing the data, you could move it to a database.
You would have to create a data structure to contain the data regardless of how you store them. But the data structure for this kind of data does not have to be complex. It is just a list of values with attributes. You don't have to store them in a complex table-like structure.
Loading the data from a flat text file would also be quite easy when representing the data as a single list.
public class DataTable {
private List<Entry> table = new ArrayList<Entry>();
public double getValue(Sex sex, MaritalStatus maritalStatus, AgeInterval ageInterval, Type type) {
for (Entry entry : table) {
if (entry.sex == sex && entry.maritalStatus == maritalStatus && entry.ageInterval == ageInterval && entry.type == type) {
return entry.value;
}
}
throw new IllegalArgumentException("Unknown value");
}
public void load(String filename) {
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(filename)));
String line;
while ((line = reader.readLine()) != null) {
StringTokenizer t = new StringTokenizer(line, ":");
table.add(new Entry(
Sex.valueOf(t.nextToken()),
MaritalStatus.valueOf(t.nextToken()),
AgeInterval.valueOf(t.nextToken()),
Type.valueOf(t.nextToken()),
Double.valueOf(t.nextToken())));
}
} catch (IOException e) {
throw new IllegalStateException("Failed to read the data file", e);
}
}
}
enum Sex {M, F}
enum MaritalStatus {SINGLE, MARRIED}
enum AgeInterval {I16_21, I22_35, I35_55, I55}
enum Type {GD, NGD} // Whatever this is ...
class Entry {
Sex sex;
MaritalStatus maritalStatus;
AgeInterval ageInterval;
Type type;
double value;
Entry(Sex sex, MaritalStatus maritalStatus, AgeInterval ageInterval, Type type, double value) {
this.sex = sex;
this.maritalStatus = maritalStatus;
this.ageInterval = ageInterval;
this.type = type;
this.value = value;
}
}
The data file would look like this:
M:SINGLE:I16_21:GD:1.10
F:SINGLE:I16_21:GD:1.20
...
You could represent it as XML, but that might be a little heavy for such numeric data. But the XML would allow you to be fairly descriptive and self documenting. Then later you could easily parse this into Java(or another language of your choice).
Partial XML example:
<dataset>
<gd>
<16to21>
<single>
<male>1.10</male>
<female>1.20</female>
</single>
<married>
<male>0.90</male>
<female>0.80</female>
</married>
</16to21>
...
</gd>
<ngd>
...
</ngd>
One way that you could break up the fields is gender, age, marital_status, GD_VS_NGD, the data inside the table, and some identifier for the time period that you are using this data for unless you do not need to keep records of the data.