so I'm writing a program that will read a csv file and put each individual line of the file into an array. I would like to know if it would be possible to name a singular array that was created in the while loop. I would also love to know if you have any ideas on how I'd be able to separate the lines (containing the rows of the csv file) by the columns of the csv file.
This is my code:
package sample.package;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
public class SampleClass {
/**
* #param args
*/
public static String fileLocation; //Used to declare the file path
public static void readAndArray(String fileLocation) throws IOException { //Method to read a file and put each line into an array
BufferedReader lineRead = new BufferedReader(new FileReader(fileLocation)); //Read the file and be able to parse it into separate lines
String line = lineRead.readLine(); //Put line into variable so it isn't a boolean statement
while ((line = lineRead.readLine()) !=null) { //Make it possible to print out array as BufferedReader is used
String[] oneLine = new String[] {line}; //Put parsed line in new array and parsing that data for individual spots in array
System.out.println(oneLine[0]); //Print out each oneLine array
}
lineRead.close(); //Neatly close BufferedReader and FileReader
}
public static void main(String[] args) throws IOException{
readAndArray("filePath"); //Initialize method by inputting the file path
}
}
Thanks so much guys!
First of: Welcome to stackoverflow!
I assume that your question relates to some sort of educational programming task, if not: There are a number of libraries dealing with CSV files (with additional features like reading header rows and read row entries by header/column name, etc).
But ... why should it be a complicated task to write a CSV parser, i mean, its basically just values separated by commas, phhh!?
To cut a long story short: There is RFC 4180, but don't expect that all your .csv files stick to it. Quoting Wikipedia:
The CSV file format is not fully standardized. The basic idea of
separating fields with a comma is clear, but that idea gets
complicated when the field data may also contain commas or even
embedded line-breaks. CSV implementations may not handle such field
data, or they may use quotation marks to surround the field. Quotation
does not solve everything: some fields may need embedded quotation
marks, so a CSV implementation may include escape characters or escape
sequences.
After you are aware of and hopefully understand the provided warning about compatibility the following code example completely ignores it and provides a quick and dirty solution (don't use it in production, seriously ... how to not be fired 101: use a well tested library like opencsv or Apache Commons CSV):
public static void main(String[] args)
{
Path path = Paths.get("some path"); // TODO Change to path to an CSV file
try (Stream<String> lines = Files.lines(path))
{
List<List<String>> rows = lines
// Map each line of the file to its fields (String[])
.map(line -> line.split(","))
// Map the fields of each line to unmodifiable lists
.map(fields -> Collections.unmodifiableList(Arrays.asList(fields))
// Collect the unmodifiable lists in an unmodiable list (listception)
.collect(Collectors.toUnmodifiableList());
// Ensure file is not empty.
if (rows.isEmpty())
{
throw new IllegalStateException("empty file");
}
// Ensure all lines have the same number of fields.
int fieldsPerRow = rows.get(0).size();
if (!rows.stream().allMatch(row -> row.size() == fieldsPerRow))
{
throw new IllegalStateException("not all rows have the same number of fields");
}
// Assume the file has a header line appearing as the first line.
System.out.printf("Column names: %s\n", rows.get(0));
// Read the data rows.
rows.stream()
.skip(1) // Skip header line
.forEach(System.out::println);
}
catch (IOException | UncheckedIOException e)
{
e.printStackTrace(); // TODO Handle exception
}
}
This code assumes:
Fields are separated by commas
A line is considered to be terminated by any one of a line feed ('\n'), a carriage return ('\r'), or a carriage return followed immediately by a linefeed (behaviour of BufferedReader used by Files.lines())
All lines have the same number of fields
The first line is a header line
The implementor was too lazy to mind fields enclosed in double quotes
Quick and dirty ... and violates the RFC ;)
Related
I downloaded my extended listening history from Spotify and I am trying to make a program to turn the data into a list of artists without doubles I can easily make sense of. The file is rather huge because it has data on every stream I have done since 2016 (307790 lines of text in total). This is what 2 lines of the file looks like:
{"ts":"2016-10-30T18:12:51Z","username":"edgymemes69endmylifepls","platform":"Android OS 6.0.1 API 23 (HTC, 2PQ93)","ms_played":0,"conn_country":"US","ip_addr_decrypted":"68.199.250.233","user_agent_decrypted":"unknown","master_metadata_track_name":"Devil's Daughter (Holy War)","master_metadata_album_artist_name":"Ozzy Osbourne","master_metadata_album_album_name":"No Rest for the Wicked (Expanded Edition)","spotify_track_uri":"spotify:track:0pieqCWDpThDCd7gSkzx9w","episode_name":null,"episode_show_name":null,"spotify_episode_uri":null,"reason_start":"fwdbtn","reason_end":"fwdbtn","shuffle":true,"skipped":null,"offline":false,"offline_timestamp":0,"incognito_mode":false},
{"ts":"2021-03-26T18:15:15Z","username":"edgymemes69endmylifepls","platform":"Android OS 11 API 30 (samsung, SM-F700U1)","ms_played":254120,"conn_country":"US","ip_addr_decrypted":"67.82.66.3","user_agent_decrypted":"unknown","master_metadata_track_name":"Opportunist","master_metadata_album_artist_name":"Sworn In","master_metadata_album_album_name":"Start/End","spotify_track_uri":"spotify:track:3tA4jL0JFwFZRK9Q1WcfSZ","episode_name":null,"episode_show_name":null,"spotify_episode_uri":null,"reason_start":"fwdbtn","reason_end":"trackdone","shuffle":true,"skipped":null,"offline":false,"offline_timestamp":1616782259928,"incognito_mode":false},
It is formatted in the actual text file so that each stream is on its own line. NetBeans is telling me the exception is happening at line 19 and it only fails when I am looking for a substring bounded by the indexOf function. My code is below. I have no idea why this isn't working, any ideas?
import java.util.*;
public class MainClass {
public static void main(String args[]){
File dat = new File("SpotifyListeningData.txt");
List<String> list = new ArrayList<String>();
Scanner swag = null;
try {
swag = new Scanner(dat);
}
catch(Exception e) {
System.out.println("pranked");
}
while (swag.hasNextLine())
if (swag.nextLine().length() > 1)
if (list.contains(swag.nextLine().substring(swag.nextLine().indexOf("artist_name"), swag.nextLine().indexOf("master_metadata_album_album"))))
System.out.print("");
else
try {list.add(swag.nextLine().substring(swag.nextLine().indexOf("artist_name"), swag.nextLine().indexOf("master_metadata_album_album")));}
catch(Exception e) {}
System.out.println(list);
}
}
Find a JSON parser you like.
Create a class that with the fields you care about marked up to the parsers specs.
Read the file into a collection of objects. Most parsers will stream the contents so you're not string a massive string.
You can then load the data into objects and store that as you see fit. For your purposes, a TreeSet is probably what you want.
Your code will throw a lot of exceptions only because you don't use braces. Please do use braces in each blocks, whether it is if, else, loops, whatever. It's a good practice and prevent unnecessary bugs.
However, everytime scanner.nextLine() is called, it reads the next line from the file, so you need to avoid using that in this way.
The best way to deal with this is to write a class containing the fields same as the json in each line of the file. And map the json to the class and get desired field value from that.
Your way is too much risky and dependent on structure of the data, even on whitespaces. However, I fixed some lines in your code and this will work for your purpose, although I actually don't prefer operating string in this way.
while (swag.hasNextLine()) {
String swagNextLine = swag.nextLine();
if (swagNextLine.length() > 1) {
String toBeAdded = swagNextLine.substring(swagNextLine.indexOf("artist_name") + "artist_name".length() + 2
, swagNextLine.indexOf("master_metadata_album_album") - 2);
if (list.contains(toBeAdded)) {
System.out.print("Match");
} else {
try {
list.add(toBeAdded);
} catch (Exception e) {
System.out.println("Add to list failed");
}
}
System.out.println(list);
}
}
I Have a formatted text file called cars.txt; It's separated by tabs.
Name Length Width
truck1 18.6 8.1
suv1 17.4 7.4
coupe1 14.8 5.4
mini1 14.1 5.0
sedan1 16.4 6.1
suv2 17.5 7.3
mini2 14.3 5.2
sedan2 16.5 6.2
I need to read in this information so it can be used for calculations later on.
This is my current idea but I am having a hard time piecing together what I need to execute.
public class Class{
public void readFileIn(){
Scanner sc = new Scanner(new FileReader("cars.txt");
try{
while (sc.hasNextLine()){
if (/**something that catches strings*/){
method1(string1, double1, double2);
method2(double1, double2);
}
}
}catch(FileNotFoundException exception){
System.out.println("File dosen't exist");
}
}
}
Scanner and Buffer Reader are not used very often anymore as Java provides a better way to achieve tha same result with less code.
I can see at least three possible approaches to solve your problem:
approach 1: if you can use at least Java 8, then I would suggest to use the java.nio.file libraries to read the file as a stream of lines:
Stream<String> linesStream=Files.lines("cars.txt");
Then depending on what you need to do, you could use either forEach that will loop on each line of the stream:
linesStream.forEach(e -> e.myMethod());
Or Java Collectors to execute the calculation that you need to. A good tutorial about Collectors can be found here. You can use collectors also to separate your string etc...
approach 2: you can use Apache Commons libraries to achieve the same goal. In particular you could use FileUtils and StringUtils. For instance:
File carFile=new File("cars.txt");
LineIterator lineIterator=lineIterator(carFile);
for(String line : lineIterator) {
String[] my values=StringUtils.split(line);
//do whatever you need
}
approach 3: use Jackson to transform your file into a json or a java object that you can then use for your own transformations. Here is an example explaining how to convert a CSV to JSON. With a bit of digging in the Jackson documentation, you could apply it to your case.
First of all, i recommend you create an Entry class that represents your data.
private class Entry {
private String name;
private double length;
private double width;
// getters and setters omitted
#Override
public String toString() {
// omitted
}
}
Next, create a method that takes a String as an arguments and is responsible for parsing a line of text to an instance of Entry. The regex \\s+ matches any whitespace characters and will split your line to its individual columns. Remember that in production, Double.valueOf can throw an RuntimeException if your are not passing a valid String.
Finally, you can read the file, here using the Java 8 stream API. Skip the first line since it includes the column header and not actual data.
private void readFile() throws Exception {
Path path = Paths.get(/* path to your file */);
Files.readAllLines(path).stream().skip(1).map(FileReadTest::toEntry)
.forEach(this::action);
}
In my example, i am just printing each entry to the console:
private void action(Entry entry) {
System.out.println(entry);
}
Resulting output:
Entry[name='truck1', length=18.6, width=8.1]
Entry[name='suv1', length=17.4, width=7.4]
Entry[name='coupe1', length=14.8, width=5.4]
Entry[name='mini1', length=14.1, width=5.0]
Entry[name='sedan1', length=16.4, width=6.1]
Entry[name='suv2', length=17.5, width=7.3]
Entry[name='mini2', length=14.3, width=5.2]
Entry[name='sedan2', length=16.5, width=6.2]
Here's an example of how to properly read a text file - replace the charset with the one you need.
try (final BufferedReader br = Files.newBufferedReader(file.toPath(), StandardCharsets.UTF_8)) {
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
} catch (IOException e) {
e.printStackTrace();
}
Once you have the individual lines, you can split them by whitespace: str.split("\\s+");
You get an array with three entries. I guess you can figure out the rest.
I have 2 columns in an excel sheet, A and B, and I need to divide column A (randomly) into 2 and calculate the average of the first and second halves of column B separately. Then I need to divide column A again but each time moving with one cell downwards and calculate the averages etc. Can this be done using Java only? Since I am finding you have to use software like Apache and I have never used them.
Thanks
Yes, you can save the Excel data as a text file and use classes like java.io.File, java.util.Scanner to read the text file.
A simple approach would be to save the Excel file as a comma-separated values (.CSV) file.
Since you have never used a framework before, I will show you how to use the OpenCSV framework.
First you need to add this framework to your Build Path. Drag the downloaded .jar files to your workspace in Eclipse. Open the project's properties, Select Java Build Path, Select Libraries tab, and then Add a Library from Workspace and select that jar file.
To use it, your class will look something like this,
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import au.com.bytecode.opencsv.CSVReader;
public class CSV {
private List<String[]> fileContent;
public CSV(String filePath) {
try {
readCSV(filePath);
} catch (IOException e) {
System.out.print("There was an error reading this CSV file. Error follows:");
e.printStackTrace();
}
}
public void readCSV(String csvPath) throws IOException {
// Store coupon objects in array list
CSVReader csvReader = new CSVReader(new FileReader(csvPath));
// Store each row into a String list
List<String[]> csvData = csvReader.readAll();
csvReader.close();
fileContent = csvData;
}
public void printFileContent() {
System.out.println("");
for (String[] row : fileContent.subList(1, fileContent.size())) {
for (String s: row) {
System.out.print(s + " ");
}
System.out.println();
}
System.out.println("");
}
}
Inside the main method, you can use a simple
CSV csvFile = new CSV("csv/Path/Here.csv");
// Verify Results
csvFile.printFileContent();
As always, please use my code as a reference. It is always better to learn how to write code yourself, especially when this question pertains to a homework assignment.
I have two files:
Grader.getFileInfo("data\\studentSubmissionA.txt");
Grader.teacherFiles("data\\TeacherListA.txt");
Both contain a list of math problems, but the TeacherList is unsolved in order to check that the StudentSubmission was not altered from the original version.
studentSubmission is sent to the Grader class and the method currently looks like this:
public static void getFileInfo(String fileName)
throws FileNotFoundException {
Scanner in = new Scanner(new File(fileName))
while (in.hasNext()) {
String fileContent = in.nextLine();
}
and the TeacherFiles method looks like
public static void teacherFiles(String teacherFiles)
throws FileNotFoundException{
Scanner in = new Scanner(new File(teacherFiles));
while (in.hasNext()){
String teacherContent = in.nextLine();
String line = teacherContent.substring(0, teacherContent.indexOf('='));
}
I don't know how to get these methods to another method in order to compare them since they're coming from a file and I have to put something in the method signature to pass them and it doesn't work.
I tried putting them in one method, but that was a bust as well.
I don't know where to go from here.
And unfortunately, I can't use try/catches or arrays.
Is it possible to send the .substring(0 , .indexof('=')) through the methods?
Like line = teacherFiles(teacherContent.substring(0 , .indexof('='))); Is it possible to do this?
Think in more general terms. Observe that your methods called getFileInfo and teacherFiles, respectively are the very same except a few nuances. So why do not we think about finding the optimal way of merging the two functionalities and handling the nuances outside of them?
It is logical that you cannot use arrays as you need to know the number of elements of your array before you initialize it and your array would have already been initialized when you read the file. So using an array for this task is either an overkill (for example you allocate 1000 elements in your memory and you use only 10 elements) or insufficient (if you create an array of 10 elements, but you would need 1000). So, due to the fact that you do not know the number of rows in advance you need to use another data structure for your task.
So create the following method:
public static AbstractList<String> readFile(String filePath) throws FileNotFoundException, IOException {
Scanner s = new Scanner(new File(filePath));
AbstractList<String> list = new ArrayList<String>();
while (s.hasNext()){
list.add(s.next());
}
s.close();
return list;
}
Then use the method to read the student file and to read the teacher file. Store the results into two separate AbstractList<String> variables, then iterate through them and compare them as you like. Again, think in more general terms.
I have a class defined like this, with the appropriate getter and setter methods...
public class Album {
private int id;
private String artist;
private String name;
private int published;
}
I also have a .csv file that stores this content for a number of Albums. In the file, one line represents one Album.
I'm trying to read the information from the .csv file, and then use the setters of the Album class to assign the values. Here is my code...
public Map<Integer, Album> load() {
Scanner scanner = new Scanner(fileName);
Map<Integer, Album> loadedAlbums = new HashMap<Integer, Album>();
while(scanner.hasNextLine()) {
Album album = new Album();
String[] albumDivided = scanner.nextLine().split(",");
//in the .csv file every unit of information is divided by a comma.
album.setId(Integer.parseInt(albumDivided[0])); //this is line 11.
album.setArtist(albumDivided[1]);
album.setName(albumDivided[2]);
album.setPublished(Integer.parseInt(albumDivided[3]));
loadedAlbums.put(album.getId(), album);
}
return loadedAlbums;
}
However, trying to use this code, I get the following Exception:
java.lang.NumberFormatException: For input string: "albums.csv" at line 11.
Could you please help me to understand the cause of this problem.
Well the problem is described to you by the Exception...
A NumberFormatException would have been triggered by one of your Integer.parseInt() lines. The line of your code that is triggering the exception is Line 11 (as per the exception message) - not sure which one this is but its probably the first Integer.parseInt() line.
Your code is trying to convert the value "albums.csv" to a number, which is obviously isn't. So somewhere in your CSV file you must have a line that contains the value albums.csv where it is expecting a number.
Hope this helps pinpoint the problem.
Since you don't want the whole solution here is a hint to resolve your problem:
You should take a look at the API documentation of the Scanner class. Take a really close look on the constructor that expects a single String parameter (as you use it in your code).
As far as I can tell, albumDivided[0] will containt "1." which will not be able to parse to an integer because of the dot. Either remove the dot from your csv file, or create a new string that removes the dot before you parse it to Integer. The approach might look something like this:
String newString;
for(int i=0;i<albumDivided[0].length-1;i++){ //length -1 to remove the dot
newString = newString + albumDivided[0].charAt(i); //get the string stored in albumDivided[0] and add each char to the new string
}