We Keep Coding

Weighted mean of a column in csv file in Java

I am trying to compute the number of elements in a column present in a CSV file using which I would like to compute the mean.
This is how I tried to find the length of the column, but it gives me the length of each data in the column. It's something very silly but I have been stuck with this for a long time.
public static void main(String[] args) throws Exception {
String splitBy = ";";
int x;
BufferedReader br = new BufferedReader(new FileReader("bank.csv"));
String line;
while ((line = br.readLine()) !=null) {
String[] b = line.split(splitBy);
x = b[10].length();
System.out.println("length " + x);
}
br.close();
}
This is the CSV file:
https://raw.githubusercontent.com/pandutruhandito/practice_bank_marketing_data/master/bank/bank-full.csv

You will want something like the following. A few notes.
If a value is always required then you can simply count the number of lines in the CSV to give you the number of values in the column. Otherwise you'll need to check if there is an actual value.
You were grabbing the month column, remember indexes start at 0.
public static void main(String[] args) throws Exception {
final String splitBy = ";";
int totalValue = 0;
int columnCount = 0;
final BufferedReader br = new BufferedReader(new FileReader("C:/test/bank-full.csv"));
String line;
// Throw away headers.
line = br.readLine();
while ((line = br.readLine()) != null)
{
String[] splitCSV = line.split(splitBy);
if (!splitCSV[9].equals(""))
{
totalValue += Integer.parseInt(splitCSV[9]);
columnCount++;
}
}
br.close();
System.out.println("Average is " + totalValue / columnCount);
}

b[10].length() is incorrect: it gives you the size of string of the 10th element in a particular line (and it changes in the loop)
line.length(): lengh of the line
b.length: number of elements in a line
if you want the number of columns, you can just only keep the number of lines read (just count your readlin())
Or you can also keep everything in a table.
beware: some csv have , or ; , and perhaps inside " (if there is string)

You have to use x = b.length; which will give you exact length per line.
public static void main(String[] args) throws Exception {
String splitBy = ";";
int x;
BufferedReader br = new BufferedReader(new FileReader("bank.csv"));
String line;
while ((line = br.readLine()) !=null) {
String[] b = line.split(splitBy);
x = b.length;
System.out.println("length " + x);
}
br.close();
}

Why does my summation program throw a NumberFormatException?

I want to read the numbers from the file and sum up the total, but I cannot seem to process the data properly. It is successfully outputting the numbers but is not successfully summing them up.
My code:
import java.io.*;
import java.util.*;
public class Q1 {
public static void main(String[] args) throws IOException {
StringBuffer str = new StringBuffer();
FileWriter output = new FileWriter("number.txt");
Random r = new Random();
for (int i = 1; i < 101; i++) {
str.append(r.nextInt(100) + " ");
}
output.write(str.toString());
System.out.println(str.toString());
output.close();
FileReader reader = new FileReader("number.txt");
BufferedReader input = new BufferedReader(reader);
String line = input.readLine();
int total = 0;
while (line != null) {
System.out.print(line);
total += Integer.parseInt(input.readLine());
}
System.out.println(total);
}
}
And the stacktrace:
Exception in thread "main" java.lang.NumberFormatException: null
at java.lang.Integer.parseInt(Integer.java:542)
at java.lang.Integer.parseInt(Integer.java:615)
at Q1.main(Q1.java:42)

You don't append a ending line char to the output.
Try out this snippet:
StringBuffer str = new StringBuffer();
FileWriter output = new FileWriter("number.txt");
Random r = new Random();
for (int i = 1; i < 101; i++) {
str.append(r.nextInt(100)).append('\n');
}
output.write(str.toString());
System.out.println(str.toString());
output.close();
FileReader reader = new FileReader("number.txt");
BufferedReader input = new BufferedReader(reader);
String line;
int total = 0;
while ((line = input.readLine())!=null) {
total += Integer.parseInt(line);
}
System.out.println(total);
If the output looks weird, examine where the output is being created. That is usually where you will find the problem. Also added in some small changes to give you an idea of a different way you could go about it that might make your life easier.

Replace letters with *

Making a hangman style of game
I have the random word now. How do I replace the letters of the word with an asterix * so that when the program starts the word is shown as *.
I assume that when someone inputs a letter for the hangman game you get the index of that character in the word and then replace the corresponding *.
public class JavaApplication10 {
public static String[] wordArray = new String[1];
public static String file_dir = "Animals.txt";
public static String selectedWord = "";
public static char[] wordCharacter = new char[1];
Scanner sc = new Scanner(System.in);
public static void main(String[] args) throws IOException {
wordArray = get_word(file_dir);
selectedWord = select_word(wordArray);
System.out.println(selectedWord);
}
public static String[] get_word(String file_dir) throws IOException {
FileReader fileReader = new FileReader(file_dir);
BufferedReader bufferedReader = new BufferedReader(fileReader);
List<String> lines = new ArrayList<String>();
String line = null;
while ((line = bufferedReader.readLine()) != null) {
lines.add(line);
}
bufferedReader.close();
return lines.toArray(new String[lines.size()]);
}
public static String select_word(String[] wordArray) {
Random rand = new Random();
int lines = Math.abs(rand.nextInt(wordArray.length)- 1);
return wordArray[lines];
}
}

If you know how many lines are there you could use Random method in java with a specific range to pick out a line at random.
Then you could read the file line-by-line till you reach that random line and print it.
// Open the file
FileInputStream fstream = new FileInputStream("testfile.txt");
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));
String strLine;
int counter=0;
//While-loop -> Read File Line By Line till the end of file
//And will also terminate when the required line is printed
while ((strLine = br.readLine()) != null && counter!=randomValue){
counter++;
//You need to set randomValue using the Random method as suggested
if(counter==randomValue)
// Print the content on the console
System.out.println (strLine+"\n");
}
//Close the input stream
br.close();

Assuming Java 8:
// Loading ...
Random R = new Random(System.currentTimeMillis());
List<String> animals = Files.readAllLines(Paths.get(path));
// ...
// When using
String randomAnimal = animals.get(R.nextInt(animals.size()));

Answer of your first question :
First you have to get the total number of lines
Then you have to generate a random number between 1 and that total number.
Finally, get the required word
try {
InputStream is = new BufferedInputStream(new FileInputStream("D:\\test.txt"));
byte[] c = new byte[1024];
int count = 0;
int readChars = 0;
boolean empty = true;
while ((readChars = is.read(c)) != -1) {
empty = false;
for (int i = 0; i < readChars; ++i) {
if (c[i] == '\n') {
++count;
}
}
}
int noOfLines = count+1;
System.out.println(noOfLines);
Random random = new Random();
int randomInt = random.nextInt(noOfLines);
FileReader fr = new FileReader("D:\\test.txt");
BufferedReader bufferedReader =
new BufferedReader(fr);
String line = null;
int counter =1;
while((line = bufferedReader.readLine()) != null) {
if(counter == randomInt)
{
System.out.println(line); // This the word you want
}
counter++;
}
} catch (Exception e) {
// TODO: handle exception
e.printStackTrace();
}
finally {
//is.close();
}

Reading CSV file without third-party libraries

I'm trying to read a csv file into either an ArrayList or a String [][] array. In this I'm trying to read it into a list and then form the list, using a tokenizer, into an array. The csv file have 7 columns (A - G) and 961 rows (1-961). My for loop for the 2D array keeps returning a null pointer, but I think it should be working..
public class FoodFacts
{
private static BufferedReader textIn;
private static BufferedReader foodFacts;
static int numberOfLines = 0;
static String [][] foodArray;
public static String aFact;
static int NUM_COL = 7;
static int NUM_ROW = 961;
// Make a random number to pull a line
static Random r = new Random();
public static void main(String[] args)
{
try
{
textIn = new BufferedReader(new InputStreamReader(System.in));
foodFacts= new BufferedReader(new FileReader("foodfacts.csv"));
Scanner factFile = new Scanner(foodFacts);
List<String> facts = new ArrayList<String>();
String fact;
System.out.println("Please type in the food you wish to know about.");
String request = textIn.readLine();
while ( factFile.hasNextLine()){
fact = factFile.nextLine();
StringTokenizer st2 = new StringTokenizer(fact, ",");
//facts.add(fact);
numberOfLines++;
while (st2.hasMoreElements()){
for ( int j = 0; j < NUM_COL ; j++) {
for (int i = 0; i < NUM_ROW ; i++){
foodArray [j][i]= st2.nextToken(); //NULL POINTER HERE
System.out.println(foodArray[j][i]);
}
}
}
}
}
catch (IOException e)
{
System.out.println ("Error, problem reading text file!");
e.printStackTrace();
}
}
}

Initialize your foodArray as foodArray = new String[NUM_ROW][NUM_COL]; before using it.
Also, there is no need for inner for loop as you are reading one row at a time.
use numberOfLines as row:
while ( factFile.hasNextLine() && numberOfLines < NUM_ROW){
fact = input.nextLine();
StringTokenizer st2 = new StringTokenizer(fact, ",") ;
//facts.add(fact);
while (st2.hasMoreElements()){
for ( int j = 0; j < NUM_COL ; j++) {
foodArray [numberOfLines][j]= st2.nextToken();
System.out.println(foodArray[numberOfLines][i]);
}
}
numberOfLines++;
}
Alternatively, I think you can use split to get all columns as once e.g.
while ( factFile.hasNextLine() && numberOfLines < NUM_ROW){
fact = input.nextLine();
foodArray [numberOfLines++] = fact.split(",");
}
One question: Is there any specific purpose for declaring all variables as static class variables? Most of them fit as local variable inside the method e.g. numberOfLines?

You can use this String [][] foodArray = csvreadString(filename); method. It actually reads the file twice, but I don't know how to get the csv dimension without reading the data (you need the dimension in order to initialize the array), and this is very fast in comparison to other methods that I tried.
static public class PairInt {
int rows = 0;
int columns = 0;
}
static PairInt getCsvSize(String filename) throws Throwable {
PairInt csvSize = new PairInt();
BufferedReader reader = new BufferedReader(new FileReader(new File(filename)));
String line;
while ((line = reader.readLine()) != null) {
if (csvSize.columns == 0) {
csvSize.columns = line.split(",").length;
}
csvSize.rows++;
}
reader.close();
return csvSize;
}
static String[][] csvreadString(String filename) throws Throwable {
PairInt csvSize = getCsvSize(filename);
String[][] data = new String[csvSize.rows][csvSize.columns];
BufferedReader reader = new BufferedReader(new FileReader(new File(filename)));
for (int i = 0; i < csvSize.rows; i++) {
data[i] = reader.readLine().split(",");
}
return data;
}

Find a specific line of a text file (not by line_number) and store it as a new String

I am trying to read a text file in java using FileReader and BufferedReader classes. Following an online tutorial I made two classes, one called ReadFile and one FileData.
Then I tried to extract a small part of the text file (i.e. between lines "ENTITIES" and "ENDSEC"). Finally l would like to tell the program to find a specific line between the above-mentioned and store it as an Xvalue, which I could use later.
I am really struggling to figure out how to do the last part...any help would be very much apprciated!
//FileData Class
package textfiles;
import java.io.IOException;
public class FileData {
public static void main (String[] args) throws IOException {
String file_name = "C:/Point.txt";
try {
ReadFile file = new ReadFile (file_name);
String[] aryLines = file.OpenFile();
int i;
for ( i=0; i < aryLines.length; i++ ) {
System.out.println( aryLines[ i ] ) ;
}
}
catch (IOException e) {
System.out.println(e.getMessage() );
}
}
}
// ReadFile Class
package textfiles;
import java.io.IOException;
import java.io.FileReader;
import java.io.BufferedReader;
import java.lang.String;
public class ReadFile {
private String path;
public ReadFile (String file_path) {
path = file_path;
}
public String[] OpenFile() throws IOException {
FileReader fr = new FileReader (path);
BufferedReader textReader = new BufferedReader (fr);
int numberOfLines = readLines();
String[] textData = new String[numberOfLines];
String nextline = "";
int i;
// String Xvalue;
for (i=0; i < numberOfLines; i++) {
String oneline = textReader.readLine();
int j = 0;
if (oneline.equals("ENTITIES")) {
nextline = oneline;
System.out.println(oneline);
while (!nextline.equals("ENDSEC")) {
nextline = textReader.readLine();
textData[j] = nextline;
// xvalue = ..........
j = j + 1;
i = i+1;
}
}
//textData[i] = textReader.readLine();
}
textReader.close( );
return textData;
}
int readLines() throws IOException {
FileReader file_to_read = new FileReader (path);
BufferedReader bf = new BufferedReader (file_to_read);
String aLine;
int numberOfLines = 0;
while (( aLine = bf.readLine()) != null ) {
numberOfLines ++;
}
bf.close ();
return numberOfLines;
}
}

I don't know what line you are specifically looking for but here are a few methods you might want to use to do such operation:
private static String START_LINE = "ENTITIES";
private static String END_LINE = "ENDSEC";
public static List<String> getSpecificLines(Srting filename) throws IOException{
List<String> specificLines = new LinkedList<String>();
Scanner sc = null;
try {
boolean foundStartLine = false;
boolean foundEndLine = false;
sc = new Scanner(new BufferedReader(new FileReader(filename)));
while (!foundEndLine && sc.hasNext()) {
String line = sc.nextLine();
foundStartLine = foundStartLine || line.equals(START_LINE);
foundEndLine = foundEndLine || line.equals(END_LINE);
if(foundStartLine && !foundEndLine){
specificLines.add(line);
}
}
} finally {
if (sc != null) {
sc.close();
}
}
return specificLines;
}
public static String getSpecificLine(List<String> specificLines){
for(String line : specificLines){
if(isSpecific(line)){
return line;
}
}
return null;
}
public static boolean isSpecific(String line){
// What makes the String special??
}

When I get it right you want to store every line between ENTITIES and ENDSEC?
If yes you could simply define a StringBuffer and append everything which is in between these to keywords.
// This could you would put outside the while loop
StringBuffer xValues = new StringBuffer();
// This would be in the while loop and you append all the lines in the buffer
xValues.append(nextline);
If you want to store more specific data in between these to keywords then you probably need to work with Regular Expressions and get out the data you need and put it into a designed DataStructure (A class you've defined by our own).
And btw. I think you could read the file much easier with the following code:
BufferedReader reader = new BufferedReader(new
InputStreamReader(this.getClass().getResourceAsStream(filename)));
try {
while ((line = reader.readLine()) != null) {
if(line.equals("ENTITIES") {
...
}
} (IOException e) {
System.out.println("IO Exception. Couldn't Read the file!");
}
Then you don't have to read first how many lines the file has. You just start reading till the end :).
EDIT:
I still don't know if I understand that right. So if ENTITIES POINT 10 1333.888 20 333.5555 ENDSEC is one line then you could work with the split(" ") Method.
Let me explain with an example:
String line = "";
String[] parts = line.split(" ");
float xValue = parts[2]; // would store 10
float yValue = parts[3]; // would store 1333.888
float zValue = parts[4]; // would store 20
float ... = parts[5]; // would store 333.5555
EDIT2:
Or is every point (x, y, ..) on another line?!
So the file content is like that:
ENTITIES POINT
10
1333.888 // <-- you want this one as xValue
20
333.5555 // <-- and this one as yvalue?
ENDSEC
BufferedReader reader = new BufferedReader(new
InputStreamReader(this.getClass().getResourceAsStream(filename)));
try {
while ((line = reader.readLine()) != null) {
if(line.equals("ENTITIES") {
// read next line
line = reader.readLine();
if(line.equals("10") {
// read next line to get the value
line = reader.readLine(); // read next line to get the value
float xValue = Float.parseFloat(line);
}
line = reader.readLine();
if(line.equals("20") {
// read next line to get the value
line = reader.readLine();
float yValue = Float.parseFloaT(line);
}
}
} (IOException e) {
System.out.println("IO Exception. Couldn't Read the file!");
}
If you have several ENTITIES in the file you need to create a class which stores the xValue, yValue or you could use the Point class. Then you would create an ArrayList of these Points and just append them..