We Keep Coding

Read a CSV file and Write a Json file in Java

I modified my code:
private static final String SAMPLE_CSV_FILE_PATH = "src/main/resources/testCSV.csv";
private static final String OUT_PUT_CSV_PATH = "src/main/resources/outCSV.csv";
public static void main(String[] args) throws IOException {
Reader reader = Files.newBufferedReader(Paths.get(SAMPLE_CSV_FILE_PATH));
CSVReader csvReader = new CSVReader(reader);
List<String[]> records = csvReader.readAll();
Writer writer = Files.newBufferedWriter(Paths.get(OUT_PUT_CSV_PATH));
CSVWriter out = new CSVWriter(writer);
int i = 1;
int total = 0;
while(i < records.size()){
String[] result = records.get(i);
for(int j =1; j<= Integer.parseInt(result[1]); j++){
String pattern="00000";
DecimalFormat myFormatter = new DecimalFormat(pattern);
String output = myFormatter.format(j);
writer.append(result[0]+output+"\n");
total++;
}
i++;
}
out.flush();
out.close();
System.out.println(total);
}
Now I am using the first CSV file to generate the serial number, Something like:
NAIS00001
NAIS00002
...
NAIS00625
Then I write these serial numbers into a new CSV file. But there is only one column. 6 millions data in one column... How can I star a new column?

Your Filewriter is not writing in append mode, so your file is being overwritten each time it goes through the outer loop. It's not a problem with the file size.
Try this:
FileWriter fileWriter = new FileWriter("src/main/resources/testOutPut.json", true);
Documentation

Writing an ArrayList to BufferedWriter using charArray()

The code snippet splits a CSV file into multiple CSV files and writes the first column content to the child CSV files. What I observed with this code is the column header "UNIQUE ID" is only appearing in FIRST CSV file. The following CSV files only contains data without the header. In order to get header to all files I thought of using an ArrayList so that I can put the header at the first index of ArrayList and rest of data afterwards. But I failed miserably.
I require suggestion or help for how to modify the code so that all the child files should have an additional UNIQUE IDENTIFIER row along as the first row with the column data. I am pasting the code which I tried and didn't work. Child csv should look like this This is what I am getting
public static void myFunction(int lines, int files) throws FileNotFoundException, IOException {
String inputfile = "C:/Users/Downloads/CONSOLIDATED.csv";
BufferedReader br = new BufferedReader(new FileReader(inputfile));
String strLine = null;
for (int i = 1; i <= files; i++) {
FileWriter fstream1 = new FileWriter("C:/Users/Downloads/FileNumber_" + i + ".csv");
BufferedWriter out = new BufferedWriter(fstream1);
for (int j = 0; j < lines; j++) {
strLine = br.readLine();
if (strLine != null) {
String strar[] = strLine.split(",");
ArrayList<String> al=new ArrayList<String>();
al.add(0,"Unique Identifier");
al.add(1,strar[0]);
char c[] = al.toString().toCharArray();
out.write(c);
out.newLine();
}
}
out.close();
}
br.close();
}

Your problem is that you are not keeping the headers out of the loops. Try something reading the first line before the main loop and store the headers in the List. then, every time you create a new file, before starting the inner loop, write the header in the first line of each file.
public static void myFunction(int lines, int files) throws FileNotFoundException, IOException {
String inputfile = "C:/Users/Downloads/CONSOLIDATED.csv";
BufferedReader br = new BufferedReader(new FileReader(inputfile));
String strLine = br.readLine(); //here you have the headers
String[] headers=strLine.split(",");
for (int i = 1; i <= files; i++) {
FileWriter fstream1 = new FileWriter("C:/Users/Downloads/FileNumber_" + i + ".csv");
BufferedWriter out = new BufferedWriter(fstream1);
out.write(headers[0]);
for (int j = 0; j < lines; j++) {
out.newLine();
strLine = br.readLine();
if (strLine != null) {
String strar[] = strLine.split(",");
out.write(strar[0]);
}
}
out.close();
}
br.close();
}

Sum of the values of the rows of each column read from a CSV file

I am trying to read from a CSV file which has 10 rows and 4 columns and after reading the data, I want to add all the rows of each column.For each row of each column there is a value. Suppose, if the row1 of column1 is 0, it's value would be 3 and same goes on for each row of each column.For example, if 10 rows of a column are-
1
2
3
4
5
0
1
0
0
0
then the sum should be 16. Here is the code so far-
public static void main(String[] args) throws Exception
{
String line=null;
try{
BufferedReader br=new BufferedReader(new FileReader(filepath));
while((line=br.readLine())!=null){
String[] distance=line.split(",");
System.out.println(line);
}
br.close();
}catch(IOException e){
e.printStackTrace();
}
}
Now, I want to find the sum of all those values per column. Help in solving this problem would be appreciated. Thanks

Disclaimer: You must use a CsvReader (see here), the following is only to demonstrate a solution.
Computing the column sum in a single-pass:
public class CsvReader {
private static final int NUMBER_OF_COLUMNS = 4;
public static void main(String[] args) throws IOException {
Scanner lineScanner = new Scanner(Paths.get("path.to.your.csv.file"));
int[] sumPerColumn = new int[NUMBER_OF_COLUMNS];
while (lineScanner.hasNextLine()) {
String line = lineScanner.nextLine();
String[] values = line.split(",");
for (int col = 0; col < values.length; col++) {
sumPerColumn[col] += Integer.parseInt(values[col]);
}
}
System.out.println(Arrays.toString(sumPerColumn));
}
}

Use ArrayList to store the lines you have read, then write a utility method that sum the desired row from this data. Like this:
public static void main(String[] args) throws Exception
{
String line=null;
// here we create a container for readed and splitted lines
ArrayList<String[]> lines = new ArrayList<String[]>();
try{
BufferedReader br=new BufferedReader(new FileReader(filepath));
while((line=br.readLine())!=null){
String[] distance=line.split(",");
// here we add each readed and splitted line
lines.add(distance);
System.out.println(line);
}
br.close();
}catch(IOException e){
e.printStackTrace();
}
// here we sum for a particular column.
int columnToSum = 4;
int sum = sumRows(lines, columnToSum );
System.out.println("The sum of column "+ columnToSum +" is: " + sum);
}
public static int sumRows(ArrayList<String[]> lines, int columnToSum)
{
int sum = 0;
for (String[] line: lines) {
sum = sum + Integer.parseInt(line[columnToSum]);
}
return sum;
}

How can I read only one thing in from a textfile?

I can read in from the file and am able to change the amount of lines given by changing the number in the for loop but I don't want all the numbers in my file displayed side by side like that. I need them all going down one by one randomly.
public class Assignment2 {
public static void main(String[] args) throws IOException
{
// Read in the file into a list of strings
BufferedReader reader = new BufferedReader(new FileReader("textfile.txt"));
List<String> lines = new ArrayList<String>();
String line = reader.readLine();
while( line != null ) {
lines.add(line);
line = reader.readLine();
}
// Choose a random one from the list
Random r = new Random();
for (int p = 0; p<3; p++)
{
String randomString = lines.get(r.nextInt(2));
System.out.println(lines);
}
}
}

I think what you want to print is
String randomString = lines.get(r.nextInt(2));
System.out.println(randomString);
To display only the first 20 random lines from this list of maybe 100
for (int i = 0; i < 20; i++) {
int rowNum = r.nextInt(lines.size ());
System.out.println(lines.get(rowNum);
}

Weighted mean of a column in csv file in Java

I am trying to compute the number of elements in a column present in a CSV file using which I would like to compute the mean.
This is how I tried to find the length of the column, but it gives me the length of each data in the column. It's something very silly but I have been stuck with this for a long time.
public static void main(String[] args) throws Exception {
String splitBy = ";";
int x;
BufferedReader br = new BufferedReader(new FileReader("bank.csv"));
String line;
while ((line = br.readLine()) !=null) {
String[] b = line.split(splitBy);
x = b[10].length();
System.out.println("length " + x);
}
br.close();
}
This is the CSV file:
https://raw.githubusercontent.com/pandutruhandito/practice_bank_marketing_data/master/bank/bank-full.csv

You will want something like the following. A few notes.
If a value is always required then you can simply count the number of lines in the CSV to give you the number of values in the column. Otherwise you'll need to check if there is an actual value.
You were grabbing the month column, remember indexes start at 0.
public static void main(String[] args) throws Exception {
final String splitBy = ";";
int totalValue = 0;
int columnCount = 0;
final BufferedReader br = new BufferedReader(new FileReader("C:/test/bank-full.csv"));
String line;
// Throw away headers.
line = br.readLine();
while ((line = br.readLine()) != null)
{
String[] splitCSV = line.split(splitBy);
if (!splitCSV[9].equals(""))
{
totalValue += Integer.parseInt(splitCSV[9]);
columnCount++;
}
}
br.close();
System.out.println("Average is " + totalValue / columnCount);
}

b[10].length() is incorrect: it gives you the size of string of the 10th element in a particular line (and it changes in the loop)
line.length(): lengh of the line
b.length: number of elements in a line
if you want the number of columns, you can just only keep the number of lines read (just count your readlin())
Or you can also keep everything in a table.
beware: some csv have , or ; , and perhaps inside " (if there is string)

You have to use x = b.length; which will give you exact length per line.
public static void main(String[] args) throws Exception {
String splitBy = ";";
int x;
BufferedReader br = new BufferedReader(new FileReader("bank.csv"));
String line;
while ((line = br.readLine()) !=null) {
String[] b = line.split(splitBy);
x = b.length;
System.out.println("length " + x);
}
br.close();
}