We Keep Coding

Program ignoring the last row in the file in calculation

I have a text file with data that looks like this (TestData.txt):
Name|Test1|Test2|Test3|Test4|Test5|Test6|Test7|Test8|Test9|Test10
John Smith|82|89|90|78|89|96|75|88|90|96
Jane Doe|90|92|93|90|89|84|97|91|87|91
Joseph Cruz|68|74|78|81|79|86|80|81|82|87
Suzanne Nguyen|79|83|85|89|81|79|86|92|87|88
Richard Perez|100|84|73|81|92|84|95|96|95|100
Ivan Dyer|77|91|90|75|97|94|76|89|90|92
Craig Palmer|91|84|98|89|82|75|78|96|100|97
Madeline Rogers|75|79|78|93|91|76|80|88|100|81
Chelsea Roxas|87|94|89|96|95|85|88|92|86|86
Jasper Bautista|100|83|93|100|98|97|96|97|97|98
Tyler Perez|82|89|90|78|89|96|75|88|90|96
My code parses the file and does some calculations with it.
However, in the method arrangeList() within which calls another method called getTestAvg() (calculates column means), the program ignores Tyler Perez's scores.
I noticed that the results I am getting were inaccurate so I went and printed the whole 2d array with all the test scores and the last column is nowhere to be found.
My entire code is below and I hope someone could point out what causes this.
I keep getting an IndexOutOfBounds error whenever I try to switch N (# of students) and M (# of tests) to see what happens. At first, I have 10 students and 10 tests, and all the calculations were correct, but when I added another student, the calculations became inaccurate.
I apologize in advance if my code isn't as well-designed as I'm not an experienced programmer.
import java.util.*;
import java.io.*;
public class TestAverages
{
private static int[] grades;
private static int[] testTotal;
private static int N;
private static double classTotal;
private static int M;
public static void main(String[] args) throws FileNotFoundException
{
File input = new File("TestData.txt");
Scanner in = new Scanner(input);
parseFile(in);
}
public static void parseFile(Scanner in) throws FileNotFoundException
{
TestAverages t = new TestAverages();
in.nextLine();
double total = 0.0;
ArrayList<Double> testScores = new ArrayList<Double>();
int index = 0;
while(in.hasNextLine())
{
String line = in.nextLine();
String[] data = line.split("\\|");
String name = data[0];
grades = new int[data.length - 1];
N = grades.length;
for(int i = 0; i < N; i++){
grades[i] = Integer.parseInt(data[i + 1]);
testScores.add((double)grades[i]);
}
System.out.println(name + "\t");
System.out.println("Student Average: " + t.getStudentAvg(grades) + "%\n");
total += t.getStudentAvg(grades);
M++;
}
t.arrangeList(testScores);
System.out.printf("\nClass Average: %.1f%%\n", t.getClassAvg(total));
}
public double getStudentAvg(int[] grades)
{
double total = 0.0;
double avg = 0.0;
int N = grades.length;
for(int i = 0; i < N; i++){
total += grades[i];}
avg = total / N;
return avg;
}
public double getClassAvg(double total)
{
double classAvg = total / M;
return classAvg;
}
public double[][] arrangeList(ArrayList testScores)
{
double[][] tests = new double[N][N];
int len = tests.length;
for(int i = 0; i < len; i++)
{
for(int j = 0; j < len; j++)
{
tests[i][j] = (Double) testScores.get(i*N + j);
}
}
for(int i = 0; i < len; i++)
{
double avg = getTestAvg(tests, i);
System.out.printf("\nTest " + (i + 1) + " Average: %.1f%%\n", avg);
}
return tests;
}
public double getTestAvg(double[][] testScores, int index)
{
double testAvg = 0.0;
for(int i = 0; i < N; i++)
{
testAvg += testScores[i][index];
}
return testAvg / N;
}
}
Here are the numbers I'm supposed to be getting (top) compared to what my program outputs (bottom).

As the other responses already stated, you had quite the issue with your variables and loops. I now changed N to # of students and M to # of tests to be as you stated in your question.
Next time, maybe try to improve your variable naming, so you don't get confused. (e.g. switch out n and m for s (students) and t (tests), if you like your variable names short).
This should work now. Just check against your code to see the changes.
import java.util.*;
import java.io.*;
public class TestAverages {
private static int[] grades;
private static int n = 0; // amount of students
private static int m; // amount of tests
public static void main(String[] args) throws FileNotFoundException {
File input = new File("TestData.txt");
Scanner in = new Scanner(input);
parseFile(in);
}
public static void parseFile(Scanner in) throws FileNotFoundException {
TestAverages t = new TestAverages();
in.nextLine();
double total = 0.0;
ArrayList<Double> testScores = new ArrayList<Double>();
while (in.hasNextLine()) {
String line = in.nextLine();
String[] data = line.split("\\|");
String name = data[0];
grades = new int[data.length - 1];
m = grades.length;
for (int i = 0; i < m; i++) {
grades[i] = Integer.parseInt(data[i + 1]);
testScores.add((double) grades[i]);
}
System.out.println(name + "\t");
System.out.println("Student Average: " + t.getStudentAvg(grades) + "%\n");
total += t.getStudentAvg(grades);
n++;
}
t.arrangeList(testScores);
System.out.printf("\nClass Average: %.1f%%\n", t.getClassAvg(total));
}
public double getStudentAvg(int[] grades) {
double total = 0.0;
double avg = 0.0;
for (int i = 0; i < grades.length; i++) {
total += grades[i];
}
avg = total / grades.length;
return avg;
}
public double getClassAvg(double total) {
double classAvg = total / n;
return classAvg;
}
public double[][] arrangeList(ArrayList<Double> testScores) {
double[][] tests = new double[n][m];
for (int i = 0; i < n; i++) {
for (int j = 0; j < m; j++) {
tests[i][j] = (Double) testScores.get(i * m + j);
}
}
for (int i = 0; i < m; i++) {
double avg = getTestAvg(tests, i);
System.out.printf("\nTest " + (i + 1) + " Average: %.1f%%\n", avg);
}
return tests;
}
public double getTestAvg(double[][] testScores, int index) {
double testAvg = 0.0;
for (int i = 0; i < n; i++) {
testAvg += testScores[i][index];
}
return testAvg / n;
}
}

You need to account for the different sizes. I think you want primarily the number of TESTS (not students), but you can't just use len for both index bounds.
double[][] tests = new double[N][M];
for(int i = 0; i < N; i++)
{
for(int j = 0; j < M; j++)
{
tests[i][j] = (Double) testScores.get(i*N + j);
}
}
Note that it didn't just resize the array, but it changed the loop conditions to loop the proper amount for inner and outer loop.

In the line
double[][] tests = new double[N][N];
of function arrangeList
you make your test array as N X N.
I believe youh should do something like
double[][] tests = new double[M][N];
It's just a suggestion as in your code it seems M = number of students and N = number of tests, differently from what you write in your question.
In general you should review all the method arrangeList and probably getTestAvg too (loop on N, instead of M), as the loops on variable len are intended for a N X N array, which is not the case.

Why is running time of the same algorithm different in doubling ratio test?

I am analyzing brute force Three Sum algorithm. Let's say the running time of this algorithm is T(N)=aN^3. What I am doing is that I am running this ThreeSum.java program with 8Kints.txt and using that running time to calculate constant a. After calculating a I am guessing what the running time of 16Kints.txt is. Here is my ThreeSum.java file:
public class ThreeSum {
public static int count(int[] a) {
// Count triples that sum to 0.
int N = a.length;
int cnt = 0;
for (int i = 0; i < N; i++)
for (int j = i + 1; j < N; j++)
for (int k = j + 1; k < N; k++)
if (a[i] + a[j] + a[k] == 0)
cnt++;
return cnt;
}
public static void main(String[] args) {
In in = new In(args[0]);
int[] a = in.readAllInts();
Stopwatch timer = new Stopwatch();
int count = count(a);
StdOut.println("elapsed time = " + timer.elapsedTime());
StdOut.println(count);
}
}
When I run like this:
$ java ThreeSum 8Kints.txt
I get this:
elapsed time = 165.335
And now in doubling ratio experiment where I use the same method inside another client and run this client with multiple files as arguments and wanna try to compare the running time of 8Kints.txt with above method but I get different result actually faster result. Here is my DoublingRatio.java client:
public class DoublingRatio {
public static double timeTrial(int[] a) {
Stopwatch timer = new Stopwatch();
int cnt = ThreeSum.count(a);
return timer.elapsedTime();
}
public static void main(String[] args) {
In in;
int[][] inputs = new int[args.length][];
for (int i = 0; i < args.length; i++) {
in = new In(args[i]);
inputs[i] = in.readAllInts();
}
double prev = timeTrial(inputs[0]);
for (int i = 1; i < args.length; i++) {
double time = timeTrial(inputs[i]);
StdOut.printf("%6d %7.3f ", inputs[i].length, time);
StdOut.printf("%5.1f\n", time / prev);
prev = time;
}
}
}
When I run this like:
$ java DoublingRatio 1Kints.txt 2Kints.txt 4Kints.txt 8Kints.txt 16Kints.txt 32Kints.txt
I get faster reuslt and I wonder why:
N sec ratio
2000 2.631 7.8
4000 4.467 1.7
8000 34.626 7.8
I know it is something that has to do with Java not the algorithm? Does java optimizes some things under the hood.

How to sort array in to 'bins', and print out stars for how many numbers are in that bin?

//so basically for all that is below, I'm trying to sort the random numbers that have been generated, and then 'sort' then into bins, and then for how many numbers there are in the bin, a star * will print out for every number. it will look like a histogram at the end. like this:
12 random integers in [0, 10) sorted into 2 bins:
******* 7 0.5833333 [5.0, 10.0)
***** 5 0.41666666 [0.0, 5.0)
but its like its skips that last two methods - generateBins, and printBins. how would i sort the random numbers into bins depending on the number (like above) and print a * for every number in that array bin?
public class BinSort {
final int totalBins;
final int totalRandom;
final float widthBin;
int [] storeNumbers;
int [] binCount;
public BinSort (int nBins, int nSamples, int max) {
totalBins = nBins; //total # of bins, ie 2
totalRandom = nSamples; //total random number generated, ie 12
widthBin = (float) (max/totalBins); ie 2
int [] storeNumbers = new int [max];
for (int i = 0; i < totalRandom-1; i++) {
storeNumbers[i] = Random.rand(i, max);
System.out.println(storeNumbers[i]);
}
}
void generateBins () {
int [] binCount = new int [totalBins];
for (int i=0; i < totalRandom-1; i++) {
int bin = (int)(storeNumbers[i]/ totalBins);
Math.floor(bin);
bin = binCount [i];
}
}
void printBins () {
for (int i = 0; i < binCount.length - 1; i++) {
for (int j=0; j < binCount[j]; j ++) {
System.out.print("*");
System.out.println(); }
float freq = (binCount[i]/totalRandom);
float binMin = (i * widthBin);
float binMax = (binMin * widthBin);
System.out.print(binCount[i] + freq + binMin + binMax);
System.out.println();
}
}
}

In your constructor you have
int [] storeNumbers = new int [max];
The problem here is that this will create a new local variable with the same name as your instance variable, storeNumbers. Also, the size should be totalRandom, not max. You need to create a Random object that you'll use to generate random numbers. Putting this together we get:
public BinSort (int nBins, int nSamples, int max) {
totalBins = nBins; //total # of bins, ie 2
totalRandom = nSamples; //total random number generated, ie 12
widthBin = (float) (max/totalBins); //ie 2
storeNumbers = new int [totalRandom];
Random rand = new Random();
for (int i = 0; i < totalRandom; i++) {
storeNumbers[i] = rand.nextInt(max);
}
}
This will generate totalRandom random numbers between 0 and max(exclusive) and store them the instance variable storeNumbers.
Next, in generateBins you have the same issue with
int [] binCount = new int [totalBins];
Which again will hide your instance variable binCount. The bin that a storeNumber falls into will be given by (int)(storeNumbers[i] / widthBin), and you need to increment the resulting bin by 1.
void generateBins()
{
binCount = new int[totalBins];
for (int i = 0; i < totalRandom; i++)
{
int bin = (int)(storeNumbers[i] / widthBin);
binCount[bin] += 1;
}
}
Finally, to the printing of the bins. This line
for (int j=0; j < binCount[j]; j ++)
should be
for (int j=0; j < binCount[i]; j ++)
Also, you should use printf to format the numbers you want to print.
void printBins()
{
for (int i = 0; i < binCount.length; i++)
{
for (int j = 0; j < binCount[i]; j++)
{
System.out.print("*");
}
float freq = (float)binCount[i] / totalRandom;
float binMin = i * widthBin;
float binMax = (i+1) * widthBin;
System.out.printf(" %d %.3f %.3f %.3f\n", binCount[i], freq, binMin, binMax);
}
}
Test:
public static void main(String[] args)
{
BinSort bs = new BinSort(2, 12, 10);
bs.generateBins();
bs.printBins();
}
Output:
***** 5 0.417 0.000 5.000
******* 7 0.583 5.000 10.000
Which I think is what you were looking for.
Be sure to compare your original code with the changes above and make sure you understand what the issues were and why the changes work.

Java 2D array fill - innocent optimization caused terrible slowdown

I've tried to optimize a filling of square two-dimensional Java array with sums of indices at each element by computing each sum once for two elements, opposite relative to the main diagonal. But instead of speedup or, at least, comparable performance, I've got 23(!) times slower code.
My code:
#State(Scope.Benchmark)
#BenchmarkMode(Mode.AverageTime)
#OperationsPerInvocation(ArrayFill.N * ArrayFill.N)
#OutputTimeUnit(TimeUnit.NANOSECONDS)
public class ArrayFill {
public static final int N = 8189;
public int[][] g;
#Setup
public void setup() { g = new int[N][N]; }
#GenerateMicroBenchmark
public int simple(ArrayFill state) {
int[][] g = state.g;
for(int i = 0; i < g.length; i++) {
for(int j = 0; j < g[i].length; j++) {
g[i][j] = i + j;
}
}
return g[g.length - 1][g[g.length - 1].length - 1];
}
#GenerateMicroBenchmark
public int optimized(ArrayFill state) {
int[][] g = state.g;
for(int i = 0; i < g.length; i++) {
for(int j = 0; j <= i; j++) {
g[j][i] = g[i][j] = i + j;
}
}
return g[g.length - 1][g[g.length - 1].length - 1];
}
}
Benchmark results:
Benchmark Mode Mean Mean error Units
ArrayFill.simple avgt 0.907 0.008 ns/op
ArrayFill.optimized avgt 21.188 0.049 ns/op
The question:
How could so tremendous performance drop be explained?
P. S. Java version is 1.8.0-ea-b124, 64-bit 3.2 GHz AMD processor, benchmarks were executed in a single thread.

A side note: Your "optimized" version mightn't be faster at all, even when we leave all possible problems aside. There are multiple resources in a modern CPU and saturating one of them may stop you from any improvements. What I mean: The speed may be memory-bound, and trying to write twice as fast may in one iteration may change nothing at all.
I can see three possible reasons:
Your access pattern may enforce bound checks. In the "simple" loop they can be obviously eliminated, in the "optimized" only if the array is a square. It is, but this information is available only outside of the method (moreover a different piece of code could change it!).
The memory locality in your "optimized" loop is bad. It accesses essentially random memory locations as there's nothing like a 2D array in Java (only an array of arrays for which new int[N][N] is a shortcut). When iterating column-wise, you use only a single int from each loaded cacheline, i.e., 4 bytes out of 64.
The memory prefetcher can have a problem with your access pattern. The array with its 8189 * 8189 * 4 bytes is too big to fit in any cache. Modern CPUs have a prefetcher allowing to load a cache line during in advance, when it spots a regular access pattern. The capabilities of the prefetchers vary a lot. This might be irrelevant here, as you're only writing, but I'm not sure if it's possible to write into a cache-line which hasn't been fetched.
I guess the memory locality is the main culprit:
I added a method "reversed" which works juts like simple, but with
g[j][i] = i + j;
instead of
g[i][j] = i + j;
This "innocuous" change is a performance desaster:
Benchmark Mode Samples Mean Mean error Units
o.o.j.s.ArrayFillBenchmark.optimized avgt 20 10.484 0.048 ns/op
o.o.j.s.ArrayFillBenchmark.reversed avgt 20 20.989 0.294 ns/op
o.o.j.s.ArrayFillBenchmark.simple avgt 20 0.693 0.003 ns/op

I wrote version that works faster than "simple". But, I don't know why it is faster (. Here is the code:
class A {
public static void main(String[] args) {
int n = 8009;
long st, en;
// one
int gg[][] = new int[n][n];
st = System.nanoTime();
for(int i = 0; i < n; i++) {
for(int j = 0; j < n; j++) {
gg[i][j] = i + j;
}
}
en = System.nanoTime();
System.out.println("\nOne time " + (en - st)/1000000.d + " msc");
// two
int g[][] = new int[n][n];
st = System.nanoTime();
int odd = (n%2), l=n-odd;
for(int i = 0; i < l; ++i) {
int t0, t1;
int a0[] = g[t0 = i];
int a1[] = g[t1 = ++i];
for(int j = 0; j < n; ++j) {
a0[j] = t0 + j;
a1[j] = t1 + j;
}
}
if(odd != 0)
{
int i = n-1;
int a[] = g[i];
for(int j = 0; j < n; ++j) {
a[j] = i + j;
}
}
en = System.nanoTime();
System.out.println("\nOptimized time " + (en - st)/1000000.d + " msc");
int r = g[0][0]
// + gg[0][0]
;
System.out.println("\nZZZZ = " + r);
}
}
The results are:
One time 165.177848 msc
Optimized time 99.536178 msc
ZZZZ = 0
Can someone explain me we why it is faster?

http://www.learn-java-tutorial.com/Arrays.cfm#Multidimensional-Arrays-in-Memory
Picture: http://www.learn-java-tutorial.com/images/4715/Arrays03.gif
int[][] === array of arrays of values
int[] === array of values
class A {
public static void main(String[] args) {
int n = 5000;
int g[][] = new int[n][n];
long st, en;
// one
st = System.nanoTime();
for(int i = 0; i < n; i++) {
for(int j = 0; j < n; j++) {
g[i][j] = 10;
}
}
en = System.nanoTime();
System.out.println("\nTwo time " + (en - st)/1000000.d + " msc");
// two
st = System.nanoTime();
for(int i = 0; i < n; i++) {
g[i][i] = 20;
for(int j = 0; j < i; j++) {
g[j][i] = g[i][j] = 20;
}
}
en = System.nanoTime();
System.out.println("\nTwo time " + (en - st)/1000000.d + " msc");
// 3
int arrLen = n*n;
int[] arr = new int[arrLen];
st = System.nanoTime();
for(int i : arr) {
arr[i] = 30;
}
en = System.nanoTime();
System.out.println("\n3 time " + (en - st)/1000000.d + " msc");
// 4
st = System.nanoTime();
int i, j;
for(i = 0; i < n; i++) {
for(j = 0; j < n; j++) {
arr[i*n+j] = 40;
}
}
en = System.nanoTime();
System.out.println("\n4 time " + (en - st)/1000000.d + " msc");
}
}
Two time 71.998012 msc
Two time 551.664166 msc
3 time 63.74851 msc
4 time 57.215167 msc
P.S. I'am not a java spec =)

I see, you allocated a new array for the second run, but still, did you try changing the order of "unoptimized" and "optimized" runs? – fikto
I changed order of them and optimized it a little bit:
class A {
public static void main(String[] args) {
int n = 8009;
double q1, q2;
long st, en;
// two
int g[][] = new int[n][n];
st = System.nanoTime();
int odd = (n%2), l=n-odd;
for(int i = 0; i < l; ++i) {
int t0, t1;
int a0[] = g[t0 = i];
int a1[] = g[t1 = ++i];
for(int j = 0; j < n; ++j, ++t0, ++t1) {
a0[j] = t0;
a1[j] = t1;
}
}
if(odd != 0)
{
int i = n-1;
int a[] = g[i];
for(int j = 0; j < n; ++j, ++i) {
a[j] = i;
}
}
en = System.nanoTime();
System.out.println("Optimized time " + (q1=(en - st)/1000000.d) + " msc");
// one
int gg[][] = new int[n][n];
st = System.nanoTime();
for(int i = 0; i < n; i++) {
for(int j = 0; j < n; j++) {
gg[i][j] = i + j;
}
}
en = System.nanoTime();
System.out.println("One time " + (q2=(en - st)/1000000.d) + " msc");
System.out.println("1 - T1/T2 = " + (1 - q1/q2));
}
}
And results are:
Optimized time 99.360293 msc
One time 162.23607 msc
1 - T1/T2 = 0.3875573231033026

Number of Comparisons between selection sort and bubble sort keep coming up the same

I have written this program to compare the number of operations needed to sort a random numbers using both selection and bubble sort. However, these numbers keep coming up the same and I can't figure out where my code went wrong.
static int num_comps;
public static void main(String[] args)
{
Random rnd = new Random();
// max size of array
// number of N inputs
int array_size = 32768;
int num_datasets = 12;
// allocate array once to the max size
int[] vals = new int[array_size];
// temp array with allocated array to max size
int[] tvals = new int[array_size];
// array to hold operation counts
int[] op_counts = new int[num_datasets];
int[] op_counts2 = new int[num_datasets];
// array to hold the size of each array
//
int[] arraySizes = new int[num_datasets];
int i;
int j;
int sz;
for (i = 0, sz = 16; i < num_datasets; i++, sz *= 2)
arraySizes[i] = sz;
for (int iter = 0; iter < num_datasets; iter++)
{
int curr_size = arraySizes[iter];
// load array with random values
//
for (i = 0; i < curr_size; i++)
vals[i] = rnd.nextInt(4999);
for (i = 0; i < curr_size; i++)
tvals[i] = vals[i];
// run the bubble sort algorithm
//
num_comps = 0;
bubbleSort(tvals, curr_size);
op_counts[iter] = num_comps;
//System.out.println("Num comps at " + iter + " is " + num_comps);
// run the selection-sort algorithm
num_comps = 0;
selectionSort(tvals, curr_size);
op_counts2[iter] = num_comps;
//System.out.println("Num comps at " + iter + " is " + num_comps);
}
System.out.println("Operation Counts (N vs. op Count): ");
for (int k = 0; k < num_datasets; k++)
System.out.println(arraySizes[k] + "\t\t" + op_counts[k] + "\t\t" + op_counts2[k]);
}
static void bubbleSort(int vals[], int curr_size)
{
int temp;
for (int i = 0; i < curr_size - 1; i++)
{
for (int j = 0; j < curr_size - i - 1; j++)
{
// swap
num_comps = num_comps + 1;
if (vals[j+1] < vals[j])
{
temp = vals[j];
vals[j] = vals[j+1];
vals[j+1] = temp;
}
}
}
}
static void selectionSort(int vals[], int curr_size)
{
int temp;
for(int i=0; i<curr_size - 1; i++)
{
for(int j=i+1; j<curr_size; j++)
{
num_comps = num_comps + 1;
if(vals[i] > vals[j] )
{
temp = vals[j];
vals[j] = vals[i];
vals[i] = temp;
}
}
}
}

Your selection sort algorithm does not search for the lowest value in the list. And swapping it afterwards with the index of the outer loop.
You should do something like this:
static void selectionSort(int vals[], int curr_size)
{
int temp;
for(int i=0; i<curr_size - 1; i++)
{
int lowest = i;
for(int j=i+1; j<curr_size; j++)
{
num_comps = num_comps + 1;
if(vals[lowest] > vals[j] )
{
lowest = j;
}
}
// swap lowest with current index
temp = vals[lowest];
vals[lowest] = vals[i];
vals[i] = temp;
}
}
(of course this can be optimized further)
The strength of this algorithm is not the amount of of comparisons, but this amount of swaps (which is at a minimum, I suppose).
Your bubble sort algorithm seems ok to me.
Both have the same two loops, so comparing the counts of the current implementations indeed result in the same values. But, I think you can optimize the bubble sort, to stop earlier (when no swaps were found). Again, the strength of sorting algorithms depends on the used ones, and are not necessarily the least amount of comparisons. So Using the correct algorithm for your specific task, and thereby circumventing the task-specific high cost operations, is important!