We Keep Coding

What's wrong with my 2-D Array mean and median calculation (using classes in Java)?

I'm doing a java project at school and one of the tasks that I'm stuck on is correctly calculating and displaying the mean and median of rows and columns of a 2-D array. My code is in classes so I'll try to ask this question as clearly as possible. Here's my code:
//Main class
package com.company;
import java.io.*;
public class Main {
public static void main(String[] args) {
FileALL Free = new FileALL("src/com/company/text.txt");
try{
Free.FileRead();
Free.Eraser();
Free.Writing();
Free.Meanie();
Free.Median();
}
catch (Exception e){
e.printStackTrace();
}
}
}
//FILEALL Class (the class with all of the calculations, file I/O, etc.)
package com.company;
import java.io.*;
import java.util.Arrays;
public class FileALL {
//Declare variables
private static int row = 20;
private static int col = 50;
private int i;
private int j;
private int size;
private int Mean;
private int Median;
private int elements;
private static int RawData[][] = new int[row][col];
private File f;
//New method for use in the main class
//Initialize the file
public FileALL (String FileRead) {
f = new File(FileRead);
}
//Method for file input
public void FileRead () {
try {
BufferedReader reader = new BufferedReader(new FileReader(f));
String line = reader.readLine();
while ((line != null)) {
String[] Values = line.trim().split("\\s+");
for (int j = 0; j < Values.length; j++) {
RawData[i][j] = Integer.parseInt(Values[j]);
System.out.print(RawData[i][j] + " ");
}
i++;
System.out.println();
line = reader.readLine();
}
reader.close();
} catch (IOException e) {
System.out.println("ERROR");
System.exit(0);
}
}
//This was a test for file output; I didn't really use it
/*public void FileWrite () throws IOException {
FileInputStream in = null;
FileOutputStream oot = null;
try {
in = new FileInputStream("text.txt");
oot = new FileOutputStream("OG_out.txt");
int c;
while ((c = in.read()) != -1){
oot.write(c);
}
}finally {
if (in != null){
in.close();
}
if (oot != null){
oot.close();
}
}
}*/
//Method for mean calculation
public int Meanie (){
int sum = 0;
for (i = 0; i < row; i++){
for (j = 0; j < col; j++){
if (RawData [i][j] > 0){
sum += RawData[i][j];
size++;
}
}
}
if (size != 0) {
Mean = (sum/size);
}
System.out.println("\nThe mean is " + Mean);
return Mean;
}
//Median method
public void Median (){
//Make a separate 1-D array to store the items
int [] list = new int[RawData.length*RawData[0].length];
//Initialize a new integer to keep track of the position of the items in the 1-D array
int Pos = 0;
//Iterating over the 2-D array, now adding in the new list position integer
for (i = 0; i < RawData.length; i++){
for(j = 0; j< RawData.length; j++){
list[Pos++] = RawData[i][j];
}
}
//Sort the 1-D array
Arrays.sort(list);
elements = row - Pos;
System.out.print("\nThe median is: " + MED(list));
}
//Separate method for Median
public double MED(int[]m){
int middle = m.length/2;
if (m.length%2 == 1){
return m[middle];
}
else {
return (m[middle - 1] + m[middle]) / 2;
}
}
//Method for Writing the means, medians, modes, and data sets of the array into a file created specifically for output (rows only)
//Need to figure out how to withhold the zeroes from being printed
public void Writing () throws IOException {
BufferedWriter writer = new BufferedWriter(new FileWriter("src/com/company/OG_out.txt", true));
writer.write("The mean of the input array is " + Mean + "\n\n");
int a = 1;
//I'm using 'i' here because 'i' is my integer for the row counter (for the sake of simplicity)
for (i = 0; i < row; i++){
writer.write("Data set " + a + ": " + Arrays.toString(RawData[i]) + "\n\n==================================================\n\n");
writer.flush();
a++;
}
}
//This is a short method that will erase the data that was written into the output file
public void Eraser () throws IOException {
new BufferedWriter(new FileWriter("src/com/company/OG_out.txt", false));
}
}
//Text.txt file (you can enter any set of numbers in a 2-D array fashion)
1 2 3
5
6
I'm not getting any errors of exceptions but the mean sometimes comes up incorrect (which probably means it's always incorrect) and the median always comes up as 0.0. I'll also need to do the mode of the rows and columns but I don't know where to start (maybe I'll ask a separate question).
Thanks for reading and taking the time to help me out (if you choose to do so).
EDIT: I need a maximum of 20 rows and 50 columns. Also, I got the mean calculation working fine.

RawData.length is always 20 no matter how much data you actually enter. RawData[0].length is always 50. Once you allocate the array, you can't change the length downward even if you enter less data. So there will be a lot of unused elements in your array that Java initializes to 0. This means you can't use .length to loop over the array. You will need another way to tell your algorithm how many rows and columns you really have. Otherwise all those extra 0's will be part of the computation.

If you are retrieving your data from a text file with delimited line columns and the potential for basically an unspecified number of rows (file lines) then why are you declaring your RawData[][] 2D Array to a fixed number of elements when you don't really know yet what the max number of columns and rows are within the data file unless of course you are going to handle chunks of data which it does not appear you are doing. What if you have 22 or 30 lines of data in file? You'll end up with a Array Out Of Bounds Exception.
In my opinion, I think the best way to handle this if you must have a 2D Array and the data file will contain lines of varying data columns is to do a two pass read over the file.
The first pass would be used to determine the maximum number of columns in any one file line (row) and also determining the number of lines within the file which actually contain data (not a blank line - just in case it does). Determining the number of valid data lines within the data file will allow you to set the Row quantity of your 2D Array. By determining the maximum number of columns in any one line within the data file will allow you to set the Columns quantity of your 2D Array.
File f = new File("src/com/company/text.txt");
if (!f.exists()) { System.exit(0); }
// PASS 1
int maxColumns = 0;
int rows = 0;
String line = "";
try {
BufferedReader reader = new BufferedReader(new FileReader(f));
while((line = reader.readLine()) != null){
if (!line.equals("")) {
String[] values = line.trim().split(" ");
// Get the largest number of columns on any line in file.
if (values.length > maxColumns) { maxColumns = values.length; }
rows++;
}
}
reader.close();
}
catch (IOException e) {
System.out.println("IOException ERROR\n\n" + e.getMessage());
System.exit(0);
}
// Declare and initialize the rawData[][] 2D Array.
Object[][] rawData = new Object[rows][maxColumns];
The second pass read over the file would be for applying the file data into their respective 2D Array elements and because you are dealing with varying columns I would declare the 2D Array as Object since the natural data type within a text file is String and a Object type allows you to determine if a specific array element contains 'null', however, if the 2D Array is declared as a numerical data type then a empty element would contain the numerical value of zero (0) which could be a valid data value in file. Since each column element within an Object 2D Array can contain a different data type we just need to load the required ones with our desired numerical data type and leave the ones we don't need as they are:
// PASS 2
rows = 0;
line = "";
try {
BufferedReader reader = new BufferedReader(new FileReader(f));
while((line = reader.readLine()) != null){
if (!line.equals("")) {
String[] values = line.trim().split(" ");
for (int i = 0; i < values.length; i++) {
rawData[rows][i] = values[i];
}
rows++;
}
}
reader.close();
}
catch (IOException e) {
System.out.println("IOException ERROR\n\n" + e.getMessage());
System.exit(0);
}
You can always convert the Array elemental data to the desired numerical data type when iterating through and retrieving elemental data within the array. For example:
for (int i = 0; i < rawData.length; i++) {
for (int j = 0; j < rawData[0].length; j++) {
// If the 2D Array element is empty then ignore it...
if (rawData[i][j] != null) {
// otherwise get the elemental data.
int num = (int) rawData[i][j]; //Cast the object to Integer.
System.out.println("i = " + i + " | j = " + j + " | element contains: " + num);
// .....................................
// ... Do whatever you want with num ...
// .....................................
}
}
}
Of course I show all this done within a single stream or method of code but it too can be moved into individual classes...that's for you to do.
I hope this helps in some way.

Reading from a text file into an array - getting "nulls"

I'm reading from a file and copying that into an array. My file has five lines of text, a sentence each. I get my output "Array size is 5" but nothing after that. If I do add a print line of the array, it gives me 5 nulls...
Can someone help explain what I did wrong? Thanks!
public static int buildArray() throws Exception
{
System.out.println("BuildArray is starting ");
java.io.File textFile; // declares a variable of type File
textFile = new java.io.File ("textFile.txt"); //reserves the memory
Scanner input = null;
try
{
input = new Scanner(textFile);
}
catch (Exception ex)
{
System.out.println("Exception in method");
System.exit(0);
}
int arraySize = 0;
while(input.hasNextLine())
{
arraySize = arraySize + 1;
if (input.nextLine() == null)
break;
}
System.out.println("Array size is " + arraySize);
// Move the lines into the array
String[] linesInRAM = new String[arraySize];// reserve the memory
int count = 0;
if (input.hasNextLine())
{
while(count < arraySize)
{
System.out.println("test");
linesInRAM[count] = input.nextLine();
System.out.println(linesInRAM[count]);
count = count + 1;
}
}

In this code
int count = 0;
if (input.hasNextLine())
The above hasNextLine will always be false as you have already read all the way through the file.
Either reset the scanner to the beginning of the file, or use a dynamic list e.g. ArrayList to add the elements to.

My Java is a bit rusty, but the basic gist of my answer is that you should create a new Scanner object so that it reads from the beginning of the file again. This is the easiest way to "reset" to the beginning.
Your code is currently not working because when you call input.nextLine() you're actually incrementing the scanner, and thus at the end of that first while() loop input is sitting at the end of the file, so when you call input.nextLine() again it returns null.
Scanner newScanner = new Scanner(textFile);
Then in the bottom of your code, your loop should look like this instead:
if (newScanner.hasNextLine())
{
while(count < arraySize)
{
System.out.println("test");
linesInRAM[count] = newScanner.nextLine();
System.out.println(linesInRAM[count]);
count = count + 1;
}
}

Performance improvement : How to read only last 10 lines of 100,000 files within 30mins

I have one doubt regarding which collection should I use. Have discussed a lot but wanted more inputs.
I have a source system from where 100,000s of trade files come to my application in say every 30mins. Each file having many lines of code (say 1000). My app should store and process only last 10 lines of trade details.
If I read file contents using buffer reader line by line then I have to keep on adding each line details in some collection and finally once I reach the last line somehow remove all and keep only last 10 lines. So by keeping all 1000 lines in collection even if I do not require all is a performance issue. Is there any collection or any approach to improve this.

You can use a CircularFifoBuffer:
CircularFifoBuffer is a first in first out buffer with a fixed size that replaces its oldest element if full.
Usage for keeping in memory only the last 10 lines:
CircularFifoBuffer buffer = new CircularFifoBuffer(10);
// read lines and add them to the buffer
At the end of reading the lines, the buffer only contains the last 10 lines.

Use a RandomAccessFile, and try ever larger buffers to read.
I made a tail function with a line-length-hint, to make a guess. Be aware that whether the file ends with a newline or may make a difference in the result. Also the code can be improved upon (power of two block size and so on).
File textFile = new File("...");
String[] lines = tail(textFile, "UTF-8", 10, 160);
System.out.println("#Lines: " + lines.length);
for (String line : lines) {
System.out.println(line);
}
String[] tail(File textFile, String charSet, int lines, int lineLengthHint)
throws IOException {
if (lineLengthHint < 80) {
lineLengthHint = 80;
}
RandomAccessFile in = new RandomAccessFile(textFile, "r");
try {
long fileSize = in.length();
int bytesCount = lines * lineLengthHint;
// Loop allocating a byte array hopefully sufficiently large.
for (;;) {
if (fileSize < bytesCount) {
bytesCount = (int)fileSize;
}
byte[] bytes = new byte[bytesCount];
in.seek(fileSize - bytesCount);
in.readFully(bytes);
int startIndex = bytes.length; // Position of last '\n'.
int lineEndsFromStart = 0;
boolean bytesCountSufficient = true;
while (lineEndsFromStart - 1 < lines) {
int pos = startIndex - 1;
while (pos >= 0 && bytes[pos] != '\n') {
--pos;
}
startIndex = pos; // -1 will do fine.
++lineEndsFromStart;
if (pos < 0) {
bytesCountSufficient = false;
break;
}
}
if (bytesCountSufficient || fileSize == bytesCount) {
String text = new String(bytes, startIndex + 1,
bytes.length - (startIndex + 1), charSet);
return text.split("\r?\n");
}
// Not bytesCountSufficient:
//lineLengthHint += 10; // Average line length was larger.
bytesCount += lineLengthHint * 4; // Try with more.
}
} finally {
in.close();
}
}

You could easily fashion a discarding queue which keeps only the last 10 lines. A LinkedList would be a good start for such an implementation. See this previous question on the topic.
This won't solve the problem of reading in the whole file, but getting around that means quite a bit more coding. You'd need a RandomAccessFile and search for the 10nth newline from the end. The appropriateness of this solution depends on how big the files are.

You could use a String array of size 10 and only always store the last 10 lines:
BufferedReader in = ...
String[] buffer = new String[10];
int bufferStartIndex = 0;
for (String line; (line = in.readLine()) != null;) {
buffer[bufferStartIndex++ % buffer.length] = line;
}
At the end of the for-loop, bufferStartIndex will point to the first of the 10 last lines of the file. However if the file contains less than 10 lines, then you should reset bufferStartIndex to 0.

import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.LinkedList;
import java.util.Queue;
public class Test {
private static Queue<String> bottom=new LinkedList<String>();
private static int count=0;
public static void main(String[] args) throws IOException{
func(3);
}
//function to get count, bottom n lines
private static void func(int n) throws IOException{
FileInputStream fstream = new FileInputStream("abc.txt");
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));
String strLine;
//Read File Line By Line
while ((strLine = br.readLine()) != null){
count++;
if(count<=n){
//initialize bottom as top n
bottom.add(strLine);
}else{
bottom.remove();
bottom.add(strLine);
}
}
System.out.println(count);
System.out.println(bottom.toString());
br.close();
}
}
I have used Queue to get the bottom n lines. For further details you can visit: http://blog.everestkc.com.np

A Good Method to read files in Java

I Really would appreciate it if someone can help me with this. I am trying to do external sorting and I am stuck on the part of merging. I get how I should merge it just not sure what function to use.
Right now I am trying to read in the first words of multiple small text files and store them in a string array of the size of the amount of files. So basically I will have a string array of the first word of each file. Then I determine which one is the smallest alphabetically wise and write that to a new file, after that I would read the next word of the file of that smallest word. This word would be placed in the position of the smallest word that got outputted in string array and compare it to the rest of the first word from the other file. This will keep repeating till all words are sorted.
The main problem I am running into is the fact that I was using scanner and after the first run of comparing it cant switch the smallest word with the next word in the file because scanner don't keep a point of what it has read. I know readline do but since my files are all words separated by only a white space I can't use readline. Can someone please guide me to a sufficient reading function that can't help me solve this problem.
for (int i = 0; i<B;i++)
{
try
{
BufferedReader ins = new BufferedReader(new FileReader(Run-"+ i + ".txt"));
Scanner scanner2 = new Scanner(ins);
temp3[i] = scanner2.next();
System.out.println(temp3[i]);
}
catch(IOException e)
{
}
}
for(int i=0;i<N;i++)
{
String smallest = temp3[0];
int smallestfile = 0;
for(j=0;j<B;j++)
{
int comparisonResult = smallest.compareTo(temp3[j]);
if(comparisonResult>0)
{
smallest = temp3[j];
smallestfile = j;
}
}
BufferedReader ins = new BufferedReader(new FileReader("C:/Run-"+ smallestfile + ".txt"));
Scanner scanner2 = new Scanner(ins);
if(scanner2.hasNext())
{
temp3[smallestfile]=scanner2.next();
}
}
}
catch(Exception e)
{
}

If the files are small enough read the entire file to memory, and use String.split() to separate the strings in arrays and do your magic.
If the the files are bigger, keep then open and read each byte until you find and space, then do it for all the files, compare the strings, do your magic and repeat until all the files reach the end.
EDIT :
how to read the files with BufferedReader
how to split the lines with String.split()
String line = readeOneLineFromTheCurrentFile();
String[] words = line.split(" ");

As for temporarily sorting/storing the words, use a PriorityQueue (not an array). Sorry, I'm too busy watching baseball to add more.

I'm not sure, if I understood you right, but a Scanner does keep the position in a file. You need just as many of them as there are files
import java.util.Scanner;
import java.io.File;
import java.io.FileNotFoundException;
public class so {
// returns the index of the smallest word
// returns -1 if there are no more words
private static int smallest(String[] words) {
int min = -1;
for (int i = 0; i < words.length; ++i)
if (words[i] != null) {
if (min == -1 || words[i].compareTo(words[min]) < 0)
min = i;
}
return min;
}
public static void main(String[] args) throws FileNotFoundException {
// open all files
Scanner[] files = new Scanner[args.length];
for (int i = 0; i < args.length; ++i) {
File f = new File(args[i]);
files[i] = new Scanner(f);
}
// initialize first words
String[] first = new String[args.length];
for (int i = 0; i < args.length; ++i)
first[i] = files[i].next();
// compare words and read following words from scanners
int min = smallest(first);
while (min >= 0) {
System.out.println(first[min]);
if (files[min].hasNext()) {
first[min] = files[min].next();
} else {
first[min] = null;
files[min].close();
files[min] = null;
}
min = smallest(first);
}
}
}
Tested with
a.txt: a d g j
b.txt: b e h k m
c.txt: c f i
Update:
In your example, you open and close the file inside the outer for loop. When you reopen a file the next time, it starts at the beginning of the file, of course.
To prevent this, you must keep the file open and move the scanner2 variable and its initialization in front of the outer for loop. You also need multiple Scanner variables, i.e. an array, to keep multiple files open simultaneously.

How to read a GRID of data from a text file in Java?

When I say a grid, I mean a multidimensional array. I want this because I am making a 2D game and I want to be able to load levels from data text files. Lets say, for example, I have this 2D array level[3][3]. A simple 3x3 map. And I also have a text file that reads:
1 2 3
4 5 6
7 8 9
In c++, I can simply do:
for (x=0; i<map_width; x++)
{
for (y=0; y<map_height; y++)
{
fscanf(nameoffile, "%d", map[x][y]);
}
}
And that would put all the contents of the text file accordingly into the array.
HOWEVER
I have no idea how to do this in java. Is there any sort of equivalent that will just place the data into the array accordingly? I already know about the scanner class, but I don't know how to use it. I have searched google, to no avail. It doesn't explain much.
Please help! Specifically, I want to know how to scan the file and put whatever int it reads IN THE APPROPRIATE PLACE in the array.
My current code is this, however, it throws a NoSuchElementException:
public void loadMap() {
Scanner sc = null;
try {
sc = new Scanner(inputmap);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
for (int x = 0; x < width; x++) {
for (int y = 0; y < height; y++) {
map[x][y] = sc.nextInt();
}
}
Where inputmap is the file, map[][] is a grid of data for each of the tiles on the map and width and height are pre-specified in a constructor.

Your question is very unhelpful when it comes to how your text files will be actually formatted. For example,
123
456
789
is very different from
1 2 3
4 5 6
7 8 9
and furthermore, you haven't mentioned whether they are always ints, or
1 2 3
4 5 6
a b c
etc. If you gave us a precise description of exactly what goes in these text files we could help you more. The best I can do is show you how to use Scanner to input stuff in general:
The for loop would look similar in Java, but you have to initialize a Scanner object.
Scanner input = new Scanner(myFile); //whatever file is being read
for (x=0; i<map_width; x++)
{
for (y=0; y<map_height; y++)
{
int nextTile = input.nextInt(); //reads the next int
// or
char nextTile = input.nextChar(); //reads the next char
}
}
Beyond that, I would need to know more about what is actually inside these input files.
EDIT:
I copied your for loop directly from your code, but you may want to swap the inner and outer for loops. Shouldn't the width be the inner parameter (reading left to right)?

In Java, it works similar - create a java.util.Scanner object for your file and use it's nextInt method instead of fscanf.

If you don't know the dimensions of the grid
static int[][] readFile(String filename) {
try {
File textfile = new File (GridSearchTest.class.classLoader.getResource(filename).toURI());
Scanner fileScanner = new Scanner(textfile);
int size = Integer.parseInt(fileScanner.next());
String line = fileScanner.nextLine();
int[][] grid = new int [size][size];
int i = 0; int j = 0;
while (fileScanner.hasNextLine()) {
line = fileScanner.nextLine();
System.out.println (line);
Scanner lineScanner = new Scanner(line);
while (lineScanner.hasNext()) {
grid[i][j] = Integer.parseInt(lineScanner.next());
i++;
}
lineScanner.close();
i=0;
j++;
}
fileScanner.close();
return grid;
} catch (IOException e) {
System.out.println("Error reading file: "+ e.getMessage());
System.exit(0);
};
}