We Keep Coding

Reading a text file into an array and performing a sort in Java

I have a homework question I need help with
We have been given a text file containing one word per line, of a story.
We need to read this file into an array, perform a sort on the array and then perform a binary search.
The task also says I'll need to use an overload method, but I'm unsure where
I have a bubble sort, that I've tested on a small array of characters which works
public static void bubbleV1String(String[]numbers)
{
for(int i = 0; i < numbers.length-1; i++)
{
for(int j = 0; j < numbers.length-1; j++)
{
if(numbers[j] .compareTo(numbers[j+1])>0)
{
String temp = numbers[j+1];
numbers[j+1] = numbers[j];
numbers[j] = temp;
}
}
}
}`
And my binary search which I've tested on the same small array
public static String binarySearch(int[] numbers, int wanted)
{
ArrayUtilities.bucketSort(numbers);
int left = 0;
int right = numbers.length-1;
while(left <= right)
{
int middle = (left+right)/2;
if (numbers[middle] == wanted)
{
return (wanted + " was found at position " + middle);
}
else if(numbers[middle] > wanted)
{
right = middle - 1;
}
else
{
left = middle + 1;
}
}
return wanted + " was not found";
}
Here is my code in an app class to read in a file and sort it
String[] myArray = new String[100000];
int index = 0;
File text = new File("threebears.txt");
try {
Scanner scan = new Scanner(text);
while(scan.hasNextLine() && index < 100000)
{
myArray[index] = scan.nextLine();
index++;
}
scan.close();
} catch (IOException e) {
System.out.println("Problem with file");
e.printStackTrace();
}
ArrayUtilities.bubbleV1String(myArray);
try {
FileWriter outFile = new FileWriter("sorted1.txt");
PrintWriter out = new PrintWriter(outFile);
for(String item : myArray)
{
out.println(item);
}
out.close();
} catch (IOException e) {
e.printStackTrace();
}
When I go to run the code, I get a null pointer exception and the following message
Exception in thread "main" java.lang.NullPointerException
at java.base/java.lang.String.compareTo(Unknown Source)
at parrayutilities.ArrayUtilities.bubbleV1String(ArrayUtilities.java:129)
at parrayutilities.binarySearchApp.main(binarySearchApp.java:32)
Line 129 refers to this line of code of my bubblesort
if(numbers[j] .compareTo(numbers[j+1])>0)
And line 32 refers to the piece of code where I call the bubblesort
ArrayUtilities.bubbleV1String(myArray);
Does anyone know why I'm getting a null pointer exception when I've tested the bubblesort on a small string array? I'm thinking possibly something to do with the overloaded method mentioned earlier but I'm not sure
Thanks

You are creating an array of length 100000 and fill the lines as they are read. Initially all elements will be null and after reading the file quite a number of them is likely to still be null. Thus when you sort the array numbers[j] will eventually be a null element and thus calling compareTo(...) on that will throw a NullPointerException.
To fix that you need to know where in the array the non-null part ends. You are already tracking the number of read lines in index so after reading the file that would be the index of the first null element.
Now you basically have 2 options:
Pass index to bubbleV1String() and do for(int i = 0; i < index-1; i++) etc.
Make a copy of the array after reading the lines and before sorting it:
String[] copy = new String[index];
StringSystem.arrayCopy(myArray,0,copy,0,index);
//optional but it can make the rest of the code easier to handle: replace myArray with copy
myArray = copy;
Finally you could also use a List<String> which would be better than using arrays but I assume that's covered by a future lesson.

It seems that you have some null values in your numbers array. Try to debug your code (or just print array's content) and verify what you have there. Hard to tell anything not knowing what is in your input file.

Method overloading is when multiple functions have the same name but different parameters.
e.g. (taken from wikipedia - function overloading)
// volume of a cube
int volume(const int s)
{
return s*s*s;
}
// volume of a cylinder
double volume(const double r, const int h)
{
return 3.1415926*r*r*static_cast<double>(h);
}
Regarding your null pointer exception, you've created an array of size 100000, but it's likely you haven't read in enough information to fill that size. Therefore some of the array is empty when you try to access it. There are multiple ways you can go about this, off the top of my head that includes array lists, dynamic arrays or even moving the contents of the array to another one, once you know the size of the contents (however this is inefficient).

Program is delayed in writing to a .txt file?

So, I've searched around stackoverflow for a bit, but I can't seem to find an answer to this issue.
My current homework for my CS class involves reading from a file of 5000 random numbers and doing various things with the data, like putting it into an array, seeing how many times a number occurs, and finding what the longest increasing sequence is. I've got all that done just fine.
In addition to this, I am (for myself) adding in a method that will allow me to overwrite the file and create 5000 new random numbers to make sure my code works with multiple different test cases.
The method works for the most part, however after I call it it doesn't seem to "activate" until after the rest of the program finishes. If I run it and tell it to change the numbers, I have to run it again to actually see the changed values in the program. Is there a way to fix this?
Current output showing the delay between changing the data:
Not trying to change the data here- control case.
elkshadow5$ ./CompileAndRun.sh
Create a new set of numbers? Y for yes. n
What number are you looking for? 66
66 was found 1 times.
The longest sequence is [606, 3170, 4469, 4801, 5400, 8014]
It is 6 numbers long.
The numbers should change here but they don't.
elkshadow5$ ./CompileAndRun.sh
Create a new set of numbers? Y for yes. y
What number are you looking for? 66
66 was found 1 times.
The longest sequence is [606, 3170, 4469, 4801, 5400, 8014]
It is 6 numbers long.
Now the data shows that it's changed, the run after the data should have been changed.
elkshadow5$ ./CompileAndRun.sh
Create a new set of numbers? Y for yes. n
What number are you looking for? 1
1 was found 3 times.
The longest sequence is [1155, 1501, 4121, 5383, 6000]
It is 5 numbers long.
My code:
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.Scanner;
public class jeftsdHW2 {
static Scanner input = new Scanner(System.in);
public static void main(String args[]) throws Exception {
jeftsdHW2 random = new jeftsdHW2();
int[] data;
data = new int[5000];
random.readDataFromFile(data);
random.overwriteRandNums();
}
public int countingOccurrences(int find, int[] array) {
int count = 0;
for (int i : array) {
if (i == find) {
count++;
}
}
return count;
}
public int[] longestSequence(int[] array) {
int[] sequence;
return sequence;
}
public void overwriteRandNums() throws Exception {
System.out.print("Create a new set of numbers? Y for yes.\t");
String answer = input.next();
char yesOrNo = answer.charAt(0);
if (yesOrNo == 'Y' || yesOrNo == 'y') {
writeDataToFile();
}
}
public void readDataFromFile(int[] data) throws Exception {
try {
java.io.File infile = new java.io.File("5000RandomNumbers.txt");
Scanner readFile = new Scanner(infile);
for (int i = 0; i < data.length; i++) {
data[i] = readFile.nextInt();
}
readFile.close();
} catch (FileNotFoundException e) {
System.out.println("Please make sure the file \"5000RandomNumbers.txt\" is in the correct directory before trying to run this.");
System.out.println("Thank you.");
System.exit(1);
}
}
public void writeDataToFile() throws Exception {
int j;
StringBuilder theNumbers = new StringBuilder();
try {
PrintWriter writer = new PrintWriter("5000RandomNumbers.txt", "UTF-8");
for (int i = 0; i < 5000; i++) {
if (i > 1 && i % 10 == 0) {
theNumbers.append("\n");
}
j = (int) (9999 * Math.random());
if (j < 1000) {
theNumbers.append(j + "\t\t");
} else {
theNumbers.append(j + "\t");
}
}
writer.print(theNumbers);
writer.flush();
writer.close();
} catch (IOException e) {
System.out.println("error");
}
}
}

It is possible that the file has not been physically written to the disk, using flush is not enough for this, from the java documentation here:
If the intended destination of this stream is an abstraction provided by the underlying operating system, for example a file, then flushing the stream guarantees only that bytes previously written to the stream are passed to the operating system for writing; it does not guarantee that they are actually written to a physical device such as a disk drive.
Because of the HDDs read and write speed, it is advisable to depend as little as possible on HDD access.
Perhaps storing the random number strings to a list when re-running and using that would be a solution. You could even write the list to disk, but this way the implementation does not depend on the time the file is being written.
EDIT
After the OP posted more of its code it became apparent that my original answer is not relatede to the problem. Nonetheless it is sound.
The code OP posted is not enough to see when is he reading the file after writing. It seems he is writing to the file after reading, which of course is what is percieved as an error. Reading after writing should produce a program that does what you want.
Id est, this:
random.readDataFromFile(data);
random.overwriteRandNums();
Will be reflected until the next execution. This:
random.overwriteRandNums();
random.readDataFromFile(data);
Will use the updated file in the current execution.

selecting random lines from huge text file

I have very huge text file 18000000 line 4Gbyte, and I want to pick some random lines from it, I wrote the following piece of code to do this but it is slow
import java.io.BufferedWriter;
import java.io.IOException;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Random;
import java.util.stream.Collectors;
import java.util.stream.Stream;
public class Main {
public static void main(String[] args) throws IOException {
int sampleSize =3000;
int fileSize = 18000000;
int[] linesNumber = new int[sampleSize];
Random r = new Random();
for (int i = 0; i < linesNumber.length; i++) {
linesNumber[i] = r.nextInt(fileSize);
}
List<Integer> list = Arrays.stream(linesNumber).boxed().collect(Collectors.toList());
Collections.sort(list);
BufferedWriter outputWriter = Files.newBufferedWriter(Paths.get("output.txt"));
for (int i : list) {
try (Stream<String> lines = Files.lines(Paths.get("huge_text_file"))) {
String en=enlines.skip(i-1).findFirst().get();
outputWriter.write(en+"\n");
lines.close();
} catch (Exception e) {
System.err.println(e);
}
}
outputWriter.close();
}
}
is there more elegant faster method to do this?
thanks.

There are several things that I find troublesome about your current code.
You are currently loading the entire file into RAM. I don't know much about your sample file, but the one I used crashed my default JVM.
You are skipping the same lines over and over again, more so for the earlier lines - this is horribly inefficient, like O(n^n) or something. I would be surprised if you could handle even a 500MB file with that approach.
Here's what I came up with:
public static void main(String[] args) throws IOException {
int sampleSize = 3000;
int fileSize = 50000;
int[] linesNumber = new int[sampleSize];
Random r = new Random();
for (int i = 0; i < linesNumber.length; i++) {
linesNumber[i] = r.nextInt(fileSize);
}
List<Integer> list = Arrays.stream(linesNumber).boxed().collect(Collectors.toList());
Collections.sort(list);
BufferedWriter outputWriter = Files.newBufferedWriter(Paths.get("localOutput/output.txt"));
long t1 = System.currentTimeMillis();
try(BufferedReader reader = new BufferedReader(new FileReader("extremely large file.txt")))
{
int index = 0;//keep track of what item we're on in the list
int currentIndex = 0;//keep track of what line we're on in the input file
while(index < sampleSize)//while we still haven't finished the list
{
if(currentIndex == list.get(index))//if we reach a line
{
outputWriter.write(reader.readLine());
outputWriter.write("\n");//readLine doesn't include the newline characters
while(index < sampleSize && list.get(index) <= currentIndex)//have to put this here in case of duplicates in the list
index++;
}
else
reader.readLine();//readLine is dang fast. There may be faster ways to skip a line, but this is still plenty fast.
currentIndex++;
}
} catch (Exception e) {
System.err.println(e);
}
outputWriter.close();
System.out.println(String.format("Took %d milliseconds", System.currentTimeMillis() - t1));
}
This takes ~87 milliseconds for me on a 4.7GB file running with a sample size of 30 and filesize of 50000 and took ~91 milliseconds when I changed the sample size to 3000. It took 122 milliseconds when I increased the filesize to 10,000. Tl;Dr for this paragraph = it scales pretty well, and it scales extremely well with larger sample sizes.
In direct answer to your question "is there more elegant faster method to do this?" Yes, there is. The faster way to do it is to skip lines yourself, don't load the entire file into memory, and make sure to keep using buffered readers and writers. Also, I'd avoid trying to do your own raw Array buffers or anything like that - just don't.
Feel free to step through the method I've included if you want to see more of how it works.

My first cut at an approach would be to have a look at RandomAccess files in Java cf. https://docs.oracle.com/javase/tutorial/essential/io/rafs.html. Typically random seeks will be a lot faster than reading the whole file, but you'd then need read byte by byte to get to the beginning of the next line (for example), then read that line in byte by byte to the next newline, then seek to another random location.
I'm not sure the approach would be more elegant (depends partly on how you code it I guess), but I'd expect it to be faster.

There is no efficient way to seek lines. Only thing I can think of is using a RandomAccessFile, seeking a random possition and then reading the next 200(?) characters into an array. Then do the linebreak finding and form a String.
doc

Java: Issue Reading Text file then Converting

I've got an issue getting a method to read a file, then converting it to an integer. Here is a brief explanation of the program. It is essentially a car dealership inventory that keeps track of the vehicles in the lot by keeping them written down in a text file. When the program starts it will need to read the file and put all the current cars into an array so they can be displayed. Then the rest of the program will do other things like remove cars and add news ones etc. The part I am at is when the program first starts it needs to read the file, but I can't seem to get it to work.
The text file consists of 6 lines in total; 4 numbers first then 2 words respectively. I want the method to read the first four lines and convert those into integers and store them in a temporary array. Then after that it will read the next two lines and store those in a temporary array as well. Afterwards I take all these stored values and send them to a constructor. The constructor is then stored in an Arraylist and the Arraylist can be accessed anytime. In the output it does all of this just fine. But it wants to run through the method a second time despite barriers in place to prevent this.
Here is the code. Its a class and not the main program. I will try to explain the program as best I can inside the code.
public class Vehicle {
//All the different private variables for the constructors and methods
private int intholder[], year, type, kilometres, price, loop;
private String make, model, myline, holder[];
//The Arraylist that the different vehicle objects will be stored
ArrayList<Vehicle> allCars = new ArrayList<Vehicle>();
//The Default constructor
public Vehicle(){
make = "Vehicle Make";
model = "Vehicle Model";
type = 0;
year = 0;
kilometres = 0;
price = 0;
}
//The constructor that has information sent to it
public Vehicle(int _type, int _year, int _kilometres, int _price, String _make, String _model){
make = _make;
model = _model;
type = _type;
year = _year;
kilometres = _kilometres;
price = _price;
}
//Text file information
/*
* CAR TYPE CODE:
* 1 - Sedan
* 2 - Truck
* 3 - Crossover
* 4 - SUV
* 5 - Sports
*
* There is a total of 6 lines for each car and are as follows
* 1 - int Type integer
* 2 - int Year
* 3 - int Kilometres
* 4 - int Asking price
* 5 - String Make
* 6 - String Model
*/
//The method in question. It reads through the file, converts the integers and stores them,
//stores the strings, and sends all the information to the constructor
public void readCars()throws IOException{
BufferedReader readFile = new BufferedReader(new FileReader("C:/Users/David/Desktop/FinalProject/Carlot.txt"));
//Setting the length of the temporary arrays
holder = new String[2];
intholder = new int[4];
//The main loop in the method.
do{
//Read the first 4 lines of the file and convert them to integers.
//The try catch shouldn't have to be there because the first 4 lines
//of the file are all numbers, but I put it in there to see when it was messing up.
for(int i = 0; i < 4; i++){
myline = readFile.readLine();
try{
intholder[i] = Integer.parseInt(myline);
}
catch(NumberFormatException e){
System.out.println(e);
}
//Had this in here to see how many lines down the file it would go before messing up.
System.out.println(myline);
}
//Loop to store the Strings
for(int i = 0; i < 2; i++){
myline = readFile.readLine();
holder[i] = myline;
System.out.println(myline);
}
//Sends all the data to the constructor
Vehicle V = new Vehicle(intholder[0], intholder[1], intholder[2], intholder[3], holder[0], holder[1]);
//Several if statements to determine which subclass of vehicle it is.
if(intholder[0]==1){
Sedan S = new Sedan();
allCars.add(S);
}
else if(intholder[0]==2){
Truck T = new Truck();
allCars.add(T);
}
else if(intholder[0]==3){
Crossover C = new Crossover();
allCars.add(C);
}
else if(intholder[0]==4){
SUV U = new SUV();
allCars.add(U);
}
else if(intholder[0]==5){
Sports P = new Sports();
allCars.add(P);
}
//Only break the loop if the myline equals null
}while(myline != null);
//if the loop breaks, close the file
readFile.close();
}
Now I think I know where it is going wrong. At the end of the do/while, it checks if "myline" is null. And because the last time it read the file it was still a String the loop continues. The last time it goes through the loop, everything is null so trying to convert the integer is impossible so I get errors. But I have no idea how to get it to read the file at the end of the loop without going to the next line. Here is what the text file looks like.
1
2007
150250
5000
Toyota
Corolla
2
2005
240400
4500
Chevorlet
Silverado
I can't have it read at the end of the loop because if it does and there are still more cars after the one I just did, It goes into the next line when the loop restarts everything is thrown off.
Any help is appreciated, Thanks!

Use a labeled break statement in your for loops to simply exit out of the main do while loop when myline becomes null. The way other objects are being instantiated within the loop doesn't leave much room for easy refactoring hence the use of a labeled break makes sense here.
outerloop:
do {
for (int i = 0; i < 4; i++) {
if ((myline = readFile.readLine()) == null) break outerloop;
// ..
}
for (int i = 0; i < 2; i++) {
if ((myline = readFile.readLine()) == null) break outerloop;
// ..
}
// ..
} while (myline != null);

Maybe you could use a while loop instead of a do-while loop and read the next line from the file before anything else. Something like this:
String myline = null;
while( (myline = readFile.readLine()) != null ) {
// All your logic...
}
readFile.close();
The condition of while loop does the following: first, read the next line of the file with myline = readFile.readLine(). The previous statement returns the value of myline, so now we check that it is not null with the comparison:
(myline = readFile.readLine()) != null

Java Buffered Reader Text File Parsing

I am really struggling with parsing a text file. I have a text file which is in the following format
ID
Float Float
Float Float
.... // variable number of floats
END
ID
Float Float
Float Float
....
END
etc However the ID can represent one of two values, 0 which means it is a new field, or -1 which means it is related to the last new field. The number of times a related field can repeat it self is unlimited. Which is where the problem is occurring.
As I have a method in a library which takes an ArrayList of the new Floats, then an ArrayList of an ArrayList of the related floats.
When I try and code the logic for this I just keep getting deeper and deeper embedded while loops.
I would really appreciate any suggestions as to how I should go about this. Thanks in advance.
Here is the code I have so far.
BufferedReader br = new BufferedReader(new FileReader(buildingsFile));
String[] line = br.readLine().trim().split(" ");
boolean original = true;
while(true)
{
if(line[0].equals("END"))
break;
startCoordinate = new Coordinate(Double.parseDouble(line[0]), Double.parseDouble(line[1]));
while(true)
{
line = br.readLine().trim().split(" ");
if(!line[0].equals("END") && original == true)
polypoints.add(new Coordinate(Double.parseDouble(line[0]), Double.parseDouble(line[1])));
else if(!line[0].equals("END") && original == false)
cutout.add(new Coordinate(Double.parseDouble(line[0]), Double.parseDouble(line[1])));
else if(line[0].equals("END") && original == false)
{
cutouts.add(cutout);
cutout.clear();
}
else if(line[0].equals("-99999"))
original = false;
else if(line[0].equals("0"))
break;
}
buildingDB.addBuilding(mapName, startCoord, polypoints, cutouts);
}
New Code
int i = 0;
BufferedReader br = new BufferedReader(new FileReader(buildingsFile));
String[] line;
while(true)
{
line = br.readLine().trim().split(" ");
if(line[0].equals("END"))
break;
polygons.add(new Polygon(line));
while(true)
{
line = br.readLine().trim().split(" ");
if(line[0].equals("END"))
break;
polygons.get(i).addCoord(new Coordinate(Double.parseDouble(line[0]), Double.parseDouble(line[1])));
}
i++;
}
System.out.println(polygons.size());
int j = 0;
for(i = 0; i< polygons.size(); i++)
{
Building newBuilding = new Building();
if(polygons.get(i).isNew == true)
{
newBuilding = new Building();
newBuilding.startCoord = new Coordinate(polygons.get(i).x, polygons.get(i).y);
}
while(polygons.get(i).isNew == false)
newBuilding.cutouts.add(polygons.get(i).coords);
buildings.add(newBuilding);
}
for(i = 0; i<buildings.size(); i++)
{
System.out.println(i);
buildingDB.addBuilding(mapName, buildings.get(i).startCoord, buildings.get(i).polypoint, buildings.get(i).cutouts);
}

Maybe you should use map for new floats and related floats..if got your question it should help..example:
HashMap hm = new HashMap();
hm.put("Rohit", new Double(3434.34));

I assume that a "field" means an ID and a variable number of coordinates (pairs of floats), that, judging from your code, represents a polygon in fact.
I would first load all the polygons, each into a separate Polygon object:
class Polygon {
boolean isNew;
List<Coordinate> coordinates;
}
and store the polygons in another list. Then in a 2nd pass go through all the polygons to group them according to their IDs into something like
class Building {
Polygon polygon;
List<Polygon> cutouts;
}
I think this would be fairly simple to code.
OTOH if you have a huge amount of data in the file, and/or you prefer processing the read data little by little, you could simply read a polygon and all its associated cutouts, until you find the next polygon (ID of 0), at which point you could simply pass the stuff read so far to the building DB and start reading the next polygon.

You can try using ANTLR here, The Grammar defines the format of the text you are expecting and then you can wrap the contents in a Java object. The * and + Wildcards will resolve the complexity of while and for. Its very simple and easy to use, you dont have to construct AST you can take the parsed content from java objects directly. But the only overhead is you have to add the ANTLR.jar to your path.