We Keep Coding

Java: Most efficient way to loop through CSV and sum values of one column for each unique value in another Column

I have a CSV file with 500,000 rows of data and 22 columns. This data represents all commercial flights in the USA for one year. I am being tasked with finding the tail number of the plane that flew the most miles in the data set. Column 5 contains the airplain's tail number for each flight. Column 22 contains the total distance traveled.
Please see my extractQ3 method below. First, created a HashMap for the whole CSV using the createHashMap() method. Then, I ran a for loop to identify every unique tail number in the dataset and stored them in an array called tailNumbers. Then for each unique tail number, I looped through the entire Hashmap to calculate the total miles of distance for that tail number.
The code runs fine on smaller datasets, but once the sized increased to 500,000 rows the code becomes horribly inefficient and takes an eternity to run. Can anyone provide me with a faster way to do this?
public class FlightData {
HashMap<String,String[]> dataMap;
public static void main(String[] args) {
FlightData map1 = new FlightData();
map1.dataMap = map1.createHashMap();
String answer = map1.extractQ3(map1);
}
public String extractQ3(FlightData map1) {
ArrayList<String> tailNumbers = new ArrayList<String>();
ArrayList<Integer> tailMiles = new ArrayList<Integer>();
//Filling the Array with all tail numbers
for (String[] value : map1.dataMap.values()) {
if(Arrays.asList(tailNumbers).contains(value[4])) {
} else {
tailNumbers.add(value[4]);
}
}
for (int i = 0; i < tailNumbers.size(); i++) {
String tempName = tailNumbers.get(i);
int miles = 0;
for (String[] value : map1.dataMap.values()) {
if(value[4].contentEquals(tempName) && value[19].contentEquals("0")) {
miles = miles + Integer.parseInt(value[21]);
}
}
tailMiles.add(miles);
}
Integer maxVal = Collections.max(tailMiles);
Integer maxIdx = tailMiles.indexOf(maxVal);
String maxPlane = tailNumbers.get(maxIdx);
return maxPlane;
}
public HashMap<String,String[]> createHashMap() {
File flightFile = new File("flights_small.csv");
HashMap<String,String[]> flightsMap = new HashMap<String,String[]>();
try {
Scanner s = new Scanner(flightFile);
while (s.hasNextLine()) {
String info = s.nextLine();
String [] piecesOfInfo = info.split(",");
String flightKey = piecesOfInfo[4] + "_" + piecesOfInfo[2] + "_" + piecesOfInfo[11]; //Setting the Key
String[] values = Arrays.copyOfRange(piecesOfInfo, 0, piecesOfInfo.length);
flightsMap.put(flightKey, values);
}
s.close();
}
catch (FileNotFoundException e)
{
System.out.println("Cannot open: " + flightFile);
}
return flightsMap;
}
}

The answer depends on what you mean by "most efficient", "horribly inefficient" and "takes an eternity". These are subjective terms. The answer may also depend on specific technical factors (speed vs. memory consumption; the number of unique flight keys compared to the number of overall records; etc.).
I would recommend applying some basic streamlining to your code, to start with. See if that gets you a better (acceptable) result. If you need more, then you can consider more advanced improvements.
Whatever you do, take some timings to understand the broad impacts of any changes you make.
Focus on going from "horrible" to "acceptable" - and then worry about more advanced tuning after that (if you still need it).
Consider using a BufferedReader instead of a Scanner. See here. Although the scanner may be just fine for your needs (i.e. if it's not a bottleneck).
Consider using logic within your scanner loop to capture tail numbers and accumulated mileage in one pass of the data. The following is deliberately basic, for clarity and simplicity:
// The string is a tail number.
// The integer holds the accumulated miles flown for that tail number:
Map<String, Integer> planeMileages = new HashMap();
if (planeMileages.containsKey(tailNumber)) {
// add miles to existing total:
int accumulatedMileage = planeMileages.get(tailNumber) + flightMileage;
planeMileages.put(tailNumber, accumulatedMileage);
} else {
// capture new tail number:
planeMileages.put(tailNumber, flightMileage);
}
After that, once you have completed the scanner loop, you can iterate over your planeMileages to find the largest mileage:
String maxMilesTailNumber;
int maxMiles = 0;
for (Map.Entry<String, Integer> entry : planeMileages.entrySet()) {
int planeMiles = entry.getValue();
if (planeMiles > maxMiles) {
maxMilesTailNumber = entry.getKey();
maxMiles = planeMiles;
}
}
WARNING - This approach is just for illustration. It will only capture one tail number. There could be multiple planes with the same maximum mileage. You would have to adjust your logic to capture multiple "winners".
The above approach removes the need for several of your existing data structures, and related processing.
If you still face problems, put in some timers to see which specific areas of your code are slowest - and then you will have more specific tuning opportunities you can focus on.

I suggest you use the java 8 Stream API, so that you can take advantage of Parallel streams.

displaying a certain output for a multi dice roller

I am very new to java and I set a goal for myself to make a dice rolling program (keeping it small). The end goal is to be able to roll a user-selected amount of dice and be able to have each die have a different amount of side if need be and I have it get the number of dice and how many sides each has. This is the code I made for it (Might be really bad, sorry if it is) :
public class Roller {
public final Random rando;
public final int faces;
public Roller(int faces) {
this.rando = new Random();
this.faces = faces;
}
public int roll() {
return 1 + rando.nextInt(faces);
}
//above code is not mine I built off what my friend wrote cause i didnt know if i still need it
public static void main(String[] args) {
Random rand = new Random();
Scanner scan = new Scanner(System.in);
System.out.print("How many dice do you want to roll?\n");
int D6 = scan.nextInt();
ArrayList<Integer> list = new ArrayList<>();
for (int i = 0; i < D6; i++) {
System.out.print("How many sides does die " + (i + 1) + " have?\n");
Roller dice = new Roller(scan.nextInt());
list.add(dice.roll());
}
}
}
Now I'm at the point where I want to display the ArrayList but I want to display it as
"Dice 1 rolled #
Dice 2 rolled #"
etc. and I'm lost on how to do that especially with the varying number of dice. Any help is very appreciated.

Let's assume you ran this and now have a List of values, [1, 3, 5, 2, 4] and you want to display them as you described.
In your main method, you have the list, so you could do some looping and string formatting to get your desired output. (edited to use printf() rather than String.format())
// in main...
// after list has all it's values
for (int i = 0; i < list.size(); i++) {
System.out.printf("Dice #%d rolled %d", i+1, list.get(i));
}
Note that the below statements are still valid, and can still be applied to printf(...)
To walk through it, String formatting is just a fancy way to format your strings (funny how that works out). The first %d corresponds to the first value given to format(), which is i+1. It's i+1 as opposed to plain i because otherwise you'd see "Dice #0 rolled ..." first, since you start indexing arrays and lists at 0. With the second %d in the format() call, you pass in list.get(i) which is the value in the list at the given index. This should correspond nicely to the order of the rolls.
This didn't have to be done with String formatting. I find it tends to be better and easier to read personally, but it is easily substituted with String concatenation.
//replace the print statement with this if you want
System.out.println("Dice #" + (i+1) + " rolled " + list.get(i));
It seems sloppier to me IMO, and needing to remember to leave spaces, or omit spaces between concatenated parts can be annoying.

Using loops and arrays to switch between players in a quiz game

I'm in the process of writing a very simple quiz-style boardgame that moves players around the board based on if they answer the question correctly and what they roll on the dice. I'm attempting to create and pass an array mehtod that stores the scores of player 1 and player 2, but the array doesn't seem to actually keep track of the score. For example, a fragment of some of the code is as follows:
public static int[] scorearray
{
int scoreplayer1 = 0;
int scoreplayer2 = 0;
return new int[] = {scoreplayer1, scoreplayer2};
}
public static int questions(int diceroll, int[] score)
{
String textinput = input("What's 9+10?");
int ans = Integer.parseInt(textinput);
if (ans == 19)
{
output("Fantastic answer, that's correct!");
diceroll = dicethrow(diceroll); // rolls the dice
output("Move forward " + diceroll + " squares. You are on square " + score[0]);
//I need the output above to print position 0 in the above array
score[0] = score[0] + diceroll; //array stores the cumulative score
}
else
{
output("Sorry, the answer was 19. Next player's turn.")
//This is where I need the loop to switch between players
}
In addition, I need to come up with a way of switching between player 1 and 2 while also switching to the position 1 in the above array, that is, I need to add to player two's score instead of player one's. I've been combing through this code for ages now trying to figure out how to do this but I can only come up with the idea of using a for/while loop. Other than that I'm truly stumped.
Thanks.
EDIT: It appears that my array apparently still does not store the score when being used in the method questions.
Also I have now realised I can control whose turn it is by creating another method, for example public static void activePlayer() but I'm still not sure how to use a loop (or anything else for that matter) to switch between the two. Also my concern is where I use score[0] = score[0] + diceroll; in my questions method only keeps the score (or at least attempts to; see above problem) for player one. How would I switch it to keep score for score[1]? Please.

your options here seem to be either have your questions function output the score or change your score object to be a static object instead of a static function.
public static int[] scorearray = [0,0];
or
public static int[] questions(int diceroll, int[] score)

How do you change each part of an array?

So here is the code I have right now.
public static void main(String[] args) {
Scanner keyboard = new Scanner (System.in);
int set[] = new int[5];
set[0] = (int)(Math.random()*6)+1;
set[1] = (int)(Math.random()*6)+1;
set[2] = (int)(Math.random()*6)+1;
set[3] = (int)(Math.random()*6)+1;
set[4] = (int)(Math.random()*6)+1;
System.out.println("Your current dice: " + set[0] + " " + set[1] + " " + set[2] + " " + set[3] + " " +set[4] );
System.out.println("Select a die to re-roll (-1 to keep remaining dice):");
int ask = keyboard.nextInt();
After this if the user types in let's say 1 then set[1] should change to the number zero and so it becomes x0xxx and if the user also wants the 3rd number to change then it should be x0x0x.
The x's are just the generated random numbers by the way.
How do I keep doing this? It has to be a total of utmost 5 times.

Here are the basic steps you should follow to accomplish what you want/need.
Read user input (using Scanner or something else).
Validate if the user input is a valid index for the array (this is, the input is a number with a value between 0 and 5). You can store this in a variable int x.
Change the value of the element of the array inside the index user entered to 0 (or the value you want/need). This would traduce into something like set[x] = ... (change the ... by the proper value).

The way you do one thing many times is in a loop. The key is in learning which kind of loop to use.
For things that get applied to every element, use a for-each loop. For things that need done until some condition use a while loop. For things that need done until some condition becomes false, use a do-until loop.
The thing you do is the same, that goes into the block of the loop. The thing you are "working on" changes, that is a variable which the loop will set each time it "goes through" the loop's block.
In your case
for (Die die : dice) {
die.roll();
}
where class Die looks like
public class Die {
private int value;
public Die() {
roll();
}
public void roll() {
value = (int)(Math.random()*6)+1;
}
public int getValue() {
return value;
}
}
Then, since you need "order" (first, second, third, etc...) use a data structure that can contain Objects (like your Die)
List<Die> dice = new ArrayList<>();
Arrays are nice, and you do need to know how to use them; however, there are far better ways of solving most problems by not using them.
When you really can't get around using them, use a for loop to walk each array index.

Comparing data from SQLite database - Android (java)

I made a SQLite highscores that has 3 columns; rank(int), score(long) and percentage(int). I am trying to pull all the data from the columns and compare them to the score the user just got to see if their new score made the highscores list that holds the top 10.
I want the highscores to be prioritized by percentage first and then score to break any ties. Below is where I am starting to do this. Am I on the correct track? I have never used SQLite even outside of Java so this is all brand new to me.
Thank you in advance for your help.
//Check if new record makes the top 10.
public boolean check(int rank, long score, int percentage) {
//SQL query to get last score/percentage if highscores has 10 records...
long[] scores = new long[9];
int[] percentages = new int[9];
scores = db.execSQL("SELECT " + SCORE + " FROM " + TABLE + ";"); //Error: Type mismatch
percentages = db.execSQL("SELECT " + PERCENTAGE + " FROM " + TABLE + ";"); //Error: Type mismatch
//Algorithms to compare scores and percentages go here...
}

scores = db.execSQL("SELECT " + SCORE + " FROM " + TABLE + ";");
execSQL returns void.
As per javadoc
Execute a single SQL statement that is NOT a SELECT or any other SQL statement that returns data.
You may need to use either query (or) rawQuery mechanisms to do select.