We Keep Coding

How do I avoid ConcurrentModificationException in ArrayList ONLY when iterating?

For clarification - I DO NOT want to remove anything from the ArrayList. Therefore 90% of all the answers I have found don't actually apply. I can't find anything here, or elsewhere that helps me out much!
I'm writing a Java Application to play Hangman where the opponent (computer) is essentially cheating, in the sense where it does not 'choose' a word, it has a group of words and decides if the player's guess is correct, or incorrect, depending on which of those leaves the more difficult group of words to guess from.
In a nutshell, my problem is this:
I have an ArrayList, masterList, where I have a set of words, a dictionary if you will, and various methods iterate through this to perform various tasks. My code is single threaded and one of these methods is throwing a ConcurrentModificationException when trying to access the next object in the ArrayList in the second iteration. However, I cannot find anything that actually changes the ArrayList during the iteration.
import java.io.*;
import java.util.*;
public class Main {
private ArrayList<String> masterList;
private ArrayList<String> contains;
private ArrayList<String> doesNotContain;
private HashMap<Integer, ArrayList<String>> wordLengthList;
private HashMap<Integer, ArrayList<String>> difficultyList;
private int guesses = 10;
private Scanner sc;
private FileReader fr;
private BufferedReader br;
private String guessString;
private char guessChar;
private static final String DICTIONARY = "smalldictionary.txt";
private String wordLengthString;
private int wordLengthInt = 0;
public Main(){
masterList = new ArrayList<String>();
contains = new ArrayList<String>();
doesNotContain= new ArrayList<String>();
wordLengthList = new HashMap<Integer, ArrayList<String>>();
difficultyList = new HashMap<Integer, ArrayList<String>>();
sc = new Scanner(System.in);
importTestDictionary(); //does not use masterList
br = new BufferedReader(fr);
importWords(); //Adds to masterList. Both readers closed when finished.
catalogLengths(); //Iterates through masterList - does not change it.
do{
setWordLength(); //does not use masterList
}while(!(validateLengthInput(wordLengthString))); //validation will change the set of masterList if valid.
//Main loop of game:
while(guesses > 0){
do{
getUserInput();
}while(!(validateInput(guessString)));
splitFamilies();//will change set of masterList when larger group is found. Changes occur AFTER where Exception is thrown
printDifficultyList();
}
}
private void importWords(){ //Adds to masterList. Both readers closed when finished.
try{
while(br.readLine() != null){
line = br.readLine();
masterList.add(line);
}
br.close();
fr.close();
}catch(IOException e){
System.err.println("An unexpected IO exception occurred. Check permissions of file!");
}
}
private boolean validateLengthInput(String length){ //validation will change the set of masterList if valid.
try{
wordLengthInt = Integer.parseInt(length);
if(!(wordLengthList.containsKey(wordLengthInt))){
System.out.println("There are no words in the dictionary with this length.\n");
return false;
}
}catch(NumberFormatException e){
System.out.println("You must enter a number.\n");
return false;
}
masterList = wordLengthList.get(wordLengthInt);
return true;
}
private void splitFamilies(){ //will change set of masterList when larger group is found. Changes occur AFTER where Exception is thrown
Iterator<String> it = masterList.iterator();
int tempCount = 0;
while(it.hasNext()){
tempCount++;
System.out.println("tempCount: " + tempCount);
String i = it.next(); //Still throwing ConcurrentModification Exception
if(i.contains(guessString)){
contains.add(i);
}else{
doesNotContain.add(i);
}
}
if(contains.size() > doesNotContain.size()){
masterList = contains;
correctGuess(); //does not use masterList
profileWords();
}
else if(doesNotContain.size() > contains.size()){
masterList = doesNotContain;
incorrectGuess(); //does not use masterList
}
else{
masterList = doesNotContain;
incorrectGuess(); //does not use masterList
}
}
private void printMasterList(){ //iterates through masterList - does not change it.
for(String i : masterList){
System.out.println(i);
}
}
private void catalogLengths(){ //Iterates through masterList - does not change it.
for(String i : masterList){
if(i.length() != 0){
if(!(wordLengthList.containsKey(i.length()))){
wordLengthList.put(i.length(), new ArrayList<String>());
}
wordLengthList.get(i.length()).add(i);
}
}
}
}
The line the exception is thrown from is marked above in the code. Any method using masterList is also marked, any method included that does not use it, there is no comment against.
I did read some answers and some of them suggested using Iterator to avoid the exception. This is implemented above in splitFamilies(). The original code was as below:
private void splitFamilies(){ //will change set of masterList when larger group is found. Changes occur AFTER where Exception is thrown
int tempCount = 0;
for(String i : masterList){ //This line throws ConcurrentModificationException
tempCount++;
System.out.println("tempCount: " + tempCount);
if(i.contains(guessString)){
contains.add(i);
}else{
doesNotContain.add(i);
}
}
....continue as before
tempCount is always 2 when the exception is thrown.
Maybe I'm missing something really simple, but I've tried tracing this, and cannot find out why I'm getting this exception!
I've tried to remove everything irrelevant from the code, but if anyone really wants to view the full thing, I guess I could dump all my code in the question!

The issue comes from the fact that masterList is a reference to either contains or doesNotContain after a first split. When you iterate on masterList, you actually also iterate at the same time on that other list.
So, then you add items to the lists:
if(i.contains(guessString)){
contains.add(i);
}else{
doesNotContain.add(i);
}
Here you do not only add items to contains or doesNotContain, but also potentially to masterList, which leads to the conccurentException.
To solve your issue, just make a copy of your lists, instead of : masterList = contains;
do a copy with: masterList = new ArrayList<>(contains);
And the same for doesNotContains.
Another solution which comes to mind is to reset the two lists contains and doesNotContains for each split. Since you only use them in this method, and nowhere else, remove these two lists from your Class, and defines them as private variables inside splitFamilies

ArrayIndexOutOfBoundsException converting C# to Java

I'm working on a program that will go through a list of records (IDs and Tickets) and parse them into two list respectively. It will also cross search the lists to see which IDs have a corresponding ticket based on names. Here is a link to an earlier version: here
Now, I've been rewritting with the help of some C# code from a coworker, but I'm having trouble with a parsing method. Here is the C# version:
public void parseLine(string _line)
{
if(string.IsNullOrWhiteSpace(_line)){ return;}
code = _line.Substring(0, 3);
ticketID = _line.Substring(3, 10);
string tmp = _line.Substring(13).Trim();
//get the first and last name
string [] tmp1 = tmp.Split(",".ToCharArray());
if(!(tmp1.Length > 1))
{
throw new Exception("unable to get the first and last name");
}
lastname = tmp1[0];
firstname = tmp1[1].Trim();
}
Here is my Java version:
public void parseLine(String line) throws Exception {
// code will store Ticket code *Alpha ticketId will store 10
// *digit code
code = line.substring(0, 3);
ticketId = line.substring(3, 10);
// tmp will store everything afterthe first 13 characters of
// line and trim the name(s)
String tmp = line.substring(13).trim();
// tmp1 array
String[] tmp1 = tmp.split(".*,.*");
if (tmp1.length > 1) {
throw new Exception("UNABLE TO GET NAME");
}
last = tmp1[0];
first = tmp1[1].trim();
}
This is in a seperate class, that will model the people with tickets. My main class(so far) which invokes the actual parseLine method is as follows:
public class ParkingTickets {
public static void main(String[] args) throws
FileNotFoundException, Exception {
ArrayList<TicketPeople> tickets = new ArrayList<>();
HashMap<String, List<SbPeople>> people = new HashMap<>();
File srcFile = new File("source.txt");
Scanner myScanner = new Scanner(srcFile);
while (myScanner.hasNextLine()) {
String line = myScanner.nextLine();
//System.out.println(line);
if (line.matches("^\\p{Alpha}.*$")) {
//System.out.printf("Ticket: %s%n", line);
TicketPeople t = new TicketPeople();
t.parseLine(line);
tickets.add(t);
}
myScanner.close();
}
}
}
the compiler points at the if statement in the parseLine method, and obviously the parseLine method in main class, when I tried stepping through that sepcifiv line, I see that it's starts parsing the the data from the source file but something is off. From the documentation the error means: Thrown to indicate that an array has been accessed with an illegal index. The index is either negative or greater than or equal to the size of the array.
I used an ArrayList for the ticket list and from what I understand it is a dynamic list that does not need to be set with a specific index size. I'm still learning and am having trouble understanding this exception. I would greatly appreciate any help.

Your call to split() in Java, doesn't match the split() from C#.
// String[] tmp1 = tmp.split(".*,.*");
String[] tmp1 = tmp.split(","); // <-- ",".
also, your check logic seems to have been reversed. But, I would suggest
if (tmp1.length != 2) {
throw new Exception("UNABLE TO GET NAME");
}

1) In C# it is Substring(int startIndex, int length). In java String substring(int startindex, int endindex).
2)The java code has also changed the exception logic. In C# code ther eis a not if(!(tmp1.Length > 1)), whereas in java code, if (tmp1.length > 1)

OOP Inventory program

The error I'm receiving is an Array index out of bounds exception, but I don't know why it's happening where it is.
import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;
public class Inventory
{
//Maximum amount of objects
private static int MAX_ITEMS = 100;
//Iteration from item to item
private int d_nextItem = 0;
//Array for the different objects
private Stock[] d_list = new Stock[MAX_ITEMS];
public static void main(String[] args) throws FileNotFoundException
{
Inventory inventory = new Inventory();
inventory.loadList(args[0]);
//Costs printing out,rough draft, toString not made
System.out.println("COSTS");
inventory.getTotalCost();
//Total Selling price printing out
System.out.println("SELLINGP");
inventory.getTotalSellingPrice();
System.out.println("SAMOUNT");
}
The specific error is Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0 at Inventory.main(Inventory.java:27) which points towards the inventory.loadList method in main. The error only comes up when run the program, and I don't know why its happening.
This is the loadList method, and the iteration doesn't look wrong, so how is an Array exception happening when the array is storing a reference to the objects information, not all the different strings, int and doubles.
public void loadList(String fileName) throws FileNotFoundException
{
fileName = "stock1.txt";
Scanner input = new Scanner(new File(fileName));
String newLine = null;
String name = null;
String identifier = null;
int quantity = 0;
double cost = 0.0;
double price = 0.0;
while (input.hasNextLine() && d_nextItem < MAX_ITEMS)
{
if(input.hasNext())
{
name = input.next();
}
if(input.hasNext())
{
identifier = input.next();
}
if(input.hasNextInt())
{
quantity = input.nextInt();
}
if(input.hasNextDouble())
{
cost = input.nextDouble();
}
if(input.hasNextDouble())
{
price = input.nextDouble();
}
d_list[d_nextItem]= new Stock(name,identifier,quantity,cost,price);
newLine = input.nextLine();
d_nextItem += 1;
}
}

That error means that you're not passing a parameter to the program.
args is an array containing the parameters passed to the program, the fact that index 0 is out of bounds means there are no parameters.
How exactly to do this would depend on how you're running the program.

The args[] array is special in that, when you're using it, you're invoking your program with more info, typically from the command line.
The appropriate way to populate args[] would be as follows:
java Inventory classname.txt
This way, Java will pull classname.txt into args[0].

From what I see, the code you have pasted here looks fine. So the problem might be elsewhere.
However a couple of quick changes may fix your problem.
use lists instead of an array for stock:
List stocklist = new ArrayList();
stocklist.add(...);
and make d_nextItem a local variable and initialize it before the while loop.

Exponentially increasing amounts of time to repeat a function

I've written my own math parser and for some reason it takes increasing amounts of time to parse when I tried to profile the parser.
For testing I used this input: Cmd.NUM_9,Cmd.NUM_0,Cmd.NUM_0,Cmd.DIV,Cmd.NUM_2,Cmd.ADD,Cmd.NUM_6,Cmd.MULT,Cmd.NUM_3
Single execution ~1.7ms
3000 repeats ~ 1,360ms
6000 repeats ~ 5,290ms
9000 repeats ~11,800ms
The profiler says 64% of the time was spent on this function:
this is my function to allow implicit multiplications.
private void enableImplicitMultiplication(ArrayList<Cmd> input) {
int input_size = input.size();
if (input_size<2) return;
for (int i=0; i<input_size; i++) {
Cmd cmd = input.get(i);
if (i>0) {
Cmd last = input.get(i-1);
// [EXPR1, EXPR2] => [EXPR1, MULT, EXPR2]
boolean criteria1 = Cmd.exprnCmds.contains(cmd) && Cmd.exprnCmds.contains(last);
// [CBRAC, OBRAC] => [CBRAC, MULT, OBRAC]
// [NUM_X, OBRAC] => [NUM_X, MULT, OBRAC]
boolean criteria2 = cmd==Cmd.OBRAC && (last==Cmd.CBRAC || Cmd.constantCmds.contains(last));
// [CBRAC, NUM_X] => [CBRAC, MULT, NUM_X]
boolean criteria3 = last==Cmd.CBRAC && Cmd.constantCmds.contains(cmd);
if (criteria1 || criteria2 || criteria3) {
input.add(i++, Cmd.MULT);
}
}
}
}
What's going on here??
I executed the repeats like this:
public static void main(String[] args) {
Cmd[] iArray = {
Cmd.NUM_9,Cmd.NUM_0,Cmd.NUM_0,Cmd.DIV,Cmd.NUM_2,Cmd.ADD,Cmd.NUM_6,Cmd.MULT,Cmd.NUM_3
};
ArrayList<Cmd> inputArray = new ArrayList<Cmd>(Arrays.asList(iArray));
DirtyExpressionParser parser = null;
int repeat=9000;
double starttime = System.nanoTime();
for (int i=0; i<repeat; i++) {
parser = new DirtyExpressionParser(inputArray);
}
double endtime = System.nanoTime();
System.out.printf("Duration: %.2f ms%n",(endtime-starttime)/1000000);
System.out.println(parser.getResult());
}
Constructor looks like this:
public DirtyExpressionParser(ArrayList<Cmd> inputArray) {
enableImplicitMultiplication(inputArray); //executed once for each repeat
splitOnBrackets(inputArray); //resolves inputArray into Expr objects for each bracket-group
for (Expr expr:exprArray) {
mergeAndSolve(expr);
}
}

Your microbenchmark code is altogether wrong: microbenchmarking on the JVM is a craft in its own right and is best left to dedicated tools such as jmh or Google Caliper. You don't warm up the code, don't control for GC pauses, and so on.
One detail which does come out by analyzing your code is this:
you reuse the same ArrayList for all repetitions of the function call;
each function call may insert an element to the list;
insertion is a heavyweight operation on ArrayList: the whole contents of the list after the inserted element must be copied.
You should at least create a fresh ArrayList for each invocation, but that will not make your whole methodology correct.
From our discussion in the comments I diagnose the following issue you may have with understanding your code:
In Java there is no such thing as a variable whose value is an object. The value of the variable is a reference to the object. Therefore when you say new DirtyExpressionParser(inputArray), the constructor does not receive its own private copy of the list, but rather a reference to the one and only ArrayList you have instantiated in your main method. The next constructor call gets this same list, but now modified by the earlier invocation. This is why your list is growing all the time.

Java problem time limit exceeded issue

Im coding a problem(reference --http://www.codechef.com/FEB11/problems/THREECLR/)
The below is my code
import java.io.*;
import java.util.*;
public class Main {
static String ReadLn (int maxLg) // utility function to read from stdin
{
byte lin[] = new byte [maxLg];
int lg = 0, car = -1;
String line = "";
try
{
while (lg < maxLg)
{
car = System.in.read();
if ((car < 0) || (car == '\n')) break;
lin [lg++] += car;
}
}
catch (IOException e)
{
return (null);
}
if ((car < 0) && (lg == 0)) return (null); // eof
return (new String (lin, 0, lg));
}
public static boolean iscontains(HashMap<Integer,HashSet<Integer>> resultmap,HashSet<Integer> b, int index)
{
boolean result=false;
for(Iterator<Integer> iter = b.iterator();iter.hasNext();)
{ int tmp=Integer.valueOf(iter.next().toString());
if(resultmap.get(index).contains(tmp))
result=true;
}
return result;
}
public static void main(String[] args) throws InterruptedException, FileNotFoundException {
try {
HashMap<Integer,HashSet<Integer>> pairlist = new HashMap<Integer,HashSet<Integer>>();
String input=null;
StringTokenizer idata;
int tc=0;
input=Main.ReadLn(255);
tc=Integer.parseInt(input);
while(--tc>=0)
{
input=Main.ReadLn(255);
idata = new StringTokenizer (input);idata = new StringTokenizer (input);
int dishnum= Integer.parseInt(idata.nextToken());
int pairnum= Integer.parseInt(idata.nextToken());
while (--pairnum>=0)
{
input=Main.ReadLn(255);
idata = new StringTokenizer (input);idata = new StringTokenizer (input);
int dish1=Integer.parseInt(idata.nextToken());
int dish2=Integer.parseInt(idata.nextToken());
if(pairlist.containsKey((Integer)dish1))
{
HashSet<Integer> dishes=new HashSet<Integer>();
dishes=pairlist.get(dish1);
dishes.add(dish2);
pairlist.put(dish1, dishes);
}
else
{
HashSet<Integer> dishes=new HashSet<Integer>();
dishes.add(dish2);
pairlist.put(dish1, dishes);
}
}
int maxrounds=1;
HashMap<Integer,HashSet<Integer>> resultlist = new HashMap<Integer,HashSet<Integer>>();
HashSet<Integer> addresult=new HashSet<Integer>();
addresult.add(1);
resultlist.put(1,addresult);
System.out.print("1");
for(int i=2;i<=dishnum;i++)
{
boolean found_one=false;
boolean second_check=false;
int minroundnum=maxrounds;
boolean pairlistcontains=false;
pairlistcontains=pairlist.containsKey(i);
for(int j=maxrounds;j>=1;j--)
{
if(!found_one){
if(pairlistcontains)
{
if(!iscontains(resultlist,pairlist.get((Integer) i),j))
{
for(Iterator<Integer> resultiter = resultlist.get(j).iterator();resultiter.hasNext();)
{
if(pairlist.get(resultiter.next()).contains(i))
second_check=true;
}
if(second_check==false)
{
found_one=true;
minroundnum=j;
j=0;
//second_check=false;
}
}
}
else
{
for(Iterator<Integer> resultiter = resultlist.get(j).iterator();resultiter.hasNext();)
{
if(pairlist.get(resultiter.next()).contains(i))
second_check=true;
}
if(second_check==false)
{
found_one=true;
minroundnum=j;
j=0;
//second_check=false;
}
}
second_check=false;
}
}
if((minroundnum==maxrounds)&&(found_one==false))
{
++minroundnum;
++maxrounds;
}
else
{
found_one=false;
}
HashSet<Integer> add2list=new HashSet<Integer> ();
if(resultlist.containsKey(minroundnum))
{
add2list=resultlist.get(minroundnum);
add2list.add(i);
}
else
{
add2list.add(i);
}
resultlist.put(minroundnum,add2list);
System.out.print(" ");
System.out.print(minroundnum);
}
if((tc !=-1))
System.out.println();
}
}
catch(Exception e){System.out.println(e.toString());}
}}
I have checked this code on online judges like Ideone and have been getting the desired results. But when i submit this code, I get a Time Limit exceeded error. I have tested this code on Ideone with a sufficiently large set of input and the time taken to execute was lesser than 1 second. It seems to have a bug or a memory leak that seems to have drained all the happiness from my life.
Any pointers/tips would be greatly appreciated.
Thanks
EDIT1 --
Thanks for the replies guys, I ran the code with the input generated by the following python script --
import random
filename="input.txt"
file=open(filename,'w')
file.write("50")
file.write("\n")
for i in range(0,50):
file.write("500 10000")
file.write("\n")
for j in range(0,10000):
file.write(str(random.randrange(1,501))+" "+str(random.randrange(1,501)))
file.write("\n")
file.close()
And my code took a whopping 71052 milliseconds to execute on the input provided by the above script. I have to now get down the execution time to 8000 milliseconds to the very least.
Im working on trying out replacing the HashMaps and the HashSets as suggested by rfeak and am also wondering if memoization would help in this scenario. Please advise.
EDIT2 --
Recoding my algo using arrays seems to have worked. Only just though, resubmitting the same code at different times gives me Accepted Solution and Time limit exceeded :D I have though of another way to use hashmaps to optimize a little further.
Thanks a load for the help guys !

How much memory does your program use when you run locally?
If they are running your Java program without enough memory, you could be spending a lot of time trying to garbage collect. This could destroy your 1 second time.
If you need to save time and memory (To be determined ...), I have 2 suggetions.
Replace HashSet<Integer> with BitSet. Similar interface, much faster implementation, and uses much less memory. Especially with the numbers I see in the problem.
Replace Map<Integer,X> with an X[] - The Integer key can simply be an int (primitive) index into your array. Again, faster and smaller.