Do static arraylists lose what the contain? - java

I have a program and its processing rather large amounts of data. It is comparing one static string arraylist to another checking whether a string is contained in it.
But what happens is after processing lets say 40k+ strings it begins to fail on the checking. By fail I mean it begins to not recognize that a string already exists in the other?
Is there a reason for this or is the arraylist simply too large?
Thanks
EDIT
for (int i = 0; i < arraylist1.size(); i++) {
boolean enter = true;
for (int x = 0; x < arraylist2.size() && enter; x++) {
if (arraylist1.get(i).getString().matches(arraylist2.get(x))) {
enter = false;
}
}
if (enter) {
//do something
}
}
EDIT****
Off-topic to the question but using .equals() instead of .matches() improves the performance MASSIVELY.

The simple answer is: no.
ArrayLists do not lose what is in them.
Your symptoms could be caused by a number of things, including threading/synchronization issues, subtle differences in the string, etc.
You should consider using a HashSet anyway though. It will make the "contains" check much much faster.
Using HashSet all your code above becomes:
List<String> list;
Set<String> set;
for (String str: list) {
if (!set.contains(str)) {
//do something
}
}
Much simpler and incredibly faster.
If you do need to use lists you can do the same thing but having both collections as List, the API doesn't change but performance will.

Related

How to properly use the remove() method in java for an array [duplicate]

This question already has answers here:
How do I remove objects from an array in Java?
(20 answers)
Closed 9 years ago.
Is there any fast (and nice looking) way to remove an element from an array in Java?
You could use commons lang's ArrayUtils.
array = ArrayUtils.removeElement(array, element)
commons.apache.org library:Javadocs
Your question isn't very clear. From your own answer, I can tell better what you are trying to do:
public static String[] removeElements(String[] input, String deleteMe) {
List result = new LinkedList();
for(String item : input)
if(!deleteMe.equals(item))
result.add(item);
return result.toArray(input);
}
NB: This is untested. Error checking is left as an exercise to the reader (I'd throw IllegalArgumentException if either input or deleteMe is null; an empty list on null list input doesn't make sense. Removing null Strings from the array might make sense, but I'll leave that as an exercise too; currently, it will throw an NPE when it tries to call equals on deleteMe if deleteMe is null.)
Choices I made here:
I used a LinkedList. Iteration should be just as fast, and you avoid any resizes, or allocating too big of a list if you end up deleting lots of elements. You could use an ArrayList, and set the initial size to the length of input. It likely wouldn't make much of a difference.
The best choice would be to use a collection, but if that is out for some reason, use arraycopy. You can use it to copy from and to the same array at a slightly different offset.
For example:
public void removeElement(Object[] arr, int removedIdx) {
System.arraycopy(arr, removedIdx + 1, arr, removedIdx, arr.length - 1 - removedIdx);
}
Edit in response to comment:
It's not another good way, it's really the only acceptable way--any tools that allow this functionality (like Java.ArrayList or the apache utils) will use this method under the covers. Also, you REALLY should be using ArrayList (or linked list if you delete from the middle a lot) so this shouldn't even be an issue unless you are doing it as homework.
To allocate a collection (creates a new array), then delete an element (which the collection will do using arraycopy) then call toArray on it (creates a SECOND new array) for every delete brings us to the point where it's not an optimizing issue, it's criminally bad programming.
Suppose you had an array taking up, say, 100mb of ram. Now you want to iterate over it and delete 20 elements.
Give it a try...
I know you ASSUME that it's not going to be that big, or that if you were deleting that many at once you'd code it differently, but I've fixed an awful lot of code where someone made assumptions like that.
You can't remove an element from the basic Java array. Take a look at various Collections and ArrayList instead.
Nice looking solution would be to use a List instead of array in the first place.
List.remove(index)
If you have to use arrays, two calls to System.arraycopy will most likely be the fastest.
Foo[] result = new Foo[source.length - 1];
System.arraycopy(source, 0, result, 0, index);
if (source.length != index) {
System.arraycopy(source, index + 1, result, index, source.length - index - 1);
}
(Arrays.asList is also a good candidate for working with arrays, but it doesn't seem to support remove.)
I think the question was asking for a solution without the use of the Collections API. One uses arrays either for low level details, where performance matters, or for a loosely coupled SOA integration. In the later, it is OK to convert them to Collections and pass them to the business logic as that.
For the low level performance stuff, it is usually already obfuscated by the quick-and-dirty imperative state-mingling by for loops, etc. In that case converting back and forth between Collections and arrays is cumbersome, unreadable, and even resource intensive.
By the way, TopCoder, anyone? Always those array parameters! So be prepared to be able to handle them when in the Arena.
Below is my interpretation of the problem, and a solution. It is different in functionality from both of the one given by Bill K and jelovirt. Also, it handles gracefully the case when the element is not in the array.
Hope that helps!
public char[] remove(char[] symbols, char c)
{
for (int i = 0; i < symbols.length; i++)
{
if (symbols[i] == c)
{
char[] copy = new char[symbols.length-1];
System.arraycopy(symbols, 0, copy, 0, i);
System.arraycopy(symbols, i+1, copy, i, symbols.length-i-1);
return copy;
}
}
return symbols;
}
You could use the ArrayUtils API to remove it in a "nice looking way". It implements many operations (remove, find, add, contains,etc) on Arrays.
Take a look. It has made my life simpler.
okay, thx a lot
now i use sth like this:
public static String[] removeElements(String[] input, String deleteMe) {
if (input != null) {
List<String> list = new ArrayList<String>(Arrays.asList(input));
for (int i = 0; i < list.size(); i++) {
if (list.get(i).equals(deleteMe)) {
list.remove(i);
}
}
return list.toArray(new String[0]);
} else {
return new String[0];
}
}
Some more pre-conditions are needed for the ones written by Bill K and dadinn
Object[] newArray = new Object[src.length - 1];
if (i > 0){
System.arraycopy(src, 0, newArray, 0, i);
}
if (newArray.length > i){
System.arraycopy(src, i + 1, newArray, i, newArray.length - i);
}
return newArray;
You can not change the length of an array, but you can change the values the index holds by copying new values and store them to a existing index number.
1=mike , 2=jeff // 10 = george 11 goes to 1 overwriting mike .
Object[] array = new Object[10];
int count = -1;
public void myFunction(String string) {
count++;
if(count == array.length) {
count = 0; // overwrite first
}
array[count] = string;
}
Copy your original array into another array, without the element to be removed.
A simplier way to do that is to use a List, Set... and use the remove() method.
Swap the item to be removed with the last item, if resizing the array down is not an interest.
I hope you use the java collection / java commons collections!
With an java.util.ArrayList you can do things like the following:
yourArrayList.remove(someObject);
yourArrayList.add(someObject);
Use an ArrayList:
alist.remove(1); //removes the element at position 1
Sure, create another array :)

Splitting vectors into subvectors - Java

I have a function that processes vectors. Size of input vector can be anything up to few millions. Problem is that function can only process vectors that are no bigger than 100k elements without problems.
I would like to call function in smaller parts if vector has too many elements
Vector<Stuff> process(Vector<Stuff> input) {
Vector<Stuff> output;
while(1) {
if(input.size() > 50000) {
output.addAll(doStuff(input.pop_front_50k_first_ones_as_subvector());
}
else {
output.addAll(doStuff(input));
break;
}
}
return output;
}
How should I do this?
Not sure if a Vector with millions of elements is a good idea, but Vector implements List, and thus there is subList which provides a lightweight (non-copy) view of a section of the Vector.
You may have to update your code to work with the interface List instead of only the specific implementation Vector, though (because the sublist returned is not a Vector, and it is just good practice in general).
You probably want to rewrite your doStuff method to take a List rather than a Vector argument,
public Collection<Output> doStuff(List<Stuff> v) {
// calculation
}
(and notice that Vector<T> is a List<T>)
and then change your process method to something like
Vector<Stuff> process(Vector<Stuff> input) {
Vector<Stuff> output;
int startIdx = 0;
while(startIdx < input.size()) {
int endIdx = Math.min(startIdx + 50000, input.size());
output.addAll(doStuff(input.subList(startIdx, endIdx)));
startIdx = endIdx;
}
}
this should work as long as the "input" Vector isn't being concurrently updated during the running of the process method.
If you can't change the signature of doStuff, you're probably going to need to wrap a new Vector around the result of subList,
output.addAll(doStuff(new Vector<Stuff>(input.subList(startIdx, endIdx)))));

Making an add method for an ArrayList

So I am supposed to make an add method for an array list which adds a new movie object to the list if it doesnt exist, or if it finds a movie object with a similar title within the list, it just increases the quantity property of that object. Here is what I've got so far.
public void add(String title, double rating, int releaseYear){
if(this.myMovies.size() < 1)
{
Movie mymovie = new Movie(title, rating, releaseYear);
this.myMovies.add(mymovie);
}
else
{
for(int i = 0; i < this.myMovies.size(); i++)
{
Movie temp = this.myMovies.get(i);
if(temp.Title.equals(title)){
this.myMovies.get(i).quantity++;
break;
}
else
{
Movie mymovie = new Movie(title, rating, releaseYear);
this.myMovies.add(mymovie);
break;
}
}
}
}
My problem is that this ends up not taking account of similar names and doesn't increase the quantity but just adds another object to the list. I have a strong feeling that the problem lies within my For loop but I just can't identify it. Can anyone see anything that I may be doing wrong? Thank you!
You're testing only for equality, not similarity here:
if(temp.Title.equals(title)){
Instead, you should write a helper method to test for similarity based on whatever criteria are appropriate. For example:
if (isSimilar(temp.Title, title)){
and the isSimilar method might look something like this (assuming you don't need any input validation):
private void isSimilar(String title1, String title2) {
return title1.equalsIgnoreCase(title2)
|| title1.toLowerCase().contains(title2.toLowerCase())
|| title2.toLowerCase().contains(title1.toLowerCase());
}
or, perhaps more appropriately, like this (if you implement it in the Movie class):
private void isSimilar(otherMovie) {
return title.equalsIgnoreCase(otherMovie.title)
|| title.toLowerCase().contains(otherMovie.title.toLowerCase())
|| otherMovie.title.toLowerCase().contains(title.toLowerCase());
}
...in which case your if statement would also change slightly.
Keep in mind that I don't know what you consider 'similar'; only that the movies are considered similar if the names are similar.
A couple more comments:
Fields and method names generally start with a lowercase letter (so the field Movie.Title should instead be Movie.title).
It's usually preferable to loop over a Collection using an Iterator instead of using the raw index--partly because the Iterator should always know how to loop over the Collection efficiently.
Learn to use your IDE's debugger (it's probably very easy). Then you can step through each line of code to see exactly where your program is doing something unexpected.
I would do something like this:
public void add(String title, double rating, int releaseYear){
for(Movie m: myMovies.size())
{
if(m.Title.equals(title)){
m.quantity++;
return;
}
}
// movie with same title not found in the list -> insert
this.myMovies.add(new Movie(title, rating, releaseYear));
}
By the way: variable names should start with a lowercase character (Title -> title).
I'm addressing your "similarity" requirement. If you really want to do this properly it could be a lot of work. Essentially you have two strings and want to get a measure of the similarity. I am doing the same thing for figure captions and I plan to tackle it by:
splitting the title into words
lowercasing them
using them as features for classifier4J (http://classifier4j.sourceforge.net/)
That will go a long way based on simple word counts. But then you have the problem of stemming
(words that differ by endings - "Alien" and "Aliens"). If you go down this road you'll need to read up about Classification and Natural Language Processing

how do i use a hashmap keys to an array of strings?

im currently working on a multiple class assignment where i have to add a course based on whether the prerequisites exist within the program.
im storing my courses within the program class using a hashmap. (thought i would come in handy) however, im having a bit of trouble ensuring that these preReqs exist.
here is some code ive currently got going
public boolean checkForCourseFeasiblity(AbstractCourse c) throws ProgramException
{
AbstractCourse[] tempArray = new AbstractCourse[0];
tempArray= courses.keySet().toArray(tempArray);
String[] preReqsArray = new String[1];
preReqsArray = c.getPreReqs();
//gets all course values and stores them in tempArray
for(int i = 0; i < preReqsArray.length; i++)
{
if(courses.containsKey(preReqsArray[i]))
{
continue;
}
else if (!courses.containsKey(preReqsArray[i]))
{
throw new ProgramException("preReqs do not exist"); //?
}
}
return true;
}
ok so basically, tempArray is storing all the keySets inside the courses hashmap and i need to compare all of them with the preReqs (which is an array of Strings). if the preReqs exist within the keyset then add the course, if they dont do not add the course. return true if the course adds otherwise through me an exception. keep in mind my keysets are Strings e.g. a keyset value could be "Programming1" and the required prerquisite for a course could be "programming1". if this is the case add then add the course as the prereq course exists in the keyset.
i believe my error to be when i initialize mypreReqsArray with c.getPreReqs (note: getPreReqs is a getter with a return type String[]).
it would be really great if someone could aid me with my dilemma. ive tried to provide as much as possible, i feel like ive been going around in circles for the past 3 hours :(
-Thank you.
Try something like this, you don't need tempArray. The "for each" loop looks lots nicer too. If you want to throw an Exception I would put that logic in the place that calls this method.
public boolean checkForCourseFeasiblity(AbstractCourse c)
{
for(String each : c.getPreReqs())
{
if(! courses.containsKey(each))
{
return false;
}
}
return true;
}

Tutorials for Java errors and syntax

I am asking for help on self-help, which is kind of an oxymoron. How do I bug you nice folks less by solving more of my own problems?
I am in my last week of Java programming and I am having a huge hurdle with learning Java. I have read all the books but I keep getting hung up on tiny little issues. It is like trying to build a house of cards. I only know about the parts of the syntax and the uses that the book shows. When I am combining things, I run into horrible hurdles. I try for hours of tinkering to figure them out. The sun docs only show basic uses that don't seem to help
Here is what I would like:
When I am trying something and it doesn't work like the following manipulations of an array list, I want to find a place or program that can show examples code of things like adding an additional class instance to an arrayList. Where can I learn concisely about this without having to ask a question or 2 for every syntax error? Where is the Google for Java? Is there a program that will take your errors and show you how to fix them (or offer suggestions)?
/tmp/jc_4083/Inventory.java:101: incompatible types
found : RatedDVD[]
required: java.util.ArrayList
dvdlist = temp;
^
/tmp/jc_4083/Inventory.java:110: array required, but java.util.ArrayList found
if (p != dvdlist[i]) {
^
/tmp/jc_4083/Inventory.java:111: array required, but java.util.ArrayList found
temp[i-adj] = dvdlist[i];
^
/tmp/jc_4083/Inventory.java:115: incompatible types
found : RatedDVD[]
required: java.util.ArrayList
dvdlist = temp;
Here is my code for this class if anyone is interested in looking at it for me:
//Contruct inv and allow for methods add, get, size, sort, and value
import java.util.*;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
public class Inventory
{// class Inventory
private ArrayList<RatedDVD> dvdlist;// declare dvdlist as ArrayList of RatedDVD
private int numDVDs;
public Inventory()
{// method Inventory
dvdlist = new ArrayList<RatedDVD>();
}// end method
// add & get
public RatedDVD get(int i){return dvdlist.get(i);}// method get
public void add(DVD d){
dvdlist = dvdlist d;
sort();
}// method add
public double value()
{// method value
double total = 0.0;
for (int i = 0; i < dvdlist.size(); i++)
{// for every pass thru dvdlist add total
// [DEBUG] consider enhanced for
total += get(i).feeValue();
}
return total;
}// end method value
public void sort()
{// method sort
// [DEBUG] consider optimization
int n = dvdlist.size();
for (int search = 1; search < n; search++)
{// for do the following and increment till dvdlist has been searched
for (int i = 0; i < n-search; i++)
{// for step through comparison for entire dvdlist
if (dvdlist.get(i).getName().compareToIgnoreCase(dvdlist.get(i+1).getName()) > 0)
{// if swap necessary then swap
RatedDVD temp = dvdlist.get(i);
dvdlist.set(i,dvdlist.get(i+1));
dvdlist.set(i+1,temp);
}// end if swap
}// end for compareto
}// end outer for
}// end method sort
public int size(){return dvdlist.size();}// method size
public void save() {
save(true);
}
// save it to C:\data\inventory.dat
public void save(boolean saveagain) {
try {
BufferedWriter w = new BufferedWriter(new FileWriter("c:\\data\\inventory.dat"));
for (int i = 0; i < size(); i++) {
RatedDVD dvd = get(i);
w.write( dvd.getItem() + "\n");
w.write( dvd.getName() + "\n");
w.write( dvd.getRating() + "\n");
w.write( dvd.getUnits() + "\n");
w.write( dvd.getPrice() + "\n");
w.write( dvd.value() + "\n");
w.write( dvd.fee() + "\n");
w.write( dvd.feeValue() + "\n");
w.newLine();
}
// total value of it
//w.write( value() + "\n");
w.close();
} catch (Exception ex) {
if (saveagain) {
new File("c:\\data\\").mkdir(); // make file if doesn't exist
save(false);
}
}
}
public int search(String name) {
for (int i = 0; i < size(); i++) { // check if name string is equal
if (get(i).getName().equalsIgnoreCase(name)) return i;
}
return -1; // we didn't find anything
}
// add a new dvd to the end, increasing the array size
public void add(RatedDVD p) {
RatedDVD[] temp = new RatedDVD[dvdlist.size()+1];
for (int i = 0; i < dvdlist.size(); i++) {
temp[i] = dvdlist[i];
}
temp[temp.length-1] = p; // add it at the end
dvdlist = temp;
}
// remove a DVD from the array, and shrink the array size
public void delete(RatedDVD p) {
RatedDVD[] temp = new RatedDVD[dvdlist.size()-1];
int adj = 0;
for (int i = 0; i < dvdlist.size(); i++) {
if (p != dvdlist[i]) {
temp[i-adj] = dvdlist[i];
}
else adj = 1;
}
dvdlist = temp;
}
public int highestNumber() {
int numb = 0;
for (int i = 0; i < dvdlist.size(); i++) {
if (get(i).getItem() > numb) {
numb = get(i).getItem();
}
}
return numb;
}
}// end class inventory
The dvdlist is an ArrayList, which implements the Collection interface, not an Array (BTW, and this is known as the "program to an interface, not an implementation" principle, you should decalare dvdlist as a java.util.List):
private ArrayList<RatedDVD> dvdlist;// declare dvdlist as ArrayList of RatedDVD
Have a look at the methods on the Collection interface, you'll find everything you need for adding and removing elements.
So, to add a RatedDVD, you don't need to use a temporary array of RatedDVD that won't fit anyway into an ArrayList like you're doing here:
// add a new dvd to the end, increasing the array size
public void add(RatedDVD p) {
RatedDVD[] temp = new RatedDVD[dvdlist.size()+1];
for (int i = 0; i < dvdlist.size(); i++) {
temp[i] = dvdlist[i];
}
temp[temp.length-1] = p; // add it at the end
dvdlist = temp;
}
Instead, just call the add(Object o) method on dvdlist.
To delete a RatedDVD instance, use the remove(Object o) method on dvdlist.
For the search() method, consider using contains(Object o) on dvdlist.
If you need to iterate over a collection, use an Iterator:
for (Iterator iter = dvdlist.iterator(); iter.hasNext();) {
RatedDVD ratedDVD = (RatedDVD) iter.next();
//rest of the code block removed
}
Or even faster now with Java 5+ and Generics:
for (RatedDVD ratedDVD : dvdlist) {
// rest of the code here
}
Really, you need to dig the the Collection Framework.
The compiler errors seem to be quite descriptive of what you're doing wrong, but I can see why you might be confused about how to do it right. You seem to be misunderstanding how an ArrayList is meant to be used. If you look at the docs, you will see it has methods add() and remove() that do the operations you've created add() and delete() methods for. You're attempting to treat the ArrayList as if it is a raw array. Don't do that; use the methods provided by the API. Not only will this solve your errors, but it will make your code cleaner and clearer to future programmers.
Actually, the compiler error is very clear:
/tmp/jc_4083/Inventory.java:101: incompatible types
found : RatedDVD[]
required: java.util.ArrayList
dvdlist = temp;
It says "incompatible types" and that it expected a java.util.ArrayList but found instead a RatedDVD[].
Your problem is simply that, unlike in languages like Python, Java does not treat lists and arrays interchangeably. They are completely different things - arrays are special language-level constructs, while ArrayList is a class like any other.
So you cannot assign an array to a variably of type list. You either have to decide on using only one of these two types throughout your program, or you have to convert between them manually, using methods such as java.util.Arrays.asList() and List.toArray().
It seems that you're trying to do too advanced things too fast - you should probably look at Sun's Java tutorials first - though they are quite comprehensive and can also be used as a reference for looking up language details. There is also a section about conversion between collections and arrays.
I suggest you use an IDE (like Eclipse, entirely free). It will help you through the API syntax by making suggestions as you type, and show you errors when you type them, so that you can pinpoint exact syntax errors and ask about them. In terms of asking, that is what StackOverflow is for.
Others beat me to your specific syntax question, so I'm just limiting my answer to the general question of how you get help.
To resolve compiler errors, usually it's best to start with the first one and fix it first. After fixing that, the rest of the compiler errors might also be solved, or they might be different kinds of errors.
To understand what some compiler error means, there is an article called Compile and Runtime Errors in Java (PDF) that goes through different kinds of error messages and gives examples of what kind of code may cause them. And as for runtime error messages, Java Glossary has quite a big list of them. They also have a list of compile-time error messages.
So, your problem here is that you're trying to access an ArrayList like an array, which is incorrect because Java doesn't do stuff like that. You need to use list.get(i) to get the ith element in an Array. Similarly, when you tried to set an ArrayList variable to an array, the compiler got mad at you. You need to create a new ArrayList with the contents of temp and then set dvdlist to that, eg. dvdlist = new ArrayList<RatedDVD>(temp);.
As for your continued problems: There is an API Specification for Java which tells you basically how to use all the classes that are included in the Java API. For example, ArrayList is a generic collection which has certain methods and constructors that you need to use. Java does not have operator overloading, so you can't just access elements in a List using array syntax. Also, arrays are their own data type so you can't just treat an ArrayList as an array and vice versa.
It looks like you are confused about the difference between an array and an ArrayList. An array is a static list of elements, and is constructed using the [] symbols. An ArrayList is an object in the Collections system in Java that acts like a size-modifiable array - i.e. it can be indexed into, and can be added onto, inserted into, etc. They are not interchangable.
As to where you can look, etc. If this is your first programming experience, you are not alone. Many compiler errors are less than helpful. One suggestion I can give you that I learned through many years of trial and error - start small and build up. Get a main method that compiles. Then add the first little piece (creating a class for instance). Then add the next piece, and so on. Test as you go, etc. You can google for particular compiler errors - I have been surprised what I have found. Beyond that, a lot of it is trial and error - these are things you learn from experience, and a lot of the speed of "old hands" comes from the long experience of seeing what you can do wrong over and over again, not any sort of innate intelligence. I totally understand your frustration - I felt that way when I was starting out about 15 years ago now (but I was on Borland Pascal - yuck).
One of the biggest issues beginning programmers seem to have is not being able to read and interpret error messages very well.
You would be well served by carefully examining the errors that javac (or any compiler/interpreter) provides. Maybe even start by making some mistakes that you understand in your code (ie, assign an incorrect typed value to a variable, extend a loop beyond the bounds) and see how your compiler handles these.
Try to think in object oriented terms...
It looks to me that something (classwork, I guess) has pushed you into writing an object-oriented program but it's possible that you haven't yet accepted that you will need to think in those terms.
In Java most things are objects, but Java supports primitive types, and arrays of both. It's possible to program in Java in a flat, procedural, mutable way, but also possible to write in an object-oriented functional way. It's possible to do both and get confused, which is where you may be right now.
You are trying to mix the two styles. This isn't always a bad thing, but for coursework we can safely bet the farm that your instructor will want to see more objects and fewer arrays, unless those arrays are the private internal implementation of an object.
So think of the data structures as black boxes with methods, and then see how what you are doing is implementing one yourself.
You have probably been here, but these are the things that you can do with an ArrayList. And you have an ArrayList<RatedDVD> which further restricts what you can do with it. Try to understand this first, and then fix the program to work with the available operations on an ArrayList object.

Categories