I have an ArrayList<String>, and I want to remove repeated strings from it. How can I do this?
If you don't want duplicates in a Collection, you should consider why you're using a Collection that allows duplicates. The easiest way to remove repeated elements is to add the contents to a Set (which will not allow duplicates) and then add the Set back to the ArrayList:
Set<String> set = new HashSet<>(yourList);
yourList.clear();
yourList.addAll(set);
Of course, this destroys the ordering of the elements in the ArrayList.
Although converting the ArrayList to a HashSet effectively removes duplicates, if you need to preserve insertion order, I'd rather suggest you to use this variant
// list is some List of Strings
Set<String> s = new LinkedHashSet<>(list);
Then, if you need to get back a List reference, you can use again the conversion constructor.
In Java 8:
List<String> deduped = list.stream().distinct().collect(Collectors.toList());
Please note that the hashCode-equals contract for list members should be respected for the filtering to work properly.
Suppose we have a list of String like:
List<String> strList = new ArrayList<>(5);
// insert up to five items to list.
Then we can remove duplicate elements in multiple ways.
Prior to Java 8
List<String> deDupStringList = new ArrayList<>(new HashSet<>(strList));
Note: If we want to maintain the insertion order then we need to use LinkedHashSet in place of HashSet
Using Guava
List<String> deDupStringList2 = Lists.newArrayList(Sets.newHashSet(strList));
Using Java 8
List<String> deDupStringList3 = strList.stream().distinct().collect(Collectors.toList());
Note: In case we want to collect the result in a specific list implementation e.g. LinkedList then we can modify the above example as:
List<String> deDupStringList3 = strList.stream().distinct()
.collect(Collectors.toCollection(LinkedList::new));
We can use parallelStream also in the above code but it may not give expected performace benefits. Check this question for more.
If you don't want duplicates, use a Set instead of a List. To convert a List to a Set you can use the following code:
// list is some List of Strings
Set<String> s = new HashSet<String>(list);
If really necessary you can use the same construction to convert a Set back into a List.
Java 8 streams provide a very simple way to remove duplicate elements from a list. Using the distinct method.
If we have a list of cities and we want to remove duplicates from that list it can be done in a single line -
List<String> cityList = new ArrayList<>();
cityList.add("Delhi");
cityList.add("Mumbai");
cityList.add("Bangalore");
cityList.add("Chennai");
cityList.add("Kolkata");
cityList.add("Mumbai");
cityList = cityList.stream().distinct().collect(Collectors.toList());
How to remove duplicate elements from an arraylist
You can also do it this way, and preserve order:
// delete duplicates (if any) from 'myArrayList'
myArrayList = new ArrayList<String>(new LinkedHashSet<String>(myArrayList));
Here's a way that doesn't affect your list ordering:
ArrayList l1 = new ArrayList();
ArrayList l2 = new ArrayList();
Iterator iterator = l1.iterator();
while (iterator.hasNext()) {
YourClass o = (YourClass) iterator.next();
if(!l2.contains(o)) l2.add(o);
}
l1 is the original list, and l2 is the list without repeated items
(Make sure YourClass has the equals method according to what you want to stand for equality)
this can solve the problem:
private List<SomeClass> clearListFromDuplicateFirstName(List<SomeClass> list1) {
Map<String, SomeClass> cleanMap = new LinkedHashMap<String, SomeClass>();
for (int i = 0; i < list1.size(); i++) {
cleanMap.put(list1.get(i).getFirstName(), list1.get(i));
}
List<SomeClass> list = new ArrayList<SomeClass>(cleanMap.values());
return list;
}
It is possible to remove duplicates from arraylist without using HashSet or one more arraylist.
Try this code..
ArrayList<String> lst = new ArrayList<String>();
lst.add("ABC");
lst.add("ABC");
lst.add("ABCD");
lst.add("ABCD");
lst.add("ABCE");
System.out.println("Duplicates List "+lst);
Object[] st = lst.toArray();
for (Object s : st) {
if (lst.indexOf(s) != lst.lastIndexOf(s)) {
lst.remove(lst.lastIndexOf(s));
}
}
System.out.println("Distinct List "+lst);
Output is
Duplicates List [ABC, ABC, ABCD, ABCD, ABCE]
Distinct List [ABC, ABCD, ABCE]
There is also ImmutableSet from Guava as an option (here is the documentation):
ImmutableSet.copyOf(list);
Probably a bit overkill, but I enjoy this kind of isolated problem. :)
This code uses a temporary Set (for the uniqueness check) but removes elements directly inside the original list. Since element removal inside an ArrayList can induce a huge amount of array copying, the remove(int)-method is avoided.
public static <T> void removeDuplicates(ArrayList<T> list) {
int size = list.size();
int out = 0;
{
final Set<T> encountered = new HashSet<T>();
for (int in = 0; in < size; in++) {
final T t = list.get(in);
final boolean first = encountered.add(t);
if (first) {
list.set(out++, t);
}
}
}
while (out < size) {
list.remove(--size);
}
}
While we're at it, here's a version for LinkedList (a lot nicer!):
public static <T> void removeDuplicates(LinkedList<T> list) {
final Set<T> encountered = new HashSet<T>();
for (Iterator<T> iter = list.iterator(); iter.hasNext(); ) {
final T t = iter.next();
final boolean first = encountered.add(t);
if (!first) {
iter.remove();
}
}
}
Use the marker interface to present a unified solution for List:
public static <T> void removeDuplicates(List<T> list) {
if (list instanceof RandomAccess) {
// use first version here
} else {
// use other version here
}
}
EDIT: I guess the generics-stuff doesn't really add any value here.. Oh well. :)
public static void main(String[] args){
ArrayList<Object> al = new ArrayList<Object>();
al.add("abc");
al.add('a');
al.add('b');
al.add('a');
al.add("abc");
al.add(10.3);
al.add('c');
al.add(10);
al.add("abc");
al.add(10);
System.out.println("Before Duplicate Remove:"+al);
for(int i=0;i<al.size();i++){
for(int j=i+1;j<al.size();j++){
if(al.get(i).equals(al.get(j))){
al.remove(j);
j--;
}
}
}
System.out.println("After Removing duplicate:"+al);
}
If you're willing to use a third-party library, you can use the method distinct() in Eclipse Collections (formerly GS Collections).
ListIterable<Integer> integers = FastList.newListWith(1, 3, 1, 2, 2, 1);
Assert.assertEquals(
FastList.newListWith(1, 3, 2),
integers.distinct());
The advantage of using distinct() instead of converting to a Set and then back to a List is that distinct() preserves the order of the original List, retaining the first occurrence of each element. It's implemented by using both a Set and a List.
MutableSet<T> seenSoFar = UnifiedSet.newSet();
int size = list.size();
for (int i = 0; i < size; i++)
{
T item = list.get(i);
if (seenSoFar.add(item))
{
targetCollection.add(item);
}
}
return targetCollection;
If you cannot convert your original List into an Eclipse Collections type, you can use ListAdapter to get the same API.
MutableList<Integer> distinct = ListAdapter.adapt(integers).distinct();
Note: I am a committer for Eclipse Collections.
If you are using model type List< T>/ArrayList< T> . Hope,it's help you.
Here is my code without using any other data structure like set or hashmap
for (int i = 0; i < Models.size(); i++){
for (int j = i + 1; j < Models.size(); j++) {
if (Models.get(i).getName().equals(Models.get(j).getName())) {
Models.remove(j);
j--;
}
}
}
If you want to preserve your Order then it is best to use LinkedHashSet.
Because if you want to pass this List to an Insert Query by Iterating it, the order would be preserved.
Try this
LinkedHashSet link=new LinkedHashSet();
List listOfValues=new ArrayList();
listOfValues.add(link);
This conversion will be very helpful when you want to return a List but not a Set.
This three lines of code can remove the duplicated element from ArrayList or any collection.
List<Entity> entities = repository.findByUserId(userId);
Set<Entity> s = new LinkedHashSet<Entity>(entities);
entities.clear();
entities.addAll(s);
for(int a=0;a<myArray.size();a++){
for(int b=a+1;b<myArray.size();b++){
if(myArray.get(a).equalsIgnoreCase(myArray.get(b))){
myArray.remove(b);
dups++;
b--;
}
}
}
When you are filling the ArrayList, use a condition for each element. For example:
ArrayList< Integer > al = new ArrayList< Integer >();
// fill 1
for ( int i = 0; i <= 5; i++ )
if ( !al.contains( i ) )
al.add( i );
// fill 2
for (int i = 0; i <= 10; i++ )
if ( !al.contains( i ) )
al.add( i );
for( Integer i: al )
{
System.out.print( i + " ");
}
We will get an array {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
Code:
List<String> duplicatList = new ArrayList<String>();
duplicatList = Arrays.asList("AA","BB","CC","DD","DD","EE","AA","FF");
//above AA and DD are duplicate
Set<String> uniqueList = new HashSet<String>(duplicatList);
duplicatList = new ArrayList<String>(uniqueList); //let GC will doing free memory
System.out.println("Removed Duplicate : "+duplicatList);
Note: Definitely, there will be memory overhead.
ArrayList<String> city=new ArrayList<String>();
city.add("rajkot");
city.add("gondal");
city.add("rajkot");
city.add("gova");
city.add("baroda");
city.add("morbi");
city.add("gova");
HashSet<String> hashSet = new HashSet<String>();
hashSet.addAll(city);
city.clear();
city.addAll(hashSet);
Toast.makeText(getActivity(),"" + city.toString(),Toast.LENGTH_SHORT).show();
you can use nested loop in follow :
ArrayList<Class1> l1 = new ArrayList<Class1>();
ArrayList<Class1> l2 = new ArrayList<Class1>();
Iterator iterator1 = l1.iterator();
boolean repeated = false;
while (iterator1.hasNext())
{
Class1 c1 = (Class1) iterator1.next();
for (Class1 _c: l2) {
if(_c.getId() == c1.getId())
repeated = true;
}
if(!repeated)
l2.add(c1);
}
LinkedHashSet will do the trick.
String[] arr2 = {"5","1","2","3","3","4","1","2"};
Set<String> set = new LinkedHashSet<String>(Arrays.asList(arr2));
for(String s1 : set)
System.out.println(s1);
System.out.println( "------------------------" );
String[] arr3 = set.toArray(new String[0]);
for(int i = 0; i < arr3.length; i++)
System.out.println(arr3[i].toString());
//output: 5,1,2,3,4
List<String> result = new ArrayList<String>();
Set<String> set = new LinkedHashSet<String>();
String s = "ravi is a good!boy. But ravi is very nasty fellow.";
StringTokenizer st = new StringTokenizer(s, " ,. ,!");
while (st.hasMoreTokens()) {
result.add(st.nextToken());
}
System.out.println(result);
set.addAll(result);
result.clear();
result.addAll(set);
System.out.println(result);
output:
[ravi, is, a, good, boy, But, ravi, is, very, nasty, fellow]
[ravi, is, a, good, boy, But, very, nasty, fellow]
This is used for your Custom Objects list
public List<Contact> removeDuplicates(List<Contact> list) {
// Set set1 = new LinkedHashSet(list);
Set set = new TreeSet(new Comparator() {
#Override
public int compare(Object o1, Object o2) {
if (((Contact) o1).getId().equalsIgnoreCase(((Contact) o2).getId()) /*&&
((Contact)o1).getName().equalsIgnoreCase(((Contact)o2).getName())*/) {
return 0;
}
return 1;
}
});
set.addAll(list);
final List newList = new ArrayList(set);
return newList;
}
As said before, you should use a class implementing the Set interface instead of List to be sure of the unicity of elements. If you have to keep the order of elements, the SortedSet interface can then be used; the TreeSet class implements that interface.
import java.util.*;
class RemoveDupFrmString
{
public static void main(String[] args)
{
String s="appsc";
Set<Character> unique = new LinkedHashSet<Character> ();
for(char c : s.toCharArray()) {
System.out.println(unique.add(c));
}
for(char dis:unique){
System.out.println(dis);
}
}
}
public Set<Object> findDuplicates(List<Object> list) {
Set<Object> items = new HashSet<Object>();
Set<Object> duplicates = new HashSet<Object>();
for (Object item : list) {
if (items.contains(item)) {
duplicates.add(item);
} else {
items.add(item);
}
}
return duplicates;
}
ArrayList<String> list = new ArrayList<String>();
HashSet<String> unique = new LinkedHashSet<String>();
HashSet<String> dup = new LinkedHashSet<String>();
boolean b = false;
list.add("Hello");
list.add("Hello");
list.add("how");
list.add("are");
list.add("u");
list.add("u");
for(Iterator iterator= list.iterator();iterator.hasNext();)
{
String value = (String)iterator.next();
System.out.println(value);
if(b==unique.add(value))
dup.add(value);
else
unique.add(value);
}
System.out.println(unique);
System.out.println(dup);
If you want to remove duplicates from ArrayList means find the below logic,
public static Object[] removeDuplicate(Object[] inputArray)
{
long startTime = System.nanoTime();
int totalSize = inputArray.length;
Object[] resultArray = new Object[totalSize];
int newSize = 0;
for(int i=0; i<totalSize; i++)
{
Object value = inputArray[i];
if(value == null)
{
continue;
}
for(int j=i+1; j<totalSize; j++)
{
if(value.equals(inputArray[j]))
{
inputArray[j] = null;
}
}
resultArray[newSize++] = value;
}
long endTime = System.nanoTime()-startTime;
System.out.println("Total Time-B:"+endTime);
return resultArray;
}
I've searched through many questions on this site with somewhat similar underlying concepts, however after many hours of attempting to solve this problem myself and reviewing I am still lost. If there is another question that answers this I will be more than happy to give it a look over.
Ultimately I want to create a recursive method such that it takes two lists and returns a Set of String lists:
//Example of such a function definition
private static Set<List<String>> myRecursiveMethod(List<String> listOne,
List<String> listTwo) {
}
When I say "Set of String lists" I mean specifically the following:
(Note:"AD" == "DA")
// if the two following lists are INPUTTED into myRecursiveMethod();
// listOne = ["A","B"]
// listTwo = ["C","D"]
// the following set is OUTPUTTED: [["AC","BD"],["AD","BC"]]
Such that if there were three elements in both listOne and listTwo, there would be SIX elements in the set. i.e:
// listOne = ["A","B","C"]
// listTwo = ["D","E","F"]
// OUTPUTTED: [["AD","BE","CF"],["AD","BF","CE"],["BD","AE","CF"],
// ["BD","AF","CE"],["CD","AE","BF"],["CD","AF","BE"]]
I tried writing this using a double enhanced FOR loop so I could understand the logic. My FOR loop approach is terrible and only works for the HARD-CODED limit of list.size() == 2.
// Create Lists and append elements
List<String> listOne = new ArrayList<String>();
listOne.add("A");
listOne.add("B");
List<String> listTwo = new ArrayList<String>();
listTwo.add("C");
listTwo.add("D");
// List One = ["A","B"]
// List Two = ["C","D"]
// Create new List
List<List<String>> newList = new ArrayList<List<String>>();
Integer counter = 0;
for (String s : listOne) {
counter++;
for (String p : listTwo) {
// A HARD-CODED bad implementation of this method
if (counter < 3) {
List<String> newListTwo = new ArrayList<String>();
newListTwo.add(s.concat(p));
newList.add(newListTwo);
} else if (!(counter % 2 == 0)) {
newList.get(1).add(s.concat(p));
} else {
newList.get(0).add(s.concat(p));
}
}
}
System.out.println(newList); // = [["AC","BD"],["AD","BC"]]
Also you can note that I defined List<List<String>> Rather than Set<List<String>>. This was due to my badly coded attempted which relies on the list.get() method.
So my current recursive method is as follows:
private static Set<List<String>> myRecursiveMethod(List<String> listOne,
List<String> listTwo)
{
//Base Case:
if (listOne.isEmpty){
return new HashSet<List<String>>;
}
//Recursive Case:
else {
String listOneFirst = listOne.get(0);
String listTwoFirst = listTwo.get(0);
List<String> sampleList = new ArrayList<String>();
sampleList.add(listOneFirst+listTwoFirst);
Set<List<String>> newSet = new HashSet<List<String>>(myRecursiveMethod())
newSet.add(sampleList);
return newSet;
}
}
This method only acts like this currently:
INPUT:
List One = ["A","B"]
List Two = ["C","D"]
OUTPUT:
[["AC"]["BD"]]
DESIRED OUTPUT:
[["AC","BD"],["AD","BC"]]
EDIT:
After reviewing responses my W.I.P code for the class:
private static Set<List<String>> myRecursiveMethod(List<String> listOne,
List<String> listTwo) {
//Backup Case (user enters an empty list)
if (listOne.isEmpty()){
return new HashSet<List<String>>();
}
// Base Case:
if (listOne.size() == 1) {
List<String> mergedStrings = new ArrayList<>();
for (String s : listTwo) {
mergedStrings.add(listOne.get(0).concat(s));
}
Set<List<String>> builtHashSet = new HashSet<List<String>();
builtHashSet.add(mergedStrings);
return builtHashSet;
}
// Recursive Case:
else {
// Ensure original list values arn't changed.
List<String> newListOne = new ArrayList<String>(listOne);
List<String> newListTwo = new ArrayList<String>(listTwo);
//first two elements...I don't think this is correct
String listOneFirst = newListOne.get(0);
String listTwoFirst = newListTwo.get(0);
List<String> sampleList = new ArrayList<String>();
sampleList.add(listOneFirst + listTwoFirst);
//used for making recursive case smaller
newListOne.remove(0);
// Calls recursion
Set<List<String>> newSet = new HashSet<List<String>>(
myRecursiveMethod(newListOne, newListTwo));
newSet.add(sampleList);
return newSet;
}
}
I think the problem is here:
if (listOne.isEmpty){
return new HashSet<List<String>>;
}
You are correct, at some point your recursion has to end, and you have to start building the desired output. But the desired output is not a Set with an empty list. It is a Set containing some lists with some content. Thus: don't wait until listOne is empty. Instead:
if (listOne.size() == 1) {
List<String> mergedStrings = new ArrayList<>();
mergedStrings = ... merge the ONE listOne entry with all listTwo entries
Set<List<String>> rv = new HashSet<>();
rv.add(mergedStrings);
return rv;
}
In other words: you use recursion to reduce the length of the first list by one. And when only one element is left in that list, it is time to merge in the second list.
Now lets look into how to "use" that (calling the method rec for brevity); putting down some pseudo code to show the steps we need:
rec([a, b], [c,d]) -->
rec([a], [c,d]) X rec([b], [c, d]) -->
<[ac, ad]> X <[bc, bd]> -->
<[ac, ad], [bc, bd]>
"X" meaning "joining" two results from recursive calls; should be as easy as:
Set<List<String>> rec1 = rec(...);
return rec1.addAll(rec2 ...
In the below example, how would I go about comparing the contents of ArrayList al4 to either of the other ArrayLists? In the same manner I compared al1 to al2.
import java.util.ArrayList;
public class Details {
public static void main(String[] args) {
ArrayList<String> al1 = new ArrayList<String>();
al1.add("hi");
al1.add("How are you");
al1.add("Good Morning");
al1.add("bye");
al1.add("Good night");
ArrayList<String> al2 = new ArrayList<String>();
al2.add("Howdy");
al2.add("Good Evening");
al2.add("bye");
al2.add("Good night");
ArrayList<ArrayList<String>> al4 = new ArrayList<ArrayList<String>>();
al4.add(al1);
// Storing the comparison output in ArrayList<String>
ArrayList<String> al3 = new ArrayList<String>();
for (String temp : al1)
al3.add(al2.contains(temp) ? "Yes" : "No");
System.out.println(al3);
}
}
Output for al3
[No, No, No, Yes, Yes]
It depends a little on what you mean. You can't directly compare al4 with al2, because they're of different types: one is an ArrayList<String> and one is an ArrayList<ArrayList<String>>.
When you compare al1 and al2, what you're doing is determining, for each element of al1, whether it occurs anywhere in al2. I presume this is what you want. You are not deciding whether it occurs in the same place in al2.
The first thing you should do is improve this comparison, which is rather inefficient. Since you don't care about ordering of al2, you should rewrite the comparison like this:
ArrayList<String> al3 = new ArrayList<String>();
Set<String> al2set = new HashSet<String>(al2);
for (String temp : al1)
al3.add(al2set.contains(temp) ? "Yes" : "No");
This is a lot more efficient because you don't have to traverse the whole of al2 for each lookup.
Now, if you want to compare your list of lists with a list, I can only assume that you mean you want to tell, for each element of any of the lists in al4, whether it occurs in al2. If so, then you want
ArrayList<String> al3 = new ArrayList<String>();
Set<String> al2set = new HashSet<String>(al2);
for (List<String> tempList : al4)
for (String temp : tempList)
al3.add(al2set.contains(temp) ? "Yes" : "No");
This will give you one flat ArrayList<String>, recording, for each element of the lists of al4, whether that element was in al2 somewhere.
It is possible that you want your result to be an ArrayList<ArrayList<String>>, so that the result mimics the structure of al4, in which case it's only slightly more complicated:
ArrayList<ArrayList<String>> al3list = new ArrayList<ArrayList<String>>();
Set<String> al2set = new HashSet<String>(al2);
for (List<String> tempList : al4) {
ArrayList<String> al3 = new ArrayList<String>();
for (String temp : tempList)
al3.add(al2set.contains(temp) ? "Yes" : "No");
al3list.add(al3);
}
After executing this, a3list will have the same structure as al4, and each element of each list will be either "Yes" or "No", according to whether the corresponding element in al4 was contained somewhere in al2.
You can create a method like this.
public static boolean[] compare(ArrayList<String> al1, ArrayList<String> al2) {
boolean result[] = new boolean[al1.size()]; // result
int i = 0; // index
/*
* Test if an element of al1 occurs in al2
*/
for(String s : al1) {
if(al2.contains(s)) {
result[i] = true;
}
else {
result[i] = false;
}
i++; // increment
}
return result;
}
And after you can call this method and display the result
boolean result[] = compare(al1, al2);
for(int i = 0; i < result.length; i++) {
System.out.print(result[i] + " ");
}