We Keep Coding

Remove duplicates elements from ArrayList which compose of Collection [duplicate]

I have an ArrayList<String>, and I want to remove repeated strings from it. How can I do this?

If you don't want duplicates in a Collection, you should consider why you're using a Collection that allows duplicates. The easiest way to remove repeated elements is to add the contents to a Set (which will not allow duplicates) and then add the Set back to the ArrayList:
Set<String> set = new HashSet<>(yourList);
yourList.clear();
yourList.addAll(set);
Of course, this destroys the ordering of the elements in the ArrayList.

Although converting the ArrayList to a HashSet effectively removes duplicates, if you need to preserve insertion order, I'd rather suggest you to use this variant
// list is some List of Strings
Set<String> s = new LinkedHashSet<>(list);
Then, if you need to get back a List reference, you can use again the conversion constructor.

In Java 8:
List<String> deduped = list.stream().distinct().collect(Collectors.toList());
Please note that the hashCode-equals contract for list members should be respected for the filtering to work properly.

Suppose we have a list of String like:
List<String> strList = new ArrayList<>(5);
// insert up to five items to list.
Then we can remove duplicate elements in multiple ways.
Prior to Java 8
List<String> deDupStringList = new ArrayList<>(new HashSet<>(strList));
Note: If we want to maintain the insertion order then we need to use LinkedHashSet in place of HashSet
Using Guava
List<String> deDupStringList2 = Lists.newArrayList(Sets.newHashSet(strList));
Using Java 8
List<String> deDupStringList3 = strList.stream().distinct().collect(Collectors.toList());
Note: In case we want to collect the result in a specific list implementation e.g. LinkedList then we can modify the above example as:
List<String> deDupStringList3 = strList.stream().distinct()
.collect(Collectors.toCollection(LinkedList::new));
We can use parallelStream also in the above code but it may not give expected performace benefits. Check this question for more.

If you don't want duplicates, use a Set instead of a List. To convert a List to a Set you can use the following code:
// list is some List of Strings
Set<String> s = new HashSet<String>(list);
If really necessary you can use the same construction to convert a Set back into a List.

Java 8 streams provide a very simple way to remove duplicate elements from a list. Using the distinct method.
If we have a list of cities and we want to remove duplicates from that list it can be done in a single line -
List<String> cityList = new ArrayList<>();
cityList.add("Delhi");
cityList.add("Mumbai");
cityList.add("Bangalore");
cityList.add("Chennai");
cityList.add("Kolkata");
cityList.add("Mumbai");
cityList = cityList.stream().distinct().collect(Collectors.toList());
How to remove duplicate elements from an arraylist

You can also do it this way, and preserve order:
// delete duplicates (if any) from 'myArrayList'
myArrayList = new ArrayList<String>(new LinkedHashSet<String>(myArrayList));

Here's a way that doesn't affect your list ordering:
ArrayList l1 = new ArrayList();
ArrayList l2 = new ArrayList();
Iterator iterator = l1.iterator();
while (iterator.hasNext()) {
YourClass o = (YourClass) iterator.next();
if(!l2.contains(o)) l2.add(o);
}
l1 is the original list, and l2 is the list without repeated items
(Make sure YourClass has the equals method according to what you want to stand for equality)

this can solve the problem:
private List<SomeClass> clearListFromDuplicateFirstName(List<SomeClass> list1) {
Map<String, SomeClass> cleanMap = new LinkedHashMap<String, SomeClass>();
for (int i = 0; i < list1.size(); i++) {
cleanMap.put(list1.get(i).getFirstName(), list1.get(i));
}
List<SomeClass> list = new ArrayList<SomeClass>(cleanMap.values());
return list;
}

It is possible to remove duplicates from arraylist without using HashSet or one more arraylist.
Try this code..
ArrayList<String> lst = new ArrayList<String>();
lst.add("ABC");
lst.add("ABC");
lst.add("ABCD");
lst.add("ABCD");
lst.add("ABCE");
System.out.println("Duplicates List "+lst);
Object[] st = lst.toArray();
for (Object s : st) {
if (lst.indexOf(s) != lst.lastIndexOf(s)) {
lst.remove(lst.lastIndexOf(s));
}
}
System.out.println("Distinct List "+lst);
Output is
Duplicates List [ABC, ABC, ABCD, ABCD, ABCE]
Distinct List [ABC, ABCD, ABCE]

There is also ImmutableSet from Guava as an option (here is the documentation):
ImmutableSet.copyOf(list);

Probably a bit overkill, but I enjoy this kind of isolated problem. :)
This code uses a temporary Set (for the uniqueness check) but removes elements directly inside the original list. Since element removal inside an ArrayList can induce a huge amount of array copying, the remove(int)-method is avoided.
public static <T> void removeDuplicates(ArrayList<T> list) {
int size = list.size();
int out = 0;
{
final Set<T> encountered = new HashSet<T>();
for (int in = 0; in < size; in++) {
final T t = list.get(in);
final boolean first = encountered.add(t);
if (first) {
list.set(out++, t);
}
}
}
while (out < size) {
list.remove(--size);
}
}
While we're at it, here's a version for LinkedList (a lot nicer!):
public static <T> void removeDuplicates(LinkedList<T> list) {
final Set<T> encountered = new HashSet<T>();
for (Iterator<T> iter = list.iterator(); iter.hasNext(); ) {
final T t = iter.next();
final boolean first = encountered.add(t);
if (!first) {
iter.remove();
}
}
}
Use the marker interface to present a unified solution for List:
public static <T> void removeDuplicates(List<T> list) {
if (list instanceof RandomAccess) {
// use first version here
} else {
// use other version here
}
}
EDIT: I guess the generics-stuff doesn't really add any value here.. Oh well. :)

public static void main(String[] args){
ArrayList<Object> al = new ArrayList<Object>();
al.add("abc");
al.add('a');
al.add('b');
al.add('a');
al.add("abc");
al.add(10.3);
al.add('c');
al.add(10);
al.add("abc");
al.add(10);
System.out.println("Before Duplicate Remove:"+al);
for(int i=0;i<al.size();i++){
for(int j=i+1;j<al.size();j++){
if(al.get(i).equals(al.get(j))){
al.remove(j);
j--;
}
}
}
System.out.println("After Removing duplicate:"+al);
}

If you're willing to use a third-party library, you can use the method distinct() in Eclipse Collections (formerly GS Collections).
ListIterable<Integer> integers = FastList.newListWith(1, 3, 1, 2, 2, 1);
Assert.assertEquals(
FastList.newListWith(1, 3, 2),
integers.distinct());
The advantage of using distinct() instead of converting to a Set and then back to a List is that distinct() preserves the order of the original List, retaining the first occurrence of each element. It's implemented by using both a Set and a List.
MutableSet<T> seenSoFar = UnifiedSet.newSet();
int size = list.size();
for (int i = 0; i < size; i++)
{
T item = list.get(i);
if (seenSoFar.add(item))
{
targetCollection.add(item);
}
}
return targetCollection;
If you cannot convert your original List into an Eclipse Collections type, you can use ListAdapter to get the same API.
MutableList<Integer> distinct = ListAdapter.adapt(integers).distinct();
Note: I am a committer for Eclipse Collections.

If you are using model type List< T>/ArrayList< T> . Hope,it's help you.
Here is my code without using any other data structure like set or hashmap
for (int i = 0; i < Models.size(); i++){
for (int j = i + 1; j < Models.size(); j++) {
if (Models.get(i).getName().equals(Models.get(j).getName())) {
Models.remove(j);
j--;
}
}
}

If you want to preserve your Order then it is best to use LinkedHashSet.
Because if you want to pass this List to an Insert Query by Iterating it, the order would be preserved.
Try this
LinkedHashSet link=new LinkedHashSet();
List listOfValues=new ArrayList();
listOfValues.add(link);
This conversion will be very helpful when you want to return a List but not a Set.

This three lines of code can remove the duplicated element from ArrayList or any collection.
List<Entity> entities = repository.findByUserId(userId);
Set<Entity> s = new LinkedHashSet<Entity>(entities);
entities.clear();
entities.addAll(s);

for(int a=0;a<myArray.size();a++){
for(int b=a+1;b<myArray.size();b++){
if(myArray.get(a).equalsIgnoreCase(myArray.get(b))){
myArray.remove(b);
dups++;
b--;
}
}
}

When you are filling the ArrayList, use a condition for each element. For example:
ArrayList< Integer > al = new ArrayList< Integer >();
// fill 1
for ( int i = 0; i <= 5; i++ )
if ( !al.contains( i ) )
al.add( i );
// fill 2
for (int i = 0; i <= 10; i++ )
if ( !al.contains( i ) )
al.add( i );
for( Integer i: al )
{
System.out.print( i + " ");
}
We will get an array {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

Code:
List<String> duplicatList = new ArrayList<String>();
duplicatList = Arrays.asList("AA","BB","CC","DD","DD","EE","AA","FF");
//above AA and DD are duplicate
Set<String> uniqueList = new HashSet<String>(duplicatList);
duplicatList = new ArrayList<String>(uniqueList); //let GC will doing free memory
System.out.println("Removed Duplicate : "+duplicatList);
Note: Definitely, there will be memory overhead.

ArrayList<String> city=new ArrayList<String>();
city.add("rajkot");
city.add("gondal");
city.add("rajkot");
city.add("gova");
city.add("baroda");
city.add("morbi");
city.add("gova");
HashSet<String> hashSet = new HashSet<String>();
hashSet.addAll(city);
city.clear();
city.addAll(hashSet);
Toast.makeText(getActivity(),"" + city.toString(),Toast.LENGTH_SHORT).show();

you can use nested loop in follow :
ArrayList<Class1> l1 = new ArrayList<Class1>();
ArrayList<Class1> l2 = new ArrayList<Class1>();
Iterator iterator1 = l1.iterator();
boolean repeated = false;
while (iterator1.hasNext())
{
Class1 c1 = (Class1) iterator1.next();
for (Class1 _c: l2) {
if(_c.getId() == c1.getId())
repeated = true;
}
if(!repeated)
l2.add(c1);
}

LinkedHashSet will do the trick.
String[] arr2 = {"5","1","2","3","3","4","1","2"};
Set<String> set = new LinkedHashSet<String>(Arrays.asList(arr2));
for(String s1 : set)
System.out.println(s1);
System.out.println( "------------------------" );
String[] arr3 = set.toArray(new String[0]);
for(int i = 0; i < arr3.length; i++)
System.out.println(arr3[i].toString());
//output: 5,1,2,3,4

List<String> result = new ArrayList<String>();
Set<String> set = new LinkedHashSet<String>();
String s = "ravi is a good!boy. But ravi is very nasty fellow.";
StringTokenizer st = new StringTokenizer(s, " ,. ,!");
while (st.hasMoreTokens()) {
result.add(st.nextToken());
}
System.out.println(result);
set.addAll(result);
result.clear();
result.addAll(set);
System.out.println(result);
output:
[ravi, is, a, good, boy, But, ravi, is, very, nasty, fellow]
[ravi, is, a, good, boy, But, very, nasty, fellow]

This is used for your Custom Objects list
public List<Contact> removeDuplicates(List<Contact> list) {
// Set set1 = new LinkedHashSet(list);
Set set = new TreeSet(new Comparator() {
#Override
public int compare(Object o1, Object o2) {
if (((Contact) o1).getId().equalsIgnoreCase(((Contact) o2).getId()) /*&&
((Contact)o1).getName().equalsIgnoreCase(((Contact)o2).getName())*/) {
return 0;
}
return 1;
}
});
set.addAll(list);
final List newList = new ArrayList(set);
return newList;
}

As said before, you should use a class implementing the Set interface instead of List to be sure of the unicity of elements. If you have to keep the order of elements, the SortedSet interface can then be used; the TreeSet class implements that interface.

import java.util.*;
class RemoveDupFrmString
{
public static void main(String[] args)
{
String s="appsc";
Set<Character> unique = new LinkedHashSet<Character> ();
for(char c : s.toCharArray()) {
System.out.println(unique.add(c));
}
for(char dis:unique){
System.out.println(dis);
}
}
}

public Set<Object> findDuplicates(List<Object> list) {
Set<Object> items = new HashSet<Object>();
Set<Object> duplicates = new HashSet<Object>();
for (Object item : list) {
if (items.contains(item)) {
duplicates.add(item);
} else {
items.add(item);
}
}
return duplicates;
}

ArrayList<String> list = new ArrayList<String>();
HashSet<String> unique = new LinkedHashSet<String>();
HashSet<String> dup = new LinkedHashSet<String>();
boolean b = false;
list.add("Hello");
list.add("Hello");
list.add("how");
list.add("are");
list.add("u");
list.add("u");
for(Iterator iterator= list.iterator();iterator.hasNext();)
{
String value = (String)iterator.next();
System.out.println(value);
if(b==unique.add(value))
dup.add(value);
else
unique.add(value);
}
System.out.println(unique);
System.out.println(dup);

If you want to remove duplicates from ArrayList means find the below logic,
public static Object[] removeDuplicate(Object[] inputArray)
{
long startTime = System.nanoTime();
int totalSize = inputArray.length;
Object[] resultArray = new Object[totalSize];
int newSize = 0;
for(int i=0; i<totalSize; i++)
{
Object value = inputArray[i];
if(value == null)
{
continue;
}
for(int j=i+1; j<totalSize; j++)
{
if(value.equals(inputArray[j]))
{
inputArray[j] = null;
}
}
resultArray[newSize++] = value;
}
long endTime = System.nanoTime()-startTime;
System.out.println("Total Time-B:"+endTime);
return resultArray;
}

Sorting a String ArrayList with numbers Java

Hello everyone I have a code using an arraylist, these are the test inputs to be added. But after I use the sort method. The output is not the one I expected.
ArrayList<String> test= new ArrayList<>();
test.add("2,2,17");
test.add("5,4,24 ");
test.add("8,1,11");
test.add("19,0,0");
test.add("2,3,21");
test.sort(null);
Output :
19,0,0
2,2,17
2,3,21
5,4,24
8,1,11
My desired out put should be :
2,2,17
2,3,21
5,4,24
8,1,11
19,0,0
Is there a way to sort "19,0,0" to be at the end, or any number to be add to make it the end of the arrayList?

You'll want to use Collections.sort, something like this:
Collections.sort(list, new Comparator<String>() {
#Override
public int compare(String s1, String s2) {
int s1int = Integer.parseInt(s1.substring(0, s1.indexOf(",")));
int s2int = Integer.parseInt(s2.substring(0, s2.indexOf(",")));
return s1int - s2int;
}
});

You can use Collections.sort()
https://docs.oracle.com/javase/6/docs/api/java/util/Collections.html#sort(java.util.List,%20java.util.Comparator).
And define a Comparator to compare the objects in the list.

In order for the strings to be sorted alphabetically rather than numerically, you will need to implement a comparator that converts the strings to integers for the comparison. You can then use this comparator with Collections.sort() to sort your list.
A better option would be to store your integers as integers in a 2D array rather than as strings (or some kind of nested list if the dimensions are not known up-front); however, I'd need to know more about how the data is created and used before uniformly proclaiming this to e the solution.

A possible flexible solution for Strings of various 'lengths', i.e. a different number of integers separated by commas.
Example list
2,2,17
5,4,24
19,0,2
8,1,11
19,0,1,2
19,0,1,4
2,3,21
2
Result after sorting
2
2,2,17
2,3,21
5,4,24
8,1,11
19,0,1,2
19,0,1,4
19,0,2
Code
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
public class SortExample {
public static void main(String[] args) {
List<String> test = new ArrayList<>();
test.add("2,2,17");
test.add("5,4,24");
test.add("19,0,2");
test.add("8,1,11");
test.add("19,0,1,2");
test.add("19,0,1,4");
test.add("2,3,21");
test.add("2");
Collections.sort(test, new Comparator<String>() {
#Override
public int compare(String s1, String s2) {
String[] split1 = s1.split(",");
String[] split2 = s2.split(",");
final int indicesToCheck = Math.min(split1.length, split2.length);
int result = 0;
int i = 0;
while (result == 0 && i < indicesToCheck) {
result = Integer.compare(Integer.parseInt(split1[i]),
Integer.parseInt(split2[i]));
i++;
}
return result == 0 ? Integer.compare(split1.length, split2.length) : result;
}
});
for (String s : test) {
System.out.println(s);
}
}
}

How to reverse a String array based on it's index value

I want to reverse this:
customArray = myArray;
nameArray = new String[myArray.size()];
for (int i = 0; i < myArray.size(); i++) {
idArray[i] = myArray.get(i).getUnitID();
nameArray[i] = myArray.get(i).getUnitName();
}
if (sortType == SORT)
Arrays.sort(idArray, Collections.reverseOrder());
Arrays.sort(nameArray, ...);
I know how to reverse this by using Arrays.sort(stringArray, Collections.reverseOrder()); but not by index value.
How can I reverse the order of my nameArray based on it's nameArray[i]?.. or even better, since my idArray is actually a list of unique ID's I would like to sort nameArray based on the idArray.

(Update: Now reversing an array of strings and not a list of strings.)
Use Collections.reverse together with Arrays.asList to reverse the array.
Example:
public static void main(String... args) {
String[] array = {"one", "two", "three"};
Collections.reverse(Arrays.asList(array)); // reverses the underlying list
System.out.println(Arrays.toString(array)); // prints [three, two, one]
}

The only solution to your problem is to use an OOP approach. After all, Java is an OOP Language. So instead of having two arrays:
int[] unitID
String[] unitName
which you can't sort in a way that the indexes stay corresponding, write a class Unit that implements Comparable<Unit> and use one array:
Unit[] units
then
Arrays.sort(units);
will do the job.
You would need your own class:
public class Unit implements Comparable<Unit> {
private int id;
private String name;
// Constructor
// Methods
#Override
public int compareTo(Unit other) {
// sorts by id
return this.id - other.id;
// to sort by name, use this:
// return this.name.compareTo(other.name);
}
}

Try this:
import java.util.Collections;
import java.util.List;
import java.util.Arrays;
public void reverseString(String[] stringArray){
List<String> list = Arrays.asList(stringArray);
Collections.reverse(list);
stringArray= (String[]) list.toArray();
}
String[] yourStringArray = new String[]{"Sunday", "Monday", "Tuesday", "Wednesday"};
reverseString(yourStringArray);
//result would be:
//"Wednesday","Tuesday", "Monday", "Sunday"

Collections.sort vs Arrays.sort - in Java

I was doing this sort first:
List<String> items = new ArrayList<String>();
Comparator<items> ignoreLeadingThe = new Comparator<items>() {
public int compare(String a, String b) {
a = a.replaceAll("(?i(^the\\s+", "");
b = b.replaceAll("(?i(^the\\s+", "");
return a.compareToIgnoreCase(b);
}
};
Collections.sort(items, ignoreLeadingThe);
now I am doing this:
ItemObject[] io = new ItemObject[items.size()];
Comparator<ItemObject> ignoreLeadingThe = new Comparator<ItemObject>() {
public int compare(ItemObject a, ItemObject b) {
a.name = a.name.replaceAll("(?i(^the\\s+", "");
b.name = b.name.replaceAll("(?i(^the\\s+", "");
return a.name.compareToIgnoreCase(b.name);
}
};
Arrays.sort(io, ignoreLeadingThe);
When I was sorting the ArrayList up top, it acted as normal; it ignored the "The " and sorted list accordingly; but it wasn't actually effecting the output of the list.
However, the bottom code when I am sorting a regular Array (filled with Objects and not Strings), actually remove "The ". For example, "The Joker", would become "Joker".
Does anyone see what is going wrong here?

As I stated in my comment,
You're overwriting a.name and b.name. Declare separate local variables for the transformed names and use those in your compareToIgnoreCase.
... or just use one large expression. So, try something like this...
final Comparator<ItemObject> ignoreLeadingThe = new Comparator<ItemObject>() {
final Pattern pattern = Pattern.compile("(?i(^the\\s+");
public int compare(final ItemObject a, final ItemObject b) {
return pattern.matcher(a.name).replaceAll("")
.compareToIgnoreCase(pattern.matcher(b.name).replaceAll(""));
}
};

Java sort String array of file names by their extension

I have an array of filenames and need to sort that array by the extensions of the filename. Is there an easy way to do this?

Arrays.sort(filenames, new Comparator<String>() {
#Override
public int compare(String s1, String s2) {
// the +1 is to avoid including the '.' in the extension and to avoid exceptions
// EDIT:
// We first need to make sure that either both files or neither file
// has an extension (otherwise we'll end up comparing the extension of one
// to the start of the other, or else throwing an exception)
final int s1Dot = s1.lastIndexOf('.');
final int s2Dot = s2.lastIndexOf('.');
if ((s1Dot == -1) == (s2Dot == -1)) { // both or neither
s1 = s1.substring(s1Dot + 1);
s2 = s2.substring(s2Dot + 1);
return s1.compareTo(s2);
} else if (s1Dot == -1) { // only s2 has an extension, so s1 goes first
return -1;
} else { // only s1 has an extension, so s1 goes second
return 1;
}
}
});
For completeness: java.util.Arrays and java.util.Comparator.

If I remember correctly, the Arrays.sort(...) takes a Comparator<> that it will use to do the sorting. You can provide an implementation of it that looks at the extension part of the string.

You can implement a custom Comparator of Strings. Make it sort them by the substring after the last index of '.'. Then pass in the comparator and your array into
Arrays.sort(stringArray, yourComparator);
// An implementation of the compare method
public int compare(String o1, String o2) {
return o1.substring(o1.lastIndexOf('.')).compareTo(o2.substring(o2.lastIndexOf('.'));
}

Comparators are often hard to get exactly right, and the comparison key has to be generated for every comparison which for most sorting algorithms mean O(n log n). Another approach is to create (key, value) pairs for each item you need to sort, put them in a TreeMap, and then ask for the values as these are sorted according to the key.
For instance
import java.util.Arrays;
import java.util.TreeMap;
public class Bar {
public static void main(String[] args) {
TreeMap<String, String> m2 = new TreeMap<String, String>();
for (String string : Arrays.asList(new String[] { "#3", "#2", "#1" })) {
String key = string.substring(string.length() - 1);
String value = string;
m2.put(key, value);
}
System.out.println(m2.values());
}
}
prints out
[#1, #2, #3]
You should easily be able to adapt the key calculation to your problem.
This only calculates the key once per entry, hence O(n) - (but the sort is still O(n log n)). If the key calculation is expensive or n is large this might be quite measurable.

Create a Comparator and compare the string extensions. Take a look at the following
http://java.sun.com/j2se/1.4.2/docs/api/java/util/Comparator.html
Then pass in your List of strings to Arrays.sort(List, Comparator)

Create your own Comparator that treats the strings as filenames and compares them based on the extensions. Then use Arrays.sort with the Comparator argument.

String DELIMETER = File.separator + ".";
List<String> orginalList = new CopyOnWriteArrayList<>(Arrays.asList(listOfFileNames));
Set<String> setOfuniqueExtension = new TreeSet<>();
for (String item : listOfFileNames) {
if (item.contains(".")) {
String[] split = item.split(DELIMETER);
String temp = "." + split[split.length - 1];
setOfuniqueExtension.add(temp);
}
}
List<String> finalListOfAllFiles = new LinkedList<>();
setOfuniqueExtension.stream().forEach((s1) -> {
for (int i = 0; i < orginalList.size(); i++) {
if (orginalList.get(i).contains(s1)) {
finalListOfAllFiles.add(orginalList.get(i));
orginalList.remove(orginalList.get(i));
i--;
}
}
});
orginalList.stream().filter((s1) -> (!finalListOfAllFiles.contains(s1))).forEach((s1) -> {
finalListOfAllFiles.add(s1);
});
return finalListOfAllFiles;

If you just want to group the files by their extension and do not care about the actual alphabetical order, you can use this:
I think the simplest thing you can do that also works when the filenname does not have a "." is to just reverse the names and compare them.
Arrays.sort(ary, new Comparator<String>() {
#Override
public int compare(String o1, String o2) {
String r1 = new StringBuffer(o1).reverse().toString();
String r2 = new StringBuffer(o2).reverse().toString();
return r1.compareTo(r2);
}
});
Its a shame that java's string does not even have a reverse().