We Keep Coding

Remove duplicate strings from an arraylist<Object>

My program is opening a file and then saves its words and their byte distance from the file beginning . Though the file has too many duplicate words that i don't want . Also i want my list to be in alphabetical order . The problem is that when i fix the order the duplicate are messed and vice versa . Here is my code:
import java.io.*;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashSet;
import java.util.LinkedList;
import java.util.Set;
class MyMain {
public static void main(String[] args) throws IOException {
ArrayList<DictPage> listOfWords = new ArrayList<DictPage>();
LinkedList<Page> Eurethrio = new LinkedList<Page>();
File file = new File("C:\\Kennedy.txt");
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(file)));
//This will reference one line at a time...
String line = null;
int line_count=0;
int byte_count;
int total_byte_count=0;
int fromIndex;
int kat = 0;
while( (line = br.readLine())!= null ){
line_count++;
fromIndex=0;
String [] tokens = line.split(",\\s+|\\s*\\\"\\s*|\\s+|\\.\\s*|\\s*\\:\\s*");
String line_rest=line;
for (int i=1; i <= tokens.length; i++) {
byte_count = line_rest.indexOf(tokens[i-1]);
//if ( tokens[i-1].length() != 0)
//System.out.println("\n(line:" + line_count + ", word:" + i + ", start_byte:" + (total_byte_count + fromIndex) + "' word_length:" + tokens[i-1].length() + ") = " + tokens[i-1]);
fromIndex = fromIndex + byte_count + 1 + tokens[i-1].length();
if (fromIndex < line.length())
line_rest = line.substring(fromIndex);
if(!listOfWords.contains(tokens[i-1])){//Na mhn apothikevetai h idia leksh
//listOfWords.add(tokens[i-1]);
listOfWords.add(new DictPage(tokens[i-1],kat));
kat++;
}
Eurethrio.add(new Page("Kennedy",fromIndex));
}
total_byte_count += fromIndex;
Eurethrio.add(new Page("Kennedy", total_byte_count));
}
Set<DictPage> hs = new HashSet<DictPage>();
hs.addAll(listOfWords);
listOfWords.clear();
listOfWords.addAll(hs);
if (listOfWords.size() > 0) {
Collections.sort(listOfWords, new Comparator<DictPage>() {
#Override
public int compare(final DictPage object1, final DictPage object2) {
return object1.getWord().compareTo(object2.getWord());
}
} );
}
//Ektypwsh leksewn...
for (int i = 0; i<listOfWords.size();i++){
System.out.println(""+listOfWords.get(i).getWord()+" "+listOfWords.get(i).getPage());
}
for (int i = 0;i<Eurethrio.size();i++){
System.out.println(""+Eurethrio.get(i).getFile()+" "+Eurethrio.get(i).getBytes());
}
}
}

Use the TreeSet instead of ArrayList, and you'll get automatically order and no repeatings.

In the first place, why are you using ArrayList to store your list of words.
ArrayList<DictPage> listOfWords = new ArrayList<DictPage>();
You should use Set (like HashSet, TreeSet or some implementation of Set) to store your words if you don't want duplicates.
Set<DictPage> listOfWords = new Hashset<DictPage>(); //no duplicates but not sorted
Or
Set<DictPage> listOfWords = new Treeset<DictPage>(); //no duplicates and sorted as well
This would make sure that your list of words does not contain any duplicates.
And if you want them sorted straight away, you can use TreeSet which will make it more easier for you.

use this.
public void stripDuplicatesFromFile(String filename) {
try {
BufferedReader reader = new BufferedReader(new FileReader(filename));
Set<String> lines = new HashSet<String>();
String line;
while ((line = reader.readLine()) != null) {
lines.add(line);
}
reader.close();
BufferedWriter writer = new BufferedWriter(new FileWriter(filename));
for (String unique : lines) {
writer.write(unique);
writer.newLine();
}
writer.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
it takes filepath as an input, find duplicate lines and remove them. But if you have large file do not use this. I'm using this method on a very small size of a .txt file (kind of log file and order is not imported).

Sorting .csv Id's in natural order Java

I'm trying to write some code that will take in a list of IDs (numbers and letters) from a .csv file and output them to a new file with the IDs in "natural order". My files are compiling, but I am getting the error:
java.lang.NumberFormatException: For input string: "Alpha"
I think the issue is I am not accounting for both number and letter values in the .csv file. What am I doing wrong?! Sorry if my variable Id's are confusing...
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
public class IdReader {
public static String CSV_FILE_PATH = "/Users/eringray/Desktop/idreader/idData.csv";
public static void main(String[] args){
try {
BufferedReader br = new BufferedReader(new FileReader(CSV_FILE_PATH));
BufferedWriter bw = new BufferedWriter(new FileWriter(CSV_FILE_PATH + ".tsv"));
ArrayList<String> textIds = new ArrayList<>();
ArrayList<Integer> numberIds = new ArrayList<>();
String line = "";
while((line = br.readLine()) != null) {
String[] values = line.split(" ");
if(values.length == 1) {
String idAsString = values[0];
try{
int id = Integer.parseInt(idAsString);
numberIds.add(id);
}
catch(NumberFormatException e){
textIds.add(idAsString);
}
}
}
Collections.sort(textIds);
Collections.sort(numberIds);
for(int i = 0; i < textIds.size(); i++){
String stu = textIds.get(i);
String lineText = stu.toString();
bw.write(lineText);
bw.newLine();
}
for(int i = 0; i < numberIds.size(); i++){
int numValues = numberIds.get(i);
bw.write(numValues);
bw.newLine();
}
br.close();
bw.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

The exception is coming at this line
int id = Integer.parseInt(idAsString);
Clearly alpha is not an integer, so it will throw NumberFormatException. In a case, where you encounter such Strings which cannot be converted into numbers, you can either skips them or throw an exception.
Update
//Use two seperate lists, one for maintaining numbers and other for text
ArrayList<String> textIds = new ArrayList<>();
ArrayList<Integer> numberIds = new ArrayList<>();
String line = "";
while((line = br.readLine()) != null) {
String[] values = line.split(" ");
if(values.length == 1) {
String idAsString = values[0];
try {
//Parse the value. If successful, it means it was a number. Add to integer array.
int id = Integer.parseInt(idAsString);
numberIds.add(id);
}catch (NumberFormatException e){
//If not successful, it means it was a string.
textIds.add(idAsString);
}
}
}
//In the end sort both the list
Collections.sort(textIds);
Collections.synchronizedList(numberIds);
for(int i = 0; i < textIds.size(); i++){
String stu = textIds.get(i);
bw.write(stu);
bw.newLine();
}
for(int i = 0; i < numberIds.size(); i++){
int numValues = numberIds.get(i);
bw.write(numValues+"");
bw.newLine();
}
br.close();
bw.close();
I am not putting code for writing this data to a new file. I hope you can do that.
Sample Input
4
6
33
2
5632
23454
Alpha
So after running my code
numberIds will have [ 2,4,6,33,5632,23454]
textIds will have ["Alpha"]

NumberFormatException occurs because of AlphaNumeric characters in the input.
Please use isNumeric(str) metod in https://commons.apache.org/proper/commons-lang/javadocs/api-2.6/org/apache/commons/lang/StringUtils.html api to verify whether the input is numeric or not and convert to int , only it is numeric

Read a file into Map<Integer, ArrayList<Double>>

I saw some similar questions, but mine is a little different.
I define a
Map<Integer, ArrayList<Double>> fl;
My input .txt file:
1 0.56 0.57 0.73 ..
2 2.3 3.50 ...
9 4.98 0.99 ..
How to read the file into the map fl?
Thanks!

Use a Scanner and first call Scanner.readInt() that will give you the first integer.
Then call Scanner.readLine() that will give you all the remaining double in the line as a String. Split it and parse everything to double.
Repeat the same till end of file.

Here's a try.
I've compiled and run the code.
Make sure the input file is in the same directory as your project if you use an IDE.-- This only applies if you do not modify the path below.
package fileread;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
public class FileRead {
private static HashMap<Integer, ArrayList<Double>> map = new HashMap<>();
private static BufferedReader reader;
public static void main(String[] args) {
try
{
reader = new BufferedReader(new FileReader("input"));
//or reader = new BufferedReader(new FileReader("C:\\full-path-to-your-file));
String line;
while((line = reader.readLine()) != null)
{
String[] tokens = line.split(" ");
Integer i;
Double d;
ArrayList<Double> list = new ArrayList<>();
i = Integer.valueOf(tokens[0]);
for(int j = 1; j < tokens.length; j++)
list.add(Double.valueOf(tokens[j]));
map.put(i, list);
}
}catch(IOException ex)
{
//break execution
}finally
{
if(reader != null)
try
{
reader.close();
}catch (IOException ex) {
//don't break :)
}
}
for(Integer i : map.keySet())
{
ArrayList<Double> l = map.get(i);
System.out.print("Line " + i + ": ");
for(Double d: l)
System.out.print(d + " ");
System.out.println();
}
}
}

The code for parsing the file and populating the map should be like below
try {
BufferedReader bReader = new BufferedReader(new FileReader(new File("c:/input .txt")));
String line = "";
Map<Integer, ArrayList<Double>> fl = new HashMap<Integer, ArrayList<Double>>();
while ((line = bReader.readLine()) != null) {
String[] strArray = line.split(" ");
for (int i=0;i<strArray.length;i++) {
ArrayList<Double> value = new ArrayList<Double>();
int key=0;
if(i==0){
key =Integer.valueOf(strArray[0]);
}
else{
value.add(Double.valueOf(strArray[i]));
}
fl.put(key, value);
}
}
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

Checking the elements of a set for multiple integers

I want to change the form of a given graph. The graph is in the form userID, number of followers, follower1, follower2, ..followerN, delimiter '---', userID2, ...etc.
I have to replace the followers with place values from a second file of the form
ID1 place1
ID2 place2
....
by matching the IDs.
Thus, I want to check if the followers id each time exists in a set.
Both my graph and the set where I seek for the follower ids are huge.
Is there a more efficient way than the one I give you below?
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.DataInputStream;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.InputStreamReader;
import java.util.*;
public class Changer {
public static void main(String[] args) {
Set set = new HashSet();
int[][] users=new int[61578415][];
try{
FileInputStream fstream2 = new FileInputStream(args[0]);
DataInputStream in2 = new DataInputStream(fstream2);
BufferedReader br2 = new BufferedReader(new InputStreamReader(in2));
String strLine2;
while ((strLine2 = br2.readLine()) != null) {
set.add(strLine2);
}
in2.close();
fstream2.close();}
catch (Exception e){
System.err.println("Error: " + e.getMessage()+"!\n"+e.toString()+"!\n");
e.printStackTrace();
System.exit(-1);
}
try{
FileInputStream fstream = new FileInputStream("InputGraph.txt");
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String strLine;
int flag=0;
int pos=0;
FileWriter fstream3 = new FileWriter("OutputGraph.txt");
BufferedWriter out = new BufferedWriter(fstream3);
int currentUser=0,counter=0;
int theNum=0;
while ((strLine = br.readLine()) != null) {
if(strLine.equals("---")){
if(counter!=pos){
System.out.println("Error reading graph");
System.out.println("For:"+currentUser);
System.exit(-1);
}
flag=0;
pos=0;
continue;
}
theNum=Integer.parseInt(strLine);
if (flag==0){
out.write("---"+"\n");
out.write(""+theNum);
out.write("\n");
currentUser=theNum;
flag+=1;
}
else if (flag==1){
counter=theNum;
users[currentUser]=new int [counter];
flag+=1;
out.write(""+theNum+"\n");
}
else{
users[currentUser][pos]=theNum;
++pos;
Iterator it = set.iterator();
while (it.hasNext()) {
Object element = it.next();
String[] arr = (String.valueOf(element)).split(" ");
if (Integer.parseInt(arr[0])==theNum)
{theNum=Integer.parseInt(arr[1]);break;}
}
out.write(""+theNum);
out.write("\n");
}
}
in.close();
out.close();
}catch (Exception e){
System.err.println("Error: " + e.getMessage());
}
System.out.println("Graph has been read");
System.gc();
System.gc();
System.out.println("Finished");
}
}

It would be more efficient to do the for loop over intersection inside, so that you don't split and parse so much:
Iterator it = set.iterator();
while (it.hasNext()) {
Object element = it.next();
String[] arr = (String.valueOf(element)).split(" ");
int arr0 = Integer.parseInt(arr[0]);
int arr1 = Integer.parseInt(arr[1]);
for (int integer : intersection) {
if (arr0 == integer) {
out.write(integer + " " + arr1 + "\n");
}
}
}
But this changes the order the write will be called in.
However I suspect you might benefit from loading it in to (or just replacing with) a HashMap or SparseArray. It's just hard to tell give then info you have given.

For Integer detection, you can use the comparation with instanceof , but you must use objects and not primitives, e.g:
Integer a=Integer.parseInt("12345");
if(a instanceof Integer){
System.out.println("We have an integer !");
}
Another way for Integer detection is to override the equals method.

read specific substring with condition

I have this from text file:
134.897 134.4565 135.134
I read them using :
while (( line = buffreader.readLine()) != null) {
String[] parts = line.split(" ");
String x1 = (parts[0]);
String x2 = (parts[1]);
String x3 = (parts[2]);
From the String in the text file :
134.897
134.4565
135.134
I just want to take the different number between these three number :
4.897
4.4565
5.134
Given more example :
Text file :
101.435
101.423
101.322
Number I want to take :
435
423
322
My plan is I want to compare each number with others,
101.435//x1
101.423//x2
101.322//x3
if x1.substring(0)=x2.substring(0)=x3.substring(0)
then remove substring(0).
I want to loop this "if" condition until the substring are not same.
Please give me some idea, thanks

Here's the skeleton of an algorithm that could be used:
List<String> inputs = ...;
int index = 0;
while (allTheStringsHaveTheSameCharacterAtIndex(inputs, index)) {
index++;
}
List<String> outputs = takeSubstringOf(inputs, index);
I'll let you fill the blanks, which are very easy to implement with the methods offered by the String class, and loops.

You didn't answer #Steve about scenario of different length of numbers, I'll assume you want to compare numbers according to characters order. Also, I assume that the file always contains three strings in one line, saperated by one white space.
Given this, I hope this helps:
import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
public class FindDifference {
static File f = null;
ArrayList<String> inputs = new ArrayList<String>();
static String x1 = null;
static String x2 = null;
static String x3 = null;
private static ArrayList<String> getDifferenceList() {
ArrayList<String> ret = new ArrayList<String>();
if (x1 == null || x1.isEmpty()
|| x2 == null || x2.isEmpty()
|| x3 == null || x3.isEmpty()) {
ret.add(x1);
ret.add(x2);
ret.add(x3);
return ret;
}
int index = 0;
while (index < x1.length()
&& index < x2.length()
&& index < x3.length()) {
if (x1.charAt(index) != x2.charAt(index)
|| x1.charAt(index) != x3.charAt(index)) {
break;
}
index++;
}
ret.add(x1.substring(index, x1.length()));
ret.add(x2.substring(index, x2.length()));
ret.add(x3.substring(index, x3.length()));
return ret;
}
public static void main(String[] args) {
if (args.length > 0) {
f = new File(args[0]);
} else {
System.out.println("Please supply file path.");
return;
}
String line;
BufferedReader buffreader = null;
try {
buffreader = new BufferedReader(new FileReader(f));
while (( line = buffreader.readLine()) != null) {
String[] parts = line.split(" ");
x1 = (parts[0]);
x2 = (parts[1]);
x3 = (parts[2]);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
ArrayList<String> output = getDifferenceList();
try {
System.out.println(output.get(0));
System.out.println(output.get(1));
System.out.println(output.get(2));
} catch (Exception e) {
e.printStackTrace();
}
}
}
I made it SSCCE. You can use only getDifferenceList (supplying x1,x2,x3 as fields) in your code (delete "static" wherever needed).
EDIT
As I said, you can use getDifferenceList in any scope (including Android application). I copy this method separately, simply copy-paste it into any class you want, and call it passing the file path:
ArrayList<String> getDifferenceList(String filePath) {
File f = new File(filePath);
String line, x1 = null, x2= null, x3 = null;
BufferedReader buffreader = null;
try {
buffreader = new BufferedReader(new FileReader(f));
while (( line = buffreader.readLine()) != null) {
String[] parts = line.split(" ");
x1 = (parts[0]);
x2 = (parts[1]);
x3 = (parts[2]);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
ArrayList<String> ret = new ArrayList<String>();
if (x1 == null || x1.isEmpty()
|| x2 == null || x2.isEmpty()
|| x3 == null || x3.isEmpty()) {
ret.add(x1);
ret.add(x2);
ret.add(x3);
return ret;
}
int index = 0;
while (index < x1.length()
&& index < x2.length()
&& index < x3.length()) {
if (x1.charAt(index) != x2.charAt(index)
|| x1.charAt(index) != x3.charAt(index)) {
break;
}
index++;
}
ret.add(x1.substring(index, x1.length()));
ret.add(x2.substring(index, x2.length()));
ret.add(x3.substring(index, x3.length()));
return ret;
}