We Keep Coding

Java: Read from txt file and store each word only once in array + sorting

I am having a problem with my program. What i am supposed to do is:
find all words from some txt files
store each word in array only once
Then sort alphabetically
I dont know how to ensure that each word won't appear twice(or more) in my array.
For example, a sentence from one of my files: My cat is huge and my dog is lazy.
I want the words "my" and "is" to appear only once in my array, not twice.
As for the sorting, is there anything that i can use from Java ? I don't know.
Any help is appreciated!
Here is what i have done so far:
try {
File dir = new File("path of folder that contains my files")
for (File f : dir.listFiles()) {
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(f)));
String line = null;
while((line = br.readLine())!= null) {
String [] tokens = line.split(",\\s+|\\s*\\\"\\s*|\\s+|\\.\\s*|\\s*\\:\\s*");
}
}
}

Here is the modified code to have sorted unique words:
try {
TreeSet<String> uniqueSortedWords = new TreeSet<String>();
File dir = new File(
"words.txt");
BufferedReader br = new BufferedReader(new InputStreamReader(
new FileInputStream(dir)));
String line = null;
while ((line = br.readLine()) != null) {
String[] tokens = line
.split(",\\s+|\\s*\\\"\\s*|\\s+|\\.\\s*|\\s*\\:\\s*");
for(String token: tokens) {
uniqueSortedWords.add(token);
}
}
System.out.println(uniqueSortedWords);
//call uniqueSortedWords.toArray() to have output in an array
} catch (Exception e) {
e.printStackTrace();
}

ifI guess you are looking for a code something like this.
try {
ArrayList<String> list = new ArrayList<String>();
File dir = new File("path of folder that contains my files")
for (File f : dir.listFiles()) {
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(f)));
String line = null;
while((line = br.readLine())!= null) {
String [] tokens = line.split(",\\s+|\\s*\\\"\\s*|\\s+|\\.\\s*|\\s*\\:\\s*");
for(int i=0; i<tokens.length(); i++)
{ //Adding non-duplicates to arraylist
if (!list.contains(tokens[i])
{
list.add(tokens[i]);
}
}
}
Collections.Sort(list);
}
}
catch(Exception ex){}
Do not forget: import java.util.*; at the beginning of your code to use Collections.Sort();
EDIT
Even though contains is a built-in method you can directly use with ArrayLists, this is how such a method works in fact (just in case if you are curious):
public static boolean ifContains(ArrayList<String> list, String name) {
for (String item : list) {
if (item.getName().equals(name)) {
return true;
}
}
return false;
}
then to call it:
ifContains(list, tokens[i]))

You can use the combination of HashSet and TreeSet
Hashset:hashset allows null object.
TreeSet:treeset will not allow null object,treeset elements are sorted in ascending order by default.
Both HashSet and TreeSet does not hold duplicate elements.
try {
Set<String> list = new HashSet<>();
File f = new File("data.txt");
BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(f)));
String line = null;
while ((line = br.readLine()) != null) {
String[] tokens = line.split(",\\s+|\\s*\\\"\\s*|\\s+|\\.\\s*|\\s*\\:\\s*");// other alternative:line.split("[,;-!]")
for (String token : tokens) {
list.add(token);
}
}
// Add the list to treeSet;Elements in treeSet are sorted
// Note: words must have the same case either lowercase or uppercase
// for sorting to work correctly
TreeSet<String> sortedSet = new TreeSet<>();
sortedSet.addAll(list);
Iterator<String> ite = sortedSet.iterator();
while (ite.hasNext()) {
System.out.println(ite.next());
}
} catch (Exception e) {
e.printStackTrace();
}

Java FileReader: How to assign every line in the text file to a variable

I want every line in my textdoc to be assigned to a variable.
import java.io.*;
import static java.lang.System.*;
class readfile {
public static void main(String[] args) {
try {
FileReader fr = new FileReader("filename");
BufferedReader br = new Buffered(fr);
String str;
while ((str = br.readLine()) != null) {}
br.close();
} catch (IOException e) {
out.println("file not found");
}
}
}

I would suggest you create a List and store every line in a list like below:
String str;
List<String> fileText = ....;
while ((str = br.readLine()) != null) {
fileText.add(str);
}

A Java 8 solution for creating a List of lines
Path path = Paths.get("filename");
List<String> lines = Files.lines(path).collect(Collectors.toList());

why do you want to add each line to a separate variable? It is better to add the lines to a list. Then you can access any line as you want.
In JDK 6 or below
List<String> lines = new ArrayList<String>();
while(reader.ready())
lines.add(reader.readLine());
In JDK 7 or above
List<String> lines = Files.readAllLines(Paths.get(fileName),
Charset.defaultCharset());

I would do
List<string> allText= new List<String>();
While(str.hasNextLine){
allText.add(str.nextLine);
}

How to store text from a certain column in an arraylist?

I would like to store only the first column that is contained in the .txt file.
hello28 23232
hello27 23232
hello25 12321
This is the code I have so far, however at the moment it stores every line in the file; how can I make it so that only the first column is stored (The one which contains the user names of the users)?
public static boolean checkUserExists(String userName){
String line = "";
ArrayList <String> userNames = new ArrayList <String>();
try{
FileReader fr = new FileReader("investments.txt");
BufferedReader br = new BufferedReader(fr);
while((line = br.readLine()) != null) {
userNames.add(line);
}
}
catch(IOException e){
System.out.println("File not found!");
}
if (userNames.contains(userName)){
return false;
}
else{
return true;
}
}

All you need to do is just to split each line using whitespace as a delimiter and keep the first token, and repeat that for every line:
This can be achieved using the following line of code which uses the split function (see more info here http://docs.oracle.com/javase/7/docs/api/java/lang/String.html#split(java.lang.String))
line.split("\\s+");
Then, the zero-th (0) element contains the first column, as you wish to do
There you go a fully working class:
import java.io.IOException;
import java.io.BufferedReader;
import java.io.FileReader;
import java.util.ArrayList;
class white {
public static void main(String[] args) {
String line = "";
String username = "";
ArrayList <String> userNames = new ArrayList <String>();
try{
FileReader fr = new FileReader("investments.txt");
BufferedReader br = new BufferedReader(fr);
while((line = br.readLine()) != null) {
line.split("\\s+");
userNames.add(line.split("\\s+")[0]);
System.out.println(line.split("\\s+")[0]);
}
}
catch(IOException e){
System.out.println("File not found!");
}
}
}
OUTPUT:
hello28
hello27
hello25

You can extract the part of the line preceding the first space:
userNames.add(line.substring(0, line.indexOf(' ') ));

Java Remove Duplicates from file search for String Array [0]

I have a long text file.
Now I will remove duplicates from the file. The problem is that the search parameter is the first word in the list, split by ":"
For example:
The file lines:
11234567:229283:29833204:2394803
11234567:4577546765:655776:564456456
43523:455543:54335434:53445
11234567:43455:544354:5443
Now I will have this here:
11234567:229283:29833204:2394803
43523:455543:54335434:53445
I need to get the first line from the duplicates, other will be ignored.
I tried this:
Set<String> lines11;
try (BufferedReader reader11 = new BufferedReader(new FileReader("test.txt"))) {
lines11 = new HashSet<>(10000); // maybe should be bigger
String line11;
while ((line11 = reader11.readLine()) != null) {
lines11.add(line11);
}
} // maybe should be bigger
try (BufferedWriter writer11 = new BufferedWriter(new FileWriter("test.txt"))) {
for (String unique : lines11) {
writer11.write(unique);
writer11.newLine();
}
}
That is working, but it removes only when the complete line is duplicated.
How can I change it so that it looks for the first word in every line and checks for duplicates here; when no duplicate is found, save the complete line; if duplicate then ignore the line?

You need to maintain a Set<String> that holds only the first word of each line.
List<String> lines11;
Set<String> dups;
try (BufferedReader reader11 = new BufferedReader(new FileReader("test.txt"))) {
lines11 = new ArrayList<>();
dups = new HashSet<>();
String line11;
while ((line11 = reader11.readLine()) != null) {
String first = line11.split(":")[0]; // assuming your separator is :
if (!dups.contains(first)) {
lines11.add(line11);
dups.add(first);
}
}
}
try (BufferedWriter writer11 = new BufferedWriter(new FileWriter("test.txt"))) {
for (String unique : lines11) {
writer11.write(unique);
writer11.newLine();
}
}

i will write the section about adding to list
use HashMap
String tmp[] = null;
HashMap<String, String> lines = new HashMap<String, String>();
String line11 = "";
while ((line11 = reader11.readLine()) != null) {
tmp = line11.split(":");
if(!lines.containsKey(tmp[0])){
lines.put(tmp[0], line11);
}
}
so the loop will add only uinuque lines , using first word as key

You can add the data in list and take one more set in which you will add first word in that set and try add every time first of new line if it is in set, then it will not be added and return false. On that basis you can add data in list or directly in you new bufferreader.
List<String> lines11;
Set<String> uniqueRecords;
try (BufferedReader reader11 = new BufferedReader(new FileReader("test.txt"))) {
lines11 = new ArrayList<>(); // no need to give size it will increase dynamically
uniqueRecords = new HashSet<>();
String line11;
while ((line11 = reader11.readLine()) != null) {
String firstWord = line11.substring(0, firstWord.firstIndexOf(" "));
if(uniqueRecords.add(firstWord )){
lines11.add(line11);
}
}
} // maybe should be bigger
try (BufferedWriter writer11 = new BufferedWriter(new FileWriter("test.txt"))) {
for (String unique : lines11) {
writer11.write(unique);
writer11.newLine();
}
}

Compare values in two files

I have two files Which should contain the same values between Substring 0 and 10 though not in order. I have Managed to Outprint the values in each file but I need to Know how to Report say id the Value is in the first File and Notin the second file and vice versa. The files are in these formats.
6436346346....Other details
9348734873....Other details
9349839829....Other details
second file
8484545487....Other details
9348734873....Other details
9349839829....Other details
The first record in the first file does not appear in the second file and the first record in the second file does not appear in the first file. I need to be able to report this mismatch in this format:
Record 6436346346 is in the firstfile and not in the secondfile.
Record 8484545487 is in the secondfile and not in the firstfile.
Here is the code I currently have that gives me the required Output from the two files to compare.
package compare.numbers;
import java.io.*;
/**
*
* #author implvcb
*/
public class CompareNumbers {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
// TODO code application logic here
File f = new File("C:/Analysis/");
String line;
String line1;
try {
String firstfile = "C:/Analysis/RL001.TXT";
FileInputStream fs = new FileInputStream(firstfile);
BufferedReader br = new BufferedReader(new InputStreamReader(fs));
while ((line = br.readLine()) != null) {
String account = line.substring(0, 10);
System.out.println(account);
}
String secondfile = "C:/Analysis/RL003.TXT";
FileInputStream fs1 = new FileInputStream(secondfile);
BufferedReader br1 = new BufferedReader(new InputStreamReader(fs1));
while ((line1 = br1.readLine()) != null) {
String account1 = line1.substring(0, 10);
System.out.println(account1);
}
} catch (Exception e) {
e.fillInStackTrace();
}
}
}
Please help on how I can effectively achieve this.
I think I needed to say that am new to java and may not grab the ideas that easily but Am trying.

Here is the sample code to do that:
public static void eliminateCommon(String file1, String file2) throws IOException
{
List<String> lines1 = readLines(file1);
List<String> lines2 = readLines(file2);
Iterator<String> linesItr = lines1.iterator();
while (linesItr.hasNext()) {
String checkLine = linesItr.next();
if (lines2.contains(checkLine)) {
linesItr.remove();
lines2.remove(checkLine);
}
}
//now lines1 will contain string that are not present in lines2
//now lines2 will contain string that are not present in lines1
System.out.println(lines1);
System.out.println(lines2);
}
public static List<String> readLines(String fileName) throws IOException
{
List<String> lines = new ArrayList<String>();
FileInputStream fs = new FileInputStream(fileName);
BufferedReader br = new BufferedReader(new InputStreamReader(fs));
String line = null;
while ((line = br.readLine()) != null) {
String account = line.substring(0, 10);
lines.add(account);
}
return lines;
}

Perhaps you are looking for something like this
Set<String> set1 = new HashSet<>(FileUtils.readLines(new File("C:/Analysis/RL001.TXT")));
Set<String> set2 = new HashSet<>(FileUtils.readLines(new File("C:/Analysis/RL003.TXT")));
Set<String> onlyInSet1 = new HashSet<>(set1);
onlyInSet1.removeAll(set2);
Set<String> onlyInSet2 = new HashSet<>(set2);
onlyInSet2.removeAll(set1);

If you guarantee that the files will always be the same format, and each readLine() function is going to return a different number, why not have an array of strings, rather than a single string. You can then compare the outcome with greater ease.

Ok, first I would save the two sets of strings in to collections
Set<String> s1 = new HashSet<String>(), s2 = new HashSet<String>();
//...
while ((line = br.readLine()) != null) {
//...
s1.add(line);
}
Then you can compare those sets and find elements that do not appear in both sets. You can find some ideas on how to do that here.
If you need to know the line number as well, you could just create a String wrapper:
class Element {
public String str;
public int lineNr;
public boolean equals(Element compElement) {
return compElement.str.equals(str);
}
}
Then you can just use Set<Element> instead.

Open two Scanners, and :
final TreeSet<Integer> ts1 = new TreeSet<Integer>();
final TreeSet<Integer> ts2 = new TreeSet<Integer>();
while (scan1.hasNextLine() && scan2.hasNexLine) {
ts1.add(Integer.valueOf(scan1.nextLigne().subString(0,10));
ts1.add(Integer.valueOf(scan1.nextLigne().subString(0,10));
}
You can now compare ordered results of the two trees
EDIT
Modified with TreeSet

Put values from each file to two separate HashSets accordingly.
Iterate over one of the HashSets and check whether each value exists in the other HashSet. Report if not.
Iterate over other HashSet and do same thing for this.