We Keep Coding

How to count unique words in a text file?

I have implemented code to count number of:
- chars
- words
- lines
- bytes
in text file.
But how to count dictionary size: number of different words used in this file?
Also, how to implement iterator which can iterate over only letters? (Ignore whitespaces)
public class wc {
public static void main(String[] args) throws IOException {
//counters
int charsCount = 0;
int wordsCount = 0;
int linesCount = 0;
Scanner in = null;
File file = new File("Sample.txt");
try(Scanner scanner = new Scanner(new BufferedReader(new FileReader(file)))){
while (scanner.hasNextLine()) {
String tmpStr = scanner.nextLine();
if (!tmpStr.equalsIgnoreCase("")) {
String replaceAll = tmpStr.replaceAll("\\s+", "");
charsCount += replaceAll.length();
wordsCount += tmpStr.split("\\s+").length;
}
++linesCount;
}
System.out.println("# of chars: " + charsCount);
System.out.println("# of words: " + wordsCount);
System.out.println("# of lines: " + linesCount);
System.out.println("# of bytes: " + file.length());
}
}
}

To get unique words and their counts:
1. Split your obtained line from file into a string array
2. Store the contents of this string array in a Hashset
3. Repeat steps 1 and 2 till end of file
4. Get unique words and their count from the Hashset
I prefer posting logic and pseudo code as it will help OP to learn something by solving posted problem.

Example of how my code works:
File with words "aa bb cc cc aa aa" has 3 unique words.
First, turn words into a string with each word separated by "-".
String: "aa-bb-cc-cc-aa-aa-"
Get the first word: "aa", set the UniqueWordCount = 1, and then replace "aa-" with "".
New String: "bb-cc-cc-"
Get the first word: "bb", set the UniqueWordCount = 2, and then replace "bb-" with "".
New String: "cc-cc-"
Get the first word: "cc", set the UniqueWordCount = 3, and then replace "cc-" with "".
New String: "", you stop when the string is empty.
private static int getUniqueWordCountInFile(File file) throws FileNotFoundException {
String fileWordsAsString = getFileWords(file);
int uniqueWordCount = 0;
int i = 0;
while (!(fileWordsAsString.isEmpty()) && !(fileWordsAsString.isBlank())) {
if (Character.toString(fileWordsAsString.charAt(i)).equals(" ")) {
fileWordsAsString = fileWordsAsString.replaceAll(fileWordsAsString.substring(0, i+1),"");
i = 0;
uniqueWordCount++;
} else {
i++;
}
}
return uniqueWordCount;
}
private static String getFileWords(File file) throws FileNotFoundException {
String toReturn = "";
try (Scanner fileReader = new Scanner(file)) {
while (fileReader.hasNext()) {
if (fileReader.hasNextInt()) {
fileReader.nextInt();
} else {
toReturn += fileReader.next() + " ";
}
}
}
return toReturn;
}
If you want to use my code just pass getUniqueWordCountInFile() the file that has the words for which you want to count the unique words.

hey #JeyKey you can use HashMap. Here I using Iterator too. You can check out this code.
public class CountUniqueWords {
public static void main(String args[]) throws FileNotFoundException {
File f = new File("File Name");
ArrayList arr=new ArrayList();
HashMap<String, Integer> listOfWords = new HashMap<String, Integer>();
Scanner in = new Scanner(f);
int i=0;
while(in.hasNext())
{
String s=in.next();
//System.out.println(s);
arr.add(s);
}
Iterator itr=arr.iterator();
while(itr.hasNext())
{i++;
listOfWords.put((String) itr.next(), i);
//System.out.println(listOfWords); //for Printing the words
}
Set<Object> uniqueValues = new HashSet<Object>(listOfWords.values());
System.out.println("The number of unique words: "+uniqueValues.size());
}
}

Reading file and store in vector

I'm reading from the file:
name1 wordx wordy passw1
name2 wordx wordy passw2
name3 wordx wordy passw3
name (i) wordx wordy PASSW (i)
x
x word
x words
words
x
words
At the moment I can print line by line:
Line 1: name1 wordx wordy passw1
Line 2: name2 wordx wordy passw2
I plan to have access to:
users [0] = name1
users [1] = name2
users [2] = name3
..
passws [0] = passw1
passws [1] = passw2
passws [2] = passw3
..
My code is:
public static void main(String args[]) throws FileNotFoundException, IOException {
ArrayList<String> list = new ArrayList<String>();
Scanner inFile = null;
try {
inFile = new Scanner(new File("C:\\file.txt"));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
while (inFile.hasNextLine()) {
list.add(inFile.nextLine()+",");
}
String listString = "";
for (String s : list) {
listString += s + "\t";
}
String[] parts = listString.split(",");
System.out.println("Line1: "+ parts[0]);
}
How do I get the following output:
User is name1 and password is passw1
User is name32 and password is passw32
Thanks in advance.

Something like this will do:
public static void main(String args[]) throws FileNotFoundException, IOException {
ArrayList<String> list = new ArrayList<String>();
Scanner inFile = null;
try {
inFile = new Scanner(new File("C:\\file.txt"));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
while (inFile.hasNextLine()) {
list.add(inFile.nextLine());
}
int line = 0;
String[] parts = list.get(line).split(" ");
String username = parts[0];
String pass = parts[3];
System.out.println("Line" + (line + 1) + ": " + "User is " + username +" and password is " + pass);
}
EDIT: if you want to iterate through all lines just put last lines in a loop:
for (int line = 0; line < list.size(); line++) {
String[] parts = list.get(line).split(" ");
String username = parts[0];
String pass = parts[3];
System.out.println("Line" + (line + 1) + ": " + "User is " + username +" and password is " + pass);
}

First thing to do is, to add this loop to the end of your code :
for(int i = 0; i <= parts.length(); i++){
System.out.println("parts["+i+"] :" + parts[i] );
}
that will simply show the result of the split using ,.
Then adapt your code, you may want to use another regex to split() your lines, for instance a space.
String[] parts = listString.split(" ");
for documentation about split() method check this.

If you want to get that output then this should do the trick:
public static void main(String args[]) throws FileNotFoundException, IOException {
Scanner inFile = null;
try {
inFile = new Scanner(new File("F:\\file.txt"));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
Map<String, String> userAndPassMap = new LinkedHashMap<>();
while (inFile.hasNextLine()) {
String nextLine = inFile.nextLine();
String[] userAndPass = nextLine.split(" ");
userAndPassMap.put(userAndPass[0], userAndPass[1]);
}
for (Map.Entry<String, String> entry : userAndPassMap.entrySet()) {
System.out.println("User is:" + entry.getKey() + " and password is:" + entry.getValue());
}
}
By storing in a map you are linking directly each username with its password. If you need to save them into separate arrays then you can do this in the while loop instead:
List<String> users = new LinkedList<>(),passwords = new LinkedList<>();
while (inFile.hasNextLine()) {
String nextLine = inFile.nextLine();
String[] userAndPass = nextLine.split(" ");
users.add(userAndPass[0]);
passwords.add(userAndPass[1]);
}
and later transform them to arrays
users.toArray()

I recommend you use a java.util.Map, a standard API which allows you to store objects and read each one of them by a key. (In your case, string objects indexed by string keys). Example:
Let's assume this empty map:
Map<String, String> map=new HashMap<String,String>();
If you store this:
map.put("month", "january");
map.put("day", "sunday");
You can expect that map.get("month") will return "january", map.get("day") will return "sunday", and map.get(any-other-string) will return null.
Back to your case: First, you must create and populate the map:
private Map<String, String> toMap(Scanner scanner)
{
Map<String, String> map=new HashMap<String, String>();
while (scanner.hasNextLine())
{
String line=scanner.nextLine();
String[] parts=line.split(" ");
// Validation: Process only lines with 4 tokens or more:
if (parts.length>=4)
{
map.put(parts[0], parts[parts.length-1]);
}
}
return map;
}
And then, to read the map:
private void listMap(Map<String,String> map)
{
for (String name : map.keySet())
{
String pass=map.get(name);
System.out.println(...);
}
}
You must include both in your class and call them from the main method.

If you need arbitraray indexing of the read lines, use ArrayList:
First, define a javabean User:
public class User
{
private String name;
private String password;
// ... add full constructor, getters and setters.
}
And then, you must create and populate the list:
private ArrayList<User> toList(Scanner scanner)
{
List<User> list=new ArrayList<User>();
while (scanner.hasNextLine())
{
String line=scanner.nextLine();
String[] parts=line.split(" ");
// Validation: Process only lines with 4 tokens or more:
if (parts.length>=4)
{
list.add(new User(parts[0], parts[parts.length-1]));
}
}
return list;
}

Scanner from file doesn't seem to be reading file

I'm doing a Phone Directory project and we have to read from a directory file telnos.txt
I'm using a Scanner to load the data from the file telnos.txt, using a loadData method from a previous question I asked here on StackOverflow.
I noticed attempts to find a user always returned Not Found, so I added a few System.out.printlns in the methods to help me see what was going on. It looks like the scanner isn't reading anything from the file. Weirdly, it is printing the name of the file as what should be the first line read, which makes me think I've missed something very very simple here.
Console
run:
telnos.txt
null
loadData tested successfully
Please enter a name to look up: John
-1
Not found
BUILD SUCCESSFUL (total time: 6 seconds)
ArrayPhoneDirectory.java
import java.util.*;
import java.io.*;
public class ArrayPhoneDirectory implements PhoneDirectory {
private static final int INIT_CAPACITY = 100;
private int capacity = INIT_CAPACITY;
// holds telno of directory entries
private int size = 0;
// Array to contain directory entries
private DirectoryEntry[] theDirectory = new DirectoryEntry[capacity];
// Holds name of data file
private final String sourceName = "telnos.txt";
File telnos = new File(sourceName);
// Flag to indicate whether directory was modified since it was last loaded or saved
private boolean modified = false;
// add method stubs as specified in interface to compile
public void loadData(String sourceName) {
Scanner read = new Scanner("telnos.txt").useDelimiter("\\Z");
int i = 1;
String name = null;
String telno = null;
while (read.hasNextLine()) {
if (i % 2 != 0)
name = read.nextLine();
else
telno = read.nextLine();
add(name, telno);
i++;
}
}
public String lookUpEntry(String name) {
int i = find(name);
String a = null;
if (i >= 0) {
a = name + (" is at position " + i + " in the directory");
} else {
a = ("Not found");
}
return a;
}
public String addChangeEntry(String name, String telno) {
for (DirectoryEntry i : theDirectory) {
if (i.getName().equals(name)) {
i.setNumber(telno);
} else {
add(name, telno);
}
}
return null;
}
public String removeEntry(String name) {
for (DirectoryEntry i : theDirectory) {
if (i.getName().equals(name)) {
i.setName(null);
i.setNumber(null);
}
}
return null;
}
public void save() {
PrintWriter writer = null;
// writer = new PrintWriter(FileWriter(sourceName));
}
public String format() {
String a;
a = null;
for (DirectoryEntry i : theDirectory) {
String b;
b = i.getName() + "/n";
String c;
c = i.getNumber() + "/n";
a = a + b + c;
}
return a;
}
// add private methods
// Adds a new entry with the given name and telno to the array of
// directory entries
private void add(String name, String telno) {
System.out.println(name);
System.out.println(telno);
theDirectory[size] = new DirectoryEntry(name, telno);
size = size + 1;
}
// Searches the array of directory entries for a specific name
private int find(String name) {
int result = -1;
for (int count = 0; count < size; count++) {
if (theDirectory[count].getName().equals(name)) {
result = count;
}
System.out.println(result);
}
return result;
}
// Creates a new array of directory entries with twice the capacity
// of the previous one
private void reallocate() {
capacity = capacity * 2;
DirectoryEntry[] newDirectory = new DirectoryEntry[capacity];
System.arraycopy(theDirectory, 0, newDirectory,
0, theDirectory.length);
theDirectory = newDirectory;
}
}
ArrayPhoneDirectoryTester.java
import java.util.Scanner;
public class ArrayPhoneDirectoryTester {
public static void main(String[] args) {
//create a new ArrayPhoneDirectory
PhoneDirectory newTest = new ArrayPhoneDirectory();
newTest.loadData("telnos.txt");
System.out.println("loadData tested successfully");
System.out.print("Please enter a name to look up: ");
Scanner in = new Scanner(System.in);
String name = in.next();
String entryNo = newTest.lookUpEntry(name);
System.out.println(entryNo);
}
}
telnos.txt
John
123
Bill
23
Hello
23455
Frank
12345
Dkddd
31231

In your code:
Scanner read = new Scanner("telnos.txt");
Is not going to load file 'telnos.txt'. It is instead going to create a Scanner object that scans the String "telnos.txt".
To make the Scanner understand that it has to scan a file you have to either:
Scanner read = new Scanner(new File("telnos.txt"));
or create a File object and pass its path to the Scanner constructor.
In case you are getting "File not found" errors you need to check the current working directory. You could run the following lines and see if you are indeed in the right directory in which the file is:
String workingDir = System.getProperty("user.dir");
System.out.println("Current working directory : " + workingDir);
You need to also catch the FileNotFoundException in the function as follows:
public void loadData(String sourceName) {
try {
Scanner read = new Scanner(new File("telnos.txt")).useDelimiter("\\Z");
int i = 1;
String name = null;
String telno = null;
while (read.hasNextLine()) {
if (i % 2 != 0)
name = read.nextLine();
else {
telno = read.nextLine();
add(name, telno);
}
i++;
}
}catch(FileNotFoundException ex) {
System.out.println("File not found:"+ex.getMessage);
}
}

You are actually parsing the filename not the actual file contents.
Instead of:
new Scanner("telnos.txt")
you need
new Scanner( new File( "telnos.txt" ) )
http://docs.oracle.com/javase/7/docs/api/java/util/Scanner.html

Input file with names and popularity Java

So I have a file that has names along with 11 popularity ranks which looks like this. <--- (this is a link) I am a bit confused on what I am suppose to do with this next part that I have for my assignment. Generally I have a name app that looks like this:
public class Name{
private String givenName;
private int[] ranks = new int[11];
public Name(String name, int[] popularityRanks){
givenName = name;
for (int i = 0; i < 11; i++){
ranks[i] = popularityRanks[i];
}
}
public String getName(){
return givenName;
}
public int getPop(int decade){
if (decade >= 1 && decade <= 11){
return ranks[decade];
}
else{
return -1;
}
}
public String getHistoLine(int decade){
String histoLine = ranks[decade] + ": ";
return histoLine;
}
public String getHistogram(){
String histogram = "";
for (int i = 0; i < 11; i++){
histogram += ranks[i] + ": " + this.getHistoLine(i)
+ "\n";
}
return histogram;
}
}
It is not finished for the getHistoLine but that doesn't have anything to do with what I am trying to do. Generally I want to take these names in from the file and create an array of list.
How he describes it:
Create the array in main, pass it to the readNamesFile method and let that method fill it with Name objects
Test this, by printing out various names and their popularity rankings
For example, if main named the array, list, then upon return from the readNamesFile method do something like:
System.out.println( list[0].getName() + list[0].getPop(1) );
This is what my main looks like:
import java.util.Scanner;
import java.io.File;
import java.io.FileNotFoundException;
public class NameApp{
public static void main(String[] args){
Name list[] = new Name()
}
private static void loadFile(){
Scanner inputStream = null;
String fileName = "names.txt";
try {
inputStream = new Scanner (new File(fileName));
}
catch (FileNotFoundException e){
System.out.println("Error opening file named: " + fileName);
System.out.println("Exiting...");
}
while (inputStream.hasNext()){
}
}
}
I am just a bit confused how I can take the name have it send to the Name object list[] and then take the popularity ranks and send it to the Name object list[]. So when I call
list[0].getName()
it will just call the name for one of the lines... Sorry I am a bit new to the java language. Thanks in advance

You need to create a Name list correctly. I would use a List since you don't know how many names there will be;
public static void main(String[] args){
List<Name> list = new ArrayList<Name>();
loadFile();
System.out.println(list.get(0).getPop());
}
private static void loadFile(){
Scanner inputStream = null;
String fileName = "names.txt";
try {
inputStream = new Scanner (new File(fileName));
}
catch (FileNotFoundException e){
System.out.println("Error opening file named: " + fileName);
System.out.println("Exiting...");
}
while (inputStream.hasNext()){
// givenName = something
// ranks = something;
list.add(new Name(givenName, ranks);
}
}
Assuming each line is something like this (from your deleted comment)
A 1 234 22 43 543 32 344 32 43
Your while loop can be something like this
while (inputStream.hasNextLIne()){
String line = inputStream.nextLine();
String tokens = line.split("\\s+");
String givenName = tokens[0];
int[] numList = new int[tokens.lenth - 1];
for (int i = 1; i < tokens.length; i++){
numList[i - 1] = Integer.parseInt(tokens[i].trim());
}
list.add(new Name(givenName, numList);
}

Java Input from formatted file

I'm trying to getting input from a file which has various different lines.
e.g. the format is Book title, Author:Borrower second name First Name:Borrower state
here's some example lines.
The Lord of the Rings, JRR Tolkien:McInnes Elizabeth:13 11 10
Crime And Punishment, Fyodor Dostoyevsky
The Clan Of The Cave Bear, Jean M Auel
The God Of Small Things, Arundhati Roy:Robins Joshua:20 11 10
So I tried to use useDelimiter after setting up a scanner, but since some line are shorter I don't no quite what to do.

Here is a solution based on regular expressions:
import java.io.*;
import java.util.regex.*;
public class Test {
public static void main(String[] args) throws IOException {
BufferedReader br = new BufferedReader(new FileReader("data.txt"));
Pattern p = Pattern.compile("(.+?),(.+?)(?::(.+?):(\\d+ \\d+ \\d+))?");
String line;
while (null != (line = br.readLine())) {
Matcher m = p.matcher(line);
if (m.matches()) {
String title = m.group(1);
String author = m.group(2);
String borrower = m.group(3);
String data = m.group(4);
System.out.println("Title: " + title);
System.out.println("Author: " + author);
if (borrower != null) {
System.out.println(" Borrower: " + borrower);
System.out.println(" Data: " + data);
}
}
System.out.println();
}
br.close();
}
}
Given your sample input, it prints:
Title: The Lord of the Rings
Author: JRR Tolkien
Borrower: McInnes Elizabeth
Data: 13 11 10
Title: Crime And Punishment
Author: Fyodor Dostoyevsky
Title: The Clan Of The Cave Bear
Author: Jean M Auel
Title: The God Of Small Things
Author: Arundhati Roy
Borrower: Robins Joshua
Data: 20 11 10

Read the file line by line, use the [^,:] regular expression to match data in each line (sequential find will bring title, author, and borrower, state, if any).

You could use .split()
try {
FileInputStream fstream = new FileInputStream("input.txt");
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String line;
String author;
String title;
String borrower;
String date;
while ((line = br.readLine()) != null) {
(author,title) = line.split(",");
if (line.contains(":")
(title,borrower,date) = title.split(":");
/*** Do what you need to do with the values here ***/
}
in.close();
} catch (Exception e) {
e.printStackTrace();
}

Why always for such trivial tasks such oversized tools like regular expressions are suggested? Why not simply use the good old line.indexOf() or line.lastIndexOf() methods?

I would split each line (using String.split) passing in the colon as the delimiter. Then use lastIndexOf(',') on the first element returned by split in order to separate the author from the book:
public class ReadCrappyInput {
public static List<String> testData() {
List<String> lines = new ArrayList<String>();
lines.add("The Lord of the Rings, JRR Tolkien:McInnes Elizabeth:13 11 10");
lines.add("Crime And Punishment, Fyodor Dostoyevsky");
lines.add("The Clan Of The Cave Bear, Jean M Auel");
lines.add("The God Of Small Things, Arundhati Roy:Robins Joshua:20 11 10");
return lines;
}
public Map<String, String> readLine(String line) {
String[] parts = line.split(":");
int endOfTitleIndex = parts[0].lastIndexOf(',');
Map<String, String> map = new HashMap<String, String>();
map.put("title", parts[0].substring(0, endOfTitleIndex));
map.put("author", parts[0].substring(endOfTitleIndex + 1).trim());
if (parts.length > 1) {
map.put("borrower", parts[1]);
}
if (parts.length > 2) {
map.put("data", parts[2]);
}
return map;
}
public static void main(String[] args) {
ReadCrappyInput r = new ReadCrappyInput();
for (String s : testData()) {
System.out.println(r.readLine(s));
}
}
}