How can i split the user's input with tokens? - java

i need your advise in order to do something ... i want to take the user's input line as i already do in my program with scanner ... but i want to split each command(word) to tokens .... i do not know how to do that , till now i was playing with substring but if the users for example press twice the space-bar button everything is wrong !!!!!
For example :
Please insert a command : I am 20 years old
with substring or .split(" ") , it runs but think about having :
Please insert a command : I am 20 years old
That is why i need your advice .... The question is how can i split the user's input with tokens.

Well, you need to normalize you string line before splitting it to tokens. A simplest way is to remove repeated whitespace characters:
line = line.replaceAll("\\s+", " ");
(this will also replace all tabs to a single " ").

Use the StringTokenizer class. From the API :
"[It] allows an application to break a string into tokens... A
StringTokenizer object internally maintains a current position within
the string to be tokenized."
The following sample code from the API will give you an idea:
StringTokenizer st = new StringTokenizer("this is a test");
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
}
It produces the following result:
this
is
a
test

Okay, so the code that I have made is going to ask you for to insert a command, it is then going to take your command and then split it around any number of spaces (due to the Regex). It then saves these words, numbers, speech marks etc. as tokens. You can then manipulate each word in the for loop.
import java.util.Scanner;
public class CommandReaderProgram{
public static void main(String[] args){
Scanner userInput = new Scanner(System.in);
System.out.println("Please insert a command:");
String temp = userInput.nextLine();
while(temp != null){
String [] tokens = temp.split("\\s+");
for(String word : tokens){
System.out.print(word);
System.out.print(" ");
}
break;
}
System.out.print("\n");
}
}
To test that each word, number and speech mark has actually been saved as a token, change the character in the System.out.print(" "); code to anything. e.g System.out.print(""); or
System.out.print("abcdefg"); and it will put this data between each token, to prove that the tokens are indeed separate.
Unfortunately I am unable to call the token array outside of the for loop at the moment, but will let you know when I figure it out.
I'd like to hear what type of program you are trying to make as I think we are both trying to make something very similar.
Hope this is what you are looking for.
Regards.

Related

How do I make a program insert a new line every time it finds a space (or whitespace) within a string (in java)?

I'm a first year IT student and we recently had an activity that asks the programmer to create a program that will accept a sentence or a phrase, then detect if there is a space in that string and add a newline in between words that have a space after it.
For clarification, here's what the prompt should look like:
Enter a sentence or phrase: I am a student
then the output would be:
I
am
a
student
I tried to actually finish the code, but I was hit by a roadblock and got stuck. Here's my attempt though:
import java.util.*;
public class NumTwo{
public static void main (String[] args){
Scanner in = new Scanner(System.in);
String phr;
System.out.print("Enter a phrase: ");
phr = in.nextLine();
if(phr.contains(" ")){
System.out.print(phr + "\n");
}
else{
}
}
}
any comments to what i might have done wrong will be very much appreciated
EDIT:
I tried using sir Christoph Dahlen's solution which was to use String.replace, and it worked! Thank you very much.
You are currently checking weither the input line contains any spaces at all, but you have to react to every space instead.
One way would be to split phr by spaces and concatenating it with newlines.
String result = String.join(System.lineSeparator(), phr.split(" "))

How can I make it so my program, for each word read in a string, also reads the following word?

I want my Java program to do the following thing:
Whenever it reads a file like the following
Bob went to the store to buy apples.
To read each word in the string (delimited by only a single space character) and to also read the next word, but without "moving" the main reader as well. So it would do something like this:
for word in string{
print word + nextWord;
}
And its output would be
Bob went
went to
to the
the store
store to
to buy
buy apples.
Edit: IMPORTANT! I don't want to read the whole file and load it into memory. I want this operation to happen DIRECTLY on the file. Imagine I am dealing with something huge, like a whole book, or more.
No. Scanner doesn't let you peek at future input.
However, it's trivial to code:
Scanner s = new Scanner(new FileInputStream("myfile.txt");
String previous = s.next();
while (s.hasNext()) {
String next = s.next();
System.out.println(previous + " " + next);
previous = next;
}

Getting a NoSuchElementException 33

I don't know why I am getting this error. What should I do to fix it? Any help really appreciated. I excluded code that does not affect the error. Line 33 is String string1 = token.nextToken();
import java.net.*;
import java.io.*;
import java.util.*;
import java.util.Scanner;
public class Exec {
public static void main(String[ ] args){
Scanner input = new Scanner( System.in );
String user_input = " ";
//user gets asked to enter input here...
//...
//tokenize input into 2 strings and method code
user_input = input.next( );
StringTokenizer token = new StringTokenizer(user_input);
String string0 = token.nextToken();
String string1 = token.nextToken();
code = Integer.parseInt(token.nextToken());
You've run out of tokens. You should print out your String before tokenizing it as it is likely not be what you think it is. In other words what you want to do now is some debugging.
Note 1: always check first if more tokens exist with hasMoreTokens() before taking a token with nextToken().
Note 2: most of us avoid use of StringTokenizer and instead use either a Scanner or String#split(...).
You can use hasMoreTokens() method while traversing using StringTokenizer to check that tokens are actually present.This method call returns 'true' if and only if there is at least one token in the string after the current position;
StringTokenizer token = new StringTokenizer(user_input);
// checking tokens
while (token.hasMoreTokens()){
System.out.println("Next token : " + token.nextToken());
}
Also,in your code you are calling nextToken 3 times,so 3 tokens will be traversed.So, if your input string has less than 3 tokens it would fail with the exception you are getting.
It looks like you want to convert one of the tokens to integer in the call
code = Integer.parseInt(token.nextToken());
do you want to say "code = Integer.parseInt(string1)" OR "code = Integer.parseInt(string2)" instead?
Because when you say token.nextToken in integer.parseInt it would again look for next token i.e. 3rd token in this case.hope this helps.
It looks like you don't have as many tokens in user_input as you think you do. By default,
Scanner#next() reads up to the next whitespace character and StringTokenizer uses a smaller subset of whitespace characters as the delimiter, as stated in the API.
Maybe you should read user input using Scanner#nextLine() instead.

Count words in a String that is NOT in a string array without using split method

I need to count the words in a String. For many of you that seems pretty simple but from what I've read in similar questions people are saying to use arrays but I'd rather not. It complicates my program more than it helps as my string is coming from an input file and the program cannot be hardwired to a specific file.
I have this so far:
while(input.hasNext())
{
String sentences = input.nextLine();
int countWords;
char c = " ";
for (countWords = 0; countWords < sentences.length(); countWords++)
{
if (input.hasNext(c))
countWords++;
}
System.out.println(sentences);
System.out.println(countWords);
}
The problem is that what I have here ends up counting the amount of characters in the string. I thought it would count char c as a delimiter. I've also tried using String c instead with input.hasNext but the compiler tells me:
Program04.java:39: incompatible types
found : java.lang.String[]
required: java.lang.String
String token = sentences.split(delim);
I've since deleted the .split method from the program.
How do I delimit (is that the right word?) without using a String array with a scanned in file?
Don't use the Scanner (input) for more than one thing. You're using it to read lines from a file, and also trying to use it to count words in those lines. Use a second Scanner to process the line itself, or use a different method.
The problem is that the scanner consumes its buffer as it reads it. input.nextLine() returns sentences, but after that it no longer has them. Calling input.hasNext() on it gives you information about the characters after sentences.
The simplest way to count the words in sentences is to do:
int wordCount = sentences.split(" ").length;
Using Scanner, you can do:
Scanner scanner = new Scanner(sentences);
while(scanner.hasNext())
{
scanner.next();
wordCount++;
}
Or use a for loop for best performance (as mentioned by BlackPanther).
Another tip I'd give you is how to better name your variables. countWords should be wordCount. "Count words" is a command, a verb, while a variable should be a noun. sentences should simply be line, unless you know both that the line is composed of sentences and that this fact is relevant to the rest of your code.
Maybe, this is what you are looking for.
while(input.hasNext())
{
String sentences = input.nextLine();
System.out.println ("count : " + line.split (" ").length);
}
what you are trying to achieve is not quite clear. but if you are trying to count the number of words in your text file then try this
int countWords = 0;
while(input.hasNext())
{
String sentences = input.nextLine();
for(int i = 0; i< sentences.length()-1;i++ ) {
if(sentences.charAt(i) == " ") {
countWords++;
}
}
}
System.out.println(countWords);

How do I change the delimiter from a text file?

Let's say I got a textfile.txt that I want to read from. This is the text in the file:
23:years:old
15:years:young
Using the useDelimiter method, how can I tell my program that : and newlines are delimiters? Putting the text in one line and using useDelimter(":"); works. The problem is when I got several lines of text.
Scanner input = new Scanner(new File("textfile.txt));
input.useDelimiter(:);
while(data.hasNextLine()) {
int age = input.nextInt();
String something = input.next();
String somethingelse = input.next();
}
Using this code I will get an inputMisMatch error.
Try
scanner.useDelimiter("[:]+");
The complete code is
Scanner scanner = new Scanner(new File("C:/temp/text.txt"));
scanner.useDelimiter("[:]+");
while (scanner.hasNext()) {
System.out.println(scanner.next());
}
The output is
23
years
old
15
years
young
Use this code
Scanner input;
String tokenizer[];
try {
input = new Scanner(new File("D:\\textfile.txt"));
input.useDelimiter("\\n");
while(input.hasNextLine()) {
tokenizer = input.next().split(":");
System.out.println(tokenizer[0]+" |"+tokenizer[1]+" | "+tokenizer[2]);
}
}catch(Exception e){}
It will give you output like
23 |years | old
15 |years | young
You have two ways to do this:
Concatenate the string to make it one line.
delimit "newline" first, then delimit ":" each return string token.
If all you want is to get everything split up all at once then I guess you can use
useDelimiter(":\\n")
That should split on both : and newspace but it is not the most efficient way of processing data, especially if each line of text is set out in the same format and represents a complete entry. If that is the case then my suggestion would be to only split on a new line to begin with, like this;
s.useDelimiter("\\n");
while(s.hasNext()){
String[] result = s.next.split(":");
//do whatever you need to with the data and store it somewhere
}
This will allow you to process the data line by line and will also split it at the required places. However if you do plan on going through line by line I recommend you look at BufferedReader as it has a readLine() function that makes things a lot easier.
As long as all the lines have all three fields you can just use input.useDelimiter(":\n");
you probably wants to create a delimiter pattern which includes both ':' and newline
I didn't test it, but [\s|:]+ is a regular expression that matches one or more whitespace characters, and also ':'.
Try put:
input.useDelimiter("[\\s|:]+");

Categories